See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/268685502

Defect Bash - Literature Review

Conference Paper · July 2013

CITATION READS

1 2,035

2 authors:

Xuejiao Zhou Mika Mäntylä

Beijing Forestry University University of Oulu
1 PUBLICATION   1 CITATION    92 PUBLICATIONS   1,816 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

IndAcSE: Meta-analysis of Industry-Academia Collaborations in Software Engineering View project

Need for Speed View project

All content following this page was uploaded by Mika Mäntylä on 24 November 2014.

The user has requested enhancement of the downloaded file.

 Defect Bash - Literature Review

Xuejiao Zhou and Mika Mäntylä

Department of Computer Science and Engineering, School of Science, Aalto University, Espoo,
P.O. Box 19210, 00076 Aalto, Finland
{jiao.zhou, mika.mantyla}@aalto.fi

Keywords: Defect Bash, Literature Review, Software Quality, Software Testing, Verification and Validation.

Abstract: Defect bash is a co-located testing session performed by a group of people. We performed a systematic
review of the academic and grey literature, i.e. informally published writings, of the defect bash. Altogether,
we found 44 items (17 academic and 27 grey literature sources) that were identified useful for the review.
Based on the review the definition of defect bash is presented, benefits and limitations of using defect bash
are given. Finally, the process of doing defect bash is outlined. This review provides initial understanding
on how defect bash could be useful in achieving the software quality and lays foundation for further
academic studies of this topic.

1 INTRODUCTION  The process of doing successful defect bash.

This paper is a literature review of defect bash.
Testing is one of most important software quality In Section 2 we define the literature review protocol.
practices and it aims in finding defects before the Section 3 defines defect bash based on the literature
software is released. Defect bash (or bug bash) is a review. Section 4 lists the benefits and limitations of
testing event where a group of people tries to find as defect bash. The process of defect bash is derived
many defects as possible from the software. It is from the references in the section 5. Finally
widely applied in software companies (Anonymous conclusion is given in Section 6.
2, 2010; Anonymous 3, 2011; Enns, 2004;
Fitzgerald, 2012; Powell, 2009; Sagynov, 2011a, b;
Whittaker, 2012). For example Microsoft used it 2 SYSTEMATIC LITERATURE
quite often (Anonymous 2, 2010; Crowhurst, 2011; COLLECTION
Liangshi, 2010; Sahay, 2012). Marick (1997) sees
defect bash as an additional testing complementing
Systematic literature review (Kitchenham and
written automated and manual test cases. Kaner
Charters, 2007) is used to identify the definition,
(2011) listed defect bash as a technique for black-
benefits and limitations, and process of defect bash.
box testing. It is used in crowd testing (Anonymous
Combination of string (("bug bash" OR "bug
6, 2012) and open source testing (Grubbs, 2012).
bashing" OR "defect bash" OR "defect bashing")
Although, defect bash has been around for
AND “software”) is used for title, abstract and
decades (Dolan and Matthews, 1993), we could not
keywords to identify articles related to defect bash.
find a single research article that would have had
Using the search string we searched both the
primary focus on the defect bash, although it is often
academic and the grey literature. The search engines
mentioned as a side note. This is in stark contrast for
used were Google (www.google.com) for grey
example with code review practice that has been the
literature and Google Scholar (scholar.google.com)
primary focus of several academic articles.
for academic literature. The latter includes the major
In order to shed light to this popular practice, this
academic databases such as IEEE Explore, Scopus
article analyses the existing information on defect
and ACM.
bash with the following fundamental issues:
We found 19,500 items from Google (September
 The definition of defect bash; 8 2012), but only the first 200 results were checked.
 The benefits and limitations of using defect This is because in the results from 130 to 200 only
bash;

125
ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

one relevant page was found. Thus, going deeper in Ramesh, 2008), the defect bash is defined as “an ad
the Google search results would have increased the hoc testing done by people performing different
workload with very small likelihood of finding roles in the same time duration during the
relevant results. Out of the 200 results 36 results integration testing phase, to bring out all types of
were related to research needs and only 27 results defects that may have been left out by planned
from Google are used as references in this article testing. It is not based on any written testing case”.
because some results are duplicated. Classification The definition in Wikipedia (2012) is “a bug
of grey literature results according to the definition bash is a procedure where all the developers, testers,
on the types of grey literature by GreyNet program managers, usability researchers, designers,
International (2012) show the following: we found documentation folks, and even sometimes marketing
four types of grey literature (27): 3 discussion people, put aside their regular day-to-day duties and
forums, 12 blogs, and 12 company websites. pound on the product to get as many eyes on the
66 items were found in Google Scholar (10 April product as possible”.
2013). Among 66 items from Google Scholar, 30 ALLInterviews (2012) and QTP (2012) have the
items were unrelated or unavailable (broken links, or same definition of “it is an ad hoc testing where
books we did not have access); In 20 items defect people performing different roles in an organization
bash only as a term was mentioned, without test the product together at the same time. The
information on definition, benefits and limitations of testing by all the participants during defect bashing
defect bash. This left us with 17 items (academic is not based on written test cases. What is to be
articles and books) containing information on tested is left to an individual's decision and creativity.
definition, benefits and limitations. Totally 44 This is usually done when the software is close to
references are used for this article as in Table 1. being ready to release”.
The classification is done based on the major Combing the above 3 definitions, we can define
content of each reference; this means that one the defect bash as follows: It is a temporally and
reference may contain different kinds of spatially co-located group testing session, done by
information, i.e., definition, benefits and limitation, people from different roles during the integration
and process of doing bug bash. One reference may testing phase or close to software release to bring
be used in several places of this article. out all types of defects that may have been left out by
planned testing. It is not based on written test-cases.

3 DEFINITION
4 BENEFITS AND LIMITATIONS
Several definitions for defect bash were found. Next
we present three definitions that appeared as the 4.1 Benefits
most popular, i.e. these definitions were used in
several places. Table 2 presents the benefits of the defect bash. We
In Desikan and Ramesh’s book (Desikan and can see the most frequently mentioned benefit from

Table 1: Classification of found literatures.

Category n: references
Definition and benefits of 15: ALLInterviews, 2012; Aranda and Venolia, 2009; Birkinshaw and Goddard,
defect bash 2009; ChetanaS, 2011; Desikan and Ramesh, 2008; Dolan and Matthew, 1993;
QTP, 2012; Marick, 1997; Nindel-Edwards and Steinke, 2006; Slaughter and
Rahman, 2011; Wikipedia, 2012; Williams, 1998; Whittaker, 2012; Wong,
2011; Yüksel, Tüzün, Gelirli, and Bıyıklı, 2009
13: Anonymous 1, 2010; Anonymous 4, 2011; Bach,1998; Berkun, 2008; Cruden,
How to do defect bash 2011; Liangshi, 2010; Haynes, 2009; Kalra, 2007; Khan and ElMadi, 2011; Mey,
2012; Powell, 2009; Pruitt and Adlin, 2005; Spagnuol, 2007
Against defect bash 1: Lyndsay, 2011
5: Anonymous 2, 2010; Anonymous 3, 2011; Sagynov, 2011a; Sagynov, 2011b;
Report after defect bash
MarkusN, 2012
Advertisement for doing 10: Anonymous 5, 2012; Anonymous 6, 2012; Crowhurst, 2011; Enns, 2004;
defect bash Fitzgerald, 2012; Grubbs, 2012; Kaner, 2011; Sahay, 2006; Sakai, 2012; Sande, 2009
Total 44

126
 DefectBash-LiteratureReview

defect bashing is finding defects in a short time aspects are also claimed to help in team building
before the software is released. It can also bring out inside a company.
both functional and non-functional defects. The life
cycle of the bugs can be minimized as the reports 4.2 Limitations
can be verified and assigned during the defects bash.
Defect bash acts as a basic sort of usability as Though many benefits are declared in the literature,
well as acceptance testing. People can pound the still there are limitations of defect bash as below by
system from the load in the defect bash. Additionally, them:
defect bash can be used to break the system instead Limitation 1: Defect bash might cause too
of trying to conclude the system works. many duplicate defect reports. The quality of defect
Defect bash brings different people from reports can be low. Time is wasted in investigating,
different roles together in the organization for testing. diagnosing and logging the same problem several
The boundaries between roles are minimized in a co- times (Anonymous 4, 2011; Berkun, 2008; Lyndsay,
located session. Different roles also help validating 2011).
the software from end user perspective. The end Limitation 2: The blog (Lyndsay, 2011) claims
users using a software product will be quite different that in defect bash there isn’t much opportunity to
from each other in many aspects such as learn from each other. This is because many people
understanding about the product, the manner of use the system for the first time, at the same time.
using the software. Defect bash can bring in people Also the limited time period disables learning. We
who have different levels of product understanding think that the defect bash in the first time would be
to test the product together randomly, which can similar to what Lyndsay observed. However after
simulate the different approaches of the end users. It more experience both organizer and participants will
is also recognized that fresh eyes have less bias and learn how to do a defect bash more efficiently.
that fresh eyes can uncover new defects. Limitation 3: Defect bash can only predict
Learning and competitions are also mentioned as customer behavior for the first few hours (Lyndsay,
benefits of a defect bash. The built-in competitive 2011). Thus, it cannot offer information of long-term
instinct of participants should be stimulated to product use. We think that usage by different users
achieve this. Defect bash also helps in learning the even once or short period is better than nothing, and
product and learning from each other. It can be used we maybe should not expect too many feedbacks on
as unofficial demo. The learning and competition customer behaviors from defect bash as Lindsay’s
Table 2: Benefits and the references mentioning it.
Benefit References mentioning the benefit
Anonymous 1, 2010; Anonymous 2, 2010; Anonymous 3, 2011;
Finding many defects, bring out both functional Aranda and Venolia, 2009; Crudden and Lawson, 2011;
and non-functional defects, also shortening the life Desikan and Ramesh, 2008; Haynes, 2009; Karla, 2007;
cycle of the bugs Liangshi, 2010; MarkusN, 2012; Powell, 2009; QTP, 2012;
Sagynov, 2011b; Sahay, 2006; Sande, 2009; Spagnuolo, 2007
The competitive instinct of participants are Haynes, 2009; Birkinshaw and Goddard, 2009; Wong, 2011;
stimulated and good for team building Yüksel, Tüzün, Gelirli, and Bıyıklı, 2009
Saving money (no need to hire group externals) Crudden and Lawson, 2011
Help in rapid evolution of test scripts Bach, 1998
Acting as acceptance testing and usability testing Anonymous 1, 2010; Anonymous 4, 2011; Karla, 2007
Make software more valuable while enhancement
Anonymous 1, 2010; Crudden and Lawson, 2011
done
Cross boundary testing Desikan and Ramesh, 2008
Berkun, 2008; Crudden and Lawson, 2011; Desikan and Ramesh,
Learn your product and team building
2008; Haynes, 2009; Spagnuolo, 2007; Khan and ElMadi, 2011
Anonymous 1, 2010; Crudden and Lawson, 2011; Desikan and
Fresh eyes have less bias
Ramesh, 2008
Users in different levels Desikan and Ramesh, 2008
Not wait for documentation Desikan and Ramesh, 2008
Testing is also to break system Desikan and Ramesh, 2008

127
ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

observation is reasonable. 6 CONCLUSIONS

Limitation 4: Defect bash causes the strain of
available resources for setting test environment with Based on a systematic review of literature of defect
a big group of people (Lyndsay, 2011). However, bash, we make the following conclusions. First,
this limitation can be overcome by careful planning defect bash is defined as a spatially and temporally
of defect bash. co-located testing session performed by a group of
people. As academic studies of defect bash are
lacking, we also decided to search for a grey
5 PROCESS literature in addition to academic literature.
Second, we found several claimed benefits of
Defect bash process is categorized into three phases defect bash. Among them are finding many defects
in this article: preparation, defect bash session and in a short time period, learning the product, team
post-process data as shown in Figure 1. There are building, getting many roles to test the software
two kinds of roles in defect bash process, organizer(s) from different viewpoints and getting fresh eyes to
and participators. Organizers plan the defect bash, search for defects (see Section 4.1). However,
Lyndsay (2011) and handful of other authors
moderate it and analyze the report from participants.
disagreed with the benefits of defect bash, see
Participants just test the software and report findings.
Section 4.2. The most serious limitation, however,
Phase 1 – Preparation: Panic can be caused was the number of duplicated defect reports.
(Berkun, 2008) if there is no preparation before Nevertheless, this disagreement calls for empirical
defect bash. Usually the defect bash events are investigation on the benefits and limitations of
advertised in a certain period earlier (Anonymous 6, defect bash.
2012). Fitzgerald (2012) and Sakai (2012) have a Third, we presented a process for defect bash in
good list of items for the bug bash: goals, when to Section 5. There are two major roles in defect bash,
have defect bash, duration, testing target software, organizer(s) and participants that can represent
testing environment, defect reporting system, various roles from the software development
participants, instructions. Sagynov (2011a) also organization. A defect bash is divided into three
declares the evaluation method and rewards in the phases: preparation, defect bash session and post-
preparation phase. Additionally, management process data. The actions for executing each defect
support needs assured during the proration phase. bash phase in Tables 4, 6 and 8 can be used as the
Table 3 collects all items which might be needed for guidelines for doing defect bash.
the preparation. The preparation actions can be as We think that this work lays out an initial
the items in Table 4. foundation for the future studies of defect bash that
Phase 2 - Defect bash session Analyzing the are needed to understand this industrially relevant
references and the items for the doing defect bash software testing approach. Some examples of future
are collected in Table 5, and the actions in the studies are: Improved guidelines on defect bash
session can be as the items in Table 6. process, collecting data on the detected defects,
Phase 3 - Post-process the data from defect factors affecting the efficiency and effectiveness, the
bash Analyzing the references and the items for spread of knowledge in defect bash and the group
post-processing the data are collected in Table 7, and sizes for defect bash. Additionally, this literature
the actions in the session are in Table 8. review should be extended to cover ‘team
exploratory testing’ practice (Bach 2003; George,
2013; Saukkoriipi and Tervonen, 2012) that appears
to have many similarities with defect bash practice.
Furthermore, similarity between defect bash and
software review meetings exist as both defect bash
and software reviews are group based quality
assurance techniques. The main difference between
defect bash and software review is that defect bash
consists of individuals testing the software while in
software review individuals review software
artefacts’ such as requirements, design or code. As
software reviews are widely studied group based QA
Figure 1: The process of doing defect bash. method (Wiegers 2002) comparison of practices of
defect bash and software reviews should be made.

128
 DefectBash-LiteratureReview

Table 3: Items on the phase 1 – preparation.

Items of preparation References for each item
Goals Flush out the bugs (Crowhurst, 2011; Fitzgerald ,2012; Sakai, 2012)
Informed 1 month earlier (Anonymous 6, 2012), A given time (Fitzgerald, 2012;
Time Sakai, 2012), clear afternoon (Berkun, 2008; Crowhurst, 2011); Schedule in key
milestone (Mey, 2012)
Get support from management Big shot (Berkun, 2008), Haynes, 2009; Pruitt and Adlin, 2005
Where Have a bug bash headquarter (Berkun, 2008); Haynes, 2009
3-5 hours (Anonymous 1, 2010), 60 minutes (Fitzgerald, 2012; Sakai, 2012),
Duration
(Cruden and Lawson, 2011), 30 minutes (Cruden and Lawson, 2011)
Focus (Anonymous 4, 2011; Fitzgerald, 2012; Sakai, 2012), Freeze the build and a
Target software or focus
Focus (Berkun, 2008; Crowhurst, 2011)
Server (Fitzgerald, 2012; Sakai, 2012), in the same environment (Cruden and
Testing environment
Lawson, 2011)
Registered users (Sakai, 2012; Fitzgerald, 2012), everyone in the company (Cruden
Participants
and Lawson, 2011); market person, developers, technical writers (Haynes, 2009)
Defects reporting system Jiras (Fitzgerald, 2012; Sakai, 2012)
Give out $50 Amazon gift card (Sagynov, 2011a), criteria to judge the bug
Evaluation method and reward
(Anonymous 2, 2010)
How to connect the testing server, do testing and report findings (Sakai, 2012;
Instructions
Fitzgerald, 2012)
Inform earlier Participants informed earlier (Cruden and Lawson, 2011)

Table 4: Actions in phase 1 – preparation.

Performer Actions
Organizer Planning defect bash:
 Goals – detect all defects as much as possible;
 Time – when to have it?
 Support from management;
 Where to have defect bash?
 Duration – how long a session lasts? Duration can be varied depending on project, 30 minutes,
3-5 hours, and whole afternoon;
 Target software – which software build should be used? Optimizing the effort involved in
defect bash. For example, it can be classified into: Feature/component defect bash; Integration
defect bash; Product defect bash (Desikan and Ramesh 2008).
 Testing environment – in what testing environment defect bash can be done? Setting up
testing environment;
 Participants – who will participate in the defect bash?
 Reporting system – how the findings should be reported?
 Instructions on how to do it;
 Evaluation method, who will win and what is the rewards?
Sending invitation to all usually at least one week ago and invite a big manager.
Participators Not involved yet.

Table 5: Items on the phase 2 - defect bash session.

Items of doing defect bash References for each item
Build team or create a rival teams Anonymous 1, 2010; Anonymous 4, 2011; Berkun, 2008
Explain what to do briefly Anonymous 1, 2010; Anonymous 4, 2011
Show a list of known issues and the format of good report Anonymous 4, 2011; Berkun, 2008
Let participants perform on their own Anonymous 1, 2010; Haynes, 2009
Keep scores Anonymous 1, 2010; Berkun, 2008
Encourage reporting Anonymous 4, 2011
Doing ad hoc testing Anonymous 4, 2011
Snacks offered Anonymous 4, 2011, Haynes, 2009; Mey, 2012
Taking pictures for fun Haynes, 2009;

129
ENASE2013-8thInternationalConferenceonEvaluationofNovelSoftwareApproachestoSoftwareEngineering

Table 6: Actions in phase 2 - defect bash session.

Performer Actions
Organizer Create rival teams; compose a team having a good mix of experienced, new and untrained people to
participate in this exercise. Or each participant does testing him/herself independently.
Explain system briefly, show known issues and show a good error report to participants at beginning.
Provide focus of testing areas; let them use the software without interruptions. Do not let them discuss
each other during the testing.
Allow participants report as they like. Anything like issues faced, crashes encountered, questions,
comments, and general feedbacks and any suggestions or enhancements.
Keep scores who found what.
Have snacks and drinks offered and pictures may be taken for fun.
Participators Doing ad hoc testing for detecting the defects.

Table 7: Items on the phase 3 - post-process data.

Items in post-process References for each item
Analyse all the issues, filter out the known issues Anonymous 1, 2010; Anonymous 3, 2011; Haynes, 2009
Criteria to judge the winner Anonymous 2, 2010
Winners Anonymous 2, 2010; Anonymous 3, 2011
Appreciation Anonymous 2, 2010; Anonymous 3, 2011; Mey, 2012

Table 8: Actions in the phase 3 - post-process data.

Performer Actions
Analyse all the issues and suggestions and summarize them for the team.
 Issues should be compared with the error tracker to check for duplicates;
 All duplicated issues grouped into one issue;
 Filter out the known issues. The left ones will probably be new encountered defects, suggestions
Organizer
and enhancement that will be used to improve the software further.
Send appreciation email or broadcasting information to all participants to tell the results, how many new
defects, who found the most important defects and reward the person or which team (if rival teams) who
found the most important defects.
Participators Receive the acknowledgement from organizer and rewards.

REFERENCES .com/99tests-bug-bash/.
Aranda, J., Venolia, G., 2009. The secret life of bugs:
ALLInterviews, 2012. What is meant by defect bash? In going past the errors and omissions in software
link: http://www.allinterview.com/showanswers/23853 repositories. In IEEE 2009. Article ID: 978-1-4244-
.html. 3452-7/09.
Anonymous 1, 2010. How to do a test bug bash for your Bach, J.A., 1998. Microdynamics of process evolution. In
software project? In link: http://bettersoftwaretesting. IEEE Computer Society, February 1998.
blogspot.com/2010/10/how-to-do-test-bug-bash-for- Bach, J.A., 2003. Exploratory testing explained. In link:
your.html. http://www.satisfice.com/articles/et-article.pdf.
Anonymous 2, 2010. The WDK community bug bash Berkun, S., 2008. How to run a bug bash? In link:
contest 2010 is over. In link: http://www.osronline. http://www.scottberkun.com/blog/2008/how-to-run-a-
com/page.cfm?name=bugbash. bug-bash/.
Anonymous 3, 2011. Bug bash aftermath. In link: Birkinshaw, J., Goddard, J., 2009. The management
http://seleniumhq.wordpress.com/2011/01/31/bug- spectrum. In Journal Compilation, London Business
bash-aftermath/ School.
Anonymous 4, 2011. Software bug bash tips. In link: Chetanas, 2011. What is defect bash? In link: http://
http://programmers.stackexchange.com/questions/470 www.testingken.com/forum/showthread.php?t=346.
07/software-bug-bash-tips. Crowhurst, C., 2011. Bug bash 2011 – A direct response.
Anonymous 5, 2012. Bug bash on Uhuru software. In link: In link: http://www.marketingarchitects.com/2011/
https://www.odesk.com/o/jobs/job/Bug-Bash-on- 05/bug-bash-2011-a-direct-response/.
Uhuru-Software_~~7390d00ba7cef979/. Cruden, K., Lawson, N., 2011. The Power of the bug bash.
Anonymous 6, 2012. 99tests Bug bash. In link: http://99tests In link: http://www.agilequalityassurance.com/2010/

130
 DefectBash-LiteratureReview

04/the-power-of-the-bug-bash/. Pruitt, J., Adlin, T., 2005. The personal lifecycle: Keeping
Desikan, S., Ramesh, G., 2008. Software testing, principle people in mind throughout product design.
and practices. Dorling Kindersley (India) Pvt. QTP Tutorials and Interview Questions, 2012. In link:
Ltd.123-126. ISBN 978-81-7758-121-8. http://qtp.blogspot.fi/2010/03/bug-bash-defect-bash.
Dolan, R.J., Matthews, J.M., 1993. Maximizing the utility html.
of customer product testing: beta test design and Sagynov, E., 2011a. CUBRID bug bash event! In link:
management. J PROD INNOV MANAG 1993;10:318- http://www.cubrid.org/blog/news/cubrid-bug-bash-
330. Elsevier Science Publishing. event/.
Enns, N., 2004. It is bug bash day on 28 April 2004. In Sagynov, E., 2011b. CUBRID bug bash event results. In
link: http://blogs.msdn.com/b/windowsmobile/archive/ link: http://www.cubrid.org/blog/cubrid-life/cubrid-
2004/04/28/122435.aspx. bug-bash-event-results/
Fitzgerald, K., 2012. Sakai bug bashes. In link: Saukkoriipi, S., Tervonen I., 2012. Team exploratory
https://confluence.sakaiproject.org/display/3AK/Bug+ testing sessions. In International Scholarly Research
Bashes. Network, ISRN Software Engineering, Volume 2012,
George, C., 2013. Team exploratory testing: our first Article ID 324838.
experiences... In link: http://mostly-testing.blogspot.fi/ Sahay, A., 2006. Microsoft, Appin launch security bug
2013/01/team-exploratory-testing-our-first.html. bash 2006. In link http://press.xtvworld.com/
GreyNet International, 2012. http://www.greynet.org/ article12437.html.
greysourceindex/documenttypes.html. Sakai, 2012. QAE QA. Call for bug bash. In link:
Grubbs, J.C., 2012. Chicago open source bug bash. In https://oae-community.sakaiproject.org/~oae-qa#l=
link: http://www.meetup.com/chicago-open-source- Bug-Bashes/Bug-Bashes.
bug-bash/. Sande, S., 2009. Bug-bashing Bento 2.0v5 is now
Haynes, D., 2009. The search for software robustness. In available for download. In link: http://www.tuaw.
Excerpt from Pacific NW Software Quality com/2009/08/19/bug-bashing-bento-2-0v5-is-now-
Conference. available-for-download/.
Kalra, P., 2007. Bug bash. In link: http://rivr.sulekha. Slaughter, J., Rahman, M., 2011. Information security plan
com/bugbash_265217_blog. for flight simulator applications. In International
Kaner, C., Fiedler, R.L., 2011. Black box software testing. Journal of Computer Science & Technology (IJCSIT),
Introduction to test design. A survey of test techniques. Vol 3, No 3, June 2011. DOI: 10.5121/ijcsit.
In BBST Test Design. In http://www.testingeducation. 2011.3301.
org/BBST/testdesign/BBSTTestDesign2011pfinal.pdf. Spagnuolo, C., 2007. The bug bash sprint. In link:
Kitchenham, B., Charters, S., 2007. Guidelines for http://edgehopper.com/the-bug-bash-sprint/
performing systematic literature reviews in software Wang, L., 2011. Master project. Best practice for testing in
engineering. Software Engineering Group, School of development and testing groups. In
Computer Science and Mathematics, Keele University, [http://etd.dtu.dk/thesis/277238/]
Tech. Rep. EBSE-2007-01, July 2007. Whittaker, J.A., 2012. The 10-minute test plan. In IEEE
Khan, M.S.A., ElMadi, A., 2011. Data warehouse testing Software by IEEE Computer Society. Article ID: 0740
– an exploratory study. Software engineering master -7459/12.
thesis no: MSE-2011-65. In School of Computing Wiegers, Karl Eugene. Peer reviews in software: A
Blekinge Institute of Technology SE-371 79 practical guide. Boston: Addison-Wesley, 2002.
Karlskrona, Sweden. Wikipedia, 2012. Bug bash. In link:
Liangshi, 2010. Ce shi za gan, bug bash. In link: http://en.wikipedia.org/wiki/Bug_bash#cite_note-0.
http://www.51testinzg.com/html/63/n-225363.html. Williams, G., 1998. Usability process challenges in a web
Lyndsay, J., 2011. Known ways of managing ET #2 – bug product cycle. In HCT’98 Conference Companion.
bash. In link: http://workroomprds.blogspot.fi/2011 Yüksel, H.M., Tüzün, E., Gelirli, E.., Bıyıklı, E., 2009.
/12/known-ways-of-managing-et-02-bug-bash.html. Using continuous integration and automated test
Marick, B., 1997. Class Testing mistakes. In http:// techniques for a robust C4ISR system. In IEEE 2009.
www.csi-chennai.org/swtws/ws-swt/mistakes.pdf. Article D: 978-1-4244-5023-7.
MarkusN, 2012. Big bug bashing for GRASS 6! In link:
http://gfoss.blogspot.fi/2012/08/big-bug-bashing-for-
grass-6.html.
Mey, C.V., 2012. Shipping greatness. Practical lessons on
building and launching outstanding software, learned
on the job at Google and Amazon. By O’Reilly Media.
Nindel-Edwards, J., Steinke, G., 2006. A full life cycle
defect process model that supports defect tracking,
software product cycles and test iterations. In
Communications of the IIMA 2006 Volume 6 Issue 1.
Powell, C., 2009. ABAKAS bug bash. In link:
http://blog.abakas.com/2009/01/bug-bash.html.

131

View publication stats