SCRUM & Testing
“Back to the Future”
Erik van Veenendaal
www.erikvanveendaal.nl
Erik van Veenendaal
Founder and major shareholder ImproveQS
In testing since 1989 working for many different
clients and in many different roles
Author “TMap”, “The Testing Practitioner” and many
other books and papers
Vice-President International Software Testing
Qualifications Board (ISTQB) 2005 - 2008
Vice-Chair TMMi Foundation
Keynote speaker, e.g. EuroSTAR, STAReast
Winner of the European Testing Excellence
Award (2007)
Improve Quality Services BV 2
Improve Quality Services BV
www. improveqs.nl
Service organisation in the area of
Testing, Requirements Engineering and
Quality Management
Consultancy, Subcontracting and Training
Testing (TMap, TMMi) SW Process Improvement
Test Process Improvement Quality Assurance
Certification (ISTQB) IT-Auditing
- incl. Advanced !! Requirements Engineering
Inspections / Reviews & management (IREB)
Improve Quality Services BV 3
What is Agile?
Improve Quality Services BV 4
Only 28% of IT projects
Why Agile? is successful !!
Deliver business value earlier
AGILE
Divide and conquer
The requirements problem
Reliable Estimates
Formal methodology
6% 5% Minimized scope
Standard software
8% 10%
REQ ENG
infrastructure
Clear business
12% objectives
Executive 18%
support
16%
User
14% involvement
Experienced
Project Manager 6%
5%
Firm basic requirements
Other
Improve Quality Services BV 5
Agile Development Model
Iteration
High level product
requirements
High level Product
iteration
requirements
“Definition of ‘Done”
Design
Implementation & unit testing
Acceptance testing
Documentation
Acceptance
…
Improve Quality Services BV 6
Testing Challenges
It’s simple but not easy
Obvious for agile (iterative) development:
Regression testing, Test automation
Quality of specification, Traceability, Time-boxing
Less obvious to most “experts”:
Knowledge and skills of test professionals
The (new) role of developers
The involvement of the business
Management awareness
Test levels and test types
Improve Quality Services BV 7
Other Test Levels?
Improve Quality Services BV 8
What is agile testing?
• Testing in an immature (test) organization
• The great excuse !!
• Testing in an agile development project
• Re-shaping traditional practices
• Testing following the statutes of the agile
manifesto
• A different mind set to start from
Improve Quality Services BV 9
Agile Manifesto
Individuals & interactions over Processes and Tools
Working Software over Comprehensive Documentation
Customer Collaboration over Contract Negotiation
Responding to Change over Following a Plan
Agile Values (Kent Beck)
Communication, Feedback
Simplicity, Courage
Improve Quality Services BV 10
Core Agile Testing Practices
Risk based: Product Risk Analysis
communication, customer collaboration, working
software & responding to change
Reviews: Walkthroughs & Informal Reviews
communication & feedback, customer collaboration
Test strategy: (automated) unit testing & regression testing
working software over comprehensive documentation
Test design: Use informal techniques formally
responding to change, customer collaboration &
no comprehensive documentation
People: Build experienced and skilled testers
individuals and interactions over processes and tools
Improve Quality Services BV 11
The Product Risk Matrix
Exploratory Testing Test design techniques
H Couldtests
Support module Test Must Test
Review design
Review module tests Support module tests
Review module tests
x
I II
x x
M
Likelihood x
III IV
x
Exploratory Testing Test design techniques
L “Won’t Test” Shoulddesign
Review Test
L M H
Impact
Improve Quality Services BV 12
tests
Exploratory Testing PRODUCT
Simultaneous exploration, test design and execution
Finding the most important defects in the time available
What about …………
- Test charters available and reviewed
- Checklist with most common defects (heuristics)
- Experienced testers (test design techniques)
- Working in pairs during test sessions
- Test logs delivered
- Daily debriefing sessions
- Feedback loop; new risk areas
Improve Quality Services BV 13
People Skills via (exchanging) Practical
Experiences,, Coaching
Experiences
and (formal) Training
Test knowledge IT knowledge
- Test management - Software development
- Techniques - Requirements (IREB)
- Tools, etc. - Configuration mgt.
Domain Soft skills
knowledge - Communication
- Business process - Critical mindset
- User characteristics - Team player
Improve Quality Services BV 14
SCRUM in Five Minutes
Sprint – a focused effort for a 30-day period
towards fixed goals
Product Owner – responsible for compiling
and maintaining a prioritized product backlog
(a to do list)
Sprint backlog – to do list for Sprint
Scrum team – self organized and jointly
responsible (5 - 9 people)
Scrum Master – coaches the scrum team and
creates the best possible circumstances
Improve Quality Services BV 15
SCRUM Process
Review, Design
Understand requirements Test execution
Estimation sessions Report, Adjust Participate Retrospective
High level test design Coach, Support System-integration-test?
Product risk analysis (test plan) Define fit criteria End-to-end test?
Definition-of-done Daily SCRUM Non-functional tests?
Review requirements
Improve Quality Services BV 16
Experiences - Finance Domain
Domain knowledge Problem areas
extremely important Product quality hard to
Manage on process, not measure
functionality Watch-out for the V-model
Exploratory testing & also Task not well split-up
test design techniques
Business will try to
Automated regression squeeze in one more CR
testing (QTP)
Continuous time pressure
Never a dull moment
Time-to market is not 4
Great to work in a team weeks
Belbin assessments
Improve Quality Services BV 17
Experiences – Medical Domain
Product Risk Analysis Problem areas
Detailed definition-of-done Non-Functional Testing
Pair programming Synchronization
Unit tests & code coverage How much
documentation?
Exploratory testing
Automated regression test
Weekly product demo
Test levels outside sprint
Empowerment
Retrospective meetings
Improve Quality Services BV 18
Experiences – Embedded SW
Product Risk Analysis Problem areas
Detailed definition-of-done Testers profile
Agile Master Test Plan Documentation
Pair programming Coherent SCRUM and
Unit tests & code coverage other testing
Exploratory testing Works great for SW, but …
Automated regression test Reliability
No changes last 4 days
Test levels outside sprint
Design Reviews
Improve Quality Services BV 19
The Agile Tester
Focus is on delivering business value
Provides a skeptical view of the quality
of the system
Does more than “just” test
Also performs test management activities
Great communicator, coach and team player
Has courage to get out of his comfort zone
What about the test manager?
Improve Quality Services BV 20
How to start?
Process
Define an overall test strategy blue--print
blue
Define an approach to development (unit) testing
People
Define role and develop skills of SCRUM testers
Change mindset and develop testing skills of
developers
Test automation
“Only” part of phase two ….
Improve Quality Services BV 21
Finally . . .
Future: A different process and mindset
Back: Testing is not as easy as it seems
(again underestimated, pffffff ..)
Study theory, but listen to practitioners
TestingX
= Tester
Professional testers can re-use their
knowledge and skills
Improve Quality Services BV 22
Agile Humor
Thank You !!
Improve Quality Services BV 23
