Fredrik Rydberg - Can Exploratory Testing Save Lives - EuroSTAR 2010
0 likes902 views
Fredrik Rydberg, a consultant at Addq, emphasizes the importance of exploratory testing (ET) in fully verifying requirements and identifying defects that may be missed with scripted testing. His experience at Maquet shows that ET enhances efficiency and quality in testing, while session-based testing management (SBTM) has provided a structured approach to implementing ET within projects. Future improvements for ET and SBTM could include enhanced reporting and mobile tool options.
More Related Content
Similar to Fredrik Rydberg - Can Exploratory Testing Save Lives - EuroSTAR 2010 (20)
Fredrik Rydberg - Can Exploratory Testing Save Lives - EuroSTAR 2010
- 2. Presentation Fredrik Rydberg AddQ Consulting Started employment in 2001 at the Med Tech company Maquet Critical Care (Stockholm Sweden) as Test Manager and later Unit Manager for testing. During the last 3 years Fredrik has been a Test Management consultant within sectors as Banking & Finance, Utilities (electricity) and Transport.
- 3. Can exploratory testing save lives? Fredrik Rydberg [email protected] +46 735 142 143Experiences from MAQUET
- 4. ProductDescriptions Anesthesia System Ventilator ( =Respirator) http://www.youtube.com/watch?v=NbxxITf-N2Q
- 5. Description Ventilator Testning Measured value Set value P Flow V
- 6. Scripted requirement based testing at Maquet •Authorities/ standards •A ”Must” forced by security and quality •Easy to evaluate when exit criteria is met •Large amount of requirements to test –time consuming! •Mostly manual test cases •Old test cases result in few new defects!
- 7. Detailed requirements and scripted testing is absolutely needed but is not sufficient to fully verify the requirements !
- 8. Exploratory testing at Maquet •Started spontaneously approx. 16 years ago •Both requirements and parts not specified as req’s •Also non functional tests •User oriented •Cost effective •Intellectual •Suitable method for clinical experts and other expertise
- 9. Comparison Scripted testing -ET Scripted testing 1.Test Design with Test Steps 2.Execution 3.Logging in protocol
- 10. Exploratory Testing •Parallel test design and execution •Level of documentation is varying Fully scripted Less guidence Fragmentic test cases charters Freestyle exploratory testing Exploratory testing Comparison Scripted testing -ET
- 11. Pitfalls/ experiences ET at Maquet •Different testers have different ET skills •Risk of decreased motivation without management and control… •…But management and control may influence creativity negatively •Difficult to monitor what has been tested without SBTM
- 12. Important capabilities for ET, Maquet •Clinical competence •Domain and technical knowledge •Experience of previous testing/ project •Test competence (testing techniques) •Curious, patient, communication and networking skills
- 13. Variation in the project cycle (ET/ scripted) ETScripted Time (months) Test Time / month (hours)
- 14. Scripted testing YIN Slow, cold and calm Exploratory testing YANG Focused, warm and aggressive Important to find the right mix!
- 15. Time distribution ET execution18% Automatic execution (scripted) 10% Script execution27% Script test reports5% Designing Scripted Test cases25% Other15% Efficient more times7 ET7)urdefects/ho(5.0)rdefect/hou(1) test timeET%(18) test timeScripted%(67:Efficiency Comparison
- 16. Visit by James Bach (spring 2007) •2 days visit •Inspiration talk for employees •Participation in testing •”pep talk” •Improvement advice •Target of ET clearer
- 17. General Test tool support •Maquet developed tool for session based testing (“Session Based Tester”) •IBM Clear Quest, Requisite Pro •Capture Replay tool •Automatic loggings tool statistics IBM Requisite Pro
- 18. SBTM-Session Based Test Management (Method) •General method to manage, report and follow up ET •Time box approx. 90 min (60-120 min) •Supported by charters •A result possible to evaluate •Often combined with debriefing (approx. 15 min) Session-based testing is developed by Jonathan and James Bach (first presented in year 2000). See http://www.satisfice.com/sbtm for more info.
- 19. Session Based Testing Tool (Open Source) •Maquet developed tool •Started in use Aug. 2008 •Developed in Perl using SQL server •Example field New session: •Project •Duration •Which tests to be done? •Which have been tested?
- 20. Example functionality: •Add new session •Report session •Metrics: Fault intensity •Archiving •Test Reports Session Based Testing Tool (Open Source)
- 21. Advantage SBTM Maquet •Easier to manage and follow up ET •Reuse of sessions •Statistics •Complementing the Test Report •Participation from other departments •Learn from other test sessions •Common ET process
- 22. ET today at Maquet •ET recognized and powerful test approach •ET has resulted in an increased test focus at Maquet •ET is the most efficient test approach •ET contribution to Market Leading product
- 23. Future possibilities •Continue to improve ET/ SBTM •Improved debriefing/reporting •ET + SBTM training for new employees Suggestions future functionality (SBTM): •Dynamic Metrics (parameter list) •Validation / write protection of sessions •iPhone/ Android version? •Other tools?
- 24. Inspiration Exploratory testing Inspiration links: www.satisfice.com www.developsense.com www.kaner.com http://testingeducation.org/wp/ Maquet SBT Open Source: http://sites.google.com/site/sessionbasedtester/ Join the “Linked in” group “Session based testing”: http://www.linkedin.com/groups?mostPopular=&gid=3316116
- 26. Summary •ET needed to fully verify requirements •Other defects with ET compared to ST •Important to find the right mix •Increased efficiency with ET •SBTM (with tool) made ET more structured •ET significant quality contribution