Testing Tools Landscape
Download as PPTX, PDF0 likes891 views
The document discusses the importance of effective testing tools and strategies for online businesses to enhance quality and performance, emphasizing that poor testing can lead to significant losses. It outlines the challenges faced in testing, such as performance and data privacy, while recommending a collaborative approach and early testing to prevent defects. Additionally, it highlights the need for businesses to invest in quality, automation, and shared goals among team members for improved user experience and operational efficiency.
Testing Tools Landscape
- 2. Testing Tools - How Teams Make It WorkMargo Visitacion, Vice PresidentFebruary 17, 2011
- 3. Don’t lose before you get into the game……
- 4. Agenda
- 5. If you don’t plan your quality and build the right team, you lose before you get into the game…….
- 6. More business is transacted online….
- 7. But problems increase your risk and costs …Performance:It only takes a moment to lose customers74% of surveyed customers shop online….…. Yet 54% abandon shopping carts, a continuing plague for online vendors vendorsSecurityOnline banking: the future is now90.5 million Americans now bank online…..….Yet less than half of customers feel secure about their institutions’ securityScalability70% of auto policies renew in JanuaryThe global Internet population is now 1.6B and expects to grow another 42% by 2014Do you have the right team in place?
- 8. In 2010 more money was directed to toward growth and expansion – don’t let poor performance shift dollars away from innovationPoor quality diverts focus
- 9. Agenda
- 10. Siloed approach to testing won’t let you answer these questions
- 11. Companies are planning to spend more on securing their customer’s loyalty
- 12. How will you change your behaviors to improve your approach to testing??How will you “Change your ways”?Challenges in testing are growing Performance, Data privacy, complianceTesting as usual won’t work todayEarly testing is imperativeTeams look different because testing approaches have to change IT budgets look better, but must bring valueCompanies want greater flexibility for their $$$Automation MUST bring value for the money invested Testing is ripe for SaaS21% are piloting SaaS for IT lifecycle components26% are planning to adopt within the next year
- 13. Taking a team approach
- 14. Testing can begin from day one to prevent defectsPost MortemDeploymentBusiness determinesobjectives, sets requirementsEnd to end testingUATBusiness stakeholders, BAs & test team set quality and performance requirements for functional and non functional requirementsDevelopment &testing continuesContinuous build and integrationContinuous testingmanual and automatedStatic and automated performance testing startsDesign and developmentbased upon requirementsSet test planning &Automation strategy
- 15. Use automation to keep up with change Capital investments are scrutinized Infrastructure and environments are challenged Ramp up time takes too long Usability is a requirement Quick time to value is a necessity Performance testing is cyclical but non negotiableTesting is an enablerEnvironmentsansconstraints
- 16. Agenda
- 17. Costs: A drill down
- 18. Why? Investing in quality can reduce risk, cost and raise value
- 19. Plan Today For Tomorrow’s Budget
- 20. Agenda
- 21. To win the game, the business needs to be involvedRequirementsUserstories “Situational realities”Risks/opportunities
- 22. Bringing the team together needs a repeatable, measurable process
- 23. Agenda
- 24. RecommendationsDon’t leave performance testing to the end. Problems will grow as business options expand Anticipate the unexpectedLife is what happens while you’re busy making other plansLeverage automation and timingFlexible tools to performance-test, “just in time”
- 26. Testing Tools Landscape- Load TestingMatt Poepsel, VP of Performance Strategies, Compuware Gomez
- 27. Customer Expectations Are Rising For Web & Mobile Applications47% of consumers expect a Web page to load in 2 seconds or less 58% of mobile phone users expect websites to load almost as quickly or faster on their mobile phone than their PC
- 28. The Challenge Of Delivering Quality Web ExperiencesSystems management tools: “OK”…user is NOT happyThe Web Application Delivery ChainTest & monitor your site the way your customers use it:What they do (key pages and transactions)
- 29. Where they do it (geographic locations)
- 30. How they do it (browsers and mobile devices)
- 31. When they do it (normal and peak usage)Determine the impact on their behavior and your businessInconsistent geo performance
- 32. Bad performance under load
- 38. Page size too big
- 39. Conflicting HTML tag support
- 40. Too many objects
- 41. Content not optimized for device
- 42. Low cache hit rate
- 48. Code defects
- 51. Outages“Outside-in” customer point of viewNetwork resource shortage
- 53. SMS routing / latency issues
- 57. Low cache hit rateApplications are no longer what you build inside your firewall10.5Number of hosts accessed directly by the browser, per user transaction, averaged across 3,000 companies
- 58. Who Owns End-User Quality?Line of Business ManagersProduct ManagersMarketersInterface DesignersApplication DevelopersArchitectsQuality Assurance TeamsInfrastructure/Operations TeamsHelp Desk StaffQuality is a Collective Function.Everybody owns a piece of the puzzle.Avoid at all costs:Individual Success, Collective Failure
- 59. How can we work together?
- 60. Situational Assessment and ImprovementProvide this 5-minute assessment to each of the End-User Quality contributors for your application:The results provide a baseline. Share the results across the team.Identify areas of improvement and execute a plan to get better.Re-administer the assessment after 90 days.On a scale of 1 (poor) to 10 (excellent), how well are we are doing in these areas:Shared GoalsShared MetricsCommunicationsSolving problems as a teamCelebrating as a team
- 61. QuestionsIncreased conversions 10% Gomez Customers Enjoy Measurable BenefitsReduced homepage load time from 11.3 seconds to 3.4 secondsImproved page load times 23%Saved 50%+ in staff and feesReduced seven-step transaction time by 50%Reduced downtime 45% Achieved under 3 second response time and 99%+ availabilityValidated decision to consolidate three data centersFor more information visit Gomez.comorcontact us at +1 781.778.2700
Editor's Notes
- #3: Delete second Name/Title if not needed, but be sure to keep the Month Day, Year information. For a teleconference, after the date, add a period and “Call in at XX:55 p.m. Eastern time” (change the time to five minutes prior to the start of the teleconference).
- #5: Don’t lose before you get into the gameWhy load testing can make the differencePlanning performance testing today can help budget planning tomorrowDeveloping your test game plan
- #6: Source: http://www.flickr.com/photos/elzey/3985788016/
- #8: Talk about rich internet apps, more media, very appealing, but
- #10: Don’t lose before you get into the gameWhy load testing can make the differencePlanning performance testing today can help budget planning tomorrowDeveloping your test game plan
- #16: Dial
- #17: Don’t lose before you get into the gameWhy load testing can make the differencePlanning performance testing today can help budget planning tomorrowDeveloping your test game plan
- #21: Don’t lose before you get into the gameWhy load testing can make the differencePlanning performance testing today can help budget planning tomorrowDeveloping your test game plan
- #24: Don’t lose before you get into the gameWhy load testing can make the differencePlanning performance testing today can help budget planning tomorrowDeveloping your test game plan
- #28: Not only are the usage of social and mobile increasing, but customer expectations about the performance are extremely high.
- #29: Matt tie into how they do it and when they do it… normal and peak usage!
- #30: When we talk about applications now, we are increasing talking about applications that are not isolated, and consist of a number of components delivered by a Number of hosts accessed directly by the browser, and assembled at the client end. This necessitates a different approach to testing and Load testing specifically.
- #31: Common Mistake:$ cost of marketing campaigns associated with driving peak traffic – driving priorityBusiness Objectives and KPIsvs the IT team’s ability to deliver.Impact on Business:Lost sales due to the site not being availableLessons Learned (case example):Nationwide – $1.5 Mill marketing to MicrositeBOA – Used microsite to generate leads – drove traffic to find breaking point (What can they support?)Informing Business of what the site can handle before they drive traffic to it
- #32: Common Mistake:Organizations think that performance management/testing is only an IT problemBusiness team is not asked to (or able to) provide key web metrics to inform the load test planBusiness team is not informing (or being asked to inform) IT of performance objectives to allow for business goals to be metBusiness team not informing IT of business/traffic growth projections for “next year”IT is not open to involving business inputIT has to be honest about their ability to deliver what they have to deliver with what they haveImpact on Business:“Successful” load test; unsuccessful peak traffic eventLessons Learned (case example):Retail Brand Alliance – regular testing with business involvement allows for confidence, sales success, and more effective planning for next year.???
- #34: Q & a session