A/B Testing: You Might Just Be Doing it Wrong
2 likes743 views
The document outlines best practices for A/B testing, emphasizing the importance of setting goals, taking incremental steps, and achieving statistical significance. It details Kissmetrics' approach, requiring a sample size of 4,000 and 99% statistical confidence before drawing conclusions. The discussion also extends A/B testing beyond click conversions, underscoring the need for comprehensive data analysis throughout the marketing funnel.
A/B Testing: You Might Just Be Doing it Wrong
- 1. PRESENTED BY + A/B Testing You Might Just Be Doing it Wrong
- 2. + Tomasz Borys Director of Marketing, KISSmetrics @tbcali Andrew Caravella VP of Marketing, Sprout Social @andrewcaravella
- 3. + Table of Contents • The optimum strategies for A/B testing • How we A/B Test at KISSmetrics • A/B Testing beyond click conversions - having a pulse on the entire funnel and why it’s important • The influence social traffic has in the funnel
- 4. + The Optimum Strategies for A/B Testing • Set a Goal • Baby Steps • Aim for Statistical Significance • Never Lose Sight
- 5. + Set a Goal
- 6. + Baby Steps People want to move the needle and see results fast, by implementing multiple changes at once. But how can you be sure what element had an impact?
- 7. + Aim for Statistical Significance SET A GOAL Aimfor Statistical Significance getdatadriven.com
- 8. + Never Lose Sight We can caught up on what we think is best to optimize conversions or traffic… but data doesn’t lie
- 9. + How We A/B Test at KISSmetrics • 4,000 and above sample size • 99% Statistical Confidence • Blind Eye for 1 Week
- 10. + 4,000 and Above Sample Size significant or decent amount of data is needed when you’re testing beyond click conversions
- 11. + 4,000 and Above Sample Size significant or decent amount of data is needed when you’re testing beyond click conversions 99% Statistical Confidence
- 12. + Blind Eye for 1 Week Data can be very erratic the first several days, so it’s easy to hit the panic button
- 13. + A/B Testing Beyond Click Conversions
- 14. + Original
- 15. + Variant
- 16. + Signups
- 17. + Signups
- 18. + Signups
- 23. + Custom Data
- 24. + Custom Data
- 25. + Custom Data
- 30. Get a free 30-day trial sproutsocial.com