Data Science at Atlassian: The transition towards a data-driven organisation
3 likes1,094 views
Atlassian transitioned to becoming a more data-driven organization by forming a Product Growth Team to discover value in customer behavioral data and champion data-driven decision making. They built analytics infrastructure to collect and analyze large amounts of customer event data. Some challenges included educating people that analytics is about more than just counting, building new infrastructure to replace silos and improve data quality, and overcoming reluctance to move away from gut feelings. A/B testing was used to constantly improve products in a measurable way, conducting over 100 experiments last year. Overall, Atlassian applied the scientific method used in other fields like medicine to make their development more evidence-based and data-driven.
![Thank you!
Dr Ilias Flaounas
!
ilias [dot] flaounas [at] atlassian.com](https://image.slidesharecdn.com/capitalmarkets-150523094323-lva1-app6892/85/Data-Science-at-Atlassian-The-transition-towards-a-data-driven-organisation-40-320.jpg)
More Related Content
Similar to Data Science at Atlassian: The transition towards a data-driven organisation (20)
Data Science at Atlassian: The transition towards a data-driven organisation
- 1. Data Science at Atlassian The transition towards a data-driven organisation Dr Ilias Flaounas Capital Markets CRC, Sydney 13 May 2015
- 3. Atlassian - Trivia • Atlassian is an Australian enterprise software company specialised in products that support project management and collaboration. • ~12 years old • Latest valuation at $US3.3 billion in 2014, (x8 since 2010). • >1.5K employees • Headquarters in Sydney • Best place to work in Australia (The Australian Financial Review, 2014)
- 4. • ~43,000 enterprise customers • Tens of thousands more own $10 starter and free community licenses
- 5. JIRA
- 6. JIRA
- 8. Key Requirements:! • Support from upper management! • Some minimum critical mass of customers! • Serious investment in infrastructure, people, time
- 9. Product Growth Team • Team was formed ~2 years ago • Main mission: growth! • Discover value in heaps of behavioural data! • Champion data-driven decision making
- 10. Analytics Infrastructure (v.2) ~3-4TB overall (highly compressed)! 800GB per day produced (uncompressed)
- 12. We analyse events, i.e., user actions Timestamp Instance User Event Attributes 1356958862 mySuperCompany John Login 1356968863 mySuperCompany Anne Login 1356978862 mySuperCompany John dashboard view 1356988862 mySuperCompany Anne dashboard view 1357884442 anotherCompany John view issue {issueID = 1234} 1357956862 mySuperCompany John create issue {issueID = 1000} 1357957862 mySuperCompany Anne view issue {issueID = 1000} 1357958862 mySuperCompany Anne comment issue {issueID = 1000} 1357959862 mySuperCompany John create issue {issueID = 1001} … … … … …
- 13. The Challenges • Build infrastructure! - break silos! - replace existing “solutions”! - add instrumentation! - improve data quality! - deal with catch-22…
- 16. The Challenges • Educate! - “analytics is counting”! - help people ask the correct questions • Build infrastructure! - break silos! - replace existing “solutions”! - add instrumentation! - improve data quality! - deal with catch-22…
- 17. JIRA Agile! Which Agile report is used the most?
- 18. However, counting is hard… Report 1 Report 2 Report 3 Report 4 Report 5 Report 6 Report 7 Report 8 Report 9
- 20. No usage
- 21. No usage Low usage
- 22. No usage Low usage Medium usage High usage
- 23. … and beyond counting…
- 25. Predicting conversions How many variables can you measure?
- 26. Top customers none, n/a, home, self, test, personal, atlassian, “-”, na, private, abc, personal use, freelance, my company,student, testing, company, no company, private, mycompany, a, no, test company, noname, testcompany, self employed, individual, tester, me, “.” , independent, trial, myself, nocompany, xyz
- 27. Predicting conversions • Baseline set at 50% • Using only the sign-up information we have an accuracy of 65%
- 28. The Challenges • Convince - challenge gut feelings - demonstrate value • Educate! - “analytics is counting”! - help people ask the correct questions • Build infrastructure - replace existing “solutions” - data quality - catch-22…
- 29. The argument of common sense How Common Sense Fails Duncan Watts, 2011 κοινὴ αἲσθησις Aristotle, 384–322 BCE
- 31. • Source: P. Chopra, The ultimate guide to A/B testing, http://www.smashingmagazine.com, 2010
- 32. A/B Tests • A/B tests allow a constant improvement of products in a measurable and predictive way • Last year we deployed ~118 experiments. • The volume of customers, evaluators, visitors, has become a limited and valuable resource
- 33. Subject: %%first_name%%, assemble the bitbucketeers! Subject: Grab the Bitbucketeers...4, 3, 2, 1 LAUNCH!
- 37. Do A/B tests slow down development? “Move fast with stable infra” Mark Zuckerberg Facebook F8, 2014 “Pauca sed matura” i.e., “Few, but mature” Carl Friedrich Gauss 1777-1855 “Move Fast and Break Things”“Move Fast and Break Things”
- 38. Instead of a Conclusion: We simply recycle the scientific method from other fields, e.g., medicine… Patient User Medical doctor Product manager Illness Bad User Experience Diagnosis Interviews, surveys… Treatment UX modification Medical researchers Growth team Observations Data mining Medical trials A/B tests Ethics Privacy Policy Evidence-based medicine Data-driven development
- 40. Thank you! Dr Ilias Flaounas ! ilias [dot] flaounas [at] atlassian.com