SRE-iously
1 like279 views
This document discusses site reliability engineering (SRE) practices at New Relic. It describes New Relic's transition from a monolithic architecture to microservices, and the establishment of an SRE team with both embedded and dedicated roles. The SRE team aims to continuously improve the reliability of New Relic's platform. Key aspects of SRE success outlined include reliability as a feature, shared understanding, clear guidelines, and community building.
SRE-iously
- 1. ©2008–18 New Relic, Inc. All rights reserved
- 2. ©2008–18 New Relic, Inc. All rights reserved Tori Wieldt, Sr. Solutions Manager SRE-iously! Defining the Principles, Habits, and Practices of Site Reliability Engineering
- 3. ©2008–18 New Relic, Inc. All rights reserved This document and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission. Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,” “expects” or words of similar import. Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov. New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.
- 4. ©2008–18 New Relic, Inc. All rights reserved THE SRE HANDBOOK But what if you’re not Google?
- 5. ©2008–18 New Relic, Inc. All rights reserved A LITTLE BACKGROUND Ruby Monolith Siloed teams Infrequent Releases TO 300+ Microservices 50+ Engineering Teams with embedded SREs 20-70 Deploys a Day Huuuge Kafka Cluster *with embedded SREs FROM
- 6. ©2008–18 New Relic, Inc. All rights reserved HOW IT WAS *with embedded SREs On-Premises On Premises Relational Data Customers NoSQL Data Store Public Cloud Micro Services API Mobile Apps Browser
- 7. ©2008–18 New Relic, Inc. All rights reserved WE ASKED OUR STAKEHOLDERS
- 8. ©2008–18 New Relic, Inc. All rights reserved WE ASKED OUR STAKEHOLDERS Why do we have SREs at New Relic? What’s the vision for our SRE team? How can SREs most effectively contribute to the future of our platform?
- 9. ©2008–18 New Relic, Inc. All rights reserved 8©2008–18 New Relic, Inc. All rights reserved
- 10. ©2008–18 New Relic, Inc. All rights reserved Continuously improve the reliability of systems in the New Relic platform ONE GOAL 8©2008–18 New Relic, Inc. All rights reserved
- 11. ©2008–18 New Relic, Inc. All rights reserved TWO ROLES 9 “Pure” SRE Build and support our core internal platform: Container Fabric Networking Systems Partner with Eng Teams Domain Experts in: Reliability Tooling Scaling Embedded SRE
- 12. ©2008–18 New Relic, Inc. All rights reserved THREE SPHERES 10 RELIABILITYSTABILITY ENGINEERING
- 13. ©2008–18 New Relic, Inc. All rights reserved 11 • Fix team problems manually. • Track down hardware defects on servers. • Provision _______________. TOIL
- 14. ©2008–18 New Relic, Inc. All rights reserved 11 • Fix team problems manually. • Track down hardware defects on servers. • Provision _______________. TOIL Repetitive work that scales linearly.
- 15. ©2008–18 New Relic, Inc. All rights reserved 12 Encourage Best Practices • Audit runbooks • Hold “game days” • Meet with product architects • Create team risk matrices Stay Current
- 16. ©2008–18 New Relic, Inc. All rights reserved 13 Reduce Sprawl • Migrate teams to our code pipeline • Build tools • Clean up alerts • Build an integration Improve Monitoring
- 17. ©2008–18 New Relic, Inc. All rights reserved KEYS TO SRE SUCCESS 14 Reliability is a feature Reliability depends on shared understanding SRE is a challenging, cross-disciplinary practice
- 18. ©2008–18 New Relic, Inc. All rights reserved KEYS TO SRE SUCCESS 14 Reliability is a feature Query your stakeholders Reliability depends on shared understanding SRE is a challenging, cross-disciplinary practice
- 19. ©2008–18 New Relic, Inc. All rights reserved KEYS TO SRE SUCCESS 14 Reliability is a feature Query your stakeholders Reliability depends on shared understanding Develop clear, specific guidelines SRE is a challenging, cross-disciplinary practice
- 20. ©2008–18 New Relic, Inc. All rights reserved KEYS TO SRE SUCCESS 14 Reliability is a feature Query your stakeholders Reliability depends on shared understanding Develop clear, specific guidelines SRE is a challenging, cross-disciplinary practice Build a strong SRE community
- 21. ©2008–18 New Relic, Inc. All rights reserved A TEMPLATE FOR SUCCESS 15 Determine Your Goal Example: Continuously improve the reliability of the systems of our company’s platform Establish Roles Example: Pure SRE Embedded SRE 21 3 Focus Areas Examples: Stability Reliability Engineering
- 22. ©2008–18 New Relic, Inc. All rights reserved DETAILS IN OUR SRE EBOOK 16 bit.ly/NewRelicSRE @ToriWieldt
- 23. ©2008–18 New Relic, Inc. All rights reserved FIRESIDE CHAT 17 TIM O’BRIEN Managing Director, Cloud Services and Information Security
- 24. ©2008–18 New Relic, Inc. All rights reserved FIRESIDE CHAT 17 TIM O’BRIEN Managing Director, Cloud Services and Information Security
- 25. ©2008–18 New Relic, Inc. All rights reserved THANK YOU! @ToriWieldt linkedin.com/in/toriwieldt
- 26. ©2008–18 New Relic, Inc. All rights reserved WHAT SRES DO 19 Champion reliability best practices. Guide designs and processes with an eye toward resilience and low toil. Reduce technical complexity and sprawl. Drive the usage of tooling and common components. Implement software and tooling to improve resilience and automate operations.
- 27. ©2008–18 New Relic, Inc. All rights reserved