The machinery
around microservices
Tools & practices for operating
microservices @
Sangeeta Narayanan (@sangeetan)
Director, Edge Developer Experience
Dec 2016
#NetflixOffline
>83M subscribers
>1000 device types
>42B streaming hours in 2015
600 hours of original content
1000s of playbacks per second
3 AWS regions; 3 AZs each
Subscribers/year (millions)
20
40
60
80
2007 2009 2011 2013 2015
Subscribers/year (millions)
20
40
60
80
2007 2009 2011 2013 2015 2016
??
http://bit.ly/2gFd6mq
Many things have to come together for a successful launch of
this complexity
http://www.martinfowler.com/articles/microservices.html
Quality
Velocity
Availability
http://bit.ly/2fWUNd9
Increasing scale
http://bit.ly/2gVNWQT
Making
microservices work
Tools, Practices, Culture
http://bit.ly/2h7Q2NY
CDN data plane not shown
Architectural benefits
❖ Scalability
❖ Resiliency
❖ Velocity
❖ Autonomy
❖ Dynamic Routing
❖ Traffic shaping
❖ Insights, debugging
❖ Traffic failover
Custom cloud gateway
Philosophies
❖ Own your service - operate
what you build
❖ Move fast - optimize for
velocity
❖ Stay safe - prepare for failure
❖ Focus on automation
http://bit.ly/2fydxy6
Develop Deploy
Operate Support
App Development &
Delivery
Insights
Netflix engineers own the entire lifecycle of their services. They are provided a rich set of tools
and libraries to simplify their tasks.
Development
❖ Standardized platform
❖ IPC
❖ Service discovery
❖ Logging, Monitoring
❖ Configuration Management
❖ and more
❖ DevEx tooling
❖ API discovery and documentation
❖ Usage tracking and lifecycle management
❖ Error simulations and behavior overrides
http://netflix.github.io/
Deployment
❖ Automated canary analysis
❖ Self-service canaries for developers
❖ Automated regional deployments
❖ Choice of deployment strategy
❖ Pipelines for config changes
http://bit.ly/2g0H5RO http://bit.ly/2fXA0GA
Support
❖ Diverse audience
❖ Internal partners: service consumers, dependencies,
customer services
❖ External partners
❖ Production and lower environments
❖ Micro level insights
Fine grained insights via real time event stream queries
Request call graph
Request profiling
Operational Insights
❖ Aggregate health dashboards
❖ Real time alerting
❖ Anomaly detection
❖ Automated correlation analysis
Anomaly Detection
Failure testing
Drills and “game-day” exercises
Latency and error simulations
Automated traffic failover
Production Readiness Checklist
Culture
Context; not Control
http://bit.ly/2h93a1f
Highly aligned
Loosely coupled
Freedom
and
Responsibility
http://www.martinfowler.com/articles/microservices.html
People
Practices
Technology
Thank You

Making Microservices work at Netflix