FlyingCloud: Building Docker Containers with SaltStack
- 1. Copyright © MetaBrite, Inc. 2016 FlyingCloud Building Docker Containers with SaltStack Adam Feuer and George V. Reilly
- 2. Copyright © MetaBrite, Inc. 2016 The Story of Lectern • Lectern Web Service • Receives uploaded images of receipts from CookBrite mobile app • OCRs images using Tesseract + OpenCV • Extracts data about purchases • Deployed on AWS ElasticBeanstalk • Could not get app to build on Amazon Linux
- 3. Copyright © MetaBrite, Inc. 2016 Lectern v1 • ElasticBeanstalk also supports Docker • We built a Docker app on top of an Ubuntu image • Solved the build problem. Sort of. • Our build was a twisted, unmaintainable, monolithic mess of shell scripts. • Took an hour to build. Any failure reset the clock.
- 4. Copyright © MetaBrite, Inc. 2016 Lectern's old Docker image setup scripts
- 5. Copyright © MetaBrite, Inc. 2016 Lectern v2: with FlyingCloud
- 6. Copyright © MetaBrite, Inc. 2016 One Big Idea • Use Configuration Management to build Docker images • Configure rather than write code • More understandable • Reusable and repeatable
- 7. Copyright © MetaBrite, Inc. 2016 What is Docker? • Lightweight Virtualization • Run multiple containers on one host system • Isolated process group for each container • Shared kernel • Run a different Linux OS in each container • Image = filesystem snapshot • Container = running image
- 8. Copyright © MetaBrite, Inc. 2016 What is SaltStack? • Configuration management system • "Automated system administration" • Configuration, not scripts • Declarative, not imperative • Salt is implemented in Python • Though you usually write Salt States in YAML • We use Salt Masterless mode • Configure system at build time; no runtime dependency on Salt
- 9. Copyright © MetaBrite, Inc. 2016 Two Ways to Build Docker Images • Traditional • Dockerfile: static list of commands • Run commands inside container, then commit • FlyingCloud • Use SaltStack to update state of Docker image • Easy to make layers • Easy to run in Continuous Integration
- 10. Copyright © MetaBrite, Inc. 2016 Example App - layers overview
- 11. Copyright © MetaBrite, Inc. 2016 Example App - project and base layers
- 12. Copyright © MetaBrite, Inc. 2016 Example App - app layer
- 13. Copyright © MetaBrite, Inc. 2016 Example App - testrunner layer
- 14. Copyright © MetaBrite, Inc. 2016 FlyingCloud Demo See https://github.com/cookbrite/flyingcloud/tree/master/examples/flask-web-app
- 15. FlyingCloud ● Use Configuration Management to build layered Docker images ● An FC Layer can be made from any of these: ○ A Dockerfile, Salt states, or custom Python code ○ Do not confuse with Docker’s layer filesystem ● Layers are buildable and/or runnable ● Can be pushed to or pulled from a Docker repository
- 16. Copyright © MetaBrite, Inc. 2016 Kubernetes • Google’s open source platform that automates deployment, scaling, and operations of application containers built on Docker images across clusters of hosts • Our Kubernetes workflow for new services: • Build Docker images with FlyingCloud • Configure Kubernetes cluster • Test cluster in Continuous Integration • Deploy cluster to AWS
- 17. Copyright © MetaBrite, Inc. 2016 FlyingCloud is Open Source • Source: https://github.com/cookbrite/flyingcloud • Docs: http://flyingcloud.readthedocs.org • PyPI: https://pypi.python.org/pypi/flyingcloud/ • Mail: https://groups.google.com/group/flyingcloud-users • Apache License Contributors welcome! Especially: • SaltStack users • Ansible users
- 18. Copyright © MetaBrite, Inc. 2016 We are organizing the world's product information We’re Hiring! Located in Pioneer Square. • Back-end devs. We're a Python shop. • Android devs • iOS devs • Data scientists Contact • [email protected] or [email protected]