Download as PDF, PPTX
![Extending Kubernetes with Custom Resources
Example: Life Cycle of NuclioFunction Resources*
• Cluster initialization [Administrator]
o Register NuclioFunction CRD
o Deploy Nuclio Controller
o Subscribe Nuclio Controller to notifications for NuclioFunction (and related) events
• Function deployment [Application Owner]
o NuclioFunction resource is created
o Nuclio Controller is triggered by the API server
o Nuclio Controller creates necessary K8s resources (Pods, routes, etc)
• Function invocation [Application User]
o Trigger is invoked (e.g., http request)
o Scaling logic is invoked (if necessary)
o Payload is forwarded to the Pod selected to run the function invocation
29CMU CS 15719 Advanced Cloud Computing S19February 25, 2019 * Conceptual worfklow](https://image.slidesharecdn.com/719s19-l11-kubernetes-190225231738/75/Cloud-Native-Application-and-Kubernetes-29-2048.jpg)
Uploaded byAlex Glikson
Cloud-Native Application and Kubernetes
The document provides an overview of cloud-native applications and Kubernetes (k8s), detailing the architecture and principles of k8s as a robust platform for deploying container-based applications. It discusses the unique requirements of cloud-native applications, including scalability, automation, and failure management, using examples such as Netflix. The document also highlights k8s features like its scheduling mechanisms, pod architecture, and integration with custom resources to enhance application development and management.
More Related Content
![[OpenInfra Days Korea 2018] Day 2 - E4 - 핸즈온 워크샵: 서버리스가 컨테이너를 만났을 때](https://cdn.slidesharecdn.com/ss_thumbnails/e41300when-180704061740-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Cloud-Native Application and Kubernetes
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Cloud-Native Application and Kubernetes
- 1. Cloud-Native Applications and Kubernetes*(k8s) Alex Glikson 15-719, S19 February 25th, 2019 CMU CS 15719 Advanced Cloud Computing S19 1 *Trivia: Κυβερνήτης – steersman, a person who steers a ship https://en.wiktionary.org/wiki/%CE%BA%CF%85%CE%B2%CE%B5%CF%81%CE%BD%CE%AE%CF%84%CE%B7%CF%82 Icons made by Freepik from www.flaticon.com February 25, 2019
- 2. Outline February 25, 20192 Background • Cloud-Native Applications K8s Overview • K8s In a Nutshell • Architecture Applications • Unit of Deployment: Pod • Composite Applications Control Plane • Scheduler • Custom Resources and Controllers Summary • Related Projects • Main Takeaways CMU CS 15719 Advanced Cloud Computing S19
- 3. Cloud-Native Applications: Motivation •Elasticity and ubiquity of cloud infrastructure enabled new generation of applications, with a potential to disrupt their markets, or to create new markets, e.g.: o Netflix o Airbnb o Spotify o Pinterest o Snapchat o Whatsapp 3CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 4. Cloud-Native Applications: Example,Requirements • Example: Netflix o Value proposition (competitive advantage): • Low cost video streaming with superb user experience, at scale o Application properties and unique requirements: • >100 millions of users in 190 countries (most of them in the US) – Vast variance in load, within minutes (evenings, campaigns, etc) • 10s of thousands of servers – At least one server will fail every day • 1000s of daily application changes, across 100s of functions – Video streaming, catalog, recommendations, subscription, etc – ~1 update every minute 4CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 5. Cloud-Native Applications: Requirements→ Design Principles • Low Cost + Variance in Load (+User Experience) o Can’t afford over- or under-provisioning → Auto-Scaling • Hardware Failures + High Scale (+User Experience) o Accommodate HW failures without downtime → Design for Failure • Frequent Application Updates + High Scale (+User Experience) o Can’t afford redeploying everything every time → Modularity o Can’t afford testing everything every time → Stable Internal APIs • Frequent Application Updates + Low Cost (+User Experience) o Can’t afford manual QA/admin effort for each update → Automation 5CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 6. Cloud–Native Applications: Design→ Common Services Design Principle Common Service Auto-Scaling Horizontal Auto-Scaling Elastic Load Balancing Design for Failure Replication Health Monitoring Modularity Decoupling into homogenous (micro)services* Unified packaging (across dev/test/prod) with Docker API-driven composition Discovery Routing Automation Fully programmable life cycle of components* Observability (monitoring, tracing, etc) 6CMU CS 15719 Advanced Cloud Computing S19February 25, 2019 * not a service
- 7. Cloud–Native Applications: Platforms •New platforms emerged, offering common services (features) that make it easier to develop cloud-native applications o Auto-scaling, replication, load balancing, health monitoring, service discovery, application-level routing, programmability o Commonly referred as ‘Platform as a Service’ (PaaS) 7 Google App Engine CMU CS 15719 Advanced Cloud Computing S19February 25, 2019 Kubernetes
- 8. Outline February 25, 20198 Background ✓ Cloud-Native Applications K8s Overview • K8s In a Nutshell • Architecture Applications • Unit of Deployment: Pod • Composite Applications Control Plane • Scheduler • Custom Resources and Controllers Summary • Related Projects • Main Takeaways CMU CS 15719 Advanced Cloud Computing S19
- 9. Kubernetes (k8s): Open-Source, Cloud-Native& Container-Based Platform Open source platform for deployment and management of arbitrary container-based, cloud-native applications on clusters of servers 9 • Originally developed by Google, based on Borg and Omega ideas • Available to all on github.com • As of 02’2019: ~2000 contributors • Common Services • Programmable • Extensible Physical or virtual, public or private cloud (e.g., EC2,OpenStack) • Specification • Provisioning • Monitoring • Logging • HA • Scaling • Updates CMU CS 15719 Advanced Cloud Computing S19February 25, 2019 Packaging and runtime isolation using Docker • Auto-Scaling • Design for Failure • Modularity, APIs • Automation
- 10. Kubernetes in aNutshell 10 https://www.youtube.com/embed/4ht22ReBjno?start=0&end=220&autoplay=1 CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 11. Kubernetes in aNutshell 11CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 12. Kubernetes Architecture 12 Inspired by:https://x-team.com/blog/introduction-kubernetes-architecture/ Master Node API Server DB Proxy ControllersControllersControllers DB (etcd) Scheduler Worker Node Docker …Pod Pod Pod kubelet Worker Node Docker …Pod Pod Pod kubelet kubectl, dashboard, SDKs CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 13. February 25, 2019 Pod:One or More Containers + Resources + Labels Why Multi-Container Pods? • Scaling/replication unit: runtime components that must run together • Sidecar pattern: “secondary” functions deployed in separate containers – e.g., web content synchronization • Ambassador/Proxy pattern: simplify access to an external system – e.g., key-value store • Adapter pattern: simplify access from an external system – e.g., monitoring • Source: https://www.usenix.org/node/196347 13 Source: https://www.slideshare.net/ssuser6bb12d/kubernetes- introduction-71846110 Containers in a Pod often share (some) namespaces Labels app: myApp Horse + saddle CMU CS 15719 Advanced Cloud Computing S19
- 14. Event-Driven Architecture ink8s: Pod Creation 14 Source: https://blog.heptio.com/core-kubernetes-jazz-improv-over-orchestration-a7903ea92ca CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 15. Outline February 25, 201915 Background ✓ Cloud-Native Applications K8s Overview ✓ K8s in a Nutshell ✓ Architecture Applications ✓ Unit of Deployment: Pod • Composite Applications Control Plane • Scheduler • Custom Resources and Controllers Summary • Related Projects • Main Takeaways CMU CS 15719 Advanced Cloud Computing S19
- 16. From Pods toApplications: Deployment, Replica Set • Goal: Maintain a given number of replica’s of a given Pod o HA, load balancing • Example: NGiNX web server 16 Pod template Containers specification Resource type Desired # of replicas Reference by labels nginx.yml Polo team CMU CS 15719 Advanced Cloud Computing S19February 25, 2019 Replica Set properties
- 17. Composite Application Typesin Kubernetes • Deployment (Replica Set) o Maintains a given number of replica’s of a given Pod • High availability, load balancing • Daemon Set o Runs exactly one instance of a given Pod on every node • Access to unique host resources, load balancing • Stateful Set o Maintains a scalable collection of Pods with unique roles • Persistent resource and identities • Job o Enforces policies for execution and completion of groups of tasks (e.g., MPI) • Controlled parallelism, work queues, schedule-based execution, etc • Custom • Your own 17 Polo team Life guard Goal keeper Relay team CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 18. Outline February 25, 201918 Background ✓ Cloud-Native Applications K8s Overview ✓ K8s in a Nutshell ✓ Architecture Applications ✓ Unit of Deployment: Pod ✓ Composite Applications Control Plane • Scheduler • Custom Resources and Controllers Summary • Related Projects • Main Takeaways CMU CS 15719 Advanced Cloud Computing S19
- 19. Kubernetes Default Scheduler •Main goal: placement of Pods on Nodes o Triggered primarily by watching for new Pods in ‘pending’ state • FIFO scheduling o Supports priority-based arbitration and eviction • Emphasis on scale and speed o No complex logic (e.g., superlinear in number of nodes) 19CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 20. Kubernetes Default Scheduler:Placement Policies • Filter() + Prioritize() • Filtering: o Configurable set of node predicates o A node must pass all predicates to qualify for prioritization • Prioritization: o Optimization goals expressed as utility functions o Prefer nodes with maximal utility • Can target multiple optimization goals with weights 20CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 21. Placement Policies: Built-InPredicates • PodFitsResources – CPU/memory allocation • PodMatchNodeSelector – Labels match - e.g., specific HW capabilities (e.g., SSD storage) • MatchInterPodAffinity – Same/Different node as given (running) Pod(s) 21CMU CS 15719 Advanced Cloud Computing S19February 25, 2019 Node 1 8 CPUs 16 GB mem Node 2 8 CPUs 32 GB mem Node 3 8 CPUs 32 GB mem disk=SSD 4 CPU 8GB mem app=Web 3 CPU 8GB mem app=DB 4 CPU 16GB mem NodeSelector: disk=SSD ? 4 CPU 16GB mem Anti-Affinity: app=DB ?
- 22. Placement Policies: Built-InPriority Functions • InterPodAffinity – Maximal affinity with given Pods • LeastResourceAllocation – Maximal spreading • MostResourceAllocation – Maximal packing • BalancedResourceAllocation – Minimal resource fragmentation • ImageLocality – Minimal startup time 22CMU CS 15719 Advanced Cloud Computing S19 Node 1 8 CPUs 16 GB mem Node 2 8 CPUs 32 GB mem Node 3 8 CPUs 32 GB mem disk=SSD 4 CPU 8GB mem app=Web 3 CPU 8GB mem app=DB 1 CPU 8GB mem PrefferedAffinity: app=DB ?
- 23. Placement Policies: Built-InPredicates and Priorities Predicates* • PodFitsResources – CPU/memory allocation • PodMatchNodeSelector – Labels match - e.g., specific HW capabilities (e.g., SSD storage) • MatchInterPodAffinity – Same node as given (running) Pod(s) Priorities* • InterPodAffinity – Maximal affinity with given Pods • LeastResourceAllocation – Maximal spreading • MostResourceAllocation – Maximal packing • BalancedResourceAllocation – Minimal resource fragmentation • ImageLocality – Minimal startup time 23 (*) Partial List. Can also implement custom predicates/prioritizers CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 24. Placement Policies: Affinity/ Anti-Affinity Examples • Affinity = same host/rack/zone/etc, Anti-affinity = different • Example 1: o If a host fails, at most one pod out of N is down (high availability) • Replica Set with N replicas, host-level anti-affinity • Example 2: o Maintain one pod in each region, to reduce latency to clients • Daemon Set with region-level anti-affinity • Example 3: o Keep all tasks of a job in the same rack, for better performance • Job with rack-level affinity 24CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 25. Affinity/Anti-Affinity Example: WebServer + Redis 25CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 26. Placement Policies –Example: Web Server + Redis 26 Bug!! Different hosts Same host Reference by labels CMU CS 15719 Advanced Cloud Computing S19February 25, 2019 If respective Redis server is killed, the web server is not re-deployed
- 27. Outline February 25, 201927 Background ✓ Cloud-Native Applications K8s Overview ✓ K8s in a Nutshell ✓ Architecture Applications ✓ Unit of Deployment: Pod ✓ Composite Applications Control Plane ✓ Scheduler • Custom Resources and Controllers Summary • Related Projects • Main Takeaways CMU CS 15719 Advanced Cloud Computing S19
- 28. Extending Kubernetes withCustom Resources • Example: Nuclio serverless function as a custom resource type in k8s • Under the covers, Nuclio Controller maintains K8s Pods for each function February 25, 2019 28Source: https://github.com/nuclio/nuclio NuclioFunction resource instance CMU CS 15719 Advanced Cloud Computing S19 New resource type definition: NuclioFunction
- 29. Extending Kubernetes withCustom Resources Example: Life Cycle of NuclioFunction Resources* • Cluster initialization [Administrator] o Register NuclioFunction CRD o Deploy Nuclio Controller o Subscribe Nuclio Controller to notifications for NuclioFunction (and related) events • Function deployment [Application Owner] o NuclioFunction resource is created o Nuclio Controller is triggered by the API server o Nuclio Controller creates necessary K8s resources (Pods, routes, etc) • Function invocation [Application User] o Trigger is invoked (e.g., http request) o Scaling logic is invoked (if necessary) o Payload is forwarded to the Pod selected to run the function invocation 29CMU CS 15719 Advanced Cloud Computing S19February 25, 2019 * Conceptual worfklow
- 30. Outline February 25, 201930 Background ✓ Cloud-Native Applications K8s Overview ✓ K8s in a Nutshell ✓ Architecture Applications ✓ Unit of Deployment: Pod ✓ Composite Applications Control Plane ✓ Scheduler ✓ Controllers Summary • Related Projects • Main Takeaways CMU CS 15719 Advanced Cloud Computing S19
- 31. Related Project (PartialList) • Virtual Kubelet o Proxy to another container management platform • kube-batch aka kube-arbitrator o Gang scheduling (e.g., MPI), non-FIFO scheduling • Knative o Serverless web applications on Kubernetes • De-scheduler o Eviction of pods – e.g., to reduce resource fragmentation • Rapidly growing ecosystem of projects governed by CNCF o https://www.cncf.io/ 31CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 32. February 25, 2019 Summary:Main Takeaways 1. Docker + Cloud-Native ➔ K8s o Container-native platform targeting arbitrary applications 2. Pod = collection of containers o Composable into Replica Sets, Daemon Sets, Jobs, Your Own Pattern 3. K8s API server provides CRUD and watchers (“events”) o Triggering a set of controllers that orchestrate the life cycle of resources 4. K8s has a powerful, flexible and extensible Scheduler 5. K8s is fun! 32CMU CS 15719 Advanced Cloud Computing S19
- 33. QUESTIONS? February 25, 201933CMU CS 15719 Advanced Cloud Computing S19
- 34. Trends: Linux Containers •Linux features providing isolation within a single OS: o Hardware resources: cgroups • CPU, RAM, I/O devices, etc o OS resources: namespaces, e.g.: • USER, PID, NET, MNT • De-facto standard to efficiently run services/jobs at scale o Enable workload consolidation, infrastructure elasticity o Pioneered by Google (Linux features, massive internal use*) • 2 billion containers per week! (>3000 per second on average) 34CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 35. Trends: Linux Containers– Implementations • Example (open source) implementations: o LXC, Cloud Foundry, Mesos, YARN, … • Docker (Moby*) o Targeting arbitrary applications, e.g. by offering: • Flexible packaging • Seamless inter-container network communication • Configurable cluster-wide storage/network (SDN, NAS, etc) o Developer-friendly • Lightweight standalone engine • Intuitive git-style image management • Easy sharing and reuse of images - Docker Hub 35CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 36. Trends: Linux Containers– Popular Docker Images* Programming LanguageData ProcessingWeb ServerOperating System 36CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 37. 37 Cloud- Native PaaS + = Container-native Platforms (e.g., Kubernetes) Docker Pre-history 1979:chroot Job scheduling 1999: VMware 2003: Xen 2004: UnionFS 2006: cgroups 2008: LXC 2013: Docker 2017: moby containerd 2004 2006 2008 2010 2012 2014 2016 2018 2006: EC2 2012: YARN 2011: Mesos 200?: Borg 2013: Omega 2002: ns 2011: Cloud Foundry AWS Beanstalk OpenShift 2008: GAE 2007: Heroku 2014: Kubernetes 2010: Azure CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 38. Surfacing Applications toUsers: Service • Goal: Maintain a given number of replica’s of a given Pod o HA, load balancing • Example: NGiNX web server 38 Pod template Resource type Port mapping Reference by labels “Deployment” resource “Service” resource Polo team CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 39. Another Example ofComposite Application Type: Job • Goal: enforce policies for execution and completion of groups of tasks (batch jobs, scheduled jobs) 39 Relay race CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 40. Resource Orchestration inKubernetes: Controllers 40 Source: https://github.com/kubernetes/community/blob/master/contributors/devel/controllers.md Image: https://comicvine.gamespot.com/coach-kleats/4005-1744/images/ CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 41. February 25, 2019 ControllerExample - Replica Set for Deployment myD: 41 Event: User changed #replicas Event: Pod failed Note: not real code CMU CS 15719 Advanced Cloud Computing S19
- 42. • One ofthe most vibrant projects on Github o 48K stars, 16K forks, 44K PRs, 75K commits, 2000 contributors https://github.com/kubernetes/kubernetes/pulse/monthly Kubernetes: Open Source Project and Community 42 9 lines of code per minute CMU CS 15719 Advanced Cloud Computing S19February 25, 2019
- 43. Kubernetes: Commercial Ecosystem •Governed by Cloud Native Computing Foundation (CNCF) o Industry-wide consensus among leading (and competing) companies • Multiple commercial Kubernetes products and hosted offerings o Red Hat, IBM, CoreOS, etc; Google GKE, Amazon EKS, etc 43CMU CS 15719 Advanced Cloud Computing S19February 25, 2019