Introduction to automated environment management with Docker Containers - for platform specialists and developers

Introduction to automated environment management with Docker Containers - for platform specialists and developers

• 1.
  Lucas Jellema AMIS PlatformSIG - 1 October 2015 Introduction to automated environment management with Docker Containers - for platform specialists and developers
• 2.
  2 Overview
• 3.
  3 Overview • Docker –what is behind the hype? • A grasp on terminology such as Container, Build, Ship and Run – Images, registry, micro services, • Enough to participate in the discussion • How to get started and going with Docker – On Windows (or Mac OS) • What can Docker mean to me – In my personal R&D environment – In my daily work situation • What’s Oracle’s position regarding Docker? • What is Docker’s current status and near by future • Handson resulting in – A Docker enabled environment – Several containers running (Ubuntu, Wordpress, Oracle 11g XE DB, JDeveloper) – A stepping stone for creating additionals containers and interdependencies
• 4.
  4 What can Dockergive us (developers/platform specialists) • An efficient way – in terms of work and physical machine resources – to rapidly create, easily manage isolated environments – That can be created and deleted easily – That start and stop rapidly – That can be exchanged simply – That consume minimal system resources • Use cases: – Quickly try out some software (release) – without upsetting your laptop – Easily create and distribute a complete training environment – Prepare an environment for deployment to a cloud environment – Locally construct and test a potentially complex environment that is then transfered to a shared infrastructure – Implement a micro services infrastructure – Emulate/Test complex clusters on a single host
• 5.
  5 Docker - Run
• 6.
  6 Demo – Runour first container dockerhost port 80 IP 172.17.0.6
• 7.
  7 Demo – Runa second container based on same image
• 8.
  8 Demo – tworunning NGINX containers; port forwarding for one port 80 port 80 port 90 IP 172.17.0.7 IP 172.17.0.6
• 9.
  9 Demo: run containerin interactive mode using shell dockerhost /tmp/mynewfile.txt
• 10.
  10 Demo: restart containerand attach to locate my file dockerhost /tmp/mynewfile.txt
• 11.
  11 Final status demo:Running Multiple Containers in a Single Linux Host port 90 port 80 port 80 /tmp/mynewfile.txt /var/lib/docker/aufs/diff/… 6af69dd33af4…/tmp/mynewfile.txt
• 12.
  12 Run • Docker Containerruns Linux – as does the host • Container is isolated - feels as stand alone run time environment – Directory structure, IP address, users and groups • Shared resources with underlying host (and therefore other containers) – memory, CPU, host • Light weight: – Quick starting up and stopping – Leverages underlying Linux, only adds what is different/additional • Clusters of containers – Dynamic adding/removing containers from clusters can be done very quickly (Google) – Especially when containers are stateless – (no shared session state in containers; possibly in joint cache, shared file system or NoSQL database) • Management tools – to monitor and manage individual containers and clusters of containers (dynamically scale up/scale down) Docker Host Docker Container Docker Container • ip address • directories & files • users & groups • process table
• 13.
  13 When it walksand talks like a Virtual Machine… • Virtual Machine vs Container
• 14.
  14 Containers vs VirtualMachines • Containers do not run through a hypervisor – They are logically isolated from the host and each other – They run the same kernel as the host (i.e. Linux!) • Containers share resources (the kernel) with host – the rest of the OS can be unique per container (namespaces on host in mounts, process table, network resources) • Containers are based on images defined in a layered file system; all containers based on the same image share the image definition – Adding a copy-on-write file system layer – 50 containers based on the same image require same file system resources as 1 container – until they start changing or creating files • Containers start, stop, restart in seconds • Containers do not have physical resources (memory, CPU) pre-allocated – Resources are allocated at run time
• 15.
  15 Docker is ahigh level API on core Linux Kernel features • Solaris Zones and Windows Server offer similar isolation capabilities
• 16.
  16 Micro Services • Architectthe application into a set of collaborating services. • Each service implements a set of narrowly, related functions. • The services are elastic, resilient, composable, minimal, and complete. • Services communicate using standard protocols such as HTTP/REST • Services are developed and deployed independently of one another. • Each service manages its own state
• 17.
  17 Micro Services • WithDocker, each Micro Service is implemented with a single container – The micro service is not just encapsulated functionality that needs to be deployed onto some platform (such as an ESB or BPEL engine) – instead it ships complete with the fully configured engine that runs in the standardized container platform • All you need to run is: – Start container. Period. Linux Host + Docker Engine
• 18.
  18 A more complexnetwork of microservices…
• 19.
  19 Micro Services andcontainers • Containers can run on the same host – Or on different hosts • Containers can be run as a single node – Or scale up to run as many nodes • Containers can easily be replaced by other containers – That can be connected to in the same way
• 20.
  dockerhost my-little-container my-tiny-container my-data-container Simple Docker UI GUIsfor monitoring & managing a Docker runtime environment
• 21.
  21 GUIs for monitoring& managing a Docker runtime environment • Dockland - Yet another docker web UI – https://github.com/dynport/dockland • Shipyard – a web UI for Docker – https://github.com/ehazlett/shipyard • DockerUI – Web interface to interact with the Docker Remote API – https://github.com/crosbymichael/dockerui • Simple Docker UI – https://github.com/felixgborrego/docker-ui-chrome-app • Panamax – https://github.com/CenturyLinkLabs/panamax-ui
• 22.
  22 Docker – Shipand Run
• 23.
  23 Demo Ship Run MySQLand Wordpress
• 24.
  24 Running Wordpress instance bypulling two public images port 8080 port 80 dockerhostvm IP: 10.10.10.29 Docker Hub
• 25.
  25 Commit container asimage and push to registry dockerhostvm Docker Hub /tmp/mynewfile.txt Dockersig-trial:1.0
• 26.
  26 Image published onpublic Docker Hub registry
• 27.
  27 Run container basedon my published image dockerhostvm Docker Hub /tmp/mynewfile.txt Dockersig-trial:1.0 Dockersig- trial:1.0
• 28.
  28 Containers in theworld of physical logistics
• 29.
  29 Docker Containers Shipmicroservice – app, libs, configuration, platform ready to run on any Linux Host+Docker
• 30.
  30 Docker Registry • Imagescan be published to Public and Private Registries – Docker Hub is the default registry – Docker Hub contains official repositories from many projects and vendors – Private Registries can be created in the cloud and on premises • Containers can be started from such images
• 31.
  Public Docker Registry Docker Hub 31 Registry push Private Docker Registry DockerHub push
• 32.
  32 Docker & Cloud Public Docker Registry DockerHub
• 33.
  33 How to Shipa Stand-Alone product? • Create Installers + Configuration Instructions? • Make your product success dependent on platform configuration and OS settings? • Ship as a container – everything set up and ready to run! • For example: – RubiconRed – Preferred way to deliver their tool MyST: as Docker Container (image)
• 34.
  34 Docker – Build,Ship and Run
• 35.
  35 Demo Build
• 36.
  36 Demo run containerafter build
• 37.
  37 Demo Build –slightly more advanced Dockerfile
• 38.
  38 Demo – runimage created in build process
• 39.
  39 Building a DockerContainer • The Build Steps are prescribed in Dockerfile – Could be called the container recipe • Docker commands specify: – Start from Base Image – Perform commands such as ADD, COPY, RUN that modify the starting situation – The Build Context contains the contents of the folder on the Docker host that contains the Dockerfile (including sub folders) • This can become quite large • During build – files can be downloaded over HTTP using standard wget (but not copied from the host outside the build context) • All files added to the container become part of its history (and increase the size) – Specify the default CMD to run when the container is started – Optionally: tag the container as image, push it to registry and start one or more containers from the new image
• 40.
  40 Building a DockerContainer • Docker files specifies all build steps – With fairly low level commands • Start from base image - each step in the Docker Script adds a layer • A layer is – A logical ‘savepoint’ in the container history • That marks an intermediate ‘image’ – A physical directory somewhere on the Docker Host Base Image Ubuntu 14.04 FROM Ubuntu:14.04 COPY RUN WORKDIR RUN EXPOSE CMD COPY RUN RUN Final Image
• 41.
  41 Run a DockerContainer • When a Docker Container is run based on an image then all these “physical” layers are combined into a “logical” read-only file system – Union file system – Copy-on-write – Like a stack: higher layers may override lower layers • A writable layer is added for the running container Base Image Ubuntu 14.04 COPY RUN RUN Base Image Ubuntu 14.04 COPY RUN RUN Writable Layer run Base Image Ubuntu 14.04 COPY RUN RUN Writable Layer run
• 42.
  Standard Image, locally built 42 Build • Inaddition to 10Ks of reusable images to start containers from • There are a zilion Dockerfiles to leverage for building images – Download script – Add software packages and installers (because of license reasons) – Tweak the script to fit your own needs OR (preferably) – Run the script, create a local image and then create your own Docker File that takes this image as its starting point Your Own Dockerfile Your Tweaked Image
• 43.
  43 “Docker” Search resultson GitHub
• 44.
  44 For example: buildcontainer for Oracle WebLogic • Clone from GitHub to Docker host – Dockerfile – Shell scripts – Supporting files • Download RPMs for – JDK 8 – WebLogic 12.1.3 • Docker Build • Optionally use second Docker file on top of WebLogic image to create a WLS Domain Standard Oracle WebLogic Image, locally built
• 45.
  45 Build File forWebLogic Base Image Oraclelinux:7 RUN COPY RUN COPY COPY COPY
• 46.
  46 Build File forWebLogic (2) Base Image Oraclelinux:7 RUN COPY RUN COPY COPY COPY RUN RUN RUN RUN RUN
• 47.
  Base Image Oraclelinux:7 RUN COPY RUN COPY COPY COPY RUN RUN RUN RUN RUN 47 Turn containerinto image Base Image Oraclelinux:7 RUN COPY RUN COPY COPY COPY RUN RUN RUN RUN RUN docker commit <container-id> weblogic:12.1.3-dev
• 48.
  48 Build File forWebLogic Domain weblogic:12.1.3-dev Base Image Oraclelinux: 7 RUN COPY RUN COPY COPY COPY RUN RUN RUN RUN RUN Base Image Weblogic:12.1.3- dev COPY RUN Base Image Oraclelinux:7 RUN COPY RUN COPY COPY COPY RUN RUN RUN RUN RUN
• 49.
  49 Build File forWebLogic Domain (2) Base Image Weblogic:12.1.3- dev COPY RUN RUN RUN Base Image Oraclelinux:7 RUN COPY RUN COPY COPY COPY RUN RUN RUN RUN RUN
• 50.
  50 Tag container asreusable image Base Image Weblogic:12.1.3- dev COPY RUN RUN RUN Base Image Oraclelinux:7 RUN COPY RUN COPY COPY COPY RUN RUN RUN RUN RUN docker commit <container-id> weblogic-basedomain:12.1.3 oraclelinux:7 weblogic: 12.1.3-dev weblogic- basedomain:12.1.3
• 51.
  51 Build Microservice basedon WebLogic plus Custom App Microservice: custom app oraclelinux:7 weblogic: 12.1.3-dev weblogic- basedomain:12.1.3 WLST deploy EAR WLST config JMS WLST config data sources Microservice: database
• 52.
  52 Automated Configuration Management • Useof Shell Scripting to create | compose | configure environments is not exactly the latest fashion • Declarative, automated configuration management tools have us specify what we need and then make that happen – No scripting – Cross platform – Parametrized – Leveraging public catalogs of environment definitions
• 53.
  Container Build process •Regular Docker Build – From base image – Add Puppet support – Add Puppet Manifests & Modules • Start Container – Optional: Map Volume from host with large files – Run Puppet to apply Manifests – Stop Container, Tag as Image dockerhostvm Dockerfile my-base-container /files /puppet /files volume 1 Very big files Advanced Docker Container Build using Puppet 2 3 4 5 4 21 3 /puppet Manifests/Modules 5
• 54.
  54 Notes on Using Puppetwith Docker • After applying Puppet – the container can be stopped, tagged and used as base image for next Docker Build – That could EXPOSE, ENV, CMD or ENTRYPOINT • With some workarounds, Puppet apply can be made to run during Docker Build (with RUN in Dockerfile) – Less control over build context – No Volume mapping from host • There are Puppet Modules to use for automating the build pipeline of Docker (leveraging the Docker API) – To install Docker, build container, create and ship an image, run container • What applies to Puppet by and large applies to similar tools such as Chef, Salt and Ansible • Puppet Modules are available for many Oracle Database & Fusion Middleware configuration management tasks – Oracle Database (EE, SE, XE) – WebLogic, SOA Suite, OSB, BPM Suite, WLST – JDK, Opatch, VirtualBox, GlassFish, Hudson, Maven
• 55.
  55 Continuous Delivery inthe Container Era • What if development team delivers the complete, runnable environment – instead of fine grained artifacts that need to be deployed to an environment?
• 56.
  56 The Image isDelivered • Source code is built into testable and deployable units • When build and unit test is complete, a Docker Image is built • This image is the unit of delivery – It is independent of / unimpeeded by the run time platform
• 57.
  57 Automated Workflow • Sourcecode commit triggers build, test, ship and deploy of Docker Container – For example: Docker Hub can be hooked into GitHub with build trigger
• 58.
  58 Docker – Concepts inPerspective Dockerfile Image Docker Hub Registry Internet Container build push pull run commit Docker Private Registry push Text file Size = ~ KB Installation & config instructions Composed of many read-only layers Typical size = ~ hundred(s) MB Can have multiple versions (akin Git tags) Shares the image’s read-only layers 1 private writeable layer (copy-on-write) Initial size = 0 bytes Can be stopped, started, paused, etc. Free public hosting Source: Sébastien Bonami, IT Engineering Student and David Lauzon, Researcher (DevOps 2015)
• 59.
  59 Docker – internal structure Source:Sébastien Bonami, IT Engineering Student and David Lauzon, Researcher (DevOps 2015) Docker Daemon Container 1 Host OS Kernel Docker Storage Backend Container 2 ... Hardware Setups & manage the Linux containers. Stores the image and container’s data layers locally.
• 60.
  60 Quick Docker Recap
• 61.
  61 Image and ContainerSpecifics Container Base Image Ubuntu 14.04 COPY RUN RUN Writable Layer run Container “state” tag remove inspect save tar load … pullregistry
• 62.
  62 Image and ContainerSpecifics Container Base Image Ubuntu 14.04 COPY RUN RUN Writable Layer run start attach (un)pause kill stop restart remove inspect logs Container “state” export tar … Flattened, no image details
• 63.
  63 Container Details &Operations Container web Container db link docker run –d –it <image-id> /bin/bash Container xxx link link 808080 /tmp/files Shared Files /data /host_files Docker storage /software Shared Files /repos/repos 1 2 3 4 1 2 3 4 2 4 –p 8080:80 -v /data -v /tmp/files:/host_files -volumes-from xxx --name web --link db:db1 –link xxx:web_xxx
• 64.
  64 Run GUI applications inDocker Container Container docker run –d –it -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY <image-id> /bin/bash /tmp/.X11-unix /tmp/.X11-unix dockerhost GUI applications
• 65.
  65 Docker and Windows •Docker sits on Linux Containers – Windows Server 2016 will have containers too – and Docker will sit on those as well – However, today, Docker does not run on Windows
• 66.
  66 Docker cannot runon Windows - directly, without Linux VM dockerhost Container Container Container
• 67.
  67 Vagrant to therescue dockerhost Container Container Container
• 68.
  68 Vagrant to therescue • Based on simple declarative definitions… • Vagrant provisions environments through various providers – VirtualBox, VMware, AWS • Subsequently, provisioning (‘configuration management’) using shell scripts, Chef, Anisble, Salt or Puppet • Vagrant supports Docker – Create Docker Host VM, Build | Run | Manage Container • Vagrant makes host-container folder mapping and networking quite easy dockerhost Container
• 69.
  69 Vagrant Docker Provisioning •Vagrantfile defines the Container to run – including name and initial command and also synched folders (i.e. host <=> container mapping) • Dockerfile contains build recipe for the Container we want to build • DockerHostVagrantfile describes the VM to be used as Docker Host Vagrantfile DockerHostVagrantfile Dockerfile
• 70.
  70 Vagrant Docker Provisioning dockerhostvm Vagrantfile DockerHostVagrantfile Dockerfile my-little-container buildprocess Docker Hub ubuntu:14.04 /u01/readme.txt Vagrant Boxes ubuntu/trusty64
• 71.
  71 Vagrant with Docker FolderMapping dockerhostvm Vagrantfile DockerHostVagrantfile my-little-container /vagrant /vagrant /host_temp /host_data /var/lib /docker /docker_ generatedId
• 72.
  72 Vagrant on Windows –other options • Boot2Docker – a lightweight Linux distribution based on Tiny Core Linux made specifically to run Docker containers. It runs completely from RAM, weighs ~27MB and boots in ~5s Windows or Mac host
• 73.
  73 Vagrant on Windows –other options • Docker Toolbox (since August 2015) replaces Boot2Docker – Contains Docker Client for Windows, Kitematic (Docker GUI, alpha release), Docker Machine, Docker Engine and leverages Oracle VirtualBox – Still uses Boot2Docker Linux Distribution to run containers – Currently no support for Windows host-container folder mapping – No support for GUI in containers
• 74.
  74 Oracle and Docker •Oracle Linux 6 and 7 Images • WebLogic certified on Docker – Official “Docker Build-scripts in GitHub to create images” • Solaris Zones as Docker backend • Participate in OCI • Docker on Oracle Cloud??
• 75.
  75 Open Container Initiative
• 76.
  76 Summary • Build • Ship •Run • Docker – started in 2013 • Open Container Initiative
• 77.
  77 Hands On • Copyworkshop resources from USB stick to c:docker_sig • Install VirtualBox 5.0 and Vagrant 1.7.4 • Use vagrant up to build and run a simple Docker Container – This will make Vagrant create the Linux Docker Host VM • Use vagrant ssh to open a terminal in the Docker Host and start your Docker adventures – Manage containers and images – Use link, port forwarding and volume configurations • Use vagrant up to build a Docker Container with Puppet support; then start the container and use Puppet to perform advanced configuration • Run a container with an Oracle Database inside • Run a container with a WebLogic Domain inside • Bonus: – Create Linux Desktop Docker Host VM (to run GUI applications in Docker Container)