Managing Infrastructure as Code

Managing Infrastructure as Code

• 1.
  Managing Infrastructure asCode Allan Shone
• 2.
  Agenda • Look atsome legacy concepts • Run through some ideas • Check out a few products • Put together some base requirements • Browse a few modern products • Throw in some more updated concepts • Dive into suitable products • Extra bits, concerns, and ideas
• 3.
  Back in theday... • Infrastructure was extremely manual • Hosts, meta details, and further information was recorded in text files, or other plain text based systems • Hostnames based on function • Documentation was fragmented • Legacy infrastructure becomes unmanageable and can be forgotten • Nightmare to keep track of
• 4.
  Tools • Wiki /HTML Tables • Text Files • Shared drives • Internal admin server • Shared FTP • Proprietary and specific software solutions • Shell scripts
• 5.
  Problems? • Keeping hostlists up-to-date • Multiple users managing infrastructure • Recent status indicators • Cumbersome interfaces • Time consuming data interactions • What about software?
• 6.
  Ideas • Some sortof versioning • Easier interface for collaboration • Provision of host state • Start looking at automation
• 7.
  Bits and Pieces •Databases • Services • Applications • Cache • Routers
• 8.
  Software? • First, infrastructurerequires orchestration • Then, software dependencies can be pushed within each of those infrastructure items • Bare-metal is different with a separate set of requirements • The premise for both though is still of value to the general topic
• 9.
  Basic Provisioners
• 10.
  Ansible • Provides inheritance •Allows for variable configuration • Straight-forward to use with automation • Expressive with its syntax using YAML • Playbooks used for grouping of instructions • Playbooks versioned in a DIY fashion • Agentless model for deployment • Templating makes conf files a breeze
• 11.
  Simple Software # Playbookfor Application - hosts: “{{hosts}}” remote_user: root sudo: yes roles: - common - app-server # Directory Structure roles/common/handlers/main.yml roles/common/tasks/main.yml roles/common/templates/ntpd.conf roles/app-server/handlers/main.yml roles/app-server/tasks/main.yml roles/app-server/templates/apache.conf roles/app-server/vars/example.yml
• 12.
  Provisioning Infrastructure - hosts:localhost connection: local gather_facts: false tasks: - name: Provision instances ec2: key_name: my_key group: test instance_type: t2.micro image: “{{ami_id}}” count_tag: Name: Demo instance_tags: Name: Demo register: ec2 - name: Add Hosts to Host Group add_host: hostname={{ item.public_ip }} groups=ec2hosts with_items: ec2.instances - hosts: ec2hosts name: configuration play user: ec2-user gather_facts: true tasks: - name: Check NTP Service service: name=ntpd state=started
• 13.
  Drawbacks • Difficult totrack created instances • Supplier specific wrapper • Versioning is DIY • Basic in terms of complete solution
• 14.
  Chef • Builds onRuby for syntax • Fluent way of pushing modifications • Variable capabilities for ease of automation • Cookbooks used to group instructions • Cookbooks synchronised with the Chef Server • Server to client model
• 15.
  Simple Instances num_instances =10 1.upto(num_instances) do |inst| machine "my-machine-#{inst}" do add_machine_options bootstrap_options: { security_group_ids: 'test-sg', subnet_id: 'subnet-1234567', instance_type: 't2.micro' } end end
• 16.
  Resources load_balancer "test-elb" do machines[ "machine1", "machine2" ] load_balancer_options :listeners => [{ :port => 80, :protocol => :http, :instance_port => 80, :instance_protocol => :http, }] end
• 17.
  Drawbacks • Dedicated serverfor management • Uses Ruby natively, which could be a positive if you work with Ruby or don’t mind • What is required for some may not exist unless the necessary Plugin is available for it • OS and Package restrictions for nodes
• 18.
  Puppet • Simple syntaxfor configuration • Server model for deployments • Automation readily available • Parameterised configurations for easy environment setup
• 19.
  Software package { 'apache2': provider=>'apt', ensure=>'installed' } notify{ 'Apache2 is installed.': } service { 'apache2': ensure=>'running' } notify { 'Apache2 is running.': }
• 20.
  ec2_securitygroup { 'sample-group': ensure=> present, region => 'us-west-1', description => 'Group used for testing Puppet AWS module', } ec2_instance { 'sample-instance': ensure => present, region => 'us-west-1', availability_zone => 'us-west-1a', image_id => 'ami-696e652c', instance_type => 't1.micro', security_groups => ['sample-group'], } Resources
• 21.
  Resource - Finalising ec2_loadbalancer{ 'sample-load-balancer': ensure => present, region => 'us-west-1', availability_zones => ['us-west-1a', 'us-west-1b'], instances => ['sample-instance', 'another-instance'], security_groups => ['sample-group'], listeners => [{ protocol => 'tcp', port => 80, }], }
• 22.
  Drawbacks • Requires learningthe Puppet specific language for the actual infrastructure code • Complex infrastructure can become quite cumbersome to manage • Dependency based, order of execution can be tricky to control when it is required to be
• 23.
  What about hosts?
• 24.
  CloudFormation • Complete “physical”infrastructure as code • Basic JSON file for definition • Services for usage easily interacted with • Tightly coupled with AWS • Versioned and stored within the console • Ease of automation
• 25.
  Beginning { "AWSTemplateFormatVersion": "2016-01-01", "Description": "MyTemplate", "Parameters": { "KeyName": { "Description": "EC2 KeyPair", "Type": "AWS::EC2::KeyPair::KeyName", "ConstraintDescription": "must be the name of an existing EC2 KeyPair." } } }
• 26.
  Resources - SecurityGroup "Resources": { "WebServerSecurityGroup": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": "Minimal Access", "SecurityGroupIngress": [{ "IpProtocol": "tcp", "FromPort": "80", "ToPort": "80", "CidrIp": "0.0.0.0/0" }] } } }
• 27.
  Resources - Instance "Resources":{ "WebServer": { "Type": "AWS::EC2::Instance", "Metadata": { "AWS::CloudFormation::Init": { "install": { "packages": { "yum": { "httpd": [] } } } } }
• 28.
  Tying it together "Outputs":{ "WebsiteURL": { "Value": { "Fn::Join": ["", [ "http://", { "Fn::GetAtt": [ "WebServer", "PublicDnsName" ]}, "/ping" ]]}, "Description": "Website" } }
• 29.
  Drawbacks • AWS specific •JSON for the configuration can be difficult to create and maintain - No comments • Not idempotent • Templates are very large and can become quite cumbersome to follow • Most functionality can be automated through the command line interface within other tools
• 30.
  Infrastructure pieces • Softwaremanagement, host management, resources • A general tool provides one but not the other • Arbitrary scripts can shoehorn this • Duplication and Inconsistencies would become problematic with keeping data sets in different tools
• 31.
  Combinations • Software dependenciesmanaged • Hosts instantiated or made available on demand • Configurations completed between environments to allow for sand-boxed communication • Entire infrastructures brought up with a single command as replica of production
• 32.
  TerraForm • Will orchestrateand provision • Syntax is easy to grasp and maintain • Configurations can be quite simple • Parameterised capabilities for ease of scripting with environments
• 33.
  Software resource "aws_instance" "web"{ connection { user = "ubuntu" } instance_type = "m1.small" ami = "${lookup(var.aws_amis, var.aws_region)}" key_name = "${aws_key_pair.auth.id}" vpc_security_group_ids = ["${aws_security_group.default.id}"] subnet_id = "${aws_subnet.default.id}" provisioner "remote-exec" { inline = [ "sudo apt-get -y update", "sudo apt-get -y install nginx", "sudo service nginx start" ] } }
• 34.
  Resources resource "aws_elb" "web"{ name = "terraform-example-elb" subnets = ["${aws_subnet.default.id}"] security_groups = ["${aws_security_group.elb.id}"] instances = ["${aws_instance.web.id}"] listener { instance_port = 80 instance_protocol = "http" lb_port = 80 lb_protocol = "http" }} resource "aws_key_pair" "auth" { key_name = "${var.key_name}" public_key = "${file(var.public_key_path)}" }
• 35.
  Drawbacks • Tightly integratedwith vendors • Learning curve for syntax • Delays with updated services and functionality • Newcomer to the fully managed tool suite, some features are incomplete or in progress
• 36.
  ManageaCloud • Complete solution,orchestration and provisioning • Simple, re-usable configuration • Built-in versioning for deployments and infrastructure • Open choice of vendor - no requirements • Framework approach for infrastructure management
• 37.
  Macfile • Configuration template •Complete infrastructure specification • Versioned to allow for ease of use, deployment, and rollback • Simple syntax no vendor specifics
• 38.
  App Instance roles: demo_app: instance create: configuration:demo_application infrastructures: demo_application_instance: name: demo provider: amazon location: us-east-1 hardware: t1.micro role: demo_app environment: - APP_BRANCH: master
• 39.
  Resource resources: elastic_load_balancer: create bash: aws elbcreate-load-balancer --load-balancer-name infrastructure.param.name --listeners infrastructure.param.listeners --availability-zones infrastructure.param.availability-zones --region infrastructure.param.region destroy bash: aws elb delete-load-balancer --load-balancer-name infrastructure.param.name --region infrastructure.param.region
• 40.
  Resource Instance infrastructures: load balancer01: resource: elastic_load_balancer params: name: my-demo-load-balancer listeners: Protocol=HTTP,LoadBalancerPort=80,InstanceProtocol=HTTP,InstancePort=80 availability-zones: us-east-1b us-east-1c region: us-east-1
• 41.
  Associationactions: get_id: ssh: wget -q-O - http://169.254.169.254/latest/meta-data/instance-id register_lb: create bash: aws elb register-instances-with-load-balancer --load-balancer-name infrastructure.param.load-balancer-name --instances infrastructure.param.instances --region infrastructure.param.region infrastructures: register_instance: ready: role.demo_app resource: register_lb params: load-balancer-name: my-demo-load-balancer instances: role.demo_app.get_id
• 42.
  Drawbacks • Most componentsare open source, not all at the present time • No unified syntax for providers
• 43.
  What about people?
• 44.
  DevOps • Largely, DevOpscame about as a hybrid role to help manage and facilitate process change • Automation is a key aspect • Not Operations, but not Development either (but is still both) • Provide an interface between infrastructure and environments and deployments made
• 45.
  Concepts • Even withautomation, humans are still needed • Sanity checking and improving tools • Removing bottlenecks • Increasing developer and wider business productivity • Know the management tools and the details of how the infrastructure functions
• 46.
  Workflows • Very importantto focus on processes • Tools are wonderful, but processes need to be suitable for the tool of choice • Automation will bring down the Op aspects of DevOps • Cross functional efforts to bring the automation to the infrastructure • Size of infrastructure
• 47.
  Infrastructure as Code
• 48.
  Decisions • Situations makedecisions difficult • Complete solutions are not always necessary • Preference and team knowledge makes a difference • A product is not specifically good just because others use it
• 49.
  Options • There’s alwaysmore options available than time to discuss - CFengine, Salt, Heat, OneOps • It’s all about automation, and removing bottlenecks in cumbersome processes
• 50.
  Future • Abilities toshare, extend, and work better with infrastructures • Inheritance for roles, resources, and instances • Complete control with automation of infrastructure sets • Simple options for deployment strategies
• 51.
  Thank you! Allan Shone https://manageacloud.com