OSDC 2014: Jochen Lillich - Dynamic infrastructure orchestration
4 likes820 views
The document discusses dynamic infrastructure orchestration and management of the Freistil IT system, utilizing tools like Serf for cluster communication and Etcd for key-value store management. It covers various processes including application deployment, membership management, and failure detection in the orchestration environment. Additionally, it includes code snippets demonstrating the usage of Serf and Etcd for handling events and data management.
![serf node 1
$ serf agent -node=node1 -bind=srv1
==> Starting Serf agent...
==> Starting Serf agent RPC...
==> Serf agent running!
Node name: 'srv1'
Bind addr: '192.168.0.11:7946'
…
==> Log data will now stream in as it occurs:
2014/04/09 16:05:51 [INFO] agent: Serf agent starting
2014/04/09 16:05:51 [INFO] serf: EventMemberJoin: node1
192.168.0.11
2014/04/09 16:05:52 [INFO] agent: Received event: member-
join](https://image.slidesharecdn.com/dynamicinfrastructureorchestration-140425081501-phpapp01/85/OSDC-2014-Jochen-Lillich-Dynamic-infrastructure-orchestration-8-320.jpg)
![User event
$ serf event deploy-app
!
2014/04/09 17:06:32 [INFO] agent: Received
event: user-event: deploy-app](https://image.slidesharecdn.com/dynamicinfrastructureorchestration-140425081501-phpapp01/85/OSDC-2014-Jochen-Lillich-Dynamic-infrastructure-orchestration-15-320.jpg)
More Related Content
Similar to OSDC 2014: Jochen Lillich - Dynamic infrastructure orchestration (9)
OSDC 2014: Jochen Lillich - Dynamic infrastructure orchestration
- 2. Jochen Lillich @geewiz Sysadmin since 1993 CEO of freistil IT freistilbox: PaaS for Drupal & WordPress
- 3. Config Management Elimination of manual changes Consistent configuration Single source of truth DRY
- 4. Weaknesses Periodic convergence runs Central database
- 5. Orchestration Perform changes quickly Handle failure Tolerate network partitions
- 6. serf
- 7. Basics Cluster communication tool Developed by Hashicorp Simple binary,written in Go Gossip protocol
- 8. serf node 1 $ serf agent -node=node1 -bind=srv1 ==> Starting Serf agent... ==> Starting Serf agent RPC... ==> Serf agent running! Node name: 'srv1' Bind addr: '192.168.0.11:7946' … ==> Log data will now stream in as it occurs: 2014/04/09 16:05:51 [INFO] agent: Serf agent starting 2014/04/09 16:05:51 [INFO] serf: EventMemberJoin: node1 192.168.0.11 2014/04/09 16:05:52 [INFO] agent: Received event: member- join
- 9. serf node 2 $ serf agent -node=node2 -bind=srv2 … $ serf join srv1 Successfully joined cluster by contacting 1 nodes. $ serf members node2 192.168.0.12:7946 alive node1 192.168.0.11:7946 alive
- 11. Failure detection Detects failure quickly Notifies other nodes Executes handler scripts
- 13. User events Deploy application Perform convergence run Query information from the cluster
- 14. Example handler #!/bin/bash ! echo echo "New event: ${SERF_EVENT}. Data follows..." while read line; do printf "${line}n" done
- 15. User event $ serf event deploy-app ! 2014/04/09 17:06:32 [INFO] agent: Received event: user-event: deploy-app
- 16. etcd
- 17. Basics HA key-value store Developed by CoreOS Written in Go Raft consensus algorithm Pure HTTPAPI
- 18. Key-value store /release = 9607bcfeb48905d26db8f /cluster /cluster/node1 = node1.example.com /cluster/node2 = node2.example.com
- 19. Quorum Available
- 20. Quorum Available
- 21. Quorum Available
- 23. Writing $ etcdctl set /message Hello Hello $ curl -L -X PUT http://127.0.0.1:4001/v2/keys/message -d value=“Test" {"action":"set","node":{"key":"/ message","value":"Test","modifiedIndex": 15,"createdIndex":15},"prevNode":{"key":"/ message","value":"Hello","modifiedIndex": 2,"createdIndex":2}}
- 24. Reading $ etcdctl get /message Hello $ curl -L http://127.0.0.1:4001/v2/keys/message {"action":"get","node":{"key":"/ message","value":"Test","modifiedIndex": 15,"createdIndex":15}
- 25. Deleting $ etcdctl rm /message $ curl -L -X DELETE http://127.0.0.1:4001/v2/keys/message
- 26. Waiting $ curl -L http://127.0.0.1:4001/v2/keys/ release?wait=true
- 27. Waiting $ curl -L http://127.0.0.1:4001/v2/keys/ release?wait=true {"action":"set","node":{"key":"/ release","value":"9607bcfe","modifiedIndex": 16,"createdIndex":16},"prevNode":{"key":"/ release","value":"18512199","modifiedIndex": 14,"createdIndex":14}}
- 35. Leader election etcd App DB XMaster me!
- 36. Leader election etcd App DB XMaster me! Ok!
- 37. Leader election etcd App DB X DB X is master.
- 38. Leader election etcd App DB X DB X is master.
- 39. Leader election etcd App DB X DB X is master.
- 41. Leader election etcd App DB X DBY
- 42. Leader election etcd App DB X DBY Master me!
- 43. Leader election etcd App DB X DBY Master me! No,DB X is.
- 44. Leader election etcd App DB X DBY
- 48. Leader election etcd App DBY You da master!
- 49. Leader election etcd App DBYDB Y is master now.
- 51. Summary