使用丝裂原加快Ansible的速度

与Mitogen一起加快Ansible！ (Speed up Ansible with Mitogen!)

→ Russian version

→ 俄语版

Ansible is one of most popular Configuration Management Systems nowadays. After it was acquired by Red Hat in 2015 Ansible has reached numbers of thousands of contributors and became maybe one of most used deployment and orchestration tools. Its use-cases are quite impressive.

Ansible是当今最受欢迎的配置管理系统之一 。 在2015年被Red Hat收购后，Ansible已经达到了成千上万的贡献者，并可能成为最常用的部署和编排工具之一。 它的用例令人印象深刻。

Ansible works by SSH connections to remote hosts. It opens SSH session, logs in to the shell, copy python code via network and create a temporary file on remote hosts with this code. In the next step, it executes the current file with python interpreter. All this workflow is pretty heavy and there are multiple ways to make it faster and lighter.

Ansible通过与远程主机的SSH连接工作。 它打开SSH会话，登录到Shell，通过网络复制python代码，并使用此代码在远程主机上创建一个临时文件。 在下一步中，它将使用python解释器执行当前文件。 所有这些工作流程都很繁琐，有多种方法可以使其变得更快，更轻便。

One of these ways is using SSH pipelines which reuses one SSH session for copying python code of multiple tasks and prevent opening multiple sessions, which saves a lot of time. (Just don’t forget to disable requiretty settings for sudo on the remote side in /etc/sudoers)

其中一种方法是使用SSH管道 ，该管道可重复使用一个SSH会话来复制多个任务的python代码，并避免打开多个会话，从而节省了大量时间。 (只是不要忘记在/etc/sudoers的远程端禁用sudo的requiretty设置)

The new way to speed up Ansible is a great python library called Mitogen. If somebody like me was not familiar with it — this library allows fast execution of python code on a remote host and Ansible is only one of its cases. Mitogen uses UNIX pipes on remote machines while passing "pickled" python code compressed with zlib. This allows to run it fast and without much traffic. If you're interested you can read details about how it works in its "How it works" page. But we'll focus today on Ansible related part of it.

加快Ansible新方法被称为伟大的Python库丝裂原 。 如果像我这样的人不熟悉它-这个库允许在远程主机上快速执行python代码，而Ansible只是其中一种情况。 Mitogen在传递通过zlib压缩的“腌制” python代码的同时，在远程计算机上使用UNIX管道。 这样可以快速运行它，而无需增加流量。 如果您有兴趣，可以在其“工作原理”页面中阅读有关其工作原理的详细信息。 但是我们今天将重点关注其中的Ansible相关部分。

Mitogen in specific circumstances can speed up your Ansible in a few times and significantly lower your bandwidth. Let's check the most popular use cases and figure out if it's helpful for us.

在特定情况下，Mitogen可以在几倍内加快Ansible的速度，并显着降低带宽。 让我们检查最流行的用例，并找出是否对我们有用。

The most popular use cases for me to run Ansible are: creating configuration files on a remote host, packages installation, downloading and uploading files from and to a remote host. Maybe you want to check other use cases, please leave comments to this article.

对于我来说，运行Ansible的最流行用例是：在远程主机上创建配置文件，打包安装，从远程主机下载文件以及将文件上传到远程主机。 也许您想检查其他用例，请对本文发表评论。

让我们开始吧！ (Let's start rolling!)

Configuring Mitogen for Ansible is pretty simple: Install the Mitogen module:

为Ansible配置Mitogen非常简单：安装Mitogen模块：

pip install mitogen

Then either configure environment variables or set configuration options in ansible.cfg file, both options are fine: Let's assume /usr/lib/python2.7/site-packages/ansible_mitogen/plugins/strategy is your path to installed Mitogen library.

然后配置环境变量或在ansible.cfg文件中设置配置选项，两个选项都很好：假设/usr/lib/python2.7/site-packages/ansible_mitogen/plugins/strategy是您安装Mitogen库的路径。

export ANSIBLE_STRATEGY_PLUGINS=/usr/lib/python2.7/site-packages/ansible_mitogen/plugins/strategy
export ANSIBLE_STRATEGY=mitogen_linear

or

要么

[defaults]
strategy = mitogen_linear
strategy_plugins = /usr/lib/python2.7/site-packages/ansible_mitogen/plugins/strategy

Prepare Ansible in virtualenv, with and without Mitogen enabled:

在启用和不启用Mitogen的情况下，在virtualenv中准备Ansible：

virtualenv mitogen_ansible
./mitogen_ansible/bin/pip install ansible==2.7.10 mitogen
virtualenv pure_ansible
./pure_ansible/bin/pip install ansible==2.7.10

Please pay attention that Mitogen 0.2.7 doesn't work with Ansible 2.8 (for May 2019)

请注意，Mitogen 0.2.7不适用于Ansible 2.8(2019年5月)

Create aliases:

创建别名：

alias pure-ansible-playbook='$(pwd)/pure_ansible/bin/ansible-playbook'
alias mitogen-ansible-playbook='ANSIBLE_STRATEGY_PLUGINS=/usr/lib/python2.7/site-packages/ansible_mitogen/plugins/strategy:$(pwd)/mitogen_ansible/lib/python3.7/site-packages/ansible_mitogen/plugins/strategy ANSIBLE_STRATEGY=mitogen_linear $(pwd)/mitogen_ansible/bin/ansible-playbook'

Now let's try the playbook that creates file on remote:

现在，让我们尝试在远程创建文件的剧本：

---
- hosts: all
  gather_facts: false
  tasks:
    - name: Create files with copy content module
      copy:
        content: |
          test file {{ item }}
        dest: ~/file_{{item}}
      with_sequence: start=1 end={{ n }}

And run it with Mitogen and without while creating 10 files:

并与Mitogen一起运行，而无需创建10个文件：

time mitogen-ansible-playbook file_creation.yml -i hosts -e n=10 &>/dev/null

real    0m2.603s
user    0m1.152s
sys     0m0.096s

time pure-ansible-playbook file_creation.yml -i hosts -e n=10 &>/dev/null

real    0m5.908s
user    0m1.745s
sys     0m0.643s

Right now we see improvement in x2 times. Let's check it for 20, 30, ..., 100 files:

现在，我们看到了2倍的改进。 让我们检查一下20、30，...，100个文件：

time pure-ansible-playbook file_creation.yml -i hosts -e n=100 &>/dev/null

real    0m51.775s
user    0m8.039s
sys     0m6.305s

time mitogen-ansible-playbook file_creation.yml -i hosts -e n=100 &>/dev/null

real    0m4.331s
user    0m1.903s
sys     0m0.197s

Eventually, we improved execution time in more than 10 times!

最终，我们将执行时间缩短了十倍以上！

Now let's try different scenarios and see how it improves:

现在让我们尝试不同的情况，看看它如何改进：

• Scenario of uploading files from the local host to remote (with copy module):

  将文件从本地主机上传到远程(带有copy模块)的方案：

• Scenario of creating files on the remote host with copy module:

  使用copy模块在远程主机上创建文件的方案：

• Scenario with fetching files from the remote host to local:

  从远程主机到本地获取文件的方案：

Let's try the last scenario on a few (3) remote hosts, for example uploading files scenario:

让我们在几台(3)远程主机上尝试最后一个方案，例如，上载文件方案：

As we can see the Mitogen saves us both time and bandwidth in these scenarios. But if the bottleneck is not Ansible, but for example I/O of disk or network, or somewhere else, then it's hard to expect from Mitogen to help of course.

如我们所见，在这些情况下，Mitogen为我们节省了时间和带宽。 但是，如果瓶颈不是Ansible，而是例如磁盘或网络的I / O或其他地方，那么Mitogen当然会提供帮助。

Let's run for example packages installation with yum/dnf and python modules installation with pip. Packages were pre-cached to avoid dependencies on network glitches:

让我们运行例如使用yum / dnf进行软件包安装以及使用pip进行python模块安装的示例。 程序包已预先缓存，以避免依赖于网络故障：

---
- hosts: all
  gather_facts: false
  tasks:
    - name: Install packages
      become: true
      package:
        name:
          - samba
          - httpd
          - nano
          - ruby
        state: present

    - name: Install pip modules
      become: true
      pip:
        name:
          - pytest-split-tests
          - bottle
          - pep8
          - flask
        state: present

With Mitogen it takes 12 seconds, as well as with pure Ansible.

使用Mitogen和纯Ansible都需要12秒。

In Mitogen for Ansible page you can see additional benchmarks and measurements. As the page declares:

在Mitogen for Ansible页面中，您可以看到其他基准和测量结果。 如页面所声明：

Mitogen cannot improve a module once it is executing, it can only ensure the module executes as quickly as possible

Mitogen无法在模块执行后对其进行改进，只能确保模块尽快执行

That's why it's important to find where your bottlenecks are and if they are related to Ansible operations, Mitogen will help you to solve it and speed your playbooks up significantly.

因此，找到瓶颈在哪里以及与Ansible操作相关的瓶颈很重要，Mitogen将帮助您解决瓶颈并显着提高您的游戏手册。

翻译自: https://habr.com/en/post/453446/