Managing Meaningful

Inventories
Will Tome Alan Rominger
Sr. Solutions Architect Sr. Software Engineer
Share your automation story

1. How did you get started with Ansible?

2. How long have you been using it?

3. What's your favorite thing to do when you Ansible?

What is a meaningful inventory?

A meaningful inventory is…

- Up to date
- Aligned with process
- Logical
- Flexible
- Inclusive
 hosts

Anatomy of an Inventory file [atl]

10.172.0.33
10.172.4.13

hosts
● Host name can be any of: IP [sfo]
web1.ansible.com
address, Hostname, Alias db2.ansible.com
host_vars
● Groups [web]
web3 ansible_host=192.168.1.32
a. What - An application, stack or web1.ansible.com
microservice.
[prod] group
b. Where - A datacenter or region, to talk to web1.ansible.com
local DNS, storage, etc. db2.ansible.com

c. When - The dev stage, to avoid testing on [web:vars] group_vars

production resources. doc_root=/var/www/mysite/

[usa:children]
● Groups can be nested and define atl
variables sfo nested groups

https://github.com/willtome/managing-meaningful-inventories/
 ---
Working with Patterns - name: my really cool playbook
What hosts apply to a configuration or process? hosts: all

...
---
- name: ONE OR MORE GROUPS
hosts: sfo:atl ---
--- - name: INTERSECTION
... - name: EXCLUDE A GROUP hosts: atl:&prod
hosts: all:!atl
...
--- ...
- name: VARIABLES
hosts: “{{ HOSTS }}” ---
--- - name: COMBINATIONS
... - name: WILDCARD hosts: “atl:sfo:!{{excluded}}”
hosts: *.com
...
...
Using host_vars and group_vars Folders

● Architecture is “vars plugin”

○ host_group_vars group_vars/
├── sfo.yml
● Subdirectory in playbook or ├── atl.yml
└── prod/
inventory folders └── app.yml
└── vault.yml
● Defines variables for the host or host_vars/
├── web3.yml
group └── web1.ansible.com/
└── network.yml
● Irrelevant of inventory type
└── vhosts.yml
● To set something for all hosts,
use “all.yml” group vars
Where does your inventory live?

?
Why use Dynamic Inventory?

Don’t reinvent the wheel

PUBLIC / PRIVATE
CLOUD PUBLIC / PRIVATE
CMDB CLOUD

Keep up with change

ANSIBLE AUTOMATION ENGINE

Capture all systems

home

HOSTS
INVENTORY CLI

Integrate with everything

MODULES PLUGINS
NETWORK
ANSIBLE DEVICES

Extend Ansible Core PLAYBOOK

Dynamic Inventory Example - azure_rm
(Azure Resource Manager)

azure_rm.yml or foo.azure_rm.yml Pre-requisites

plugin: azure_rm # install dependency
auth_source: env pip install azure

set_private_env.sh
#!/bin/sh

Specify credentials by export AZURE_SUBSCRIPTION_ID=<private>

environment variables export AZURE_CLIENT_ID=<private>
export AZURE_SECRET=<private>
export AZURE_TENANT=<private>

https://github.com/willtome/managing-meaningful-inventories/
 Debugging Inventory
# Check inventory plugin docs locally
ansible-doc -t inventory azure_rm $ ansible-doc -t inventory -l
advanced_host_list Parses a 'host list' with ranges
# See inventory JSON representation auto Loads and executes an inventory plugin
ansible-inventory -i azure_rm.yml --list aws_ec2 EC2 inventory source
aws_rds rds instance source

# Run playbook azure_rm Azure Resource Manager inventory plugin

ansible-playbook -i azure_rm.yml debug.yml cloudscale cloudscale.ch inventory source

constructed Uses Jinja2 to construct vars and groups

# Multiple inventories docker_machine Docker Machine inventory source

...
ansible-playbook -i <1> -i <2> debug.yml
ansible-playbook -i <dir> debug.yml

Inventory plugins that Ansible “sees”

See ansible.cfg env
ANSIBLE_INVENTORY_PLUGINS
 How ansible processes inventory?
Some plugins are enabled by default
Set in ansible.cfg or ANSIBLE_INVENTORY_ENABLED env variable
ansible.cfg
[inventory]
enable_plugins = host_list, script, auto, yaml, ini, toml

Auto uses inventory file to load inventory plugins that are not enabled
aws_ec2.py
aws_ec2.yml class InventoryModule(BaseInventoryPlugin, Constructable, Cacheable):
plugin: aws_ec2
regions: NAME = 'aws_ec2'
- us-east-1
def __init__(self):
filters: super(InventoryModule, self).__init__()
tag:Environment: dev ...
Writing Inventory Plugins
Overview of Developing a Custom Inventory Plugin
Demo plugin: “basic”
https://galaxy.ansible.com/alancoding/basic
● Python content
○ Add DOCUMENTATION
○ Inherit from base class
● Decide namespace (ex. basic)
○ Name file `basic.py`
○ Set `NAME = “basic”`
● Define `verify_file` method, if desired
● Define `parse` method
● Make it available for use

https://github.com/willtome/managing-meaningful-inventories/
DOCUMENTATION = r'''
name: Inventory Plugin Basics
plugin_type: inventory
Documentation
author:
● This section is required
- First Last (@username)
short_description: Used for instructive purposes. ● Functional impacts:
version_added: "2.10" ○ Options documentation is used
description:
by get_option() method
- Demonstrates basics of a custom inventory plugin.
options: ○ Environment variables can be
count:
used for authentication
description: The number of hosts to make.
type: integer
parameters
required: True ○ Types will be enforced in
default: 1
Ansible 2.9
required: False
env:
- name: HOST_COUNT
requirements:
- python >= 3.4
''' https://github.com/willtome/managing-meaningful-inventories/
 Imports
Ansible
from ansible.module_utils.six.moves.urllib.parse import urljoin
module utils
from ansible.module_utils.urls import Request, ConnectionError, urllib_error

from ansible.errors import AnsibleParserError

from ansible.plugins.inventory import BaseInventoryPlugin Ansible Core
libraries
from base64 import b64encode
import json

Other dependencies

https://github.com/willtome/managing-meaningful-inventories/
 Custom Inventory Plugin Interface Can add other
mixins for added
● Inherit from base class (must be InventoryModule) functionality
● In parse, call methods on inventory
○ add_group(group) Group must be created to add host

○ add_host(host, group=None, port=None)

○ add_child(group, entity) Group or Host
○ set_variable(entity, varname, value) Class name must be “InventoryModule”

class InventoryModule(BaseInventoryPlugin, Constructable, Cacheable):

Group or Host
NAME = 'my_custom_plugin' The plugin name

def parse(self, inventory, loader, path):

Copy and paste these lines super(InventoryModule, self).parse(inventory, loader, path)
exactly, your logic comes after self._read_config_data(path)

https://github.com/willtome/managing-meaningful-inventories/
 Use the Inventory Data Methods to Build Inventory
def parse(self, inventory, loader, path, cache=True):
super(InventoryModule, self).parse(inventory, loader, path) Copy and paste these lines
self._read_config_data(path) exactly, your logic comes after

root_group_name = self.inventory.add_group('root-group') Group must be present to add hosts

for i in range(self.get_option('count')): get_options requires correct DOCUMENTATION

group_name = self.inventory.add_group('group_{}'.format(count))
self.inventory.add_child(root_group_name, group_name)
Parse
host_name = self.inventory.add_host('host_{}'.format(count)) Logic
self.inventory.add_child(group_name, host_name)

self.inventory.set_variable(
root_group_name, 'hashed_password', _hash_password(self.get_option('password'))
)

https://github.com/willtome/managing-meaningful-inventories/
Share your Custom Inventory Plugin
☹ Put it in a share folder
○ /usr/share/ansible/plugins/inventory

😐 Point to it in your Ansible config

🙂 Use relative path (Ansible 2.8+)

○ Next to playbook (inventory_plugins/)

😃 Add to a collection (Ansible 2.9+)

○ stuff/plugins/inventory/foo.py
○ Upload to Ansible Galaxy
Troubleshooting Inventory
 Troubleshoot Inventory Loading
Custom
ansible-playbook -i sqlite.yml debug.yml
inventory
plugin
Project
sqlite.py
.
├── sqlite.yml from ansible.errors import AnsibleError, AnsibleParserError

├── hosts.db from ansible.plugins.inventory import BaseFileInventoryPlugin

import sqlite3
└── sqlite.py
import os

sqlite.yml class InventoryModule(BaseFileInventoryPlugin):

plugin: sqlite NAME = 'sqlite'

db_path: hosts.db def verify_file(self, path):

db_table: hosts super(InventoryModule, self).verify_file(path)

return path.endswith(('sqlite.yml', 'sqlite.yaml'))

https://github.com/willtome/managing-meaningful-inventories/
 Troubleshoot Inventory Loading
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with auto plugin: inventory config
'/home/wtome/inventory/sqlite.yml' specifies
unknown plugin 'sqlite'

[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with yaml plugin: Plugin configuration

YAML file, not YAML inventory

[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with ini plugin: Invalid host pattern
'---' supplied, '---' is normally a sign this is a
YAML file.

[WARNING]: Unable to parse /home/wtome/inventory/sqlite.yml as an inventory source

 Troubleshoot Inventory Loading
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with auto plugin: inventory config
'/home/wtome/inventory/sqlite.yml' specifies
unknown plugin 'sqlite'
Auto plugin
is being
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml usedplugin: Plugin configuration
with yaml
The plugin (.py)
YAML file, not YAML inventory
cannot be found

[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with ini plugin: Invalid host pattern
'---' supplied, '---' is normally a sign this is a
1) file.
YAML Plugin (.py) is not in the correct location
(./inventory_plugins:~/.ansible/plugins/inventory:/usr/share/ansible/plugins/inventory )

2) PluginUnable
[WARNING]: nameto(sqlite) does not match NAME in plugin
parse /home/wtome/inventory/sqlite.yml (.py) source
as an inventory
 Troubleshoot Inventory Loading
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with auto plugin: inventory config
'/home/wtome/inventory/sqlite.yml' specifies Ansible is trying
Invalid format
unknown plugin 'sqlite' the yaml plugin
for yaml plugin
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with yaml plugin: Plugin configuration
YAML file, not YAML inventory

[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with ini plugin: Invalid host pattern
'---' supplied, '---' is normally a sign this is a
1) Invalid
YAML file. YAML syntax
2) Intended failure because you weren’t using the YAML plugin anyways
[WARNING]: Unable to parse /home/wtome/inventory/sqlite.yml as an inventory source
 Troubleshoot Inventory Loading
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with auto plugin: inventory config
'/home/wtome/inventory/sqlite.yml' specifies
1) Invalid
unknown pluginINI syntax
'sqlite'

2) Intended failure because you weren’t using the INI plugin anyways
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with yaml plugin: Plugin configuration
YAML file, not YAML inventory

[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with ini plugin: Invalid host pattern
'---' supplied, '---' is normally a sign this is a
YAML file.
Ansible is trying
the ini plugin
[WARNING]: Unable to parse /home/wtome/inventory/sqlite.yml as an inventory source
 Troubleshoot Inventory Loading
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with auto plugin: inventory config
'/home/wtome/inventory/sqlite.yml' specifies
unknown plugin 'sqlite'

1) All enabled
[WARNING]: * Failedplugins
to parse failed to produce a valid inventory
/home/wtome/inventory/sqlite.yml with yaml plugin: Plugin configuration
YAML file, not YAML inventory
2) Playbook will continue but only localhost will be available
[WARNING]: * Failed to parse /home/wtome/inventory/sqlite.yml with ini plugin: Invalid host pattern
'---' supplied, '---' is normally a sign this is a
YAML file.

[WARNING]: Unable to parse /home/wtome/inventory/sqlite.yml as an inventory source

 Troubleshoot Inventory Loading
Relative path solution for custom inventory plugins:

. .
├── hosts.db ├── hosts.db
├── sqlite.py ├── inventory_plugins
└── sqlite.yml │ └── sqlite.py
└── sqlite.yml

This make the sqlite inventory plugin available.

WARNINGS:
Folder inventory_plugins is relative to playbook, so commands
- ansible-config
- ansible-inventory
Will not identify your plugin without additional work (--playbook-dir=./)
Advanced Inventory Options
 Common / Shared Inventory Plugin Functionality
“Constructed” Functionality

● Compose
○ Set hostvars based on jinja2 expressions of other hostvars
● Conditional groups (sometimes just “groups”)
○ Assign hosts to groups based on True/False evaluation of hostvars
● Keyed Groups
○ Create groups based on the value of hostvars (can combine jinja2)
● Filters
○ Limit the hosts returned, specific to each API
● Local Cache
○ Avoids slow API calls between multiple playbook runs
Constructed dev example (foreman): https://github.com/ansible/ansible/pull/62542/files
 Simple AWS EC2 “Constructed” Example

example.aws_ec2.yml
● regions filters the returned hosts
plugin: aws_ec2

● compose creates hostvars by templating regions:

- us-east-1
other hostvars
compose: # Set individual hostvars
ec2_state: state.name
● groups (“conditional groups” elsewhere) groups:

groups hosts by boolean expression ec2: true # conditional groups

platform_undefined: platform is not defined
keyed_groups: # Create groups for each region
● keyed_groups creates groups with names - key: placement.region
from templating hostvars prefix: aws_region

https://github.com/willtome/managing-meaningful-inventories/
 Keyed Groups AWS Regions / Zones

keyed.aws_ec2.yml Hostvars of an ec2 machine:

plugin: aws_ec2 "http://ipv4-address.compute-1.amazonaws.com":

keyed_groups: {
"...",
- key: placement.region
"placement": {
parent_group: regions "availability_zone": "us-east-2c",
prefix: '' "group_name": "",
separator: '' "region": "us-east-2",
"tenancy": "default"
- key: placement.availability_zone
},
separator: '' "..."
parent_group: '{{placement.region}}' },

https://github.com/willtome/managing-meaningful-inventories/
 Keyed Groups AWS Regions / Zones

keyed.aws_ec2.yml Static
“regions” regions
plugin: aws_ec2
keyed_groups:
- key: placement.region Regions us_east_1 us_west_1
parent_group: regions
prefix: ''
Availability us_east_1a us_east_1b us_west_1a us_west_1b
separator: ''
zones
- key: placement.availability_zone
separator: ''
Hosts
parent_group: '{{placement.region}}'

https://github.com/willtome/managing-meaningful-inventories/
Inventory Plugins In
Ansible Tower
(use cases)
Sourcing Inventory in Ansible Tower * venv = custom virtual
environment (if needed)

What you need to get inventory from…

1. a Tower built-in type
○ UI
2. an Ansible inventory plugin not built-in
○ SCM inventory file + venv* + UI
3. a custom inventory plugin
○ SCM inventory file + venv* + UI +
SCM inventory plugin (relative path)
2. a custom inventory script
○ UI
Linode Inventory Plugin (Shipped in Ansible but not Tower)

● Put inventory file in source control

https://github.com/ansible/test-playbooks
inventories/linode.yml
plugin: linode

● Create linode custom credential type

● Create custom virtual environment
○ pip install linode_api4
● Create Inventory Source from project
○ + credential, venv, etc.
● Update inventory source
Custom Inventory Plugins in Tower (Not Shipped with Ansible)

● Relative tree structure is advised (use symlinks if you have to)

# sqlite example from earlier

.
├── hosts.db ● Add both to source control
├── inventory_plugins
│ └── sqlite.py
○ Inventory plugin
└── sqlite.yml ○ Inventory file

● Collections coming soon in Ansible Tower

○ Add inventory file to source control
○ Add collections/requirements.yml
 Drive your Automation → Drive your Business

Ansible Automation is glue. Inventory drives

your automation which drives your
business.

Inventory plugins give you more power than

ever to integrate Ansible with your
infrastructure and business systems.

Combine multiple SOT (sources of truth) to

get complete data and a bigger picture.

Use Ansible as a feedback loop to keep all

systems accurate and current.
https://github.com/willtome/managing-meaningful-inventories/