Ansible - Inventory, Variables, and Modules (Part 2.2 of 14)

Welcome back. Today we will cover Host and Group variables, followed by modules. I originally planned to cover plays in this section, but realized it makes more sense to cover when we cover playbooks (part 5).

Host and Group Vars

Sometimes you need to store additional information associated with a host or a group. Ansible supports this through variables (referred to as vars in the Ansible documentation). Let’s take a look at some host variable examples.

1
2
3
4
5
[proddatabase_servers]
sql1.softwarecorp.com sql_port=8888
[devdatabase_servers]
devsql1.softwarecorp.com sql_port=1433

In the above example, we’ve assigned a variable named sql_port to the host sql1.softwarecorp.com with a value of 8888 and a variable of the same name to devsql1.softwarecorp.com with a value of 1433. While this can be useful, it’s more likely that you will be setting variables at a group level instead on individual hosts.

1
2
3
4
5
6
7
8
[prodapp_servers]
appserver1.softwarecorp.com
appserver2.softwarecorp.com
[prodapp_servers:vars]
http_port=443
proxy=proxy.softwarecorp.com
maxRequestPerChild=900

In the above example, we set 3 variables (http_port, proxy and maxRequestPerChild) on the prodapp_server group. These variables are applied to each host that is part of the prodapp_server group. There are a few rules around variables to keep in mind. First we’ll look at variables precedence is handled.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[appserver:children]
prodapp_servers
devapp_servers
[prodapp_servers]
appserver1.softwarecorp.com port=443 proxy=proxy2.softwarecorp.com
appserver2.softwarecorp.com port=443
[devapp_servers]
devappserver1.softwarecorp.com
devappserver2.softwarecorp.com
[production_servers:children]
appserver
[production_server:vars]
proxy=proxy.softwarecorp.com

Variables have an order of precedence, with host variables having the highest precedence. Group precedence lowers as you move further from the host, so a variable defined on the host appserver1.softwarecorp.com would override the value set on the production_servers group, but would not override a variable set on the appserver group. The next rule is how variables work when hosts are in multiple groups.

1
2
3
4
5
6
7
8
9
10
11
12
[app_servers]
appserver1.softwarecorp.com port=443
appserver2.softwarecorp.com port=443
devappserver1.softwarecorp.com
devappserver2.softwarecorp.com
[production_servers]
webserver1.softwarecorp.com
webserver2.softwarecorp.com
sql1.softwarecorp.com
appserver1.softwarecorp.com proxy=proxy.softwarecorp.com
appserver2.softwarecorp.com proxy=proxy.softwarecorp.com

Hosts in multiple groups will have their variables merged together, so in the above example appserver1.softwarecorp.com will have proxy and port variables.

Ansible will allow you to define group and host variables in their own file or files in addition to setting them in the inventory file. These files can have no extension or end in ‘.yml’, ‘.yaml’, or ‘.json’. Assuming you have an inventory path (an ansible configuration we will cover in a later part) of /etc/ansible/hosts you can create host and group variable files in directories as follows.

1
2
3
/etc/ansible/host_vars/appserver1.softwarecorp.com
/etc/ansible/group_vars/production_servers
/etc/ansible/group_vars/all

Ansible will apply all variables defined in the directory (matching the above schema) /etc/ansible/host_vars/appserver1.softwarecorp.com to the host appserver1.softwarecorp.com and /etc/ansible/group_vars/production_server and /wtc/ansible/group_vars/all to all members of the production_server group (and child groups). Note that we’ve used the special all group covered in the previous post to assign variables to all hosts.

Lastly, we still need to know how to define variables within these files. Fortunately for us, this process is pretty straightforward.

1
2
3
--- production_server
port: 888
proxy: proxy.softwarecorp.com

As you can see it’s simply a key value pair separated by a colon (:) and a space. The three dashes (—) indicate a comment and are ignored by Ansible.

Modules

Modules are the core working component of Ansible. Modules are commands that perform some operation and can return information or data. There’s a extensive library of existing modules. Ansible has modules for managing networks, windows, cloud resources(AWS, Azure, etc), source control and more. You can find a full listing in the Module Index Should you need a module that doesn’t exist, you can create your own using powershell or python. We will cover this in Building modules (Part 11).

Modules can be called as part of a play (covered below) or ad-hoc via the shell. For these examples, we are going to assume we have the inventory file from the last post. Lets take a look at 3 examples of ad-hoc ansible commands.

1
2
3
$ ansible all -m ping
$ ansible webservers ping
$ ansible devweb_servers,devapp_server ping

Well that’s pretty simple, but lets review. The first command is passing the argument all. This tells ansible to run the module against all hosts in our inventory file (This is the special all group covered in the previous post). In the second example, we are passing webservers and in the third example we are passing both devweb_servers and devapp_server. Ansible will use the host defined in these groups to run the command against. Next we pass -m followed by the module name (in this example the module is ping). I bet some of you noticed that in the second and third examples, -m was omitted. This is because modules that take no parameters do not require -m. Let move to something more complex.

1
$ ansible devwebservers -a "/sbin/reboot" -f 10

Now this is a bit more complex. It should be clear by now, but devwebservers is telling ansible to use the devwebservers group in our inventory file. You’ll notice that we do not have a -m or a module. When this is omitted, Ansible will use the command module by default. Next we pass -a followed by '/sbin/reboot'. This is passing the argument to the command module. This will run the command /sbin/reboot on each host in the devwebservers group. Next up is -f which specifies that we want to run this command in 10 parallel forks. Now that’s a pretty useful parameter if you need a nightly/weekly system reboot against a large number of machines. Since this is clearly not a windows command, it would fail if we had a windows machine in our inventory list. Lets move on to another example.

1
ansible all -m win_chocolatey -e 'state="latest"'

Now this is a favorite of mine. Lets break it down. We are targeting all servers in our inventory file running the module [![win_chocolatey] (http://docs.ansible.com/ansible/latest/win_chocolatey_module.html)]. Next we have -e. This allows up to specific variables in a key/value or YAML/JSON format for use by the module. In this example, we are telling the win_chocolatey module we want the state to be the latest (which will attempt to update all chocolatey installs).

You can also get documentation on a specific module by using the ansible-doc command. Let look at what we get if we user the win_chocolatey module.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
[root@AnsibleVM ~]# ansible-doc win_chocolatey
> WIN_CHOCOLATEY (/usr/lib/python2.7/site-packages/ansible/modules/windows/win_chocolatey.py)
Installs packages using Chocolatey (http://chocolatey.org/). If Chocolatey is missing from the system, the module will install it. List of packages can be found at http://chocolatey.org/packages
Options (= is mandatory):
- allow_empty_checksums
Allow empty checksums to be used.
[Default: False]
- force
Forces install of the package (even if it already exists).
Using `force' will cause ansible to always report that a change was made.
(Choices: True, False)[Default: False]
- ignore_checksums
Ignore checksums altogether.
[Default: False]
- ignore_dependencies
Ignore dependencies, only install/upgrade the package itself.
[Default: False]
- install_args
Arguments to pass to the native installer.
[Default: (null)]
= name
Name of the package to be installed.
- params
Parameters to pass to the package
[Default: (null)]
- source
Specify source rather than using default chocolatey repository.
[Default: (null)]
- state
State of the package on the system.
(Choices: present, absent, latest, reinstalled)[Default: present]
- timeout
The time to allow chocolatey to finish before timing out.
[Default: 2700]
- upgrade
If package is already installed it, try to upgrade to the latest version or to the specified version.
As of Ansible v2.3 this is deprecated, set parameter `state' to "latest" for the same result.
(Choices: True, False)[Default: False]
- version
Specific version of the package to be installed.
Ignored when `state' is set to "absent".
[Default: (null)]
EXAMPLES:
# Install git
win_chocolatey:
name: git
state: present
# Upgrade installed packages
win_chocolatey:
name: all
state: latest
# Install notepadplusplus version 6.6
win_chocolatey:
name: notepadplusplus.install
version: '6.6'
# Install git from specified repository
win_chocolatey:
name: git
source: https://someserver/api/v2/
# Uninstall git
win_chocolatey:
name: git
state: absent
MAINTAINERS: Trond Hindenes (@trondhindenes), Peter Mounce (@petemounce), Pepe Barbe (@elventear), Adam Keech (@smadam813)
METADATA:
Status: ['preview']
Supported_by: curated

Now this tells us a lot of information about what parameters can be used and some example of calling the module.

Next time we’ll be installing Ansible and creating a couple machines for testing

Ansible - Inventory, Variables, and Modules (Part 2.1 of 14)

Welcome to part 2 of my Ansible series. If this is your first time here, you might consider starting with part 1 I’ve broken this part into two post to make it easier to consume. Part 2 will cover inventory, variables, modules and plays. These components are critical to understanding how Ansible works and the components of playbooks. This part covers mostly oncepts with supporting examples. When we get to Ad-hoc Ansible examples, we’ll get to see Ansible in action. So without further ado, lets get started.

Inventory

Ansible needs to know what hosts to execute modules against. This list of hosts is called an inventory file. It’s a flat text file format, but does support some grouping and organization options. Here’s a simple sample.

1
2
3
4
5
6
7
8
9
10
11
12
13
webserver1.softwarecorp.com
webserver2.softwarecorp.com
devwebserver1.softwarecorp.com
devwebserver2.softwarecorp.com
sql1.softwarecorp.com
devsql1.softwarecorp.com
appserver1.softwarecorp.com
appserver2.softwarecorp.com
devappserver1.softwarecorp.com
devappserver2.softwarecorp.com
mail.softwarecorp.com
dc1.softwarecorp.com
dc2.softwarecorp.com

Our simple sample has 13 hosts without any organiation. This is simpe enough to start running ad-hoc ansible commands against, but what it can’t do is target a specific server or specific servers. What we need is a bit more organization to our inventory.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
[web_servers]
webserver1.softwarecorp.com
webserver2.softwarecorp.com
devwebserver1.softwarecorp.com
devwebserver2.softwarecorp.com
[database_servers]
sql1.softwarecorp.com
devsql1.softwarecorp.com
[app_servers]
appserver1.softwarecorp.com
appserver2.softwarecorp.com
devappserver1.softwarecorp.com
devappserver2.softwarecorp.com
[mail_server]
mail.softwarecorp.com
[domain_controllers]
dc1.softwarecorp.com
dc2.softwarecorp.com
[production_servers]
webserver1.softwarecorp.com
webserver2.softwarecorp.com
sql1.softwarecorp.com
appserver1.softwarecorp.com
appserver2.softwarecorp.com
[dev_servers]
devwebserver1.softwarecorp.com
devwebserver2.softwarecorp.com
devsql1.softwarecorp.com
devappserver1.softwarecorp.com
devappserver2.softwarecorp.com

Well that looks a lot better. We have a bunch of meaningful groups, but you’ll notice in the above example that a number of hosts that exist in multiple groups. Ansible support is perfectly happy with this, but can result in some unexpected behavior related to variables (we’ll cover these later in this post). While the above is a functional inventory file, odds are we don’t really want to repeat hostnames across multiple groups as this leads to added maintenance costs and possibility of maintenance errors. Luckily Ansible supports groups within groups.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
[prodweb_servers]
webserver1.softwarecorp.com
webserver2.softwarecorp.com
[devweb_servers]
devwebserver1.softwarecorp.com
devwebserver2.softwarecorp.com
[proddatabase_servers]
sql1.softwarecorp.com
[devdatabase_servers]
devsql1.softwarecorp.com
[prodapp_servers]
appserver1.softwarecorp.com
appserver2.softwarecorp.com
[devapp_servers]
devappserver1.softwarecorp.com
devappserver2.softwarecorp.com
[mail_server]
mail.softwarecorp.com
[domain_controllers]
dc1.softwarecorp.com
dc2.softwarecorp.com
[production_servers:children]
prodweb_servers
prodapp_servers
proddatabase_servers
[dev_servers:children]
devweb_servers
devapp_servers
devdatabase_servers
[webserver:children]
prodweb_servers
devweb_servers
[appserver:children]
prodapp_servers
devapp_servers
[databaseserver:children]
proddatabase_servers
devdatabase_servers

While this looks a bit more complex, it is actually far easier to maintain and use. If you have a new server, or retire a server, you simple add or remove the hostname from the appropriate group and the associated groups are automatically updated when used. It does require some up front planning and organization when setting up groups. Ansible can also handle multiple inventory files, which allows you to create separate inventory files which aids when maintaining larger inventories. Ansible allows you to specify multiple inventory files via passing -i followed by a comma separated list of inventory files.

1
ansible -i /etc/ansilbe/inventory/databases,/etc/ansible/webservers

Note: Groups can have multiple parents and multiple children, but cannot have circular relationships.

In the real work, especially in a large enterprise, it’s unlikely that you are going to want to maintain large inventories by hand. It’s not easily maintained, it doesn’t scale and has a high risk of being inaccurate. In addition, complex grouping can become challenging to maintain properly. Fortunately, ansible has the concept of dynamic inventory.

Dynamic Inventory

Ansible provides a method for gathering inventory from various cloud providers, LDAP, Cobbler, or enterprise CMDB (Configuration Management database) software. To use dynamic inventory, you specify a path to a script file that when executed returns inventory information in a JSON format instead of passing a path to an inventory file. Ansible ships with a number of provider scripts for systems like EC2/Eucalyptus, Rackspace Cloud, and OpenStack. Each of these require some initial setup and can be used along with parameter’s to generate inventory files with just the hosts you want. Here’s a couple of example calls for cobbler, Amazon EC2 and Openstack.

1
2
3
ansible webserver -m ping
ansible -i ec2.py -u ubuntu us-east-1d -m ping
ansible -i openstack.py all -m ping

Setting up these dynamic inventory providers is a bit out of the scope of this article and outside my experience. More information about dynamic inventory and examples can be found in Ansible’s documentation as well as documentation on how to create your own custom dynamic inventory source

Another option is to create inventory files thought an ansible module. This gives a great deal of flexibility to create inventory files with just the hosts and variables needed. It also allows you to check the content and helps troubleshooting. This is the method my company has gone with as we interfacing with a custom CMDB (and different instances of the CMBD) and operate across multiple datacenters. We’ll cover building modules in part 12

Lastly there are 2 built in groups: all and ungrouped. All contains all hosts while ungrouped contains only hosts that are not defined in a group.

The second post of part 2 will cover Host and Group Variables, Modules and Plays. Click here to continue

Ansible, Part 1 of many

5 months ago, I got the opportunity to dive headfirst into the DevOps world. I’ve always played on the edge of IT and Software development, but this was the first time I’ve gotten to focus deeply on DevOps. It’s been a challenging and rewarding 5 months so far and I’ve gotten to learn a number of DevOps tools. One of our main tools is Ansible.

This is the start of my series on Ansible and over the next few months I will will provide

  • A high level overview (This post)
  • in-depth explanation of Inventory, Variables, and Modules (Part 2)
  • Getting Ansible Setup (part 3)
  • Ad-hoc Ansible Examples (Part 4)
  • Creating playbooks (Part 5)
  • Inventory, Hosts, Facts and Delegation (part 6)
  • Under the hood (part 7)
  • Security and Vault/Password Management (Part 8)
  • Debugging Playbooks (Part 9)
  • Advanced Playbook Features (Part 10)
  • Advanced Playbook Design (Part 11)
  • Building Modules (Part 12)
  • Tips, Tricks and Best Practices on using Ansible.(Part 13)
  • Ansible Tower (Part 14)

So what the heck is Ansible

In simplest terms, it’s an automation engine and library of modules. It’s kinda like C#, you’ve got a runtime (engine) and a class libraries (modules). With C#, you can use these components to create either more components or an application. With Ansible, you create playbooks which can use other playbooks, plays and modules to accomplish a task often in a repeatable manner. Also like C#, these a million and one things you can do accomplish a task, some ways are better than other while some are just about trade-offs. It can be used to accomplish a variety of goals such as provisioning, configuration management, app deployment, continuous delivery, security and compliance, orchestration and more. It also has a very rich module library for integrating with other systems like AWS, Cisco, F5, GCP, vmware, windows, and many more.

Ok, so why should I use Ansible

If you have repeatable tasks, and you do, you can leverage Ansible to automate and simplify these task and get back to more creative and rewarding work. In reality, why you’d use Ansible is to automate the repetitive tasks to ensure they are done automatically and consistently. This could be doing CI or Deployments, or maybe it’s automating updating images for AWS/Azure or maybe you need to collect information on remote systems for auditing and compliance purposes. Ansible is so flexible that the reason you’d use it really depends on what you are trying to accomplish.

What if my company has custom systems to integration with

That’s not a problem. While there are a ton of integrations already developed, you can develop your own components using wither powershell or python. There’s a small bit of ceremony on how to write these modules, it’s fairly easy to learn. My experience already included adding modules to integrate with 2 different systems, 1 3rd party and 1 internally developers.

But what about cost and support

Well it’s open source, but there is premium support available though RedHat. There’s a ton of documentation available and a great active community that holds various events and meetups.

Now onward to Part 2.

Hello World

Hello all, if you’ve gotten here and this is the only post, well congratulations you found me before I have finished getting things setup. I apologize for all the broken links and other missing content, but check back in a week or 2 and there should be some real content.