Best practices to build great Ansible playbooks

I have been building playbooks for two years and I always try to make them understandable. However, one of our playbooks in a big project became very complex and is now very hard to understand and to maintain. I didn’t want to make that mistake again so I made a list with my personal good practices to prevent building complex playbook that I share with you here.

TL;DR: If you don’t want to build unintelligible playbooks that will cost you a lot in the long run, you should read my following suggestions.

[Edit]: I’m writting a book about Ansible, click here if want a free draft.

Make it simple

I am a web developer not an OPS. I build Node.js/AngularJs and Symfony applications. I remember my first times doing some provisioning, it was running with Puppet. It was brilliant but also very complex. I had a really hard time understanding how it worked and debugging it wasn’t the funniest part in my life. I hated working with provisioning at that time. I still make those nightmares about it…

I was then introduced to Ansible and I completely changed my mind about provisioning \o/. Now I love it. And I want beginners to feel the same as I do. Therefore I really focus on making my playbooks truly simple so they can be understandable by everyone. So if you also believe that automated provisioning should be applied by everyone, then think about beginners when you write your code and make everything simple. Everybody should be able to use your playbooks.

Baby Ansible

(Thanks Justin Nemmers for the picture !)

Most of the time my playbooks are used to provision servers running a webapp and since I always run them on fewer than 5 servers at a time, I don’t try to make them quick (to execute), speed is not a big deal for me. But I guess execution time is an important factor when you have to run your playbook on thousands of servers.

Therefore, I think that my best practices fit only for people who use Ansible in the same way as I do.

Avoid using tags

no tags

I often read that we should tag everything, but I disagree.

TL;DR: It can be OK to tag a role in a playbook, but no one should use tag inside a role.

Why do tags exist in Ansible ? Let’s see what the Ansible documentation says:

If you have a large playbook it may become useful to be able to run a specific part of the configuration without running the whole playbook.
Both plays and tasks support a “tags:” attribute for this reason.

There are ways to avoid tags

I use Ansible to provision my servers or to udpdate a configuration on some servers. Playbooks have to be idempotent and it’s better to check idempotency at each execution. So the only reason why I could be using tags is to gain execution time.
But as I said earlier, time is not a big deal for me. Therefore I have no interest in using them.

If you are building your playbook and you want to run only some roles to gain time, just comment the unnecessary roles in the playbook. You don’t need tags for that.

Sometimes, you may have to perform a specific action. Like during the “heartbleed crisis”, you had to update your bash on almost every server. But even in this kind of situation, you don’t use tags, you use a specific new playbook.

When you run a playbook using tags you are often targeting specifics roles like the example from the Ansible documentation

- name: be sure ntp is installed
  yum: pkg=ntp state=installed
  tags: ntp

- name: be sure ntp is configured
  template: src=ntp.conf.j2 dest=/etc/ntp.conf
  notify:
    - restart ntpd
  tags: ntp

- name: be sure ntpd is running and enabled
  service: name=ntpd state=running enabled=yes
  tags: ntp

In this case I’d rather have a playbook “ntp” that run only the ntp role than using the tags.

Which one of the following commands tells us the most clearly what is going to happen ?

ansible-playbook playbook.yml -i hosts/production --tags="ntp"

or

ansible-playbook ensure-ntp-is-working.yml -i hosts/production

The first one is talking about ntp. But we don’t know the purpose of running this command. With the second one we know that it is to ensure that ntp is working.

Think about beginners ! Make their life easier, don’t use tags.

not suited

Tags come from the dark side

So tags are not useful. But there is worse. They are harmful to your playbook because they add complexity. When you add a tag, this tag is here for a purpose. Therefore, every time they see a tag in your playbook, people have to understand the purpose of it. As a result, your playbook is becoming more difficult to understand and to maintain.

Here is a part of one of our nginx’s role.

- name: add repo file
  template: src=nginx.repo dest=/etc/yum.repos.d/nginx.repo mode=644 owner={{ app_user }}
  tags:
  - nginx
  - packages

- name: install
  yum: name=nginx enablerepo=nginx state=latest
  tags:
  - nginx
  - packages
  - yum

- name: create working directories
  file: path={{ item }} state=directory mode=755 owner={{ app_user }}
  with_items:
    - "{{ etc_path }}/nginx"
    - "{{ etc_path }}/nginx/conf.d"
    - "{{ log_path }}/nginx"
    - "{{ tmp_path }}/nginx"
  tags:
  - nginx
  - filesystem

- name: remove nginx conf file
  file: path=/etc/nginx/nginx.conf state=absent
  tags:
  - nginx
  - yum-cleaning

- name: disable service as sudo_user
  service: name=nginx enabled=no
  tags:
  - nginx
  - services
  - yum-cleaning

There are six different tags (nginx, packages, yum, filesystem, yum-cleaning, service) making at least 6 ways of using the role. If you want to test your playbooks (and you should), you will have to test every combination that your tags permit instead of one single run without tags. It’s already hard to test the playbooks because they depend on the Ansible version and the OS version. If you add tags, it becomes really annoying.

In this role, if we want to test every possible combination, we will have to test 2^6 = 64 combinations !

GOT maths

The same part of the role without the tags is pretty simple. We know that every task will be run anytime without any condition or specific purpose to understand.

- name: add repo file
  template: src=nginx.repo dest=/etc/yum.repos.d/nginx.repo mode=644 owner={{ app_user }}

- name: install
  yum: name=nginx enablerepo=nginx

- name: create working directories
  file: path={{ item }} state=directory mode=755 owner={{ app_user }}
  with_items:
  - "{{ etc_path }}/nginx"
  - "{{ etc_path }}/nginx/conf.d"
  - "{{ log_path }}/nginx"
  - "{{ tmp_path }}/nginx"

- name: remove nginx conf file
  file: path=/etc/nginx/nginx.conf state=absent

- name: disable service as sudo_user
  service: name=nginx enabled=no

It’s nearly impossible to remember all the tasks of your playbook that have a particular tag. So it’s never crystal clear what your are doing when you run your playbooks using tags.

random

One role, one goal

Avoid tasks within a role that are not related to each others. Don’t build “common role“. It’s ugly and it’s bad for the readability of your playbook.

one role one goal

Instead try to make more little roles with explicit names.

See this playbook:

- name: My playbook
  hosts: all
  sudo: true
  vars_files:
    - vars/main.yml

  roles:
    - common #What the hell does this role do?
    - Stouts.nodejs
    - Stouts.mongodb

And this one:

- name: My playbook
  hosts: all
  sudo: true
  vars_files:
    - vars/main.yml

  roles:
    - ubuntu-apt
    - create-www-data-user
    - Stouts.nodejs
    - Stouts.mongodb

The last one is easier to understand.

Convention on naming your role.

I really like roles that can be run on many OSs and for many kinds of application. I try to make my roles as generic as I can. But, if in order to work on many OSs or for many applications, the role becomes too complex, then I prefer using a more specific but simpler role.

simplicity

If your role works only on some OSs or for a specific kind of application, you should explicitly say so in the role’s name. By making so, just by reading the role’s name we know how specific it is.

We often find the name of the author in the name of a role but having it doesn’t help us a lot to understand how a role works. So I think it shouldn’t be in the role’s name.

Taking into account what I just said, my convention on how to name a role is OS-Application-Purpose.

A few examples:
debian-symfony-nginx: I can quickly understand that this role provision nginx for Symfony’s applications on every OS of the Debian’s family.
ubuntu-mongodb: The role will install and configure MongoDB on Ubuntu.
nodejs: Here the role will install Node.js on every kind of OS.

Make explicit the dependencies of a playbook

A playbook should be read like a great story in a book. You read the roles from top to bottom and you understand everything that is going on with the playbook, period.

no dependencies

If you want to quickly understand what a playbook does which solution do you prefer?

- name: My playbook with dependencies
  hosts: all
  sudo: true
  vars_files:
    - vars/main.yml

  roles:
    - elasticsearch
    - drupal

or

- name: My playbook without dependencies
  hosts: all
  sudo: true
  vars_files:
    - vars/main.yml

  roles:
    - java
    - elasticsearch
    - git
    - apache
    - mysql
    - php
    - php-mysql
    - composer
    - drush

When you read the second one, you see every roles involved in the playbook and you don’t have to dig to get the information.

One server, one run to provision it the first time

You can have many playbooks to manage your server in your daily operations like the ensure-ntp-is-working.yml playbook.
But you should be able to provision it the first time with only one playbook.

You shouldn’t do something like this:

ansible-playbook -i hosts/myserver playbook_part1.yml
ansible-playbook -i hosts/myserver playbook_part2.yml
ansible-playbook -i hosts/myserver playbook_part3.yml

Instead, you should be able to provision your server with:

ansible-playbook -i hosts/myserver playbook.yml

one playbook

If you have more than one playbook, in most of the time you will have to remember the order in which you have to run them. Testing and checking idempotency are also more pratical having one playbook instead of many.

Explore Ansible Galaxy

There are many great roles over there. Instead of rewriting everything go forking! Look at how other people do, you will learn faster.

Don’t use requirements.txt

When you type ansible-galaxy install -r requirements.txt, it uses the tag of the Github repository to git clone it. A git tag doesn’t set a version of the code. The code behind a tag can be updated. So you can’t know for sure what you will end up downloading by this way. And this is bad for many reasons including security.

danger

Instead use GIT and add the roles as submodules. It allows you to update the roles while being confident about the version of the code you download.

Put the community’s roles in a separate folder

If you want to be able to update the roles you found on ansible-galaxy or directly on Github you should put it in a separate folder so you can quickly find your roles (that you can change) and the community role (that you shouldn’t change). You will end with something like:


├── group_vars
├── hosts
├── roles
│   ├── community
│   │   ├── composer
│   │   ├── ubuntu-apt
│   │   ├── ubuntu-mysql
│   │   └─ ubuntu-symfony-nginx
│   ├── my_app-monitoring
│   └─ ubuntu-rabbitmq
├── vars
└─ playbook.yml

If you really want to make a change to a community role, you have two options:
– You make a pull-request for this change.
– You make the change and you move the role in the directory with your own roles.

If you choose to separate your roles, you have to tell Ansible where it can find them with the ansible.cfg file like for example:


[defaults]
roles_path = devops/provisioning/roles/community
inventory = devops/provisioning/hosts

Don’t use ‘vagrant provision’

If you are a Vagrant user, you may be using the ansible provisioner. I know that the command is very convenient but it will confuse beginners a lot because there are differents concepts involved:
– Creating the VM (Virtualbox)
– Configuring the VM (Vagrant)
– Provisioning the VM (Ansible)

It’s already not easy to understand each role of Virtualbox and Vagrant during the creation of the VM. If you mix it with the role of Ansible, you are not helping…

not helping

And honestly, you won’t lose so much time using

ansible-playbook playbook.yml -i hosts/vagrant

instead of

vagrant provision

At Theodo, we use OpenStack to create VMs for our staging environment. There is also plugin to create OpenStack’s VM from the Vagrantfile. Same story here, don’t use this plugin because it’s highly confusing.

One day one brillant trainee spent almost the whole day trying to create and provision one OpenStack VM via the Vagrantfile. Because we were doing vagrant up and vagrant provision for the dev environment, he wanted to do the same for the staging one to have consistency.

true story

Keep it light

This is the same as any other part of your code: you should delete every file and directory that is not mandatory.

about perfection

Check out our open source projects

We have a small open source organization on Github, have a look at out our website. Our goal is to provide simple-to-use tools to make the provisioning with Ansible easier and easier. By doing this, we hope that we will help a lot of people learning and enjoying Ansible. If you like the spirit of this organization, we are looking for people to help us.Just make a nice PR and you will be welcome! =).

help hand

Other great tips

I really liked this article about “Ansible (Real Life) Good Practices“.
You will find other opinions about tags… ;).

It will explain why you should:
Use a module when available
Set a default for every variable
– Use the ‘state’ parameter
– Prefer scalar variables
– Tag every task and role

You can find some other tips here like “Beware of default” in these slides.

Thanks for reading, if you want to share your bests tips with us, they are very welcome! The fastest way is to ping me on Twitter.

If you need help to build a nice Agile/Devops working environment, we will love helping you!


You liked this article? You'd probably be a good match for our ever-growing tech team at Theodo.

Join Us

  • Strahinja Kusutdic

    I almost stopped reading after you wrote “Don’t use tags” and “Tags Are Useless”. They are useless if you use them like in the example you gave, since those tags don’t make any sense. Why on earth would you ever execute a tag ‘filesystem’, or ‘packages’ by itself, since they won’t do anything meaningful. A good example of using tags in an example you wrote would be to have a tag for e.g. ‘nginx’ role which will be called ‘nginx-config’, which will only push the latest nginx configuration and reload nginx if needed, so that you don’t have to execute all tasks, just to push the new config. Or for example for ‘zabbix_agent’ role you can use ‘zabbix_agent_scripts’, which only pushes new scripts to all the nodes, without going through all zabbix_agent setup tasks. And my favorite usage of tags is for iptables. Our roles configure iptables themselves, so each role that needs to configure some iptables rules has a task for that and that task has an ‘iptables’ tag. So when we want to configure iptables on hosts, we just run that tag and it configures iptables instantly, instead of going over all roles and their tasks.

    Also it’s much easier to tag the whole role (which I recommend you always do) like this:

    roles:
    – { role: ‘nginx’, tags: ‘nginx’ }
    – { role: ‘php-fpm’, tags: ‘php-fpm’ }

    than to edit playbooks and comment out roles if you just want to execute one or more roles. It’s clean and simple and harder to make a mistake (because you don’t edit anything).

    And the ‘ntp’ tag example is a very bad example of using a tag, as well as using a playbook for that. ‘ntp’ should be a role and when adding a role to a play, it should be tagged like mentioned above.

    I’m not a fan of dependencies as well, but there is one situation where I like to use them. When there is a common repository for two roles, for example you want to have a ‘postgres’ and ‘pgbouncer’ roles which both need the official postgres repo to install packages, in that case I have a ‘postgres_repo’ role which is a dependency for those two roles.

    I don’t agree with not using the ‘vagrant provision’. If you setup ansible as the provisioner, you just have to type ‘vagrant up’ and you are all setup. How can that be harder on any developer than to do ‘vagrant up, ‘vagrant ssh’, then cd into some directory and run ansible-playbook …? If you do it right, the developer doesn’t even have to know ansible is being used. The most awesome thing about vagrant is that you can have a dev environment with one command (or two if you count the git clone).

    Regarding other small tips which are mentioned in the ‘Ansible (Real Life) Good Practices’ article, I agree with all of them except one ‘Set a default for every variable’. You should set a default for every variable of a role for which makes sense to have a default. For example if you have a ‘postgres’ role which can install and configure multiple versions of postgres (9.1, 9.2, …) I would recommend that you make the postgres_version (or however you call it) unset and make the role fail instantly if the variable is not set. The reason for this, is that you cannot guess which postgres version the user will need, but also you will be able to see from group_vars/host_vars what version is installed.

  • Torkel Lyng

    I use tags, dependencies and let vagrant do provisioning.

    Tags dramatically decreases the time if you want to rerun some parts, or skip some parts (downloads for example*).

    Dependencies simplifies playbooks.

    Vagrant do provisioning because the users using the Virtualbox doesn’t necessarily know what ansible is, neither is that an requirement.

  • Maxime Thoonsen

    Hello Strahinja,
    Thanks for the long comment, it’s very interesting!

    I almost stopped reading after you wrote “Don’t use tags” and “Tags Are Useless”.

    I updated my post so it sounds less radical.. =).

    A good example of using tags in an example you wrote would be to have a tag for e.g. ‘nginx’ role which will be called ‘nginx-config’, which will only push the latest nginx configuration and reload nginx if needed, so that you don’t have to execute all tasks, just to push the new config.

    You are right, I could have taken better examples, yours are better. I took the ntp one because it was in the Ansible documentation and the other one from a personal experience to underline my point. If you agree that tags increase complexity, you will try to avoid them as much as you can, and my point was to provide some tips to avoid using them.

    Let’s take the nginx case that I also encountered. Your “nginx” role has two purposes. The first is to install and setup nginx. The second one the to manage the nginx conf and the vhosts. In this case why not spliting it into two differents roles: “nginx-setup” and “nginx-conf” ? In your “setup” playbook you would have both roles and you would have a “update-nginx-conf” playbook with only the “nginx-conf” role. Then you can perform your daily updates of your nginx conf without using tags and running the entire playbook. What do you think ?
    For me it is more clear to do it this way AND we avoid using tags which add complexity: I got a lot a critics about this tags things but no one has ever mentioned tests ^^. I don’t know if you can do the same for iptables and your scripts so it may be mandatory to use tags for those cases.
    But I think you should still consider using tags less. I will add a quuote from the “Ansible (Real life) Good Pratices”:

    Keep the number of tags low, document the usage of tags, maybe have a closed set, and double check the spelling.

    I’m not a fan of dependencies as well, but there is one situation where I like to use them. When there is a common repository for two roles, for example you want to have a ‘postgres’ and ‘pgbouncer’ roles which both need the official postgres repo to install packages, in that case I have a ‘postgres_repo’ role which is a dependency for those two roles.

    I also changed the subtitle of this part to: “Make Explicit The Dependencies Of A Playbook”. It’s true that sometimes you need dependencies, but it shouldn’t make the playbook difficult to understand.

    I don’t agree with not using the ‘vagrant provision’. If you setup ansible as the provisioner, you just have to type ‘vagrant up’ and you are all setup. [..]

    This is purely pedagogic strategy. We are a fast growing startup, there is a lot of new people arriving every month.
    We have to help them understanding a lot of new concepts including those ones.
    Empirically I have noted that “vagrant provision” isn’t helping people to understand how the whole thing works so I made this unpopular decision to remove the provisioning from the Vagrantfile.
    But I agree that “vagrant provision” is far more convenient!!

    [..]You should set a default for every variable of a role for which makes sense to have a default.[..]

    What you say makes also a lot of sense.
    I like to have tools that work quickly out of the box (like the vagrant provision :) ) and having default var everywhere helps for that.
    You can be working on a little project where you don’t really care about the postgres_version.
    In the other hand all your arguments are great so…

    I hope my answers make sense, thank you again for your message. I would continue this conversation with great pleasure =).

    Maxime

  • Maxime Thoonsen

    Hello Torkel,

    I replied to Strahinja, so you might find some arguments in my answer.

    Dependencies simplifies playbooks.

    If you doesn’t write dependencies in your playbook, it removes lines so yes it simplifies the playbook in some manner but you lose a lot of readability, don’t you think so ?

    Vagrant do provisioning because the users using the Virtualbox doesn’t necessarily know what ansible is, neither is that an requirement.

    If you doesn’t want them to understand it, all right but if you want your team to understand the tools they use, you should consider not using the provisioning in the Vagrantfile.

    Maxime

  • Strahinja Kustudic

    Let’s take the nginx case that I also encountered. Your “nginx” role has two purposes. The first is to install and setup nginx. The second one the to manage the nginx conf and the vhosts. In this case why not spliting it into two differents roles: “nginx-setup” and “nginx-conf” ? In your “setup” playbook you would have both roles and you would have a “update-nginx-conf” playbook with only the “nginx-conf” role. Then you can perform your daily updates of your nginx conf without using tags and running the entire playbook. What do you think ?

    That works if you have a few hosts, but if you have more of them, it just doesn’t scale. Feedback I gave you should work with 10 and with 500 hosts. When you have more hosts your playbooks will usually be grouped by hosts/groups, and not by function, which means you wont have separate playbooks for setup and for configuration as you suggest. So for example for one web server this is how a playbook would look like:

    ---
    - hosts: web-server1
    roles:
    - { role: 'common', tags: 'common' }
    - { role: 'mysql', tags: 'mysql' }
    - { role: 'nginx', tags: 'nginx' }
    - { role: 'php-fpm', tags: 'php-fpm' }
    - { role: 'zabbix_agent', tags: 'zabbix_agent' }
    - { role: 'iptables', tags: 'iptables' }

    This playbook is used for setup and for configuration update, etc. If you had a different playbook for all different things, you would end up in a plethora of playbooks which would make everything very complex and messy, especially if you have a lot of hosts. Also if you have a ‘config’ role for all these roles, it would also be harder to read and understand.

    The other reason why you shouldn’t split setup and configuration roles is because, why would you install nginx without configuring it? Nginx configuration is part of the nginx setup. If you don’t configure nginx, it won’t work. When you see my nginx role you know it will do everything which is related to nginx, you don’t have to think if something is missing. One nginx role can easily have nice defaults if you just want to install nginx without adding specific vhosts, so even in this situation it’s easier to have one role, instead of two. So one ‘nginx’ role with ‘nginx-config’ tag is less complex on any day for me, than two roles, with two playbooks.

    This is purely pedagogic strategy. We are a fast growing startup, there is a lot of new people arriving every month.
    We have to help them understanding a lot of new concepts including those ones.
    Empirically I have noted that “vagrant provision” isn’t helping people to understand how the whole thing works so I made this unpopular decision to remove the provisioning from the Vagrantfile.

    Than this is more a tip how to make people learn Ansible faster, than how to use vagrant. From your article you get a feeling that ‘vagrant provision’ used with ansible is bad.

    I like to have tools that work quickly out of the box (like the vagrant provision :) ) and having default var everywhere helps for that.
    You can be working on a little project where you don’t really care about the postgres_version.
    In the other hand all your arguments are great so…

    I understand where you are going and what you say makes sense on a small project, but it doesn’t if you go beyond small. Also the title of the article is “Best practices to build great Ansible playbooks”, but you don’t mention anywhere that you are talking about a smaller project (and development oriented), that is at least how I got the feeling from your comments, but not from the article itself.

  • Maxime Thoonsen

    Thanks for the new answer.
    I see that you like the common role.. ^^.

    That works if you have a few hosts, but if you have more of them, it just doesn’t scale. Feedback I gave you should work with 10 and with 500 hosts.

    I don’t get it. What does it change if you have 10 or 500 hosts in our situation? In your case you have one playbook, in mine 2-4 but not 500 or .. ? I don’t see the scaling problem.

    If you had a different playbook for all different things, you would end up in a plethora of playbooks which would make everything very complex and messy, especially if you have a lot of hosts.

    We don’t need to split every roles, ‘php-fpm’ doesn’t need to be splited for example. Spliting roles add indeed complexity but it can be wisely made. I’m seeking for simplicity so of course there is a balanced decision to take.

    And this decision is also about how much execution time you need to gain by using tags OR small targeted playbooks. Here we are in two very different situation if we have 10 or 500 hosts.

    Also the title of the article is “Best practices to build great Ansible playbooks”, but you don’t mention anywhere that you are talking about a smaller project (and development oriented), that is at least how I got the feeling from your comments, but not from the article itself.

    If you have a non sensitive playbook to share with me so I can understand why it is so different I’ll be very happy =)

    Maxime

  • Strahinja Kustudic

    If you have 500 servers, you will not have one playbook, you will have a lot more playbooks which will run plays on specific groups or similar types of servers. The reason why you shouldn’t have one playbook for all host is because it’s hard to execute that big playbook on a subset of hosts, since you would usually use ‘–limit’.

    But an even bigger reason is that it takes time for Ansible to parse the whole playbook before it starts to do anything, For example our playbook with all hosts took around 30s to parse, and 30s is really a long time especially if you want to execute just a few tasks which execute in 2 seconds. So as I said what you usually do for convenience is have a playbook for groups of servers, e.g. website1.yml (which would have web, DB, caching servers), website2.yml (same thing), gaming_servers.yml (and if this it to big, you could also split it into smaller playbooks), payment_servers.yml, monitoring.yml, hadoop.yml, elk,yml, …, and besides these playbooks you usually have some special purpose playbooks for some complex deploys, or for some special things like deploying security patches, changing root keys, etc. So in your case you would have 2-4 times more playbooks than we have, and this makes a difference, 15 playbooks or 50 is a big difference, not to mention if those numbers are even higher.

    Of course you should have one playbook called e.g. site.yml which includes all playbooks so that you can easily run Ansible on all hosts if you need to.

    I would like to mention that 90% of the time when we use tags, we use them to execute a specific role, e.g.:
    ansible-playbook -i hosts webserver1.yml -t mysql

    For the other 10% we use them for iptables, or app deploys since our roles usually support deploying.

    I don’t have any playbook/role examples, but I will share a few things and how we do them in a near future and I’ll let you know when we do if you are interested :)

  • Fred Mitchell

    The issue of dealing with a large number of servers is nontrivial at best. If those servers are being managed in the cloud, then there are many things that can run concurrently. For instance, you might need to provision your Elasticsearch cluster, MongoDB cluster, and many different Application clusters, which can be done simultaneously.

    I’ve run into trouble with variable contention among different playbooks when you try to run multiple ones using includes, as well.

    I have created Ansible Powerplay to get around some of these issues.

    https://github.com/flajann2/ansible-powerplay

    I think tags are a good idea if used judiciously, and don’t quite understand why they’re so “bad”. Of course, anything can be abused. And commenting out sections? I think that is an especially bad idea, especially if you are working in teams. I have on more than one occasion accidentally checked in playbooks with sections commented out, and then wondered why the provisioning went wrong.

    With judicious use of tags, you should have no “need” to test every possible combination, or at least, not have to deal with a combinatorial explosion of the same.

    For example, with provisioning servers in AWS, one part of the playbook is responsible for creating the nodes of the cluster, and the other part responsible for installing and configuring the services running on those clusters. Tags will allow me to switch off the creation part that only needs to be done once anyway, when I don’t need it.

    I would agree somewhat with tags embedded in roles. I don’t generally think it’s a good idea, but I would not completely rule it out either. I think one should think very carefully about using them in that context. Too many potentials for abuse there.

    I do have a question for everyone here who does AWS provisioning. I use AWS tags on the instances to define them for the rest of the playbook, but there is an annoying latency between when the AWS tags are defined and when they become visible to the playbook. Are there any good workarounds for this, outside of sitting in an “endless” loop waiting for the tag to “show up”?

  • Maxime Thoonsen

    Hello Fred,

    Tags are bad because they add complexity, everytime you have a tag, you need to understand why it is here. Commenting out the code is only for dev purpose, no one should commit commented part of a playbook (in most of the case).

    “one part of the playbook is responsible for creating the nodes of the cluster, and the other part responsible for installing and configuring the services running on those clusters.”
    => My solution is to have 2 playbooks. It’s really more simple to understand and you don’t need tags.

    For your last question I haven’t encountered this situation, may be you can provide more details on how you do your AWS provisioning step by step so it will be more easy to understand your problem ? Maybe your playbook is public ?

    Thank for this interesting comment !

    Maxime

  • Fred Mitchell

    Thanks for your response.
    The playbooks are fairly involved, and proprietary, as they manage over 100 AWS instances in 5 VPCs. So, I can’t really show them…

    But basically, I have one role that filters on the instance tags and does a set_facts so that subsequent plays can use the facts as inventory.

    Problem is, from the time the instance is created and AWS-tagged to the time the tags are visible in boto is something on the order of 2 minutes or so, so by the time it comes to configuration they are skipped, in which case they have to be rerun.

    I was wondering if there is a “clean” way to wait for the AWS tags to become visible. I could write some code to do this, of course, but was hoping there is a more elegant solution to this.

    Back to the tags — I use tags only only in the roles: section, so each type of
    role has an associated tag. Makes sense for how I structured things across multiple Ansible playbooks, all run by PowerPlays.

    So I can pass in an Ansible tag from the command-line of powerplay with something like: “powerplay play -p development –skip-tags instance”, so that all the instance roles across all playbooks that are being run are skipped, which is what I want if the stack itself has already been deployed and all I want to run is the configuration.

  • Maxime Thoonsen

    Hello Fred,

    What if you split your playbook so there is one for the creation of instances and the others playbooks where there isn’t the instance roles ?

  • Fred Mitchell

    It still does not solve the latency problem between when the AWS tag is set and when it becomes visible to boto.

  • Maxime Thoonsen

    Hello, I don’t have a solution for this problem. I’m starting to work with boto so if I found one, I’ll ping you 😉

  • Marek Koch

    First of all thank you very much for the comprehensive post! Also I like the style. Pics/gifs are spot-on 😉
    I am just starting to use ansible in a virtualized in-house environment with 100 nodes. Anyhow I like to comment on your suggested naming convention. I am a huge fan of having some sort of convention. Imho it is only worthwhile if you follow it as strictly as possible. Your examples given kind of do the opposite. Sorry in advance for being picky.

    debian-symfony-nginx is fine to the OS-Application-Purpose scheme although a one word purpose like nginx, where nginx is also an application in a broader sense, is not really helpful.

    ubuntu-mongodb – well where is your purpose?

    nodejs – here you get rid of two of your suggested prefixes. Make it at least all-nodejs. If I had a convention like that and a colleague would just write nodejs, how should I know if he is still developing that or was he just lazy.

    To complete the feedback sandwich 😉 it is >really< awesome that you link another article worth reading and furthermore that that article shares a different view on tags. Much appreciated!

  • Maxime Thoonsen

    Hi Marek, I was hoping that people gave me feedbacks on this convention far sooner, so thank you very much! =).
    I totally agree, we can’t be sur that people are not just lazy if they wrote ‘nodejs’. ‘all-nodejs’ or ‘anyOS-nodejs’ can be indeed a more explicit solution.
    We can also be more clear on the purpose of ‘ubuntu-mongo’ but I don’t really see how. Is ‘ubuntu-install_and_configure_mongodb’ better for you ? Do you have any suggestion ?

    I think, I will update the post thanks to your feedbacks. What’s your twitter account ?

  • Pingback: Implementing Ansible – Chefkoch's Ansible blog()

  • Fred Mitchell

    That would be most appreciated. In the meantime I will just tool around the problem. Thanks.

  • vikram fugro

    Can’t agree more with “Don’t use ‘vagrant provision’” and it holds true in a general sense for any tech. Using any tool for what it is best at and what is was envisioned for, makes life of an operator/developer much much much easier and things are very easy to imagine, rather than clubbing too many into one leading to a one big monolithic beast!.

  • Keyboard Interrupt

    Thank you for this Post, it was a great Read :)

    I Think you should drop the OS portion in the Name entirely.
    If you i.e. start wit a role named `ubuntu-mysql` but later extend it so it can also be used on Debian, Centos or another Distribution, the Name breaks. Just name the role `mysql`

    Instead you could use the `meta/main.yml` https://galaxy.ansible.com/intro#meta and the `README.md` to clarify which Operating Systems are supported

  • Damodaram Gmr

    Hi All, this is my error and Please help me.

    ERROR! Attempted to read “/etc/ansible/hosts” as YAML: ‘AnsibleUnicode’ object has no attribute ‘keys’
    Attempted to read “/etc/ansible/hosts” as ini file: /etc/ansible/hosts:1: Expected key=value host variable assignment, got: server1.abc.com

  • vlcinsky

    Nice article on how to keep things practically simple. Thanks.
    The pictures in the text are funny, but sometime rather distracting.

  • ASA

    Very nice artiicle.

  • Norbert Varzariu

    Nice post, good thoughts there. I like your concepts, thank you for this write up.
    One little big piece you might want to add is “Use molecule to test your roles”.
    Molecule is really a quantum leap in ansible role development, you can easily set it up so that a check of your role will run upon every pull request or so.
    https://molecule.readthedocs.io/en/latest/

  • Er Balvinder Singh

    Great post, clear and precise. Love the way of explaining, me as a beginner and trying sincle yesterday to learn. It explained a lot . Keep posting more stuff like this. I have also sign up for your book. Waiting for the completion of boo. Good luck.

  • davi koscianski vidal

    I take it that his is a pretty old post, but do you still advocates for git submodules?
    I can’t see an advantage of it over git tags/revisions on requirements.{txt,yml}.

    Could you clarify, please?