Open Credo

November 4, 2015 | Software Consultancy

Reusing ansible roles with private git repos and dependency management

Writing reusable roles for Ansible is not an easy task but one that’s worth doing. This post should walk you through the basics of writing reusable roles with dependencies backed by public and private git repositories.

WRITTEN BY

Adrian Ursu

Adrian Ursu

Reusing ansible roles with private git repos and dependency management

Why do it?

One of the great advantages of Ansible is that it is easy to learn and straightforward to use. That means you are able to achieve a specific task in a relatively short period of time and without complete knowledge of the full capabilities of the system.

Sounds good, doesn’t it? It is, with one exception. Getting up and running with a limited knowledge of the features and best practices makes people (which nowadays are always in a hurry) create superficial and inflexible roles.

To quote the documentation about roles: “You absolutely should be using roles. Roles are great. Use roles. Roles! Did we say that enough? Roles are great.”.

One of the reasons why roles are great is that they are REUSABLE. At least they are meant to be. You can compare an Ansible role with a Opscode Chef Cookbook, considering the functionality offered, even if Ansible can offer so much more.

Having well written and flexible roles has its advantages:

  1. Smaller roles with dependencies. Easier to test and manage.
  2. Role version management (usually tracked by git tags or branches)
  3. When you need it, just pick up an existing role, don’t create a new one.
  4. Reusable Ansible Roles versioned in git with dependencies using tags/branches is a great subject to talk about at parties.

What you should keep in mind

Creating reusable Ansible roles by following best practices is not as easy as it sounds but it’s definitely worth taking that challenge. You start by identifying reusable components (this might be from packages that you find yourself installing and configuring over and over again to deployment strategies ).  After you pined what you want to reuse, as you are writing the role, you are soon faced with taking decisions on how flexible and configurable you want that role to be.

From my experience, I’ve seen two types of extremes that you don’t want to fall into:

  • making everything in the role configurable (overdoing it). This leads to roles that are hard to read, understand and maintain.  
  • having almost everything hardcoded. (not doing it enough). This won’t help you with reusability.

Finding a balance between these extremes, best practices for writing and testing roles is not in the scope of this post. These topics will be analysed in later articles.

Not everything is meant to be reusable

One more important thing to keep in mind is that some roles are more reusable than others. For example a good candidate for role reusability is a role for installing and configuring Artifactory, or a role which will download, install and run Spring Boot Applications on different machines.

Before using Ansible, I used Opscode Chef.  One of the things that I really liked about it was the Chef Supermarket. You could easily find  good quality and maintained cookbooks. Ansible Galaxy tries to do something similar for roles to what Chef Supermarket does for chef cookbooks.

What is Ansible Galaxy

Ansible Galaxy can refer to one of  two things:

  • a website for sharing and downloading Ansible roles
  • a command line tool that facilitates working with roles

The Ansible Galaxy Website – is a nice idea, but one that is in a relatively incipient state. It tries to apply the open source concept to ansible roles, by gathering, centralising and connecting roles from GitHub repositories while providing quality indicators through a rating system.

 In my opinion, the Ansible Galaxy Website is still in the early stages of forming a mature and responsible community.  Also, from my experience, the quality of most roles found there is questionable, mostly because they lack contributions, maintenance by the owners and feedback from users. The biggest reason however, could be that Ansible is so easy to get up and running with, that few people bother to “get it right”.

I am certain that in the near future this will change.

The ansible-galaxy command line tool – is something that I want to focus the discussion on because it has a lot of advantages. Some of them are:

  • can create the base structure for new role (role scaffolding) ensuring that you comply with the standards
  • has a yaml format for advanced control over role requirements
  • can pull and install roles from the Ansible Galaxy website
  • can pull and install roles from private source code repositories (Github, Bitbucket) and archives from web servers
  • gives you the possibility to specify where roles should be downloaded and even to rename them

Let’s go thorough some of the features of the ansible-galaxy CLI and finish with an example of two dependent roles.

Create a new role role structure

    ansible-galaxy adrianursu$ ansible-galaxy init sample-role
    - sample-role was created successfully

This will lead to the following file structure:

ansible-galaxy adrianursu$ tree sample-role/
sample-role/
├── README.md
├── defaults
│   └── main.yml
├── files
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── tasks
│   └── main.yml
├── templates
└── vars
    └── main.yml

For more information about directory layout  and content organisation you should take a quick look here.

Installing roles

Roles can be pulled and installed in multiple ways by using the CLI. The easiest and most straightforward way of doing it is to use the ansible-galaxy website:

ansible-galaxy install username.rolename

The location where roles are downloaded can be configured in your ansible.cfg file as the roles_path.

Using roles with public and private git repositories

A not so much advertised feature of ansible galaxy CLI is the ability to download roles from private git repositories. This is something very useful in corporate environments where code can’t be open sourced and automation is a must.

A new YAML format to define the role requirements file was introduced since version 1.8. This new feature adds the possibility to specify remote repositories from where to download roles.

                #command to install roles from a requirements file
		ansible-galaxy install -r requirements.yml

The new YAML format offers a great degree of freedom and configuration to specify role dependencies. Below I will go through a simple setup for two roles:

  • sample_role_oracle-java ( ansible role to setup Oracle Java Development Kit )
  • sample_role_launchpad-ppa-webupd8 (ansible role to setup webupd8 apt repo)

Here we have two roles, one for installing Oracle JDK and one for installing the webupd8 apt repository. Here, the webupd8 apt repository is a precondition for installing Oracle jdk on Debian based distributions. We know, from the Ansible docs, that role dependencies are always executed before the role that includes them, and are recursive.

For demonstrating this setup, a simple vagrant project was created here. In order to run this, make sure that you have Vagrant and the vagrant-hostsupdater plugin installed.

Clone the sample Bitbucket repository
git clone https://bitbucket.org/ursuad/ansible_galaxy_git_repos
Check out the content of our requirements.yml file
- src: git+https://bitbucket.org/ursuad/sample_role_oracle-java
  version: v1.1.6

You can see that there is no dependency in the requirements file on the sample_role_launchpad-ppa-webupd8 role. This is not needed, because the dependency is specified on the meta/main.yml file of the sample_role_oracle-java role:

dependencies:
  - { role: 'git+https://bitbucket.org/ursuad/sample_role_launchpad-ppa-webupd8,v1.0.0'}

This is very important to remember because that’s where the dependencies for a role must be. You, as the user of a role, in this case “sample_role_oracle-java”, must only know that you want to run a specific tag for that role (In this case it’s v1.1.3, but it might be release-v1). It is the job of the developer of the role to make sure that version v1.1.6 of the oracle-java role needs version v1.0.0 of the ppa-webupd8 role, and it’s the job of ansible-galaxy to download this dependency automatically.

Let’s install the roles defined in the requirements.yml file
# install the roles specifying the roles path
ansible_galaxy_git_repos adrianursu$ ansible-galaxy install -r requirements.yml -p roles/
- executing: git clone https://bitbucket.org/ursuad/sample_role_oracle-java sample_role_oracle-java
- executing: git archive --prefix=sample_role_oracle-java/ --output=/var/folders/bf/0jy8vjln2pz1047jykdw4tq40000gp/T/tmprEfvly.tar v1.1.6
- extracting sample_role_oracle-java to roles/sample_role_oracle-java
- sample_role_oracle-java was installed successfully
- adding dependency: sample_role_launchpad-ppa-webupd8
- executing: git clone https://bitbucket.org/ursuad/sample_role_launchpad-ppa-webupd8 sample_role_launchpad-ppa-webupd8
- executing: git archive --prefix=sample_role_launchpad-ppa-webupd8/ --output=/var/folders/bf/0jy8vjln2pz1047jykdw4tq40000gp/T/tmpQusBAt.tar v1.0.0
- extracting sample_role_launchpad-ppa-webupd8 to roles/sample_role_launchpad-ppa-webupd8
- sample_role_launchpad-ppa-webupd8 was installed successfully

By specifying the -p parameter, we tell ansible to download the dependent roles in the roles directory. That way, we have a working shippable product which is not necessarily dependent on those git repos. This comes in handy when, for example, your company has a private repository for ansible roles and you have multiple clients. You don’t want to give those clients access to the repositories, but you can still give them certain versions of different roles required in a specific project.

If you try to run the same command again, you will see that you get:

ansible_galaxy_git_repos adrianursu$ ansible-galaxy install -r requirements.yml -p roles/ 
- executing: git clone https://bitbucket.org/ursuad/sample_role_oracle-java sample_role_oracle-java
- executing: git archive --prefix=sample_role_oracle-java/ --output=/var/folders/bf/0jy8vjln2pz1047jykdw4tq40000gp/T/tmpCIOdR5.tar v1.1.6
- extracting sample_role_oracle-java to roles/sample_role_oracle-java
- error: the specified role sample_role_oracle-java appears to already exist. Use --force to replace it.
- sample_role_oracle-java was NOT installed successfully.
- you can use --ignore-errors to skip failed roles.

This is because ansible-galaxy detects that you already have those roles in your folder, so it doesn’t download them again.

Where to go next?

This is just a small example of how you can reuse ansible roles with ansible-galaxy. Even if we barely scratched the surface, this should give you a good starting point for writing reusable and good quality roles. 

A good exercise in doing this is to start contributing, rating and give feedback on existing roles on the Ansible Galaxy Website or publish your own.  Open Source is based on community feedback and contributions. This will improve the quality of your roles and help the website develop a larger and more mature audience. Ansible Galaxy CLI will help you in maintaining a standard role structure and in managing role dependencies from public and private sources.  

RETURN TO BLOG

SHARE

Twitter LinkedIn Facebook Email

SIMILAR POSTS

Blog