Terraform Provider Development

It is expected that this separation will grant increased ownership to community teams supporting providers, allowing them to independently evolve and gain sufficient velocity to match the pace of development found in systems such as Amazon Web Services and Google Cloud Platform.

As part of transforming and expanding the Terraform ecosystem, HashiCorp also outlined the Terraform Provider Development Program. Primarily a self-service programme, this is aimed at technology vendors looking to support the provisioning of their infrastructure resources through Terraform. As part of this program Hashicorp have identified key agencies with solid experience in developing Terraform providers to support vendors going through this process.

OpenCredo are proud to have been recognised as an organisation with key expertise in the development and testing of Terraform providers. Having contributed to a variety of existing providers (Google and Vault amongst others), as well as having developed new providers from scratch for various clients, we thought it might be nice to distill some of our insight and thoughts in this area in article form.

Overview

Terraform is used to create, manage, and manipulate infrastructure resources. The types of resources it is able to control varies greatly – including both physical and virtual machines, network switches, containers, as well as PaaS and SaaS offerings. As long as the target provider has an API, and can logically be understood to consist of composable entities, it’s typically fair game for inclusion.

End users interact with Terraform by describing and defining their infrastructure setup as a set of interconnected resources within Terraform definition files. It is the responsibility of the Terraform Provider layer to translate between these user definitions, and the ultimate calls needing to be made under the covers to provider specific APIs in order to realise these entities.

At the time of writing Terraform supports more than 70 different providers including the likes of AWS, GCP and Azure in the pure play cloud area, CloudFlare and Heroku in the SaaS and PaaS areas, and many more.

The business benefits of a custom provider

Programmable infrastructure is becoming a key foundation of modern DevOps workflows. In our opinion, Terraform is one of the de-facto tools of choice in this space. The ability to programmatically control your internal, and in some cases proprietary, infrastructure and services through Terraform has the potential to yield some interesting business benefits including:

Reduce risk and increase the reliability of managed infrastructure

Infrastructure automation significantly reduces the risk of deployment when compared to manual human actions. Manual configuration is error prone and can often lead to outages and downtime – resulting in disgruntled team members and customers struggling to find ways to get their system back up and running. Bespoke automation scripts often end up complex, unreliable and difficult to manage. The simple nature of Terraform configuration and powerful deployment model provides a easy-to-use framework for safely performing these actions.

Gain broader adoption by customers and the community

Terraform is already used by many organisations to manage their infrastructure. It is increasingly a desirable core skill for DevOps engineers, with HashiCorp, Terraforms creator, named in Gartners Cool Vendors in DevOps 2016 report. Instead of developing a bespoke tool which requires specialist skills to operate, creating a Terraform provider aligns your provisioning process with other infrastructure resources and makes it easier for engineers to incorporate into automation pipelines. 

With many organisations looking to operate in a hybrid world, having a tool which can manage both public cloud providers like AWS and your on-premise infrastructure makes it much easier to tackle such challenges. We have seen a significant change in attitude from organisations who are shying away from bespoke UI and human driven tools, towards those which promote automation and DevOps processes. These tools are helping to drive DevOps transformation and automation change programmes.

But why not …

We have worked with a few different clients who have tried other tools in this space. The section below provides a high level summary of some of the main alternatives we see being used, along with our thoughts on the matter.

Note: The Terraform vs Others section of the Terraform website goes through a list of common tools in this space and describes why Terraform is different. The guys from TerraGrunt have also written a great article which goes into significantly more detail around the factors which prompted them to choose Terraform over many other leading contenders as their infrastructure as code tool of choice.

Configuration management tools like Ansible, Chef, Puppet?

In short, whilst configuration management tools can be used to perform similar tasks to Terraform, this was not part of their original idiomatic design and purpose. Their sweet spot is in installing and managing software on existing servers. They are good at this and we believe this is what they should be used for.

We see two distinct phases when it comes to deploying infrastructure. The first is provisioning: the macro level orchestration phase (creation, destroying and managing of core infrastructure: VMs, networks, key services etc). We see this role being firmly played by Terraform. The second phase is the micro level configuration phase, which installs and configures software on these servers and resources. This is where we see Terraform handing off to a specific configuration management tool in order to complete this process.

Infrastructure specific management tools like AWS CloudFormation, OpenStack HEAT?

Whilst these tools do operate at the macro level orchestration phase, they often result in a plethora of different tools being required to manage different parts of your infrastructure. This can become a nightmare from a maintenance perspective. Most organisations will operate in some form of hybrid environment and having one tool and process capable of managing the whole estate in a consistent manner is a major benefit.

Many of these tools also lack some of the more sophisticated lifecycle and management features of Terraform. For example being able to have a separate plan and apply phase within your lifecycle to allow you to see what the impact is of a change before actually making it.

Conclusion

If you are considering developing the capability to support the programmatic creation, destruction and management of your own internal infrastructure or service, the development of a custom Terraform provider could be a good fit for you. Whilst helping to save time, reduce risk and increase the reliability of managing your infrastructure, its popularity within the DevOps community also makes it a good choice in terms of being more easily embraced and adopted into a DevOps workflow.

If you are looking for help with writing a custom Terraform provider, we would obviously be more than happy to help, please do just reach out!

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.