Why should you upgrade your Terraform provider to the latest version of the API

At the end of 2022, after more than a year of development, we saw the release of version 1.0 of HashiCorp’s Terraform Plugin Framework – the next SDK for building Terraform Providers after SDKv2. While it was confirmed that the two would co-exist, it was clear that SDKv2 would be taking the back seat with most feature developments being put to a halt, in order to focus on the Terraform Plugin Framework – which is set to become the foundation for future Terraform plugin development features and tooling.

So if SDKv2 isn’t coming to the end of life, why should you spend the time and resources to migrate to the Framework? 

In this blog, we will explore the risks of not migrating, the benefits of doing so and the difficulties and challenges that follow.

The Risks of Not Migrating to the Terraform Plugin Framework

With HashiCorp themselves recommending companies migrate to the framework and stating that most feature developments have stopped on SDKv2, it is inevitable that those who choose not to will eventually face several business critical risks. Such risks may include:

Security Vulnerabilities: You’re safe for now. HashiCorp has committed to keeping SDKv2 up-to-date with security patches while Terraform 1 is the latest release. But Terraform 2 will arrive someday and support will end. One morning you’ll wake up and find your infrastructure exposed to some new vulnerability and there won’t be an SDKv2 version to help you. The fix would be a time-consuming migration from SDKv2 to the Framework. It could be best to get this out of the way before it’s urgent. 

Limited Functionality: As well as missing out on the new features offered by the Framework now, any further improvements would not be available to SDKv2 users. This will ultimately limit your ability to take advantage of improved tools, performance optimizations, and additional capabilities that can enhance your infrastructure management.

Limited Support: You’ll always have documentation for SDKv2, but as more and more people migrate to the Framework, your usual troubleshooting resources (like StackOverflow) will offer less help for novel problems.

Increased Technical Debt: By delaying upgrades, the company accumulates technical debt, making future upgrades more difficult and time-consuming. As the gap between the current version and the latest version widens, the upgrade process becomes more complex and may require addressing multiple breaking changes and compatibility issues all at once.

It is important for companies to regularly evaluate and prioritise upgrades to their Terraform provider to mitigate these risks and ensure they are operating on a secure, reliable, and up-to-date infrastructure provisioning platform.

The Benefits of Migrating to the Terraform Plugin Framework

Having been designed from the feedback of thousands of providers, the Framework already offers significantly better functionality than that available in SDKv2. Key benefits to migrating include:

Standardised Development: The Plugin Framework also offers a standardised approach and set of tools for building plugins, ensuring consistency and compatibility across different providers and provisioners. However, the approach is far more friendly to Go developers, particularly those who might be new to writing Terraform Providers or even to the programming language. A complete list of the advantages at a technical level is available here. Knowing whether a value is truly unknown, or just matches its zero-type is a very welcome feature. 

Community Collaboration: This flipside of the Limited Support disadvantage: customer feedback has directly influenced the Terraform Plugin Framework, meaning it was built with real-world problems in mind. As well as the Framework itself, tooling is being designed to specifically target Providers built with the Framework, such as the plugin-docs tool.

Upgradability: By utilising the Plugin Framework, developers ensure that their plugins are compatible with the evolving Terraform ecosystem. When Terraform 2.0 is released, developers can update their plugins to align with the latest features, improvements, and changes introduced by Terraform, allowing users to leverage the full potential of the latest Terraform releases.

THE DIFFICULTIES OF MIGRATING TO THE TERRAFORM PLUGIN FRAMEWORK

While it is clear that it should be a business priority to migrate, it is not necessarily a trivial task. If your Provider is particularly complex, HashiCorp has given you the opportunity to “mux” together Framework-based Resources and SDKv2-based Resources, so you don’t have to do everything in one giant release. The following are a few challenges you might run into: 

Complexity: If you do opt for the ‘muxing’ route, your already-complex Provider is about to get even more complicated. Some utilities (such as DefaultFunc) need to be removed from all your code – even the Resources you aren’t yet upgrading. Your test code will need to be looked at too. Terraform gently recommends avoiding this approach for simplicity, my personal opinion is that you should only approach muxing if you really, really can’t do everything in one go.

Compatibility with Existing Configurations: Upgrading the provider API may require adjustments to your existing Terraform configurations to align with the new version. This can involve updating resource or data block configurations, adjusting input parameters, or revising module dependencies. Ensuring that your existing infrastructure configurations work seamlessly with the upgraded provider can be a complex task, especially if you have a large and intricate infrastructure setup. HashiCorp recommends an extra layer of testing here to protect against changes in behaviour.

Limited Documentation or Examples: There are a fair few examples of Providers using the Framework and HashiCorp’s documentation about Providers in situ is comprehensive. However, at the time of writing, there aren’t a huge number of resources to help you perform a migration. HashiCorp has a lot of useful documentation, but this is (necessarily) only useful for the general case. We couldn’t find any blogs or unofficial guides to help us through our own patches of weirdness. 

Risk of Regression: Upgrading to a new provider API version introduces the possibility of encountering regressions or new bugs that were not present in the previous version. It is important to thoroughly test your infrastructure configurations after the upgrade to ensure that everything functions as expected.

FAKE NEWS

Out of interest, we checked what ChatGPT had to say about migrating your Terraform provider from SDKv2 to Framework. While there were some valid points, a few were inaccurate – or just missed the point. Be careful if you see (anything like) the following elsewhere:

Risks of not migrating:

“Compatibility Issues: As Terraform evolves, new versions may introduce changes to the core functionality or configuration language. Not upgrading the provider may result in compatibility issues with newer versions of Terraform. This can lead to conflicts, errors, or incompatibilities between the infrastructure code and the Terraform runtime, thus making it challenging to manage and maintain the infrastructure effectively.”

HashiCorp has confirmed that while Terraform 1.0 is supported, SDKv2 will be supported too.

“Decreased Stability and Reliability: Outdated provider versions may contain bugs or stability issues that have been addressed in subsequent releases. By not upgrading, the company continues to operate with potentially unreliable or less stable infrastructure provisioning and management processes, increasing the likelihood of encountering unexpected behaviour or failures.”

This one isn’t necessarily untrue, but SDKv2 has been tried and tested for a long time now. It’s unlikely to have any particularly nasty bugs.

Benefits of migrating:

“Ecosystem Integration: The Plugin Framework enables seamless integration with the Terraform ecosystem. Plugins developed using the framework can be used alongside official and community-supported providers, allowing users to combine resources and modules from various sources within their Terraform configurations.”

Providers are separate from each other and so a version change in one doesn’t affect others. With some work, you might even be able to run Terraform with multiple versions of the same provider.

“Extensibility and Customization: The Plugin Framework empowers developers to extend Terraform’s functionality by creating custom providers and provisioners tailored to their specific infrastructure needs. It allows users to define and manage resources not supported by the official providers, enabling a broader range of infrastructure management scenarios.

Improved Automation: With the Plugin Framework, developers can automate complex workflows and interactions with external systems. This includes integrating with various APIs, cloud platforms, or configuration management tools, enabling seamless automation and orchestration of infrastructure provisioning and management processes.”

These two entries are absolutely true of both SDKv2 and the Framework. They are not benefits of migrating.

Difficulties in migrating:

“Breaking Changes: New API versions may introduce breaking changes that require modifications to your Terraform configurations or infrastructure code. These changes could involve syntax updates, deprecated attributes, or changes in behaviour that may impact how your infrastructure is provisioned. Ensuring compatibility and addressing these breaking changes can be time-consuming and may require code updates and testing.”

Not necessarily untrue, but you have a nice suite of tests to protect against this sort of thing, right?

“Third-Party Dependencies: Upgrading the provider API might require you to update and align other third-party dependencies, such as plugins or modules, to their compatible versions. Managing and coordinating these dependencies across your infrastructure stack can add complexity to the upgrade process.”

Once again, Providers are separate from each other and so a version change in one doesn’t affect others.

CLOSING THOUGHTS

As a consultancy that has helped several companies develop and upgrade their providers, we would highly recommend investing in migrating to the Terraform Plugin Framework and facing the difficulties of doing so now rather than delaying and dealing with much more critical risks to your businesses later. If you would like to book a free discovery session to get advice, then do reach out to andy@opencredo.com

This blog is written exclusively by the OpenCredo team. We do not accept external contributions.