It’s time to talk about Chef Infra upgrades! Last time around, we covered why it’s important to run on the latest client, and reviewed some of the improvements that have been made in the last few years of releases. Today we’re going to dive into some practical advice on how to go about undergoing a Chef Infra Client upgrade. Specifically, we’ll be covering how you can evaluate whether your cookbooks will be compatible with the latest versions, and how to use the tools provided in Chef Workstation to quickly iterate on and test those cookbooks against a target release.
To be very clear, this write-up is specific for upgrading your cookbooks which is often done in conjunction with a major client upgrade. We’ll touch on the client upgrade process briefly here, and our next blog will cover the upgrade process itself in greater detail.
Great question! In the past you needed to download multiple versions of ChefDK that mapped to the Chef Infra Client version you were going from, to the version you were going to. Then you’d run foodcritic against each cookbook and start traipsing through the output.
With the introduction of Chef Cookstyle, that’s not a thing anymore! We can simply set a target version in our cookbook’s .rubocop.yml (in this example we’ll be going from 12 to 13) and get rolling! It’s important to note when you specify a version constraint like we’ve done, that only the warnings specific to this version will display. When you change the target version or release the specification, you’ll have more to fix.Please check out the Chef Cookstyle documentation for a complete review!
Ok, now what?
For this example and to get you familiar with the process, please clone this repository (github.com/ChefRycar/bad_dates) and have the latest version of Chef Workstation for your OS platform installed (downloads.chef.io/chef-workstation/).
The ones that cookstyle can autocorrect of course! What’s that? Cookstyle has an autocorrect feature in it? Yes it does. We’ll get to that shortly, but first we want to see which deprecations exist in our cookbook and then we’ll perform a frequency analysis.
To check offenses that exist when upgrading from Chef Infra Client 12 to 13.latest:
cd ~/path/to/bad_dates> cookstyle .
You’ll now see output that looks like this:/tmp/bad_dates# cookstyle .Inspecting 9 files.WWWWW...Offenses:metadata.rb:8:1: R: ChefSharing/InsecureCookbookURL: Insecure http Github or Gitlab URLs for metadata source_url/issues_url fieldsissues_url 'http://github.com/ChefRycar/bad_dates/issues'^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^*snip*recipes/windows.rb:13:10: W: ChefDeprecations/ChocolateyPackageUninstallAction: Use the :remove action in the chocolatey_package resource instead of :uninstall which was removed in Chef Infra Client 14+action :uninstall^^^^^^^^^^9 files inspected, 18 offenses detected
Now let’s do a quick frequency analysis:/tmp/bad_dates# cookstyle . --format offenses 9/9 files |============================================== 100 ==============================================>| Time: 00:00:003 ChefDeprecations/ChefRewind3 ChefDeprecations/DeprecatedYumRepositoryProperties2 ChefSharing/InsecureCookbookURL1 ChefCorrectness/NodeNormal1 ChefDeprecations/ChefWindowsPlatformHelper1 ChefDeprecations/ChocolateyPackageUninstallAction1 ChefDeprecations/CookbookDependsOnCompatResource1 ChefDeprecations/CookbookDependsOnPartialSearch1 ChefDeprecations/CookbookDependsOnPoise1 ChefDeprecations/EpicFail1 ChefDeprecations/NodeSet1 ChefDeprecations/WindowsTaskChangeAction1 Style/StringLiterals--18 Total
Now that’s useful information! At this point if we really want to dive in, we can compare the output from the analysis with all of the rules that currently exist in Cookstyle (github.com/chef/cookstyle/blob/master/docs/cops.md). For the sake of brevity, we won’t do that here. Instead we’ll look at the autocorrect logic.
To use Cookstyle to autocorrect your cookbook:
/tmp/bad_dates# cookstyle -a .Inspecting 9 files.WWWWW...Offenses:metadata.rb:8:1: R: [Corrected] ChefSharing/InsecureCookbookURL: Insecure http Github or Gitlab URLs for metadata source_url/issues_url fieldsissues_url 'http://github.com/ChefRycar/bad_dates/issues'^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^metadata.rb:9:1: R: [Corrected] ChefSharing/InsecureCookbookURL: Insecure http Github or Gitlab URLs for metadata source_url/issues_url fieldssource_url 'http://github.com/ChefRycar/bad_dates'^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^*snip*recipes/windows.rb:9:11: W: [Corrected] ChefDeprecations/WindowsTaskChangeAction: The :change action in the windows_task resource was removed when windows_task was added to Chef Infra Client 13+. The default action of :create should can now be used to create an update tasks. action [:change, :create] ^^^^^^^recipes/windows.rb:13:10: W: [Corrected] ChefDeprecations/ChocolateyPackageUninstallAction: Use the :remove action in the chocolatey_package resource instead of :uninstall which was removed in Chef Infra Client 14+ action :uninstall ^^^^^^^^^^9 files inspected, 18 offenses detected, 15 offenses corrected
From 18 offenses down to 3 in only a few seconds. This will clearly scale.
Incredibly important note here: Once you’ve used Cookstyle’s autocorrect feature, you should absolutely, without exception, run your cookbook through your full suite of local and integration testing using tools like Test Kitchen and its integration in your CI/CD pipeline.
This is by far the most common and important question our customers ask us with respect to this process. It is Chef’s opinion that when working through the cookbook version and client upgrades, you perform them one major version at a time. In this example, we’re moving from 12.x compatibility to 13.latest compatibility. By doing this you’re introducing code changes required in 13 that are supported in 12, thus giving you the confidence and control to upgrade cookbook by cookbook instead of all at once. Once your codebase has full compatibility with version 13, you can seamlessly upgrade the Chef Infra Client version running in your estate. This will be covered in our next blog!
Once you’re completely stable on the version of Infra Client that you just moved to, you repeat the process from 13 => 14, 14 => 15, and finally 15 => 16. A quick and easy way to do this is by modifying the TargetChefVersion attribute in the .rubocop.yml in your cookbook to the version of the client you’re intending to upgrade to.
“Can I upgrade from Chef Infra Client 12 to 16?” Yes, you can. But because you’ll no longer have backward compatibility with your codebase, you’ll need to introduce the new code and new client version in an extremely controlled fashion. This can be done, but isn’t advised unless thoughtfully planned out and vetted by one of Chef’s Customer Architects.
How should I release these changes in a controlled fashion? Let’s continue… By now you should still have 2 offenses present in the bad_dates example cookbook we provided.
To take a look at how to correct what’s left, run the following command without the offenses formatter and make the necessary changes:
/tmp/bad_dates# cookstyle .Inspecting 9 files.WR......Offenses:metadata.rb:12:1: W: ChefDeprecations/CookbookDependsOnPartialSearch: Don't depend on the deprecated partial_search cookbook made obsolete by Chef 13depends 'partial_search'^^^^^^^^^^^^^^^^^^^^^^^^recipes/default.rb:9:1: R: ChefCorrectness/NodeNormal: Do not use node.normal. Replace with default/override/force_default/force_override attribute levels.node.normal['antipattern'] = true^^^^^^^^^^^9 files inspected, 2 offenses detected
You’ll now see we have a couple offenses that weren’t autocorrected by cookstyle. However, you’ll always be given a path forward or an explanation for what’s presented. In this case, we see partial_search is remediated by moving to Chef Infra Client 13, which is exactly what we’re doing! We also see a warning against using node.normal and should defer to our attribute precedence documentation to choose a way forward.
Once the offenses are corrected and after a series of successful integration testing, you have a cookbook that is completely compatible with Chef Infra Client 13. Above all, it’s backwards compatible with your existing Chef Infra Client 12 nodes. You can now begin releasing this cookbook into your pre-production environments with the goal of promotion to your production environment. After your cookbook code is sorted, you may now upgrade the Chef Infra Client itself to the target version. Rinse and repeat until you’re at Chef’s latest kit!
“This sounds like an insurmountable challenge.” Like all things in life, you get better with practice. The goal should be to never let yourself get more than 1 major version behind what is currently supported by Chef. This starts with shoring up how you perform local development.
Once you’re on Chef Client 15/16 and the latest Chef Workstation, you can use Test Kitchen attributes (https://docs.chef.io/workstation/config_yml_kitchen/) to be declarative about the versions you want to write code for. Adding the following to your kitchen.yml will ensure that the cookbook is compiled on the target version, as well as any deprecations will present as errors, forcing the developer to keep their code compatible.
In your kitchen.yml:
# Set target version (latest by default) https://docs.chef.io/workstation/config_yml_kitchen/#new-provisioner-settingsproduct_version: latest # Set deprecations as errors (https://docs.chef.io/workstation/config_yml_kitchen/#provisioner-settings)provisioner: deprecations_as_errors: true
You can also add Cookstyle autocorrection to run automatically in local development, or add it into your CI/CD pipeline’s linting/correction phase.
Using these simple methods will enable you to correct deprecations and offenses as part of the natural lifecycle of your codebase, rather than allowing it to become a monumental effort.
Next up, we’ll look at the other half of upgrades: upgrading Chef Infra Client itself in your environments. Until then, here are some resources to dig deeper into the upgrade process.