During ChefConf 2021 Online Stephanie Laingen, DevOps Consultant/Software Engineer from TapHere!Technology, presented an informative session on using Chef and GitLab together as part of an Agile Developmet Workflow. This blog post is an overview of the session.
Agile development relies on a robust Continuous Integration/Continuous Delivery (CI/CD) strategy. It should have the capability to handle the entire development, testing, and production cycle while balancing the risks against deployment velocity. CI/CD pipelines allow deploying new code quickly and sustainably. There are many CI/CD automation tools (Jenkins, Azure DevOps, Docker, etc.) that you can use to build agile workflows.
Chef integrates seamlessly with these tools to deliver continuous deliver. Chef uses immutable and universal artifacts that can be validated and deployed to diverse environments which makes it easy to automate workflows. But what exactly do we mean by agile workflows? And how can we automate workflows using CI/CD pipelines?
A workflow defines the processes involved in developing and deploying code. It standardizes coding practices making it easier for different dev teams to collaborate. A workflow that is efficiently designed will improve code resilience which means that-
Most development teams use CI/CD automation tools to create and maintain individual pipelines in different workflows meant for different applications. Automating workflows enable agility and accelerate deployment velocity. It can greatly improve developer productivity by streamlining processes and routines. The developer can spend less time fixing broken code and focus on building functional code without compromising quality or security.
Building agile workflows is simple and quick using Chef’s ability to define and deploy “policy as code.” It allows developers to create policyfiles and policy groups using code that can be easily adapted and modified as needed. This also helps maintain automated and continuous enforcement of run lists with minimal human interaction.
As Stepahnie explains, "Chef is policy as code. And since code is malleable, it can adapt as often as our needs do. And if you are a developer, you know that needs often do change on the fly. So, it's incredibly helpful to be able to be agile in that way and make changes that are necessary.”
Pipelines run a set of commands, so you need a workstation/machine to build and run the pipeline. The CI/CD pipelines enable a machine to run commands automatically. To understand how Chef enables continuous delivery with pipelines, let us take the example of updating cookbooks on a live testing/dev node. We can create an automated workflow in three quick steps using Chef and a CI/CD automation tool of your choice, we use GitLab in this example.
Using GitLab pipelines with Chef’s standardized coding practices you can create workflows to deliver changes instantly and automatically to all your managed nodes. GitLab pipelines use YAML documents that describe the steps to be executed when certain actions occur within a repository, for example, merge request creation or approval. The different steps within the workflow can be simplified and automated using pipelines as shown in the image below.
Workflow updating cookbook code using GitLab pipelines (highlighted in red)
In this example, the cookbook repository will automatically run cookstyle and test kitchen, eliminating the need for peers to test everything manually. If the pipeline run is successful, the feature branch is merged into the development branch. The same process is followed to merge the development branch into the master branch.
The pipeline file in the cookbook repository will automatically upload the cookbook to the Chef server upon a successful merge. The cookbook repository pipeline will then trigger the policyfile pipeline to run Chef install, and Chef update commands on all the policyfiles in the development branch. This will also update the lock files to pull in the new version of the cookbook to be tested.
The policyfiles contain the run list for the cookbooks including cookbook version and location. You can use either a strict version pinning or a loose version pinning in this workflow. Loose version pinning allows you to define an approximate range of
version values, this means that the version does not have to be manually changed in the file before running chef update and chef install commands.
Loose Version pinning adds another level of automation into the workflow. “If you don't want certain policy files to be updating to include newer versions of cookbooks, you can just write those policy files using stricter version pinning so that you know that policy files that you do want to be updated can be written with a looser version pinning so you can just run those Chef update commands and only update the policy files that you want,” says Stephanie.
The pipeline will commit the changes to the development branch and use the Chef push command to push the new lock files to the dev policy group on the chef server. The dev node will automatically run the updated cookbook code at the next check-in because it is attached to the dev policy group.
The workflow described here is platform agnostic and can be easily adapted to any CI/CD automation tool. Chef and GitLab pipeline work together to create an agile, efficient and functional routine that allows developers to deploy code to production faster with minimal risk. The repeated testing and linting in the workflow ensure the code is working as it should, such a test-driven environment builds confidence in the code and the development process across all the teams involved.
Watch the entire session “Automatically Deliver Chef: An Agile Workflow” for a detailed walkthrough of the workflow. Or download a copy of the user guide here.