After taking Chef Essentials or spending time on learn.chef.io, you might still feel there’s a gulf between what you’ve learned and the details of automating your real-world infrastructure. In this article, I’ll outline some common starting points for writing your own Chef recipes, and point you to frequently-used tools and reference resources.
What should you automate first? While the list of possibilities is endless, it’s good to start by replacing something you already do with an automated alternative. In my experience, there are three great places for introducing automation into your system: automating your runbook, automating away an outage, and automating a simple application installation.
Without automation, most companies provision servers using a formalized list of steps and scripts. These scripts and runbooks are a great place to start building our first Chef cookbooks, because we can quickly turn them into code and improve the speed and consistency of our server provisioning process. What’s more, you can prevent your server configuration from drifting off of the baselines you’ve set in your code by running your Chef cookbooks regularly during the server’s lifecycle.
If you already use scripts as part of your provisioning process, you can quickly turn them into a chef recipe using resources like bash and powershell_script. By adding these scripts you’ve centralized your provisioning code on your Chef server and enabled users to call a build script while bootstrapping a new server. You can do even more, since Chef’s DSL utilizes desired system state. You can replace parts of your script with Chef resources, which allows Chef server to detect the current system state and only make changes when needed, for instance during initial provisioning or to revert configuration drift. You can also utilize guards to test and control when scripts run.
For automating a manual runbook, you can break apart the steps in your runbook and replace them with Chef Resources to perform the task in that step. Two common resources used in runbooks are the Package resource, which is used for installing a software package, and the Service resource, which ensures critical services are running and configured to auto-start on reboot.
You might also want to check out the Chef Supermarket for cookbooks that help you automate common tooling that exists across your infrastructure, such as the users cookbook, ntp, monitoring tools such as nagios, and antivirus tools such as McAfee. We also recommend using the chef-client cookbook to automate the chef-client, turning it into a scheduled task that will apply your Chef runlist regularly without manual intervention.
Most engineers can think of an outage they’ve had to respond to that could have been avoided or minimized with system automation. These outages are often great targets for your first automation code, as increasing system reliability and infrastructure security immediately produces value for your infrastructure.
I’ve seen four common causes of outages that are great candidates for automation: delicate systems prone to human error such as DNS or storage; monitoring systems that are not reporting correctly; configuration drift due to manual changes; and security vulnerabilities.
Chef’s increasing stability around error-prone systems can start by managing their components with cookbooks publicly available on the Chef Supermarket, and managing their configuration files with Chef using resources such as templates. Monitoring tooling on your endpoints can be managed with a Chef cookbook that ensures the monitoring service is up and connecting properly every time each of your endpoints connects to the Chef server. Automating around configuration drift can include things like using a cookbook that maintains a standard template for the configuration of host files, storage configurations, or application configuration files, or that ensures that important services like ntp, antivirus, and monitoring are set to be running and health-checked every time the chef-client checks in with the server.
Automating the detection and remediation of security problems is another easy point to start delivering value through automation. We’ve seen great customer success using Chef and InSpec to detect vulnerability to Windows ransomware. Getting familiar with Chef and the InSpec testing language will allow you to quickly write tests and remediation for vulnerabilities the next time you have to deal with a zero day security issue that affects your infrastructure.
My final recommendation for beginning your automation journey is automating a common, straightforward application that is widely used in your environment. For many Chef users, this is a web application that does not require large amounts of database support as part of its installation. Writing a cookbook to install such an application involves installing the framework required by the application. The Chef Supermarket can be searched for well-supported cookbooks for many web application hosting tools, such as IIS, apache, nginx, and tomcat. You’ll then use the remote_file resource to copy the artifact onto the local filesystem and resources such as execute, powershell_script, or the tar cookbook to unpack it into the location it will be served from.
A similar scenario to this is patching an application that isn’t part of your operating system’s built in patch management tool. You can take advantage of the remote_file and package or execute resources to pull down a patch hosted on a central server, and apply it with any needed arguments to the endpoint. Best practices recommend using guards such as not_if or only_if when using the execute resource, to make sure you aren’t running an installation that is already in place on the server.
Now that you have some ideas for where to start, I want to go over a few places I’ve seen new users run into friction when writing their first code. This isn’t a declaration that it’s not possible; there are definitely Chef users who have jumped in the deep end with great success, but starting with these sorts of projects will likely be challenging for a beginner.
Complex applications – databases, Kafka, and Sharepoint are common examples. If it takes a great many delicate steps to install manually, it will require a great deal of care and precision to automate. Additionally, applications which require a complex set of multi-node interactions, such as having to quiesce a database during a certain stage of the install, can be very tricky for first-time automators.
Application installations that require manual steps – many organizations rely on Windows applications that don’t have good command line tooling, and can be difficult or impossible to automate. Some large enterprise applications can fit in this space as well.
Chef has some great resources available to folks in the community looking to get started. Learn.chef.io is a great place to get step-by-step tutorials to get you started with Chef’s entire ecosystem of products. Our documentation page is an excellent, searchable resource for finding specific information on writing Chef code or maintaining your Chef infrastructure. Chef’s YouTube channel has a great collection of resources for Chef users of all levels, and our Joy of Automating series of tutorial videos are specifically aimed at users trying to bridge the gap between learning the Chef basics and writing real-world code.