On February 24th I presented a live webinar on Test Driven Cookbook Development. Watch the recording below to hear me explain what unit and integration tests are and how they apply to the domain of cookbook development. You’ll see me demonstrate how to build cookbooks by using a test-driven approach. At the end of the presentation, you’ll know when to apply unit and integration tests, how to use the Chef DK tools effectively, and where you can go to improve your skills even more. At the end of this post, I’ve included some Q&A from the presentation.
franklin@chef.io or training@chef.io
The Chef DK contains almost all of the tools to complete the work that I went through with all of you. Test Kitchen communicates with a virtualization tool vagrant that asks Virtual Box to manipulate test instances. Virtual Box requires the extension pack installed to provide support for guest extensions.
The important part about the tests are that they setup expectations on what the cookbook accomplishes. Those goals can be different. If we were to change the cookbook we created in this demonstration not to start the service of the web server then we would change the expectations to match the state we want. That may be that the package is installed, the file structure happens to be in place, and that the service is there, available, but not running.
The developer builds a cookbook, that employs this one, would probably then take care of starting the service and setting expectations for their ‘customers’. This may be the expectations we defined here in the demonstration.
My advice: think about the intention of the cookbook and how it serves the customer. That customer could be someone in the organization.
The current Chef DK cookbook generator tools (0.11.2) defaults to use ServerSpec. While I really enjoy the performance and expanded capabilities of InSpec it requires you to make a number of changes to a generated cookbook. When InSpec becomes the default you will be pleasantly surprised to see that they are very similar to each other and your time spent with ServerSpec will only make you a better InSpec developer.
Every example I showed was on CentOS linux. The current Test Driven Cookbook Development content focuses currently on that platform. However, a lot of testing tools will work for the Windows platform. InSpec has some really great feature support upcoming that goes above-and-beyond ServerSpec. But ServerSpec still is able to do all that we did in this webinar (i.e. run a command) and ChefSpec is platform independent.
Check out the ServerSpec documentation of all the different Resource Types it supports: http://serverspec.org/resource_types.html.
It is worth it! Writing the tests and running them locally is only part of the value. You want the expectations you defined to be asserted by a CI sever like Delivery or Jenkins. This will give you an impartial source to execute the tests to catch any issues not discovered on your personal machine or a teammates. This will help reduce the “Works on my machine!” problem and give an entire team insight into when things stopped working.
Either tool and strategy would work and both should be employed as they compliment each other. Where I start depends on how well I know the problem I am trying to solve.
When I am not familiar with the implementation or less familiar with the domain of the application I am deploying I prefer the route of coming from the outside-in with my testing. That means integration testing. It’s easier for me because I can more easily define what success means and defining an integration test helps me clearly state that success.
When I am more familiar with an implementation or application domain I will take the route of coming from inside-out and build up my confidence through unit testing. There it’s easier for me to validate my work as I go – but that’s because I know the work I want to do.
I answered this question incorrectly during the webinar. Cucumber, http://cukes.info/, is a framework that allows you to express your expectations in a natural language. Cucumber will read in the language, compare them to a body of regular expressions and then map the sentence you wrote to code to invoke. When you write in Cucumber you have the power of RSpec. You could easily add ChefSpec or ServerSpec to your Cucumber step definitions.
What I should have said during the webinar is that I do not know of any active projects that have built a library of step definitions that you could employ. That project may be other there. I would encourage you to find it and if it does not exist I encourage you to make it.
The Pragmatic Programmer released an RSpec book many years ago that is now six years old. It is still a valid tome to discuss the value of RSpec and its uses but it has not been updated with the latest syntax and features currently supported by RSpec.
The most accurate documentation for RSpec can be found at their documentation project site: https://relishapp.com/rspec. It is hard to navigate so I encourage you to set aside some time and browse around instead of immediately seeking a particular answer.
At ChefConf I have proposed a talk to get into some of the details about RSpec. Exploring the advanced areas of the language like helpers, shared contexts, shared examples, and a lot of other nuances with the language.