Today I’d like to share a cookbook that we’ve developed, chef-splunk. This cookbook replaces an old internal Splunk cookbook that has served us well for quite some time, but that one was becoming more difficult to modify and extend. There is a community splunk cookbook already. However after review, it appears to be a fork and update of our old one, and follows several of the patterns in our old one that we want to correct. We do plan to work with the maintainers of that cookbook to consolidate the two.
The chef-splunk cookbook’s README.md is comprehensive and covers all the things about how it works and how to use it. We wanted to write this post to illustrate some of the new(er) patterns from the community that we followed in development. There will be a longer-form blog post to explain some of these and more in greater detail. This post is to highlight some of these patterns for those who dig into the cookbook.
Definitions Aren’t Bad
The installation for Splunk server and client (forwarder) is largely the same, save the package to download, and the package / file name. We had two recipes that handled this all very similarly in the previous cookbook, and have refactored that logic into a reusable definition. One benefit is that we greatly reduced the recipe complexity for handling package installation on multiple platforms, which leads to only one place to update for adding new platforms.
The definition code for splunk_installer is in the GitHub repository. Examples of use are theinstall_forwarder and install_server recipes. This is reused in the upgrade recipe, too.
Composable Recipes
The new cookbook makes use of composable recipes. That is, there are recipes that contain only one or two resources that were repeated throughout the server and client recipes before. This allows consumers of the cookbook to choose which recipes they need for their use case, or they can use the default and take on all our opinions.
The default, client, and server recipes use the other recipes directly. In the default use case of the cookbook (the default recipe), an attribute controls whether the client or server recipe is included. In turn, theclient and server recipes include the user, service, and setup_auth recipes in the order required to get the various resources in Splunk configured.
These recipes are our opinions about how to do this, and follow along with the Splunk deployment instructions. However, not all sites are created equal, and some users may have different needs. The important thing here is to use the primitives provided (recipes, the splunk_installer definition mentioned earlier), and compose a wrapper cookbook to suit local needs.
The recipes can all be viewed in the GitHub repository.
Search via Attribute
Chef search is a powerful feature, and when it comes to using it in recipes, a common technique is to search for nodes with a particular role applied in the run list or expanded run list. Unfortunately, this requires knowledge of the run list, and using the same role name. Some folks end up making the role name or the entire search query itself into an attribute. This works alright too, but then it means the consumer has to modify the attribute. This isn’t so bad, but considering that a lot of recipes are written as large monolithic lists of resources, it means extracting that and adapting it to a custom wrapper can be painful.
We felt that a better approach to this is to use an attribute that indicates the intent. This is actually used in some of Chef’s older cookbooks, such as rsyslog and ntp. The attribute in question here isnode['splunk']['is_server'], and the search looks like this:
search(:node, "splunk_is_server:true")
Or with knife:
knife search node splunk_is_server:true
While not grammatically pure (in English), it does clarify intent. This is used in the client recipe, with a little more syntax to get Splunk servers in the node’s environment and sort the results. However, since that recipe is composed of others it could be rewritten in a wrapper cookbook.
“Heavyweight” LWRPs
A long-time feature of Chef is the “lightweight” resource and provider DSL. This feature makes it easier to write Chef resources for recipes without as much Ruby ceremony. However, the DSL does have limitations. First, the resource name is derived from the name of the cookbook, combining the cookbook name and the resource filename from the resources directory. That is, the aws cookbook has a resources/elastic_ip.rb file for managing Elastic IPs in AWS EC2. This is aws_elastic_ip in recipes. For various reasons, one may wish to use a different name. In our cookbook, we wanted a resource for managing “Splunk Apps” namedsplunk_app, but we used the name chef-splunk (namespaced as a splunk cookbook exists), which means the resource would be chef_splunk_app.
Second, another limitation is that “lightweight” resources and providers cannot subclass other resources and providers. We didn’t have this particular issue, but it is addressed with the technique we used for splunk_app.
Enter “heavyweight” “lightweight” resources! This technique allows using the LWRP DSL within a regular Ruby class. It simply entails creating the resource and provider files in the libraries directory, e.g.,libraries/splunk_app_resource.rb. The actual code in a resource is:
require 'chef/resource/lwrp_base' class Chef::Resource::SplunkApp < Chef::Resource::LWRPBase self.resource_name = 'splunk_app' # resource code! end end
The code in libraries/splunk_app_provider.rb is:
require 'chef/provider/lwrp_base' class Chef::Provider::SplunkApp < Chef::Provider::LWRPBase # provider code! end
Then in a recipe, one can do this:
splunk_app 'bistro' do splunk_auth 'admin:notarealpassword' cookbook_file 'bistro-1.0.2.spl' checksum 'SHA256-checksum-of-bistro' action [:install, :enable] end
(Bistro is a Splunk App for Chef)
Testing
The cookbook itself is tested using commonly used tools by the Chef
community.
- ChefSpec (unit/spec)
- Test Kitchen (integration)
- Foodcritic and RuboCop (linting)
We don’t currently have post-convergence testing, as we encountered cross-platform shell issues in Test Kitchen’s “BATS” busser, and haven’t had a chance to revisit that.
Available Immediately
Our new chef-splunk cookbook is available for use now, and like all our open source projects it is released under an Apache 2.0 software license. We hope it is useful to those running Splunk in their infrastructure, and we plan to extend it with more primitives. Also, stay tuned as we’ll work with the maintainers of splunk to consolidate effort and make future announcements here.
- http://community.opscode.com/cookbooks/chef-splunk
- https://github.com/opscode-cookbooks/chef-splunk