We are excited to announce a new integration between Google Cloud Platform (GCP) and Chef. For the ever increasing number of Chef customers using GCP, these GCP specific cookbooks are consistent, cohesive and comprehensive. They enable Chef practitioners to quickly, declare and request resources from GCP.
The work for these Chef cookbooks to automate GCP is largely driven by GCP. The GCP team has created cookbooks to provide a curated experience for Chef practitioners to define and provision GCP specific Compute Engine install / docs | source, Container Engine install / docs | source, Cloud Storage install / docs | source, Cloud SQL install / docs | source and Cloud DNS Services install / docs | source, along with a single authentication mechanism across the services mentioned above. However, If you want all of the cookbooks you can get a bundled cookbook here
Custom resources give you the ability to write a recipe declaring your database layer with GCP SQL, your application layer (including network resources such as load-balancers and firewall rules) with Google Compute Engine, then leverage GCP DNS to create new entries for the instance. With this type of recipe, you could put this in your development pipeline, run automated tests on a clean installation from your master branch every night, giving you a way to test exactly how your production instances would be in a fraction of the cost.
With this type of repeatability, you can also scale up and down your stack with only a small amount of extra code allowing you to give segregated instances for your Dev team, or even each Developer. In turn, this gives your performance/QA teams, something that closely resembles production. This enables your performance/QA teams to verify that the application can handle the load of traffic you expect in the future. The possibilities of repeatability and a simple declaration of the resources you need are enormous.
How do I use this
As with most new technologies the words are great, but how do I actually use this? Do I need to learn a new tool? Do I need to learn a new language? Luckily, no. If you are a versed practitioner of Chef, the resources that GCP and Chef have created should look very natural to you. In order to put these into practice, let’s take a simple use case of a web application. Undoubtedly you know that applications on the web move fast. New features, changes, and options need to get to market as quickly as possible let’s take a practical example here:
https://gist.github.com/jjasghar/9820ab1e2fcbb4a46d2f6f7aad25cf92
I’d like to highlight a few things here. If you clicked on the link above, it may initially seem daunting, but as you read through it, it really is extremely straightforward.
First thing first, if you look at line 4, you’ll notice I declare the project as ‘jj-test-kitchen.’ If you don’t know, in GCP every project is a way to separate groupings of resources. You can be part of multiple project, and in this case, we are demoing using my test-kitchen instance. The next thing I’d like to highlight is lines 6-12. This is the ability to authenticate to the GCP backends. This is a dependant cookbook for all of the cookbooks we have created, allowing for updates to go in tandem with any changes that are required from GCP. (This is a good thing, and I hope more cloud providers start adopting this process. Stay tuned for a follow up blog post about this possible new paradigm.)
As you continue down the recipe, you’ll see it’s pretty self explanatory, we create a disk, network, region, internal address, and machine type. Then on line 54 onwards we ask GCP for the machine wiring up all together. Yes, this only creates one machine, but that’s fine, but this is idempotent. So in other words, you could run this once, get the machine created, but the next time you’ll notice that it’ll ask, see that everything is created and not make any changes! You could easily expand this with some simple ruby and create more advanced infrastructures in no time. If you take a look at line 79, you’ll notice the metadata options, this is how we inject the bootstrap script at the end of provisioning. The value on line 82 can be any shell script, I’m using a private bucket on GCP called ‘jj-test-kitchen.’ The tags on line 86 open up ports for http and https, this can be any tag you have on your networking in GCP, this is just to illustrate the option. Finally the service account scopes allow for the internal communication to the GCP bucket above. If you change around anything with jj-test-kitchen to your own project, and add a bootstrap script, you’ll amazed on how easy it it to provision compute resources.
If you’re curious, my bootstrap script just does the following:
#!/bin/bash curl -L https://chef.io/chef/install.sh | sudo bash -s -- -P chef
You could easily add a way to inject PEMs, and other secrets and have it run an initial chef-client run. (I may have just given away a portion of the demo for the webinar here. ;) )
GCP Cookbooks on Chef Supermarket
We have published these cookbooks to the Official Chef Supermarket within the GCP section on October 4th, 2017 and they are ready for prime time. I say this as a practitioner of Chef for multiple years that this new way of declaring resources a pattern is something that we’ll hopefully start seeing more. The ability to have a sanctioned resources from the Cloud Provider, will give first class integrations instead of the reverse engineering required by people downstream.
Getting Started: Automating GCP with Chef
In this recorded webinar, see a demo of a practical use case that shows you how to adopt this strategy at your organization.