A Supermarket of Your Own: Running a Private Supermarket

A little bit of history…

When Chef unveiled Supermarket in 2014 it was intended to be a public community cookbooks site where Chef Community Members could browse cookbooks from anywhere. We were honored that it was received so well!

However, we soon discovered a problem. Some companies create very company-specific cookbooks that they cannot share publicly (often for license or security reasons), yet they still want a way to share them internally.

Before Private Supermarket there were two main ways of sharing internal cookbooks. The first was by storing cookbooks on Github Enterprise, but the problem with this approach is it is hard to browse cookbooks without knowing exactly what to search for. The second way was to use Berkshelf with Github Enterprise. The problem with this approach is that resolving dependencies from various Github Enterprise repos is slow.

At the Chef Community Summit 2014 Chef Community members requested the ability to run their own copy of Supermarket on their own infrastructure. We hadn’t thought of this need…but were certainly willing to try it! We first built a stand alone Supermarket package using Chef Omnibus Packager.

Benefits of a Private Supermarket

A private supermarket provides an easily searchable cookbook repository (with friendly GUI and command line interfaces) that can run on a company’s internal network. When Berkshelf is connected to a Private Supermarket’s API, Berkshelf runs speed up. Customers have also told us it is easier to formalize the process of releasing cookbooks. If a new cookbook version is not yet on the Private Supermarket, it is considered under development or review and has not yet been released. If it is on Private Supermarket, it is considered officially released and ready to use.

Installing a Private Supermarket

Requirements

To install a private Supermarket, you will need:

• A Running Chef Server (this is needed for using oc-id – Chef’s Oauth Provider)
• An Admin User account on the Chef Server
• A Key for the User Account on Chef Server
• A fresh VM with at least 1 GB memory (we use an Ubuntu instance on AWS)

What is oc-id?

oc_id is Chef’s Authentication/Authorization service. It allows users to use their credentials for a Chef Server to access other applications.

For an intro to oc_id, please see oc-id on Chef Server: An Introduction.

First, a user visits an application and clicks “Sign In”

Then they are redirected to the Chef Server where they sign in.

And then they are redirected back to the application, now signed in!

Preparing Your Chef Server

First, let’s prepare the Chef Server to allow your Private Supermarket to use it for authentication/authorization. Go ahead and ssh in:

$ (your workstation) ssh user@your-chef-server

Then open up this file:

$ (your chef server) sudo vim /etc/opscode/chef-server.rb

Add this content, then save and close the file:

  oc_id['applications'] = {
    'supermarket' => {
       'redirect_uri' => 'https://supermarket-hostname/auth/chef_oauth2/callback'
     }
  }

Now reconfigure your Chef Server:

$ (your chef server) sudo chef-server-ctl reconfigure

Once that completes, take a look at this file:

$ (your chef server) sudo cat /etc/opscode/oc-id-applications/supermarket.json

You should see output that looks like this:

 {
   "name": "supermarket",
   "uid": "123...989",
   "secret": "1234...897",
   "redirect_uri": "https://supermarket-hostname/auth/chef_oauth2/callback"
 }

Take special note of the “uid” and “secret” values – you will need these later. (Note: You can always get them again by looking at this file on your Chef Server, they don’t go away after you view them the first time.)

Preparing Your Supermarket

The best cookbook to use when setting up a Supermarket is the supermarket-omnibus-cookbook.

It’s good practice to use a wrapper cookbook around the supermarket-omnibus-cookbook, we will need to supply certain attributes through the wrapper cookbook.

Let’s go through the layers of the wrapper cookbook, cookbook, and internal cookbook within the Omnibus package.

Wrapper Cookbook

First there is the wrapper cookbook where we define node[supermarket_omnibus] attributes:

supermarket-omnibus-cookbook

Then there is the supermarket-omnibus-cookbook, which is what our wrapper cookbook wraps around. This cookbook will:

1. Install the Supermarket Omnibus yum or deb packages
2. Writes the node[supermarket_omnibus] attributes to /etc/supermarket/supermarket.json

Supermarket Omnibus Package internal cookbook

Finally, we have the Supermarket Omnibus yum or deb package. This package has an internal chef cookbook which installs and configures the package using the attributes defined in /etc/supermarket/supermarket.json. When the package runs this internal cookbook it is very similar to running chef solo on a server.

Creating the wrapper cookbook

So let’s go ahead and create the wrapper cookbook.

On your workstation, generate a new cookbook.

$ (your workstation) chef generate cookbook my-supermarket-wrapper

Then change directories into that cookbook:

$ (your workstation) cd my-supermarket-wrapper

Now go ahead and open the metadata file of the cookbook:

$ (your workstation) vim metadata.rb

And add this line, then save and close the file. This defines our wrapper cookbook’s dependency on the supermarket-omnibus-cookbook:

  depends 'supermarket-omnibus-cookbook'

Now open up the default recipe within the cookbook:

$ (your workstation) vim recipes/default.rb

And add this content, which will execute the default recipe of the supermarket-omnibus-cookbook

  include_recipe 'supermarket-omnibus-cookbook'

Next we need to define the attributes for the Supermarket installation and how it connects to the Chef Server. One solution is to hard code attributes in the wrapper cookbook’s default recipe, but a better practice is to place the attributes in a data bag (or encrypted data bag or vault), then reference them in them recipe.

At a minimum, we must define the chef_server_url, chef_oauth2_app, chef_oauth2_secret attributes.

So let’s open up the default recipe again:

$ (your workstation) vim recipes/default.rb

And let’s pretend we have a data bag called ‘apps’, with an item in it called ‘supermarket’:

  # calling the data bag
  app = data_bag_item('apps', 'supermarket')

Then let’s add in attributes we want to use from the databag:

  # calling the data bag
  app = data_bag_item('apps', 'supermarket')
  node.set['supermarket_omnibus']['chef_server_url'] = app['chef_server_url']
  node.set['supermarket_omnibus']['chef_oauth2_app_id'] = app['app_id']
  node.set['supermarket_omnibus']['chef_oauth2_secret'] = app['secret']

Go ahead and save and close the file.

Now, let’s install the dependent cookbooks in supermarket-omnibus-cookbook.

$ (your workstation) berks install

Now we need to upload all dependent cookbooks to the Chef Server (NOTE: There is more than one way to do this, use whatever works best for you and your team’s workflow).

$ (your workstation) cd ~/.berkshelf/cookbooks
$ (your workstation) knife cookbook upload -a

Then upload the wrapper cookbook (again, there is more than one way of doing this, this is one way that works):

$ (your workstation) cd path/to/wrapper/cookbook/
$ (your workstation) knife cookbook upload -a

Now let’s bootstrap our Supermarket node with the Chef Server. If we had an ubuntu node in AWS, we would bootstrap it like this:

$ (your workstation) knife bootstrap ip_address -N supermarket-node -x ubuntu --sudo
 # -N flag defines the name of the node (in this case supermarket-node)
 # -x flag defines the username to use (the default username for ubuntu instances on AWS is ubuntu)
 # --sudo runs the bootstrap command as sudo on the node

Once bootstrapping is complete, we edit the new supermarket node

$ (your workstation) knife node edit supermarket-node

And add the wrapper’s default recipe to the supermarket-node’s run list then save and quit the file.

"run_list": [
    "recipe[my_supermarket_wrapper::default]"
  ]

Now we ssh into the Supermarket node

$ (your workstation) ssh ubuntu@your-supermarket-node-public-dns

And once we’re in the node, we run chef-client. This will install and configure Supermarket.

$ (your supermarket node) sudo chef-client

Using Private Supermarket

Connecting to Supermarket

When we’re ready to use our newly spun up Private Supermarket, we need to open up /etc/hosts on our local workstation (the workstation we’ll be uploading cookbooks from to Supermarket) and add an entry for our Supermarket instance:

# Supermarket ip address Supermarket hostname
00.00.000.000 supermarket-hostname

After saving and closing the file, we visit the Supermarket hostname in the browser (Supermarket by default uses a self-signed certificate, if we receive a warning we need to accept the SSL certificate) then click the “Create Account” link. If not already logged into the Chef Server, we will be prompted to do so. Then we need to authorize the supermarket app to use our Chef Server account and we’re in!

Interacting with Cookbooks on Supermarket

The best current and Chef supported way to interact with a private Supermarket is with the knife-supermarket plugin. (NOTE: Other tools can be used and we will go over those a little later).

First, we install knife-supermarket

# If using the Chef DK
 $ (your workstation) chef gem install knife-supermarket
# If not using the Chef DK
 $ (your workstation) gem install knife-supermarket

Now we open up our knife.rb file

$ (your workstation) vim .chef/knife.rb

And define the supermarket site for our Private Supermarket, then save and close the file.

  knife[:supermarket_site] = 'https://your-private-supermarket'

Knife Supermarket commands are the same as knife cookbook site commands, only with the ability to connect with an Private Supermarket rather than just the Public Supermarket. Please consult the docs for information on all commands that can be used with Knife Supermarket.

Let’s take the time to go over how to share a cookbook to a private Supermarket. Using knife supermarket, we would run this command:

$ (your-workstation) knife supermarket share 'my_cookbook'

When we first run this command, we might see an SSL error:

$ (your-workstation) knife supermarket share 'my_cookbook'
Making tarball my_cookbook.tgz
ERROR: Error uploading cookbook my_cookbook to the Opscode Cookbook Site: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed. Increase log verbosity (-VV) for more information.

This is because Chef 12 enforces SSL by default when sharing cookbooks (Chef 11 did NOT do this). Private Supermarkets, by default, use self-signed certificates. Fortunately we can fix this error through fetching and checking the private Supermarket’s ssl certificate.

$ (your-workstation) knife ssl fetch https://your-private-supermarket

Followed by:

$ (your-workstation) knife ssl check https://your-private-supermarket

Now let’s try sharing the cookbook again and we should see a success message:

$ (your-workstation) knife supermarket share 'my_cookbook'
Generating metadata for my_cookbook from (...)
Making tarball my_cookbook.tgz
Upload complete!

Managing Private Supermarket

supermarket-ctl

We can manage services and other settings of our instance with supermarket-ctl commands.

For example, running supermarket-ctl reconfigure will do an internal Chef run based on the cookbook within the omnibus package.

supermarket-ctl make-admin username will make a supermarket user and admin user

supermarket-ctl restart will stop services if they are running, then start them again.

Here’s what we would see when using supermarket-ctl restart on a supermarket server:

$ (your supermarket node) sudo supermarket-ctl restart
 ok: run: nginx: (pid ####) 1s
 ok: run: postgresql: (pid ####) 0s
 ok: run: rails: (pid ####) 1s
 ok: run: redis: (pid ####) 0s
 ok: run: sidekiq: (pid ####) 0s

Monitoring

By default every installation of Supermarket includes a Monitoring URL at https://your_private_supermarket/status.

Other Tools With Private Supermarket

Stove

Stove is an alternate utility for releasing and managing Chef Cookbooks.

Berkshelf

Berkshelf can include multiple Supermarkets for dependency resolution. Cookbook dependency resolution is performed from the top down – the first source defined in the Berksfile will be searched for the cookbook before the second source.

This Berksfile would first look for the cookbook on the private Supermarket and, if it did not find a cookbook with the specified name, would look on the public Supermarket.

source 'https://your_private_supermarket_url'
source 'https://supermarket.chef.io'

Further Customizing a Private Supermarket

Configure an external database

Supermarket installations can use an external database (we use Amazon RDS for the Public Supermarket). To configure this, we would need to configure the following attributes in the default recipe of the wrapper cookbook.

node.set['supermarket_omnibus']['postgresql']['enable'] = false
node.set['supermarket_omnibus']['database']['user']     = 'supermarket'
node.set['supermarket_omnibus']['database']['name']     = 'supermarket'
node.set['supermarket_omnibus']['database']['host']     = 'yourcompany...rds.amazon.com'
node.set['supermarket_omnibus']['database']['port']     = '5432'
node.set['supermarket_omnibus']['database']['pool']     = '25'
node.set['supermarket_omnibus']['database']['password'] = 'topsecretneverguessit'

Configure an external cache

Supermarket installations can also use an external cache store. Public Supermarket uses redis on Amazon Elasticache. To configure this, we would need to configure the following attributes in the default recipe of the wrapper cookbook.

node.set['supermarket_omnibus']['redis']['enable'] = false
node.set['supermarket_omnibus']['redis_url'] = 'redis://your-redis-instance:6379'

Configure cookbook storage

Cookbook artifacts (the tar.gz artifacts which are uploaded to Supermarket when sharing a cookbook) can be stored either on the local filesystem of the Supermarket node (this is the default) or in an Amazon S3 bucket.

If we were using an Amazon S3 bucket, we would configure the following attributes in the default recipe of the wrapper cookbook.

node.set['supermarket_omnibus']['s3_access_key_id'] = false
node.set['supermarket_omnibus']['s3_bucket'] = 'supermarket'
node.set['supermarket_omnibus']['s3_access_key_id'] = 'yoursecretaccesskey'

Additional Configuration Options

For additional configuration options for a private Supermarket, please consult this list of default attributes included with Omnibus Supermarket. And of these attributes can be configured.

Conclusion

Supermarket – both the public and private versions – was created for the Chef Community. If you have problems, suggestions, or feature requests please consider opening an issue in the Supermarket Github Repo.

Note: The content for this blog post was originally developed for a webinar in collaboration with Kurt Fitzpatrick. This post is welcome to be shared but please attribute it to the author, Nell Shamrell-Harrington.