Have you ever wondered how you would deal with secrets – i.e. API keys, database passwords, and more – within a configuration management framework? Although data bags are incredibly useful for storing global variables to use in your Chef recipes, they do not always provide adequate security for sensitive information

One solution is to use encrypted databag items. In this case you would encrypt a data bag item containing sensitive information with a shared key. Only those people or pieces of infrastructure with the shared key would be able to decrypt the data bag.

The problem with this approach is the use of the shared key –
you would be responsible for distributing this key to the appropriate users and pieces of infrastructure. This invites security issues and is definitely not a best practice.

To avoid this security issues of a shared key, you can use Chef Vault.

What is Chef Vault?

Chef Vault is an alternative tool for managing secrets with Chef. It was developed as an open source project originally by Nordstrom, but transferred to Chef in 2015. Unlike encrypted data bags, it does not require the use of one shared key by all users and pieces of infrastructure.

How Does Chef Vault Work?

The THEORY.md file in the Chef Vault Github repo has detailed information on the how and why Chef Vault works the way it does. Let’s go over some of this now.

Let’s say you are managing a set of nodes through Chef Server

Every node managed through a Chef Server has an associated client object on the Chef Server. This is how Chef Server is aware of and managing the node.

When a node is managed through Chef Server it uses an RSA keypair. The node retains the private half of the RSA keypair (usually it’s in /etc/chef/client.pem). The Chef Server holds the public half of the keypair.

Additionally, every user account on the Chef Server also has an RSA key pair. This is how you authenticate/authorize your workstation to make modifications to the Chef Server.

You workstation will retain the private half of the key (often this is in a .chef directory) and the Chef Server will hold the public half of the keypair.

When you create an encrypted data bag with Chef Vault (hereafter referred to as a vault), it will encrypt the data bag item (hereafter referred to as the vault item) with a randomly generated secret string known as the “shared secret.”

When you create a vault, you also specify the users and nodes who will need access to the vault. Chef Vault will then encrypt a copy of the “shared secret” for each user and node using that user or node’s public key on the Chef Server. So if you have two nodes and two admin users, Chef vault with encrypt four copies of the “shared secret” – one for each node or user using the respective node/user’s public key.

When you access an encrypted item in a vault you are using 2 layers of encryption – your RSA key decrypts the shared secret, then the shared secret decrypts the encrypted vault item item.

This makes it easy to add an admin user or node without needing to find and distribute the shared key. It also means when a user or node’s access needs to be removed, you only have to remove that user or node’s key, you don’t need to create a new key and distribute it to everyone.

Using Chef Vault

Installing Chef Vault

If you are using the Chef DK…congratulations, you already have Chef Vault installed! If you are not using the Chef DK, you can install Chef Vault with

$ gem install chef-vault


Creating a Vault

Let’s create a vault. Let’s say you want to store a database password. If you prefer to create data bag items from json files, you can create a json file like this.

database.json

{“db_password”:”some_password”}

Finally, let’s define who/what you want to be able to decrypt this vault item. Let’s say you have two users who need to be able to decrypt this item from their workstations – starley and jsnow. You also have two nodes that need to be able to decrypt this item named whitewalker_node_1 and whitewalker_node_2.

You would then run this command:

$ knife vault create credentials database -A starley, jsnow -M client -S ‘name:whitewalker_node_*’ -J ./database.json


Let’s break this down.

$ knife vault create credentials database -A starley, jsnow -M client -S ‘name:whitewalker_node_*’ -J ./database.json

  # credentials is the name of the vault
  # database is the name of item within the vault which will be encrypted
  # starley and jsnow are the users we want to be able to decrypt the encrypted item from their workstations (hereafter referred to as admin users)
  # We also specify a mode for Chef Vault.  If you are using it with a Chef server, then you will need to specify it as -M client.  If you are using it with Chef Solo, you will need to specify it as -M solo
  # You define nodes that need to decrypt this item by doing a search of nodes associated with your organization on the chef server, in this case for ‘name:whitewalker_node_*’  This will find all nodes with names that begin with ‘whitewalker_node_’
  # Finally, since you are defining the content for this vault item in a JSON file, you include the filename and path to that file

NOTE: After posting this, I received feedback that searching for nodes using the -S tag can be insecure. This is because any node which has been compromised can potentially fake its own data and be picked up by the search. If this happens, the next time someone runs knife vault refresh, the compromised node could be illicitly given decryption access.

To be fully secure, you must use

-A "whitewalker_node1,whitewalker_node2"


Or, when combined with adding users as well, it would be

-A "starley,jsnow,whitewalker_node1,whitewalker_node2


When this command is run, it will create a vault named ‘credentials’ and TWO items within that data bag – ‘database’ and ‘database_keys’.

The ‘database’ item contains the actual encrypted content. If we were to look at this item from a user’s workstation who was NOT an admin user for the vault, we would see only the encrypted output.

$ knife vault show credentials database

db_password:
  cipher:         aes-256-cbc
  encrypted_data: dsiBtNHX8Sbis42yKuYBvbdNXPpu8bQfJrS20op7zoysfR8roFlzpVHyoaG2
  4yb3
  iv:             +0siNLzFHHqStM07k5JhYw==
  version:        1
id:          database

If we are on a user’s workstation who IS an admin user (pretend we are on jsnow’s workstation) we would see the decrypted content.

$ knife vault show credentials database

db_password: some_password
id:          database