Infrastructure Testing and Compliance with Chef and Terraform

Cloud resource provisioning is fast and easy to scale, but in the process of getting everything up and running in a short time, security vulnerabilities are often overlooked. These vulnerabilities usually go undetected until audited, which may happen only a few times a year. When DevOps teams are under constant pressure to increase release velocity, there is a high chance that the software they release into production includes unresolved vulnerabilities. Auditing and fixing all the potential risks can be a slow process.

When done correctly, infrastructure-as-code can greatly simplify provisioning and configuring systems at scale without compromising security. Codifying repeatable processes allows IT teams to manage infrastructure more efficiently and ensure compliance of every IT resource. This blog discusses how modern configuration management and provisioning tools, such as Chef and Terraform, use the concepts of infrastructure-as-code and policy-as-code to deploy secure instances in the cloud.  

Infrastructure as Code to the Rescue (well, partially)   

Configuration management and provisioning tools such as Terraform and Chef enable infrastructure automation using the “infrastructure as code” approach. These tools automate the process of infrastructure provisioning, configuration, and maintaining compliance of base systems – VMs and cloud instances – and the applications that run on them.

Chef and Terraform work together to deliver infrastructure automation. Terraform is commonly used by organizations to provision cloud infrastructure using code. Organizations use it to automate cluster deployment in cloud providers like AWS, Google Cloud or Azure, and Chef offers provisioners and providers for Terraform that can be used to set up how and what runs on those cloud or virtual servers.

Going Beyond Infrastructure as Code with Policy as Code

Policy as code merges infrastructure-as-code and compliance-as-code into a single workflow. Setting company-wide policies for cloud-resource provisioning enables development teams to leverage the power and speed of the cloud and ensure environments are configured correctly and are compliant. Chef’s infrastructure and compliance automation tools allow you to define security and compliance policies using code that is straightforward and human-readable. 

Because everything about your systems is outlined in code, policy files can be added to your Git repository for version control and help enforce policies across all the resources in your provisioned environments. Policy as code makes it easier to manage IT resources while maintaining compliance irrespective of scale.

For example, Chef can be used to ensure port 22 is secure and accessible only within your organization’s network. Chef can also perform identity and access management checks, such as confirming the root IAM has password policy and MFA enabled. 

Policy as code can also be applied in the context of provisioning tools like Terraform. You can leverage Terraform in local development, in your automated pipelines, or in runtime environments. Terraform uses descriptive files to define system resources, and Chef InSpec can be used with Terraform in two different ways to confirm compliance:

• Audit provisioned infrastructure: When developing Terraform code for repos, Chef InSpec can be used to verify that resources have been provisioned/updated to match the tested and approved criteria. 
• Terraform code declaration: Chef InSpec can be used for test-driven development to declare the infrastructure configuration, and Terraform can be used to provision resources accordingly. In this way, Terraform manages provisioning while InSpec ensures the provisioned resources meet the organization’s policy requirements.

Drive Cloud Security with Chef’s Pre-Built Compliance Policies for Terraform

Consider the following example to understand how Chef and Terraform work together. It's based on the template for a  basic two-tier AWS architecture. 

In this example, Terraform is used to create the environment by running terraform apply, and then a stateless NGINX server is configured behind an Elastic Load Balancer on AWS. We'll extend this example to demonstrate how you can eliminate manual testing. The detailed explanation and documentation are available on GitHub.

To begin, run inspec init profile test/verify within your Terraform project directory to create a new empty InSpec profile. 

Edit the  inspec.yml based on this specific use case and add compliance tests. You can write a test that uses the available InSpec resources for each Terraform resource, such as:

For some tests, Chef needs data from Terraform. The best hand-over at this point is to use Terraform output variables. These are variables that can be added to the system environment and made available for Chef. Define the variables you need as an output in Terraform, such as this AWS example: 

Terraform has a neat built-in command-line feature that creates a JSON file with the output you'll use in the next step:

$ terraform output --json > test/verify/files/terraform.json

Chef InSpec can load files directly from profiles located in your project directory:

# load data from Terraform output content = inspec.profile.file("terraform.json") params = JSON.parse(content)

# store vpc in variable VPC_ID = params['vpc_id']['value']

You can use the variable in various places in your Terraform code:

describe aws_vpc(vpc_id: VPC_ID) do its('cidr_block') { should cmp '10.0.0.0/16' } end

describe aws_security_group(group_name: 'terraform_example') do it { should exist } its('group_name') { should eq 'terraform_example' } its('description') { should eq 'Used in the terraform' } its('vpc_id') { should eq VPC_ID } end

These steps allow you to use any data from Terraform in Chef InSpec. You're now ready to provision the infrastructure and run InSpec checks afterward.

# Clone repository $ git clone git@github.com:chris-rock/inspec-verify-provision.git $ cd inspec-verify-provision/terraform

# Download the required plugins $ terraform init

# Run terraform to apply the changes $ terraform apply -var 'key_name=terraform' -var 'public_key_path=/Users/chris/.ssh/id_rsa.pub'

# Make terraform variables available to inspec $ terraform output --json > test/verify/files/terraform.json

# Run the inspec profile to verify the setup $ inspec exec test/verify -t aws://

The following InSpec report shows the result of the Terraform provisioning with built-in Chef compliance.

Conclusion

One of the most significant advantages of implementing policy as code with Chef InSpec is having reusable version-controlled security policies. You get better visibility and control over vulnerabilities in your implementation of infrastructure-as-code, and using Chef alongside Terraform offers convenient and powerful ways to provision and manage your systems, regardless of scale. You can use these same techniques to automate pipelines used for deploying applications, and introduce security and compliance into every phase of development. A shift-left strategy using policy as code reduces risks in the development cycle while increasing release velocity.

Learn more about how you can scale infrastructure testing with Chef. Watch the on-demand webinar here.