Chef 11 Released!

We’re incredibly proud to share the latest major release of Chef with you.

The Chef Client and Chef Server are now separate projects. We will be making separate but compatible releases moving forward.

You can download both of them from the Chef download page.

Erchef, the Chef 11 Server

The most significant new feature is that the Chef 11 Server is a complete rewrite of the core API server in Erlang, which we call Erchef. We learned a lot from running Opscode Hosted Chef, the single largest Chef Server, as well as supporting our Opscode Private Chef customers. Using these lessons and experience, we wrote the new server to be faster and more scalable, but still API compatible with the original Ruby based server.

We’ve been working on Erchef for a while, focusing on the API endpoints where the scalability benefits over the old server were the most significant. We’re all excited to now incorporate this core project into Open Source Chef and share it with you. In the coming weeks we will have more blog posts focusing on the new server for those interested in getting involved with development or just becoming familiar with the new internals.

We also migrated the WebUI from merb to Rails 3 for this release.

The official packaging format for the Chef 11 Server is the same Omnibus packaging that we use for the Chef Client. We created the Omnibus framework to provide an easy way for you to quickly install our software with consistent dependencies on multiple platforms. All of the required libraries are included in the package and everything is installed into a single directory in /opt to isolate it from the rest of your system. If you’re interested in learning more about Omnibus, be sure to check out the omnibus-chef repository, which we use with the framework to build these packages.

For those of you with extensive deployments of the open-source Chef 10 Server already, we will be relasing migration tools in the future.

Only 64-bit systems are supported at this time. Packages are available for these platforms:

Post-release update

The most recent patchlevel of the Chef Server is 11.0.4, which includes these important fixes:

• CHEF-3818 – chef-server-webui is susceptible to recent Rail’s YAML exploit
• CHEF-3813 – Server does not return cookbook dependencies in metadata

Chef 11 Client

This release contains new features and some important refactoring of existing parts of the client.

Chef::Node::Attribute has been completely overhauled. These changes fixed a number of confusing exceptions to attribute precedence, making it much easier to reason about. Previously you could define a normal level attribute on the node my using methods, such as node.my_attr("foo"). Occasionally someone would name an attribute with the same name as an internal Chef method, with unexpected results. On the other hand if you were trying to access a particular method, such as node.has_key?("foo") but you accidently left out a character, you wouldn’t get an error because Chef would make an attribute with that name.

You can now access attributes from roles and environments in a cookbook attributes file, allowing you to dynamically set other attributes based on their values. Additionally, node.platform? and other helper methods are available in attributes files now, making it easier to keep logic related to attributes in one place in the attributes file.

Files other than recipes in cookbooks are now evaluated in run_list order, with proper consideration given for cookbook dependencies. Previously they were based on the order given by ruby’s Hash implementation, which differs based on version and vendor patching. This change ensures that your runs will continue to be reproducible.

We’ve merged knife-essentials into core knife. These new sub-commands are particularly useful for working with getting cookbooks between the server and your workstation and maintaining them. They are documented on the docs site.

• knife diff
• knife download
• knife upload
• knife list
• knife show
• knife delete
• knife raw

Contributions

Long time contributor and past-MVP Andrea Campi added support for partial templates. This is a significant feature if you have templates with large sections that change based on attributes. You can now render additional templates inside a template with <%= render 'other_template.erb' %>. This functionality expands the capability of the template reasource in a huge way.

Another prior MVP and Food Fight co-host Bryan Berry provided a new chef-apply tool in this release. If you pass an individual recipe to this simple tool, it will run it on the current system. This can be used for learning Chef, testing part of a recipe, and much more.

These two features really improve the Chef experience and both Andrea and Bryan are valued members of the Chef community. You guys are the Chef 11 co-MVPs! Thanks!

Multiple MVP recipient Xabier de Zuazo continued to provide big fixes. He refactored CookbookLoader to speed it up by walking the repositories paths only once. He added support to remote_file for providing a list of URLs in case one doesn’t work. He also fixed a bug related to metadata files when updating cookbooks

Victor Lowther helped on work to add support for creating a lock file when the client runs.

Bruce Krysiak added support for specifying a group by group name on the user resource on OS X.

Matthew Horan improved the code that makes knife ssh return an exit code based on the success of the remote command.

Autif Khan patched Chef::REST.new() to read raw keys from an argument.
Chris Roberts helped fix issues with precendence when attributes are set in knife.rb.

We shipped Chef 10.18.2 without an MVP, so we’re awarding it post-release to Fletcher Nichol. Fletcher has been around the Chef community for a while and contributed to a lot of related projects. He recently developed a test harness for Chef named jamie that was so awesome that we’re merging it into the test-kitchen project. Thanks Fletcher!

Breaking Changes

We did our best to maintain API compatibility between the Chef 11 Server and the Chef 10 Server, so you can run Chef 10 clients against a Chef 11 Server. With any major release, there are some important changes to be aware of. For complete descriptions of the breaking changes in this release, see the Chef 11 Breaking Changes page on the Opscode docs site.

Gem Packaging

We’re researching adding signature verification to our gems, but for now you can refer to this sha256sum for the Chef Client 11.0.0 gem if you’re interested:

2cef7db770aea59cd1d4d2c5fbea608d6ef32c24e4a2fc7e548aee1835e1fb36

Release Notes

Bug

• CHEF-581 – Delayed scripts don’t run on failure
• CHEF-867 – Use exclusive file locks with chef-client
• CHEF-1804 – Values of nested Node attributes disappear after iterating
• CHEF-2591 – Chef::ChecksumCache not working due to :skip_expires => true
• CHEF-2627 – Knife SSH should return exit code based on whether or not ssh command is successful or not
• CHEF-2792 – XSS vulnerability in messages field on login page
• CHEF-2903 – Attribute files not loaded in deterministic order
• CHEF-2923 – Cookbook Upload Fails due to Syntax Error in unrelated cookbook’s metadata file
• CHEF-3068 – Chef resources display incorrectly in log files on windows due to splitting on :
• CHEF-3376 – Chef Should Load Cookbooks In Dependency Order
• CHEF-3393 – Chef Encrypted Data Bag Error due to Different YAML Engines
• CHEF-3467 – Permissions Not Inherited from Parent on Child Object cookbook_file (Windows)
• CHEF-3477 – knife node show shows unexpected brackets for single-member arrays
• CHEF-3480 – When Encrypting Data-Bag Items, Use Different IV Per Encrypt
• CHEF-3555 – knife cookbook site install fails due to not allowing string format cookbook_path
• CHEF-3561 – Error inspectors seem to quash template error contextual information
• CHEF-3589 – Why Run code runs code in {{converge_by}} blocks after the provider action is complete
• CHEF-3604 – Chef::Provider::Service::Init should pass why run assertions if a custom command is provided
• CHEF-3617 – Chef::RunLock incorrectly assumes the full path to the run lock file exists before opening for writing
• CHEF-3619 – Chef still has obsolete ‘rake/rdoctask’ require
• CHEF-3632 – All providers have whyrun enabled by default due to RemoteDirectory
• CHEF-3638 – knife cookbook upload with–all flag fails
• CHEF-3639 – ‘knife index rebuild’ no longer works with Chef Server 11
• CHEF-3640 – bookshelf is logging to the wrong directory
• CHEF-3641 – lock down permissions on omnibus generated private keys
• CHEF-3643 – WebUI: cookbook version show generates “undefined method `close!’ for nil:NilClass” error
• CHEF-3647 – changing a user’s password in chef-server-webui changes their public key to undefined
• CHEF-3648 – WebUI: creating client fails with ‘Could not create client: 500 “Internal Server Error”‘
• CHEF-3653 – Unable to show data bag items in webui
• CHEF-3654 – Adding role to node run list in webui gives undefined method `empty?’ for nil:NilClass
• CHEF-3655 – Webui node show run_list: ERROR: Unable to create Chef::RunList::RunListItem from String
• CHEF-3656 – Last check-in display shows > entity in webui for /status
• CHEF-3657 – Unable to add cookbook version constraints to an environment via webui
• CHEF-3658 – Admin clients should not be able to de-admin the last admin user
• CHEF-3662 – knife client reregister fails against Chef 11 Server
• CHEF-3666 – Postgresql recipe is overzealously guarding configuration files behind bootstrap variable
• CHEF-3673 – Bookshelf URL is not configurable
• CHEF-3674 – Psql doesn’t work by default, since /opt/chef-server/embedded/bin is not in the path
• CHEF-3675 – Bookshelf cannot be disabled
• CHEF-3680 – json error: (eval):3:in `keys’
• CHEF-3688 – Stale attribute read protection does more harm than good
• CHEF-3689 – Client registration fails on existing clients
• CHEF-3699 – Getting ISE when depsolver runs
• CHEF-3702 – Postgresql foreign key integrity error on cookbook upload in Erchef alpha
• CHEF-3710 – chef_wm: exception handling for bad_headers in chef_wm:malformed_request/2
• CHEF-3716 – Erchef 1.0.21 knife cookbook upload succeeds, but cookbook content gives 404 from chef-client
• CHEF-3718 – peg 2.2.0 version of systemu for win ruby 1.9 compat
• CHEF-3724 – node.recipe? appears to be broken
• CHEF-3782 – Bad regular expression make sandbox test fail in pedant
• CHEF-3783 – compatibility for chef/dsl/recipe not found in chef/mixin/recipe_definition_dsl_core.rb
• CHEF-3792 – chef-apply is still referred to as chef-recipe in places
• CHEF-3793 – knife is broken in latest Chef 11 client beta
• CHEF-3799 – Cannot call puts on a VividMash
• CHEF-3802 – omnibus package symlinks not created for chef-apply, chef-shell
• CHEF-3806 – when setting node attributes, after a set_unless call subsequent set calls behave like set_unless (and correspondingly for other attribute levels)

Improvement

• CHEF-707 – Change “gid” to “group” for the User Resource
• CHEF-2936 – Cookbook attribute file to have access to expanded attributes
• CHEF-2984 – Remove moneta dependency
• CHEF-3021 – Convert chef-server-webui from merb to Rails 3.2
• CHEF-3197 – Undefined methods on a node should not be assumed to be attributes
• CHEF-3249 – Chef support for template partials
• CHEF-3385 – Remove support for multiple notifications in one call to Resource#notifies
• CHEF-3392 – Serialize Encrypted Data-Bag Item Values with JSON Instead of YAML
• CHEF-3438 – Allow Chef::REST.new() to read raw keys
• CHEF-3487 – Refactor CookbookLoader to walk @repo_paths only once
• CHEF-3497 – Allow knife.rb to implicitly provide all knife related options
• CHEF-3499 – Allow access to platform? and friends in attribute files
• CHEF-3500 – ruby_block resource’s action “create” is counter-intuitive
• CHEF-3556 – Knife search should assume you’re searching for nodes, and make a fuzzy query if the query is not in solr syntax
• CHEF-3576 – support raspbian as a platform
• CHEF-3603 – Chef::Provider::Service::Init should support specification of alternate ‘init_command’
• CHEF-3616 – Include cipher in Encrypted Data Bag Item v1 Format
• CHEF-3628 – knife upload with no parameters should prompt before uploading EVERYTHING
• CHEF-3663 – Knife’s config file lookup doesn’t work when the current directory or parent is a symlink
• CHEF-3681 – the LWRP DSL should automatically create and converge a new run_context
• CHEF-3715 – Remove caching of SHA256 sums in Chef::ChecksumCache

New Feature

• CHEF-2004 – knife environment show should have an attribute selection option (-a / –attribute)
• CHEF-3375 – remote_file support for URL lists to use as mirrors
• CHEF-3520 – Bring knife-essentials commands into Chef proper
• CHEF-3571 – Add a chef-apply ruby script into chef gem for running a single recipe without modifying the node’s run_list

Task

• CHEF-2925 – Rename “shef” to “chef-shell” (or similar)
• CHEF-2992 – Move seen_recipes out of node context into run context
• CHEF-3408 – Add deprecation notices for Chef::ShellOut
• CHEF-3417 – Remove cookbook file support from remote file
• CHEF-3542 – Remove subtractive “knockout” merge from Deep Merge for Chef 11