Chef 0.8.2 Release

Ladies and gentlemen, on behalf of the entire Chef community, I’m proud to announce the release of Chef 0.8.2. This release marks a major milestone for Chef – we’ve re-factored the Chef Server to support a more robust authentication mechanism, have a completed REST API, made Search a robust and integral feature, added Data Bags, and have a fully functional command-line client that quickly became the only way I want to interact with Chef.

This release represents the hard work of 28 different contributors, the vast majority of whom volunteered their time and effort – fixing bugs, adding features, and extending the number of providers we support. On top of that, countless others have given of themselves freely in IRC and on the mailing lists, answering questions and helping each other solve their problems and build great infrastructure. As proud as I am of all of the technical accomplishments in this release, I can’t even begin to express how proud I am that so many would give so much to make Chef great – thank you all for being the living heart of a community I’m thrilled to be a part of every day.

With so many people involved, it’s difficult to choose just one to highlight as the MVP for this release. When I reflected on it, one name stood out above the rest as doing more to make sure that this monster of a release would see the light of day, and that anyone who wanted to start testing it would have the resources they needed to do so. That name is Scott M. Likens (Damm on irc). Scott has been running pre-release versions of Chef 0.8 from the first moment it was possible to do, and he’s been constantly giving feedback, filing tickets, and most importantly, helping everyone and anyone get up to speed on all the things it contains. He seems an almost omnipresent force for good within the IRC channel, always ready to walk someone through boot-strapping a complicated piece of pre-release software. Without Scott, the testing of this release would have taken much longer, and its quality would have been much, much lower. So here’s to you, Scott – the Chef community owes you a real debt.

A quick run-down of what’s new:

• The Chef Server REST API, which provides API level access to the full functionality of the Chef Server.
  • To prove it, we made the Web UI utilize the REST API exclusively.
  • Knife, the command line utility originally written by Josh Sierles of 37signals, has been embraced and expanded to cover every API end-point of the Chef server (and a whole lot more – like search-driven parallel SSH). Managing your infrastructure from vi and emacs has never been this easy.
• We have replaced OpenID as the Chef Server Authentication mechanism with per-request digital signatures. We absolutely love how this has turned out, and we think you will too.
• Cookbooks and Roles are both uploaded directly to the Chef Server via the API.
• Search is now stable and fully functional, opening up a whole new realm of cross-infrastructure utility. We now utilize Solr and RabbitMQ, rather than Ferret and Stomp.
• Data Bags allow you to store arbitrary data within the Chef Server, and have it fully indexed for Search. This allows even more flexibility in separating the data that drives your recipes from the definition themselves.
• Shef, the Chef REPL, has been introduced. Shef can be used to interactively run chef resources, as a debugger for running chef clients, and to astound your friends.
• Roles can now contain roles, thanks to the effort of the esteemed Thom May. Thanks, Thom – this is one of my new favorite features.

All this does come at a cost – your Chef 0.7 (and earlier) environments are not backwards compatible with Chef 0.8.

If you’re ready to upgrade, head over to the wiki for instructions on upgrading your Chef 0.7 environment to Chef 0.8.

That’s a lot of stuff, but it only puts a dent in the total amount of goodness that’s in this release. Let’s review what all our mad-cap contributors have been up too, shall we?

Alan Johnson fixed the Cron resource to function well with non-numeric time entries.

Bryan McLellan, added support for Ubuntu’s Upstart service manager, patched the mount provider to work with SMB and CIFS mounts, and did numerous other packaging and bug fixes. He also found the time to put out a number of unofficial releases in the 0.7 series, and continued his tradition of giving valuable advice and feedback.

Caleb Tennis continued to provide updates to our ability to properly manage Gentoo Portage builds.

Cary Penniman, of RightScale, extended Chef’s metadata support to better enable the creation of interactive UIs.

Daniel DeLeo kept up a furious pace during the development of Chef 0.8. He was responsible for numerous improvements, such as: Shef, ensuring the completeness and speed of the test suite, SCM and Deploy fixes, and the RabbitMQ-based message queue infrastructure – and he did all of that before we hired him. Thanks, Dan – and we’re super glad to have you on-board at Opscode.

The enigmatic dreamcat four added support for managing Users and Groups on Mac OS X, helped create new rack configuration files for the re-factored Web UI and API, and fixed a number of bugs in our development environment.

Eric Hankins provided some fixups to the logging system, fixed a bug in the Cron provider that resulted in possible dead-locks, and provided insight into the gem building process.

Ian Meyer from Etsy made it so when you said you wanted ‘1’ backup of a file, Chef actually makes one. He also provided insight into some dependency issues, cleaned up some un-needed dependencies, and fixed a bug with setting the log level.

Ivan Pirlik fixed a bug in an early version of Chef 0.8 that was stopping recursive directory transfers from working, and patched the Web UI to reflect a change in how we encode node names in the API.

James Golick enabled use of the Cron provider on FreeBSD. He’s also giving a talk about Chef at the Mountain West Ruby Conference, which I’m excited to watch.

Jan Zimmek added support for managing packages and services on ArchLinux. Nice work, Jan!

Jesse Nelson fixed the route provider so that it properly configures Red Hat and CentOS machines.

Joe Williams added an mdadm provider for managing software RAID arrays on Linux, an erl_call provider for interacting with Erlang processes, and a Python easy_install package provider.

Kris Rasmussen from Aptana made it so that delayed actions execute in the order they are triggered in the recipe, adding more consistency and predictability to Chef. He also helped ensure that the Git provider always fetches all remote tags and branches.

Mario Giammarco added support for SuSE’s Zypper package manager.

Matthew Kent provided significant feedback and improvements to our Deep Merge strategy – including porting the test suite! How great is that? He also updated the Red Hat init scripts, improved Lightweight Resources and Providers, and improved the debugging output.

Matthew King provided a fix for the chef-repo ticket that caused it to fail when no upstream Git origin was present.

Mathias Meyer, the most cupcake-obsessed developer on the planet, made the template resource respect the cookbook option, and improved the scm provider to allow the use of symbols for the underlying provider rather than the verbose constant.

Pavel Valodzka fixed the deploy resource to no longer have an implicit trailing slash for the cached-copy, making it decidedly less brittle.

In tandem with his co-worker Bryan McLellan, Peter Crossley led the effort to make editing JSON via the Web UI far less perilous – going so far as to re-build it for us. Thanks, Pete.

Tollef Fog Heen extended the not/only_if meta parameters so that they honor the same arguments as a normal execute resource. For fans of Chef’s backup functionality, Tollef made it possible to backup files to a directory structure rather than in-place. He also enabled SSL certificate verification, made accessing the node and @node variables more consistent, fixed daemonizing bugs, and patched a number of potential security issues. In addition to his code contributions, Tollef has been invaluable in ensuring that this release is well packaged for Debian and Ubuntu, and is a consistently rational voice in favor of quality and safety.

A huge thank you to everyone who contributed to this release. As always, we’re on the mailing list, IRC, and via e-mail if you need any help.

Full release notes follow:

Release Notes – Chef – Version 0.8.2

Bug

• [CHEF-219] – Attribute files need include_attribute
• [CHEF-406] – typos in specs, fix resulting failures
• [CHEF-438] – Chef::Provider::Execute doesn’t honor user attribute for not_if/only_if
• [CHEF-440] – Running chef-client with a JSON file should override the node’s run_list on server
• [CHEF-454] – Centos4 yum provider failure
• [CHEF-470] – role attribute deep merge only goes one level deep
• [CHEF-491] – Should suport SSL verification
• [CHEF-530] – chef-client daemon dies with segfault
• [CHEF-584] – launching chef-client init script hangs chef-solo
• [CHEF-607] – cookbook loader doesn’t get attributes in correct order
• [CHEF-616] – rake install in chef-repo breaks if there is no git origin
• [CHEF-623] – Nodes are able to be created without names
• [CHEF-629] – Users must be able to set the action on the SCM resource used by deploy
• [CHEF-633] – HTTP Request uses Chef::REST incorrectly
• [CHEF-634] – UI fails silently when unable to save node
• [CHEF-635] – Portage incorrectly detects currently installed packages
• [CHEF-638] – Deploy with a revision should make that revision current if it already exists
• [CHEF-639] – git resource fails on subsequent checkouts of the same repostiry
• [CHEF-642] – Services will always issue a WARN when status is not present
• [CHEF-643] – Fix CHEF-570, as it doesn’t catch package -revisions
• [CHEF-644] – Shebang lines should respect user’s ruby
• [CHEF-645] – cd to /tmp breaks merb bootloader in features
• [CHEF-650] – Freebsd and Yum package shortcut resources don’t exist
• [CHEF-654] – chef-client -j against a self signed cert fails
• [CHEF-657] – Deploy with revision strategey did not clean up cache on rollback
• [CHEF-659] – UI expands some escaped characters from JSON, then fails to encode them again
• [CHEF-664] – libxml required but not a gem dependency
• [CHEF-668] – cron resource seems to always run for non-numeric time entries
• [CHEF-673] – uuidtools gem is required with a silent rescue and not dependend on by chef server
• [CHEF-680] – When rake upload_cookbooks fails, it leaves behind a tempdir that causes the next run to fail
• [CHEF-681] – 500 error when trying to retrieve a file w/o a default dir
• [CHEF-688] – Deploy revision strategy does not fetch tags
• [CHEF-693] – Nodes should auto-expand
• [CHEF-694] – role override and default attributes don’t get indexed
• [CHEF-695] – No backups kept if backup is set to 1
• [CHEF-697] – Cookbook metadata does not convert booleans to "required" or "optional" on ruby 1.8.7 as shown by spec failures
• [CHEF-703] – chef search: make the default always be to iterate over the results
• [CHEF-704] – Ruby block device does not have a default action
• [CHEF-706] – mount provider fails to mount samba/cifs devices (Device does not exist)
• [CHEF-710] – knife fails silently if you lack the EDITOR env var
• [CHEF-713] – missing alias for rpm_package (and probably also freebsd_package, yum_package)
• [CHEF-722] – Link provider doesn’t understand paths requiring expansion
• [CHEF-723] – creating ssl-certs with fqdn "*.example.com" should name w/ wildcard instead of *
• [CHEF-725] – ‘rake gem’ fails calling ‘rake package’ on chef-solr
• [CHEF-732] – Remove references to chef-indexer
• [CHEF-734] – LWRP resources should look for provider named same by default
• [CHEF-736] – chef-solr gem rake install task does not use sudo
• [CHEF-737] – Starting chef-server with ‘-c2’ fails to create a couch database
• [CHEF-741] – Web UI allows creation of a user with no name
• [CHEF-742] – Cron provider hangs installing new crontab
• [CHEF-744] – rake upload_cookbook and upload_cookbooks tasks should complain when run from the wrong location.
• [CHEF-747] – webui needs a binary so you can run the slice outside of git
• [CHEF-749] – cache is not maintained with run_list; so we load items from the cache that are no longer set to run in the run_list
• [CHEF-751] – mixlib-authentication needs version tags
• [CHEF-753] – chef server slice should finish activating
• [CHEF-754] – Chef::Config.cookbook_path is in reverse order of override application
• [CHEF-756] – Specifying the run list for instance data with knife should ignore commas
• [CHEF-758] – Webui prints generated keys with spaces and not newlines
• [CHEF-762] – chef-solr gem builds from chef root but doesn’t include the lib dir
• [CHEF-764] – webui crashes when attributes are Fixnum
• [CHEF-765] – when creating a user in the web-ui, it causes an indexer failure to occur in solr and thus users never get indexed.
• [CHEF-766] – Chef should print the error message generated by the Chef Server when an HTTP Exception occurs
• [CHEF-774] – Chef-server does not log anything to server.log
• [CHEF-775] – if a client has a hostname with an _ such as hasoffers_3.adappsolutions.com it gets truncated improperly and mangled into hasoffers.3.adappsolutions.com and then returns a 403 during chef-client run
• [CHEF-777] – hostnames with an _ in it, create 2 node entry’s and then fail during a chef-client run
• [CHEF-779] – rake upload_cookbook shouldn’t try to upload cookbooks that don’t exist in the local repo
• [CHEF-796] – Change to mixlib-log breaks Chef::Log.level() usage
• [CHEF-799] – remote_directory does not work
• [CHEF-800] – The response from /search contains nil in some situations while the Chef::Search::Query library tries to process every result even it’s nil
• [CHEF-801] – Deprecated dependencies in chef-server
• [CHEF-807] – SVN provider uses undefined local variables to create an error message, obscuring the true cause of the error
• [CHEF-808] – Rakefile for mixlib-authentication uses obsolete cucumber task syntax
• [CHEF-811] – knife subcommand options are invalid, not processed
• [CHEF-812] – file backup permissions less secure than file
• [CHEF-813] – Spec failure: ‘Chef::Application::Knife run should exit 2 if run without a sub command’ actually returns 1
• [CHEF-814] – Spec failure: ‘Chef::Application::Knife run should exit 2 if run without a sub command’ actually returns 1
• [CHEF-815] – knife cookbook upload fails when cookbooks are in an SVN repo
• [CHEF-818] – rake install: no longer require sudo
• [CHEF-822] – Web UI differentiates between "user not found" and "wrong password"
• [CHEF-824] – The WebUI is not detecting thin
• [CHEF-825] – The WebUI is a little excited about wanting you to change the password and gramatically off
• [CHEF-826] – rest fails with an obscure error if node_name is not determinable
• [CHEF-828] – Mixlib CLI – Preserve ARGV after @opt_parser.parse!
• [CHEF-832] – chef-solr does not install any binaries what so ever.
• [CHEF-837] – Delayed actions excute in unpredicatable order
• [CHEF-840] – client admin field should be checkbox not text box
• [CHEF-841] – knife keeps trying to upload a cookbook despite HTTP 401
• [CHEF-842] – duplicate copies of FileEdit: file_edit.rb & fileedit.rb
• [CHEF-843] – FileEdit permission issues
• [CHEF-850] – rake spec in ‘chef’ project tries to create directory /new/home/adam
• [CHEF-851] – specs leave a /tmp/foo directory lying around
• [CHEF-854] – shef executable needs to be added to the gemspec
• [CHEF-855] – ruby-openid required by chef-server-webui, and missing in rakefile
• [CHEF-858] – Provider detection broken if Resource type and Provider name matches
• [CHEF-859] – route provider will incorrectly configures centos/rhel networking
• [CHEF-860] – chef-web-ui status button shows 500 error
• [CHEF-864] – Unable to reindex chef via knife or chef-solr-rebuild
• [CHEF-870] – does not close stderr when daemonising
• [CHEF-873] – Template provider doesn’t respect the template resource’s cookbook option
• [CHEF-877] – fix :default and :required in lwrp
• [CHEF-879] – knife data bag show groups fails … with something like undefined method `keys’ for ["http://localhost:4000/data/groups/sysadmin"]:Array
• [CHEF-880] – chef-server-api should require admin privileges to update data bag items
• [CHEF-896] – file_backup_path include prefix in log output
• [CHEF-897] – package provider does not tell you about which package is the cause of problems
• [CHEF-898] – chef-solr-indexer needs to depend on uuidtools
• [CHEF-900] – security vulnerability in 0.8 webui
• [CHEF-902] – mixlib-authentication fails to generate SHA1
• [CHEF-904] – "Could not create work tree dir" on chef deploy
• [CHEF-906] – chef-server-webui refers to JSONeditor which doesn’t exist
• [CHEF-908] – Indexer fails on node properties that have invalid XML character sequences
• [CHEF-916] – on chef-server startup, webui_user.rb:203:in `create_design_document’: uninitialized constant Chef::Couchdb (NameError)
• [CHEF-918] – chef-solo fails to download remote recipes because application/solo.rb no longer uses open-uri
• [CHEF-919] – chef-server-webui requires merb-param-protection but rakefile doesn’t list it.
• [CHEF-922] – knife ssh should accept -a ec2.public_hostname
• [CHEF-927] – Override and Default attributes get clobbered at recipe load time
• [CHEF-928] – deep_merge issues with empty strings
• [CHEF-929] – typo in provider/deploy/revision causes rescue from StandardError instead of Chef::Exceptions::FileNotFound
• [CHEF-931] – deep_merge mixin spits out a warning
• [CHEF-932] – git provider fetch strategy will not get all updates
• [CHEF-934] – chef-server’s old gemspec is still in the source tree
• [CHEF-935] – chef-server-webui data bag item editing is broken
• [CHEF-936] – chef no longer requires deep_merge gem
• [CHEF-937] – chef-server-api and Chef::Solr::Query bans queries for api_users
• [CHEF-969] – Api client should expect 409 not 403 in save when one with same name already exists
• [CHEF-970] – Need both class and instance methods for chef_server_rest
• [CHEF-974] – undefined chef_server_rest
• [CHEF-975] – client.pem is mode 644 by default
• [CHEF-977] – `knife configure` should prompt for validation_client name and validation_key
• [CHEF-978] – Knife should not show HTTP Request Returned 404 Not Found: Cannot load node foo as WARN when the node gets created successfully
• [CHEF-980] – BULK DELETE (cookbook/node/client/role) defaults to deleting everything
• [CHEF-989] – Search for client in the webui returns 404 when trying to show the client in the search result

Improvement

• [CHEF-291] – No high-level cookbook endpoint in REST API
• [CHEF-358] – Refactor the REST API
• [CHEF-374] – drop outdated contrib/
• [CHEF-555] – Default adapter for chef-server merb should be thin instead of mongrel
• [CHEF-573] – Add a verbose setting to allow logging to a TTY
• [CHEF-601] – Extend metadata spec based on wiki comments
• [CHEF-617] – Install to chef repository on a remote machine
• [CHEF-626] – Template Context should have a #node method so users don’t have to remember to use @node
• [CHEF-646] – Enable Chef::Provider::Cron under FreeBSD
• [CHEF-647] – Deploy resource should always run symlinks before migrate.
• [CHEF-656] – 0.8 Integration tests should setup Vhosts and users for nanite
• [CHEF-666] – Chef should have separate Rabbitmq users for "nanite" and "mapper" roles
• [CHEF-667] – Specs run really slooooow because of ohai
• [CHEF-669] – optional rubygems?
• [CHEF-670] – knife should be smarter about the command line arguments
• [CHEF-671] – SCM providers should be able to specify the group
• [CHEF-676] – redhat init script update
• [CHEF-684] – Should be possible for roles to be created without anything in the run_list
• [CHEF-687] – shef: irb/REPL mode for chef
• [CHEF-698] – Validate JSON in the Web UI
• [CHEF-699] – solo and client modes in shef
• [CHEF-705] – chef-repo with rake install should update roles via the api
• [CHEF-709] – Support for backup up files in another directory than the original file
• [CHEF-728] – Data bags could be Mash, not Hash
• [CHEF-748] – chef-server-webui – config.ru and bin script
• [CHEF-750] – Refactor the caching code so logic for what gets cached and why is wrapped in a class
• [CHEF-755] – "knife create_client" could accept :admin option
• [CHEF-760] – Use AMQP drivers directly for indexing queues
• [CHEF-776] – users and groups for mac os x
• [CHEF-805] – Deploy resource’s scm_provider should accept a short string/symbol as name instead of a fully qualified class name
• [CHEF-806] – knife refactored should show all the sub commands with —help
• [CHEF-865] – Implement retry logic when making server requests
• [CHEF-881] – make :name_attribute work in lwrp
• [CHEF-882] – slowdown due to debugging in ruby_block provider
• [CHEF-943] – add section for knife ssh to man page
• [CHEF-944] – chef-solr needs LICENSE information
• [CHEF-951] – update redhat distro files for 0.8
• [CHEF-956] – Embed the Chef::VERSION as X-Chef-Version in HTTP requests
• [CHEF-971] – Changes to webui error handling

New Feature

• [CHEF-209] – make use of Joshua Sierles’s thorfile for recipe management
• [CHEF-328] – CRUD Cookbooks via API
• [CHEF-390] – Software raid provider
• [CHEF-417] – Add rake task to update roles in running server.
• [CHEF-444] – document using the chef-server API from a script using ‘knife’ as an example
• [CHEF-447] – Zypper provider – suse support
• [CHEF-474] – Switch chef-server syntax highlighting to coderay
• [CHEF-505] – Nested Roles
• [CHEF-536] – Add server side key-pair creation
• [CHEF-537] – Authenticate Signed API Requests
• [CHEF-538] – Allow for the creation of API Clients
• [CHEF-539] – User Accounts
• [CHEF-540] – Allow association of user accounts with OpenIDs
• [CHEF-542] – Cookbook Uploading
• [CHEF-543] – Web UI should call the API exclusively
• [CHEF-551] – erl_call provider
• [CHEF-576] – provider for python easy_install
• [CHEF-598] – Upstart service provider
• [CHEF-696] – Cache file checksums
• [CHEF-708] – add data bags support to Chef DSL
• [CHEF-731] – Add a screenrc to start a chef environment in screen
• [CHEF-769] – knife should be able to specify the config file location
• [CHEF-809] – man page for knife
• [CHEF-856] – knife configure should allow you to create a new api user
• [CHEF-866] – @node and node should be available in attributes
• [CHEF-912] – deep_merge should be a Chef::Mixin
• [CHEF-948] – pacman provider / resource – archlinux support
• [CHEF-949] – service daemons – archlinux support
• [CHEF-957] – arch as platform – archlinux support

Task

• [CHEF-661] – mixlib-authentication not available on apt.opscode.com
• [CHEF-674] – uuidtools gem needs to be packaged for debian
• [CHEF-740] – debian package for nanite
• [CHEF-759] – Package bunny for debian + ubuntu
• [CHEF-787] – Make bunny DFSG-free
• [CHEF-921] – Remove deprecated gem dependencies
• [CHEF-990] – bulk deletes are not spec tested for clients, cookbooks, or roles