Chef 11 In-Depth: Attributes Rewrite

Chef’s attributes system is frequently cited by power users as one of
their favorite features. Chef users love having the flexibility to tune
their applications based on a node’s role or environment. Since we first
introduced the current attributes system in Chef 0.8, however, we’ve
found a few ways to make attributes work even better, but to do so we
needed to make some incompatible changes.

Read Vs. Write

The primary change in the revamped attributes implementation is that
read and write have been separated syntactically. When node attributes
were updated to support multiple precedence levels in Chef 0.8, an
important design constraint was maintaining compatibility with the prior
API. In Chef 0.7, for example, you would set attributes like this:

node["attribute"] = "value"

Chef 0.8 introduced the precedence-based API we’re familiar with today:

node.default["attribute"] = "default value"
node.set["attribute"] = "use this value"
node.override["attribute"] = "No, I really mean this"

Supporting both APIs was very difficult and required a lot of
complexity in the implementation, and as a result attributes would
sometimes exhibit surprising behaviors. To fix these bugs, we needed to
simplify the code, and that meant dropping support for the Chef
0.7-style API. Now you must specify which precedence level you want to
write to when setting attributes; accessing attributes on the node
directly is only for reading. When reading attributes, a merged view of
the components is generated. The merged attributes are made read-only,
because if you were able to write to them, any changes would be lost the
next time the attributes were regenerated. In practice, you’ll see
something like this if you try to write to the read-only merged
attributes:

node[:foo] = "oh no"
> Chef::Exceptions::ImmutableAttributeModification 
> Node attributes are read-only when you do not specify which precedence level to set.
> To set an attribute use code like `node.default["key"] = "value"'

Stricter API

We also took the opportunity to make the attributes API more
strict. The new implementation still allows
you to interact with attributes using Hash-like syntax with either
String or Symbol keys, or with method calls via method_missing. When
using the method_missing API, you now can only set values using setter
syntax. For example, node.default.an_attribute("value") is no longer
valid; you need to use code like node.default.an_attribute = "value"
instead. The reasoning for this is best explained by example.

Attribute Mishaps

Lamont Granquist > this probably is an issue though:
Lamont Granquist >
================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/REDACTED/recipes/default.rb
================================================================================

NoMethodError
-------------
Undefined node attribute or method `has_key' on `node'

Dan DeLeo > @Lamont did `has_key` work before? should be `has_key?` with the question mark, no?

Steven Danna > has_key probably worked by mistake?
Steven Danna > do you have a "has_key" attribute?

Lamont Granquist > hah

Lamont Granquist >
% knife node show i-d2da7caa -l -a has_key
has_key:  [REDACTED]

Steven Danna > @Lamont lolol

And the best part?

$ git blame FILE
...
b32f1ac5 (NAME REDACTED 2011-04-11 17:38:04 -0700 104) 

Role and Environment Attributes Visible Everywhere

One problem we noticed users running into was that Chef would give
unexpected results when you used logic to compute an attribute in an
attributes file. For example, suppose we want to set a virtual server’s
host based on two other attributes. Even simple code like this wouldn’t
work as expected:

node.default[“server_name”] = node[“app_name”] + “-” + node[“app_environment”]

This didn’t work because Chef internally managed the precedence of role
attributes vs. attribute files by waiting to apply role attributes until
after the attributes files were evaluated. If you had set app_name or
app_environment in a role (or environment), you’d get the wrong
result. In Chef 11, the attributes implementation now maintains role and
attribute file-sourced attributes separately, so role and environment
attributes are readable while attribute files are evaluated.

Ordered Evaluation of Attributes Files

In addition to setting attributes in roles, another common way to
customize attribute values is by designing cookbooks to work together.
Suppose you have the MySQL cookbook from the community site and a
“mysql-with-our-tweaks” cookbook that modifies the other cookbook’s
behavior for your needs. The natural place to tweak attributes with your
site-specific modifications is in the attributes file of your “tweaks”
cookbook. In Chef 10 and earlier, this could be frustrating because Chef
would load attributes files in essentially random order, so you had to
manually track dependencies between attributes files using the
include_attribute directive. In Chef 11, we’ve fixed this. Attributes
(and other cookbook components, aside from recipes) are now evaluated in
order based on your run list and cookbooks’ dependencies; all of a
cookbook’s dependencies appear before it in the final sort order, but
the overall order is otherwise controlled by the run list.

Chef solo users should be aware that this feature is driven by
dependencies specified in cookbook metadata. This means that chef-solo
will now only load cookbooks that are directly in the run_list or
reachable via the dependency chain. Additionally, chef-solo users will
now see errors when a nonexistent cookbook is specified as a dependency.

Debugging Attributes

One nice side-effect of this change is that you can more easily debug
where attribute values are getting set. To get a list of all the values
set for a given key, use debug_value(:foo, :bar). For example:

# Role File:
default_attributes "test" => {"source" => "role default"}
override_attributes "test" => {"source" => "role override"}

# Attributes File:
default[:test][:source]  = "attributes default"
set[:test][:source]      = "attributes normal"
override[:test][:source] = "attributes override"

# Recipe:
require 'pp'
pp node.debug_value(:test, :source)

# Output:
[["set_unless_enabled?", false],
 ["default", "attributes default"],
 ["env_default", :not_present],
 ["role_default", "role default"],
 ["force_default", :not_present],
 ["normal", "attributes normal"],
 ["override", "attributes override"],
 ["role_override", "role override"],
 ["env_override", :not_present],
 ["force_override", :not_present],
 ["automatic", :not_present]]

Force Default and Override

An important consequence of splitting the environment and role
attributes is that attributes you set in recipe files no longer
overwrite values set in roles or environments. For most users, this
isn’t a problem becuase they only set attributes in recipes to work
around these problems. Some users, however, have adopted a workflow that
heavily relies on cookbooks for specific applications setting different
values than what the role or environment specifies. For these users, the
above changes were a step backward. To mitigate this problem, we’ve
added force_default and force_override attribute levels. These are
available in attributes files, too, so you can keep all of your
attribute logic in one place.

# attributes/default.rb
default["attribute"] = "a value set from a role will replace me"
force_default["attribute"] = "I will crush you, role attribute!"
# default! is an alias for force_default
default!["attribute"] = "The '!' means I win!"

Enjoy

Breaking changes are never easy, and we didn’t make these choices
lightly. We believe we’ve done a good job of keeping the API and
behavior mostly intact while removing the unexpected “gotchas” from the
previous design. We think you’ll find the new attributes system will
work just the way you’d expect with less opportunity for hidden mistakes
to go unnoticed, and make Chef even more powerful.