Roles in chef are un-versioned. Early in our adoption of chef we defined our roles as a run list and a collection of attributes. Our infrastructure is set up such that almost all of our environments share a single chef server and each node periodically checks in with the chef server and updates itself as appropriate.

We wanted a way for roles to propagate between our environments step by step, validating the role met our requirements at each step before being promoted to the next step. i.e. A role should be deployed to the 'development' environment and then 'integration', 'uat', 'staging' and finally 'production'.

Initially we broke our environments a few times when we updated our role definitions as these definitions were shared across all environments. To address this we attempted to use an ugly naming schema of roles, suffixing a version. So we ended up with a scenario where we had the 'myrole_v4' role in 'development', the 'myrole_v3' role in the 'uat' environment and the 'myrole_v1' role in the 'production' environment. This approach did not feel right. Finally we realized that Chef already has a mechanism for versioning artifacts - namely cookbooks.

Role Cookbooks

Today our roles simply include a single recipe, the role recipe. The role recipe then uses "include_recipe" to include other recipes in the required order. This allows us to rely on chefs builtin version resolution mechanisms to version our roles. Our role and associated 'role cookbook' looks something not unlike the following;

roles/foo.rb

name "foo"
description "Foo Server"
run_list("recipe[mybiz-foo]")

cookbooks/mybiz-foo/metadata.rb

name "mybiz-foo"
description "Sets up the Foo Server"
version "0.4.2"
...
depends "ntp"
depends "git"
depends "foo"

cookbooks/mybiz-foo/recipes/default.rb

include_recipe "ntp"
include_recipe "git"
include_recipe "foo"

Using this approach allowed us to have a single role "foo". However the associated cookbook/recipe may have a different version in different environments. This allowed us to easily control the evolution of the role and the promotion of the role between different environments.

Wrapper Cookbooks

Role cookbooks were great at being able to version the run list of our roles but it did not solve the problem of attributes in our roles. The attribute data was what we used to customize the cookbooks for our particular business. We had struggled with the interaction between the place the attributes are specified (i.e. node, environment, role, cookbook) and the precedence levels (i.e. default, normal, override and automatic). So we decided to simplify.

For every cookbook and/or recipe we wanted to customize we created a separate 'wrapper' recipe that set the required attributes and then included the recipe from the original cookbook. This resulted in a layout that looked like the following;

cookbooks/mybiz-bar/metadata.rb

name "mybiz-bar"
description "Sets up the Bar Server"
version "0.1.3"
...
depends "bar"

cookbooks/mybiz-bar/recipes/default.rb

node.override['bar']['port'] = 80
node.override['bar']['interface'] = '0.0.0.0'
node.override['bar']['host'] = 'bar.mybiz.example.com'
node.override['bar']['max_threads'] = 250

include_recipe "bar"

We also decided on a few simple rules for setting attributes;

• Wrapper cookbooks should only ever set attributes using the 'override' precedence.
• Cookbooks should set attributes using the 'default' precedence if a wrapper cookbook is allowed to override the attribute.
• Cookbooks may set attributes using the 'override' precedence if they are publishing attribute data for other cookbooks to use but do not expect the other cookbooks to override the attribute data.

Using this technique, there was effectively only one place to look for cookbook customisations and you did not have to think about the precedence rules. It felt much easier to manage our cookbooks.

Role or Wrapper Cookbook?

While I presented role and wrapper cookbooks as separate concepts and separate cookbooks, this need not be the case. If the recipe in a wrapper cookbook is only used within a single role cookbook and it is relatively simple then we tend to move the wrapper recipe into the role cookbook. Only when there is complex logic required or a cookbook is used by multiple roles do we bother creating a separate wrapper cookbook.

Environments

Our environments do still have some attribute data within them but the data tends to be used to drive a rules layer. For example, our environment attributes specify a data center key. Later, in a wrapper cookbooks we examine the data center key and set the appropriate name servers and ntp servers (neither of which are managed by chef).

Closing thoughts

These cookbook patterns are not unique to our infrastructure. The wrapper cookbook is in some ways a limited form of the "application" cookbook pattern described in the latest Food Fight show hangout. Jamie Winsor from Riot Games has also mentioned role cookbooks as did Joseph Holsten on the mailing list.

One thing we have yet to try but I am really excited about is resource patching in the wrapper cookbook to modify resources defined in the original cookbook after it has been included. Bryan Berry put together the chef-rewind gem that makes it simple to patch the resources already defined. Joshua Timberman also demonstrated a slightly more raw way of modifying already defined resources. Both look like effective ways of keeping the line between vendored cookbooks and business specific cookbooks clear.