Chef Policyfile.rb `include_policy` doesn't guarantee immutability of policy dependency

Question

Let's say I have a Polcyfile.rb in a cookbook called motd:

    name 'motd'
    default_source :chef_repo, "../"
    include_policy "Policyfile", path: "../environment"
    run_list 'motd'

and a recipes/default.rb:

file '/etc/motd' do
  content node['message']
end

I have another cookbook called environment which has a Policyfile.rb:

name 'environment'
default_source :chef_repo, "../"
run_list 'environment'

It has an empty recipes/default.rb and attributes/default.rb with:

default['message'] = 'i am a message'

I run chef install Policyfile.rb in environment dir to generate the lock file. When I run kitchen converge from motd dir and then kitchen login, I get my expected output to console:

This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento
i am a message

Now I go and update environment/attributes/default.rb to be

default['message'] = 'i am updated'

I DO NOT run chef update Policyfile.rb for environment and run kitchen converge again from motd. My expectation is that kitchen login will not reflect my update because Policyfile.lock.json in motd has not updated its revision_id for the included environment policy. But much to my surprise I indeed see the updated message in the console. I do see that Policyfile.lock.json has a new root revision_id and that cookbook_locks->environment->identifier has changed. But still, I would think that in this case, if the cookbooks in my dependency Policyfile.rb have changed and don't compute to match the hash of its Policyfile.lock.json revision_id then I should still see the old output or there should be some kind of other warning here.

I guess I'm just trying to understand the concept here more fully. On the one hand, the root revision_id for motd changed so I have achieved idempotency in one sense. But on the other hand the revision_id for environment dependency and its component cookbook don't match. Can someone explain why this makes sense?

Answer 1

It seems like there is a bug or possibly a feature in the way that chef install interacts with test-kitchen. All that chef install does is validate against version numbers, and if you update the version of the cookbook without regenerating with a chef update then you'll get an exception:

Error: Failed to install cookbooks from lockfile
Reason: (CookbookOmnifetch::CookbookValidationFailure) The cookbook downloaded for Cookbook 'xxxxxxx' = 1.2.3 {:path=>"."} did not satisfy the constraint.

If you modify the cookbook, test-kitchen will run chef install and will the synchronize the cookbook files to the virtual instance you are building and will run chef-client against them, which means any local modifications are picked up automatically without a chef update needing to be applied.

This isn't how it happens in production settings, since you need to use chef push to upload the cookbooks to the cookbook_artifacts repository, which is versioned by the sha of the cookbook. Since the client pulls down the cookbook_artifact based on the sha in the lockfile this will fail during deployment. Since test-kichen doesn't use chef push it doesn't have the immutability features you'd see in deployments.

It could be argued this is a test-kitchen feature, otherwise test-kitchen would need to switch to always running chef update every single time by default, or forcing users to manually run chef update between runs. But keeping the lockfile reflecting production so that the workflow is to iterate on the cookbook and only regenerate the lockfile when it is time to push may be a workflow feature. If anyone feels that this is an actually harmful bug, you can open an issue against test-kitchen or chef-cli in order to get it fixed (but you'll need to argue compelling workflow features, not philosophy). The way that test-kitchen isn't immutable though doesn't have any bearing on the immutability of policfiles in production settings, though, which I think is what this question is about.

Answer 2

revision_id isn't something that freezes the dependency. It does not matter whether it is a cookbook in the run list or a dependency. Only cookbook version is used for locking.

Imagine you already have a lock file with for example your environment cookbook version and path to the cookbook set.

[...]
"cookbook_locks": {
  "environment": {
    [...]
    "source": "../environment"
  }
}
[...]
"solution_dependencies": {
  "Policyfile': [
    ["motd", "= 1.2.3"],
    ["environment", "= 2.7.8"]
  ]
}
[...]

Then if and only if you change the version of the cookbook in ../environment/metadata.rb you will get the error, stating that particular version of the cookbook is not found. In this case you have to regenerate the lock file.

Any other change to the cookbook does not trigger any error. And that is expected behaviour. Imagine, if you had to regenerate lock files for every change you make, that would be a lot of overhead.

Think of revision_id in policyfile as a version of policyfile. As there is no such concept as backwards compatibility for policyfiles, there is no need to use semantic versioning, and a long unique not so randomly generated string as a version is good enough in this case. And this is why chef show-policy will show you the revisions of policyfiles applied in particular policy groups, to show that this version is deployed to this group.