In yesterday’s post I showed how you can compile a Puppet catalog from a bundle on a laptop.  Today I’m going to show how you can use Zack Smith’s catalog diff tool to assist with complex refactoring changes.

Code examples

For the purpose of describing how to use the catalog diff tool, it will be better to use an artificially simple code example.  Imagine we have all of our code in site.pp as follows:

node 'myhost.example.com' {
  file { '/tmp/myfile':
    ensure  => file,
    content => "My hostname is ${::hostname}\n",
  }
}

We will refactor this by:

  • creating a class and moving the file resource inside that class
  • adding some additional attributes to the file resource

And the code:

class myfile {
  file { '/tmp/myfile':
    ensure  => file,
    content => "My content is ${::hostname}\n",
    owner   => 'root',
    group   => 'root',
    mode    => '0644',
  }
}

node 'myhost.example.com' {
  include ::myfile
}

Now neither of these code examples are “correct” as far as Puppet best practices are concerned of course, but for the purposes of our demonstration, this shouldn’t be an issue.

About my bundle

If you would like to follow along, you’ll need to know that my bundle is very simple, with only the following files:

$ tree myproject/
myproject/
├── Gemfile
├── Gemfile.lock
└── manifests
    └── site.pp

1 directory, 3 files
$ cat myproject/Gemfile
source "https://rubygems.org"
path ".bundle"
gem "puppet", "3.3.1"

(Why am I using Puppet v3.3.1 I hear you ask? All you need to know is that there’s no good reason for this!)

$ cat myproject/.bundle/config 
---
BUNDLE_PATH: .gems
BUNDLE_BIN: .bin

Installing puppet-catalog-diff

Normally, you would install puppet-catalog-diff on your Puppet Master, but as with yesterday’s post, I find it more useful to be able to run all of this from my laptop in a bundle.  So I have chosen to simply clone the Github project to the directory on my laptop where I have all my git projects:

$ cd ~/git
$ git clone https://github.com/acidprime/puppet-catalog-diff.git

The documentation for the project is awesomely long and complicated, no doubt a function of the many amazing things it can do.

The only points I care about are:

  • The tool is delivered as a Puppet Face
  • You’ll therefore need to add its lib directory in $RUBYLIB for the Face to be available.

Again, I assume we have Puppet installed in a bundle.  I assume we have the bundle’s bin directory in $PATH and I assume we are in the root directory of the bundle.  So I add the catalog-diff tool’s lib directory to $RUBYLIB:

$ cd myproject/
$ export RUBYLIB=../puppet-catalog-diff/lib

We should now be able to access the ‘diff’ subcommand of the ‘puppet catalog’ face.

$ puppet help catalog
...
ACTIONS:
...
  diff        Compare catalogs from different puppet versions.

Note that the help for the ‘diff’ subcommand is a little misleading. While the original motivation for the tool may have been to compare catalogs from different puppet versions, it is equally useful to our task of comparing catalogs from before and after a refactor. So ‘puppet catalog diff’ really just allows you to do diff two Puppet catalogs.

We should also look at the help for the diff subcommand, and I show in this excerpt the only option I care about:

$ puppet help catalog diff
...
  --show_resource_diff           - Display differeces between resources in
                                   unified diff format

Being able to see the diffs between resources is certainly important if you’re refactoring code.

Compiling the catalogs

For more detail on how to compile the catalogs, refer to yesterday’s post.

Our YAML facts file today needs only two fact values:

$ cat ~/.puppet/var/yaml/facts/myhost.example.com.yaml
--- !ruby/object:Puppet::Node::Facts
values:
  hostname: myhost
  domain: example.com

A reminder that we set the $PATH:

$ export PATH=.bin:$PATH

And then the command to compile the catalog:

$ puppet master --manifestdir=manifests --compile myhost.example.com | sed 1d > myhost_before.json

At this point I would refactor the code, as described in the ‘Code examples’ section above, and then create the second catalog:

$ puppet master --manifestdir=manifests --compile myhost.example.com | sed 1d > myhost_after.json

Diff’ing the two catalogs

We are now ready to run the diff:

$ puppet catalog diff --show_resource_diff myhost_before.json myhost_after.json

--------------------------------------------------------------------------------
myhost_after                                                               22.5% 
--------------------------------------------------------------------------------
Old version:    1451635325
New version:    1451652391
Total resources in old: 4
Total resources in new: 5
Only in new:
        class[Myfile]
Differences as diff:
        File[/tmp/myfile]
 ",
                        }
             ensure => "file"
+            group => "root"
+            mode => "0644"
+            owner => "root"
        }
Params in old:
        File[/tmp/myfile]:

Params in new:
        File[/tmp/myfile]:
        owner = root
        group = root
        mode = 0644
Catalag percentage added:       20.00
Catalog percentage removed:     0.00
Catalog percentage changed:     25.00
Added and removed resources:    +1 / 0
Node percentage:        22.5

--------------------------------------------------------------------------------
1 out of 1 nodes changed.                                                  22.5% 
--------------------------------------------------------------------------------

Nodes with the most changes by percent changed:
1. myhost_after                                                           22.50%

Nodes with the most changes by differeces:
1. myhost_after                                                           2false

So we can easily see that we have added a new resource, Class[Myfile]. Both catalogs contain the resource File[/tmp/myfile], and we can see they differ, as expected, by the additional attributes we have passed to them.

So that’s it. Have fun.