Updated with thanks to Bill Wang for his feedback and pull request.
I have recently experimented with using Test Kitchen and Neill Turner‘s Kitchen Ansible extension to set up automated testing for Ansible roles, and in this post I document the working configuration that I ended up with.
Acknowledgements go to Neill for writing all of these extensions, as well as Martin Etmajer of Dynatrace for his DevOpsDays presentation on this topic, and the Zufallsheld blog post that I’ve borrowed a graphic from.
Kitchen CI architecture
At a high level I have used an architecture that includes Test Kitchen with the kitchen-docker driver, the kitchen-ansible provisioner, which in turn calls Neill’s omnibus-ansible as well as his kitchen-verifier-serverspec test runner to run the Serverspec tests. Use of the kitchen-verifier-serverspec means that I am not dependent on busser runner and therefore have no need to have Chef Omnibus in the picture just to run the tests, which was the case in earlier incarnations of this stack.
How to set this up
I assume that we already have an Ansible role that we want to test, and in my case, I forked a Galaxy module for Sonatype Nexus by jhinrichsen, and added the Kitchen CI configuration. My code is available at Github here.
I assume we have installed the following:
- Ruby Gems
How to use and install these is out of scope for today, but here’s what I have before we start:
$ git --version
git version 2.5.4 (Apple Git-61)
$ docker -v
Docker version 1.11.1, build 5604cbe
$ ruby -v
ruby 2.0.0p481 (2014-05-08 revision 45883) [universal.x86_64-darwin14]
$ gem -v
$ bundler -v
Bundler version 1.10.5
In Parts I & II we have looked at setting up Puppet-syntax, Puppet-lint, Rspec-puppet, and Beaker. In today’s post we look at how to automatically run these tests in the Travis CI build pipeline.
Note that Travis CI is only freely available for open source projects. If it’s the paid-for service you need, consult their pricing plan, or perhaps look at other options like Jenkins.
I must thank Mark McKinstry for showing me how to do this when he sent in a pull request to add the Travis CI configuration to one of my own Puppet Modules. One of the wonderful things about maintaining open source projects is that people sometimes turn up randomly and teach you things, and that’s what happened here.
Anyhow, let’s begin.
Setting up Travis CI
Changes to Gemfile
One of the advantages of using Travis CI is that it is easy to test Puppet modules against a matrix of Ruby and Puppet versions. This is a great advantage when maintaining a Puppet Forge module, as it allows us to easily test and advertise the versions of Puppet that we support, and we can ensure that our module is also tested against the standard set of Ruby versions.
But in order to test against different Puppet versions, our Gemfile needs to be modified to accept the Puppet Gem version as a variable.
Here’s what we’re going to do. We’ll delete the line …
… and replace it with a block:
if puppetversion = ENV['PUPPET_GEM_VERSION']
gem 'puppet', puppetversion, :require => false
gem 'puppet', :require => false
In our previous post we covered the py_
puppetlabs_spec_helper, a kind of front-end to a bunch of tools including
Today we will be looking at how to set up the Beaker framework for module testing. Beaker is the Puppet community’s preferred acceptance test harness.
Before proceeding, I must call out some of the materials that I found useful when I learnt Beaker, especially Liam J. Bennett’s great three part series, Testing puppet with Beaker, Part 2: the Windows Story, and Part 3: testing roles. There is also a great 1-hour presentation by Puppet’s David Schmidt online here, which covers a lot of the material we look at in this post and in the previous post (the Beaker-related material starts at around 40 minutes).
Once again, my focus is on helping new people to quickly stand up the Beaker framework, while assuming as little as possible by way of Ruby and any other prior knowledge; I am less concerned about writing a proper tutorial on writing Beaker and Serverspec tests. And as before, I use the real life example throughout of the Spacewalk module we are developing.
Let’s dive right into it.
Installing and configuring Beaker
To get started, we’ll need to ensure we have our prerequisites installed. In addition to Ruby Gems and Bundler, which were discussed in Part I, we’ll also need to install Vagrant. I assume that my readers have used Vagrant before; if not, it’s well worth spending 15 minutes doing the Vagrant tutorial.
Otherwise, let’s continue.
As we discussed in Part I, the
Gemfile is a file that specifies Ruby dependencies to be installed in our ‘bundle’ by Bundler, a Ruby app that manages dependencies in Ruby projects.
To install Beaker, we will add a new system testing Gem group to the
group :system_tests do
gem 'beaker', :require => false
gem 'beaker-rspec', :require => false
gem 'beaker-puppet_install_helper', :require => false
It was brought to my attention that there’s demand for a post on how to set up Beaker from scratch. Then, after looking into it, I realised there’s a case for a whole series on how to set up Puppet modules as well as Puppet roles & profiles for testing.
In this series I am going to look at setting up all of the following components for Puppet module testing: Puppetlabs_spec_helper, Puppet-syntax, Puppet-lint, and Rspec-puppet (this post); Beaker for modules (part II); Travis CI (part III), Puppet Blacksmith and additional set up required for publishing your module on the Forge (part IV); and using ModuleSync to keep all of this set up in sync when you support many modules or code bases (part V).
My aim is not to provide tutorials on how to write Rspec or Rspec-puppet or Beaker tests; there are many of those out there already. My focus is simply how to set up the various frameworks, assuming no prior knowledge from the reader.
By way of example, we will look at adding the testing of the puppet-spacewalk module that my colleague is working on. As such, this is a real-life example.
It makes sense to begin with the
puppetlabs_spec_helper gem, a wrapper around quite a number of other tools, including:
And many others.
To install and configure up the
puppetlabs_spec_helper gem we need to firstly have RubyGems, and Bundler installed. This is well-documented in the links provided. Once these are installed you should find these commands in your path:
$ bundler -v
Bundler version 1.10.5
$ gem -v
Update: Thanks to Steven Bambling for pointing out that SquidMan is also available as a Homebrew Cask.
If you have used Beaker extensively for system testing your Puppet roles and profiles, you will have no doubt had some coffees while waiting for RPMs to download that you may well have downloaded before.
I was pleasantly surprised to find that setting up a Squid Cache using SquidMan on my Mac OS X Yosemite laptop and then having Beaker point at it was fairly straightforward. Still, there are a few gotchas to justify a blog post on the subject.
Thanks go to Alexander Rumyantsev for his post on Using squid to cache RedHat/CentOS yum repositories, and also to My Private Network for their post on Setting up Squid Man.
Installing and configuring SquidMan
I downloaded SquidMan 3.6 from here, and installed as with any other DMG file (although, to be sure, I had to manually drag and drop the app into my
Having started I went to its Preferences and entered the following config:
That is, I set the port to 3128, increased the maximum object size to 256MB in case I need to deal with large RPMs, and set the cache size to 4GB, and then I went to the Clients tab:
And here I allowed Beaker to connect from whatever network it happens to be on, i.e. all. (Limit that as your needs for security dictate.) (If you forget this step, Beaker will error out during a Yum install with a 403 Forbidden error.)
After starting Squid, you can find its config file using:
$ ps -ef |grep squid
501 2955 1 0 8:17pm ?? 0:03.64 /Applications/SquidMan.app/Contents/Ma
501 7283 1 0 8:28pm ?? 0:00.00 /usr/local/squid/sbin/squid -f /Users/
501 7285 7283 0 8:28pm ?? 0:00.08 (squid-1) -f /Users/alexharvey/Library
501 13310 96095 0 8:43pm ttys003 0:00.00 grep squid