Packer provisioning for warden compatible boxes

[sykesm]

This includes a packer template and provisioning scripts to create warden-compatible boxes for vmware and virtual box. The box consists of:

• Ubuntu 12.04.4 and the latest kernel
• cgroup memory limits and accounting as required for 12.04.4
• kernel hold to prevent automatic updates (avoids VMware tools issues)
• warden-test-infrastructure provisioning via the existing project cookbooks

This is a starting point and needs some cleanup but it would be great to get some input on what's here already in support of https://www.pivotaltracker.com/story/show/68302420. I've used this box for testing warden and the dea without issue.

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: http://www.pivotaltracker.com/story/show/69539058. This repo is managed by the 'Runtime' team.

[chou]

@sykesm This looks good; we'll get someone from the runtime team to take a look.

CF Community Pair (@chou & @jfoley)

/cc @MarkKropf, @hiremaga

[hiremaga]

This looks great! Thanks @sykesm

If the warden and dea tests pass with this (i.e. we're able to create a valid warden-compatible.box), I'd love for us to switch to packer.

[hiremaga]

We took a little at this earlier today. We started by looking at Virtualbox and ran into problems so we didn't get as far as looking into VMware Fusion.

We were able to create a box successfully, but when trying to bring this up using the DEA's Vagrantfile we ran into this error:

○ → vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'warden-compatible'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: dea_next_default_1397771232083_13060
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection timeout. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
    default: /vagrant => /Users/pivotal/workspace/cf-release/src/dea_next
Failed to mount folders in Linux guest. This is usually because
the "vboxsf" file system is not available. Please verify that
the guest additions are properly installed in the guest and
can work properly. The command attempted was:

mount -t vboxsf -o uid=`id -u vagrant`,gid=`getent group vagrant | cut -d: -f3` /vagrant /vagrant
mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant` /vagrant /vagrant

Any idea why we might see this? We were running Virtualbox 4.3.10 and Vagrant 1.5.3

[sykesm]

I don't know why you're seeing that. Here are the steps I went through just now without encountering that issue:

Build the boxes

[sykesm@imac-en0 ~/packer/warden-test-infrastructure/packer]$ make
...
Build 'virtualbox' finished.

==> Builds finished. The artifacts of successful builds are:
--> virtualbox: VM files in directory: virtualbox/output
--> virtualbox: 'virtualbox' provider box: virtualbox/warden-compatible.box

Remove the existing box

[sykesm@imac-en0 ~/packer/warden-test-infrastructure/packer]$ vagrant box list
warden-compatible   (virtualbox, 0)
[sykesm@imac-en0 ~/packer/warden-test-infrastructure/packer]$ vagrant box remove warden-compatible
Removing box 'warden-compatible' with provider 'virtualbox'...

Add the new box

[sykesm@imac-en0 ~/packer/warden-test-infrastructure/packer]$ vagrant box add --name warden-compatible virtualbox/warden-compatible.box 
==> box: Adding box 'warden-compatible' (v0) for provider: 
    box: Downloading: file:///Users/sykesm/packer/warden-test-infrastructure/packer/virtualbox/warden-compatible.box
==> box: Successfully added box 'warden-compatible' (v0) for 'virtualbox'!

Bring up the vagrant box

[sykesm@imac-en0 ~/packer/dea_ng]$ cat Vagrantfile 
Vagrant.configure("2") do |config|
  # Build this box by running `rake test_vm`
  config.vm.box = "warden-compatible"
  config.vm.box_url = "https://s3.amazonaws.com/runtime-artifacts/warden-compatible.box"
  config.ssh.username = "vagrant"

  config.vm.provision "shell", inline: "sudo apt-get -q -y install libxslt-dev libxml2-dev" # For Nokogiri
  config.vm.provision "shell", inline: "sudo apt-get -q -y install libcurl4-gnutls-dev" # For
end
[sykesm@imac-en0 ~/packer/dea_ng]$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'warden-compatible'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: dea_ng_default_1397781008473_49627
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection timeout. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
    default: /vagrant => /Users/sykesm/packer/dea_ng
==> default: Running provisioner: shell...
    default: Running: inline script
...

SSH into the vagrant box and dump the mounts

[sykesm@imac-en0 ~/packer/dea_ng]$ vagrant ssh
Welcome to Ubuntu 12.04.4 LTS (GNU/Linux 3.11.0-15-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

Last login: Fri Apr 18 00:08:41 2014 from 10.0.2.2
vagrant@warden-compatible:~$ mount
/dev/mapper/warden--compatible--vg-root on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sda1 on /boot type ext2 (rw)
/vagrant on /vagrant type vboxsf (uid=900,gid=900,rw)
vagrant@warden-compatible:~$

This was done with the following levels:

[sykesm@imac-en0 ~/packer/dea_ng]$ VBoxManage --version
4.3.6r91406
[sykesm@imac-en0 ~/packer/dea_ng]$ vagrant --version
Vagrant 1.5.2

While they're a little bit older than what you're using, they don't seem like they're ancient.

[sykesm]

Looks like it's a bug with VirtualBox 4.3.10's guest additions iso. I found hashicorp/vagrant#3341 which points to https://www.virtualbox.org/ticket/12879. You might want to try this with 4.3.8 instead.

[shalako]

@sykesm I'm getting a make error running the create script.

ruby 1.9.3p484 mason in ~/workspace/warden-test-infrastructure
±  |master ✓| → ./create_vagrant_box.sh
...
+ cd packer
+ box_name=warden-compatible
+ provider=virtualbox
+ rm -f virtualbox/warden-compatible.box
+ make virtualbox/warden-compatible.box
rm -rf virtualbox/output
packer build -only=virtualbox warden-compatible.json
make: packer: No such file or directory
make: *** [virtualbox/warden-compatible.box] Error 1

[sykesm]

@shalako Do you have packer installed already? The error implies packer isn't found on your path. If you just try to run packer, do you get a help string or a message like -bash: packer: command not found?

Assuming it's not installed, if you're using home-brew, installing should be as easy as brew install packer.

[shalako]

@sykesm Thank you. The script ran, and generated artifacts in packer/virtualbox (which I will try running with vagrant and run CATs).

However, nothing was generated in packer/vmware.

[sykesm]

I just looked at the script that was put together after my PR was merged. The way it was written, it defaults to virtualbox as the provider but you can override it to vmware by adding a command line argument.

create_vagrant_box.sh vmware

That should build you a vmware image. (You'll need to make sure that the linux tools iso has already been downloaded by fusion.)

[shalako]

successfully ran dea_next/test_in_vm and warden/test_in_vm using warden-compatible virtualbox created with packer.

Will try building the vmware warden-compatible box next.

[shalako]

@sykesm As you predicted, it doesn't appear I have VMWare linux tools installed. I'm reading that you have to install them with a VM. Sounds like a chicken or egg problem. I can't build a vm without them, but I need a VM to install them. I'll keep searching; let me know if you can provide guidance.

± |master ✓| → ./create_vagrant_box.sh vmware
Build 'vmware' errored: Couldn't find VMware tools for 'linux'! VMware often downloads these
tools on-demand. However, to do this, you need to create a fake VM
of the proper type then click the 'install tools' option in the
VMware GUI.

[sykesm]

@shalako Yes, that's an unfortunate issue that happens with VMware Fusion. I don't know why they went that way.

One quick way to get the tools is to bring up bosh-lite with the fusion provider, then start the Fusion UI. If you start the UI after bringing up the VM, you should see it in your VM library. From there you can use Virtual Machine -> Update VMware Tools to force Fusion to download the iso into the library.

It would be nice if there was a way do that from the command line. If a way exists, I don't know what it is.

[shalako]

I think I got vmware tools installed. Create script appears to be running for the vmware box.