Erbot Cucumber

Erbot Cucumber is an Erbot based Cucumber test-suite runner…

Pretty much, it allows you to issue commands to a bot in an IRC channel and – when commanded, your robot slave will fire up an instance of Cucumber, spit out a URL to the IRC room that tells you where you can find the results, then proceed to BDD test the server autonomously.

When a test is complete, the bot pings the room again giving you a heads up when it is all finished.

You can run as many tests simultaneously as your resources allow.

A typical test command would likely look like the following:

<User> ,test staging
<Bot> Started. Results will populate at http://example.com/cucumber/staging.html

And then some time later:

<Bot> The test for staging is now complete... You may view results here:
      http://example.com/cucumber/staging.html

Requirements

This project assumes you have:

• A Unix-like operating system (Or, MinGW or Cygwin in your path)
• Emacs 24 or later (It will likely work on Emacs 23, but is untested)
• A working Erbot running in the above Emacs.
• The ability to run Cucumber tests on the box the bot runs on
• An HTTP server

Installation

All you need to do to install this is to load erbot-cucumber.el in the configuration file for your erbot-running version of Emacs.

So in your .emacs or init.el, you should have a line like:

(load "/path/to/erbot-cucumber.el")

(…obviously replacing /path/to/erbot-cucumber.el with the actual path of that file.)

After it is loaded, all you need to do is configure it!

Configuration

After erbot-cucumber has been loaded, you can configure it using cusomize-group on the erbot-cucumber customization group.

So that would be typing M-x customize-group, followed by ”erbot-cucumber” and hitting [Enter].

Inside of this customization group, there are a bunch of options with detailed descriptions that will allow you to modify how the bot runs.

To save and apply your changes, hit C-x C-s in the customize buffer.

Basic configuration

This configuration is required regardless of your testing requirements.

Erbot Cucumber Feature Location

This configuration tells the bot where your Cucumber test files are… This should point to a path that contains your feature files, so if you have a bunch of features in a place like /home/erbot/cucumber/features, you would need to point it to that.

Erbot Cucumber Use Feature Location P

This is a predicate that determines whether or not to invoke cucumber with the Erbot Cucumber Feature Location, this is here for those who want to configure the feature file usage manually, either by keeping them in the current working folder of the bot, or though a script.

It is easier to keep it on, but you can do more with it off. (Like create a “rerun” test that only reruns failed tests.)

Erbot Cucumber Output Path

This option is pretty important too, because it tells the bot where it should dump the results of the Cucumber test. This should be a path on the testing box that will be served by the web server at the configured Erbot Cucumber Test Result Root.

So, if you had a server serve /var/www/cucumber/ as ”http://staging.int.example.com/cucumber/”, you would want to set this to /var/www/cucumber/

You should make sure the bot has write permissions to this folder.

Erbot Cucumber Test Result Root

A URL that allows access to the configured Erbot Cucumber Output Path.

So, if you had a server serve /var/www/cucumber/ as ”http://staging.int.example.com/cucumber/”, you would want to set this to http://staging.int.example.com/cucumber/

When the bot runs for a server, it will create an output file with the server name in it, and return the file path as a URL.

For example, if you had the bot test a host configured as local:

• /var/www/cucumber/local.html would be created.
• http://staging.int.example.com/cucumber/local.html would be the link the bot spits out to the room to let you see the results.

Erbot Cucumber Exec

The command to invoke to run Cucumber. This can either be Cucumber itself with some specific command line arguments, or a script that runs it with some special requirements. (Like running it in Xpra on headless servers, or running it over SSH.)

Multiple hosts

If you want to be able to have the bot test multiple hosts, the following configuration is needed:

Erbot Cucumber Host Alist

This is where your hosts are configured.

By default there is one host called ”local” inside of a list that should look something like the following:

(local "http://127.0.0.1"
       :replace (("http://somehost" . "https://the-right-host")))

You can add or remove hosts with the ”INS” and ”DEL” buttons next to the host list.

Lets break down what each host consists of:

Name

The local you see at the beginning is the name of the host, this is the name you will refer to the host in the room… So to test this host you would invoke the bot like:

<User> ,test local

Address

This option is used to allow quick linking to the root of the project that is being tested so you can inspect it.

Think of it like a bookmark the bot can pull up for you.

Replace list

The relevant information for the replace list in the above example is as follows:

:replace (("http://somehost" . "https://the-right-host"))

This whole bit is optional if you don’t need it, so it is completely valid to have a host configured like:

(local "http://127.0.0.1")

What the replace list does is take every dotted pair, and replaces the first half before the dot with the second half after the dot in the configured Erbot Cucumber Host File.

You can have as many of these as you like… For example, if in your hosts file you have two variables defined for two servers that talk to each other like:

@root = "http://test.int.example.com"
@other_root = "http://testapi.int.example.com"

…and you wanted to test a second host where the @root is supposed to be ”https://example.com” and @other_root needs to be ”https://api.example.com”, you could configure this host like:

(prod "https://example.com"
      :replace (("http://test.int.example.com" . "https://example.com")
                ("http://testapi.int.example.com" . "https://api.example.com")))

Then when you invoked the bot like:

<User> ,test prod

It would test the right location.

Parameter list

Say you want to test something with a particular Cucumber tag, how do you do that? You can add a :parameter section to a host, which would look like so:

(prod "https://example.com"
      :replace (("http://test.int.example.com" . "https://example.com")
                ("http://testapi.int.example.com" . "https://api.example.com"))
      :parameters ("-f rerun" "--out failed-tests.txt"))

This would add the ”-f rerun” and ”--out failed-tests.txt” parameters to the end of the eventual Cucumber run.

Of course, this bit is optional as well.

Erbot Cucumber Host File

This is the file where you have your hosts configured for Cucumber globally so the above host replacing can work when you configure your hosts.

Erbot Cucumber Default Host

If you invoke the bot like:

<User> ,test prod

It will test the host configured as ”prod”. However, if you invoke it like this:

<User> ,test

It will test the host with the name configured here.

So, setting this value to ”hargleblargle” and invoking the bot like:

<User> ,test

Would be the same as if you invoked it like:

<User> ,test hargleblargle

Erbot Cucumber Reset Tree

When we perform host file string replacing, this changes a file in our Cucumber file tree…

When we test, we might also add files we don’t want into the file tree.

This configuration is a command, or bunch ‘o commands that will revert the Cucumber folder structure back to how it was before we invoked the bot, or reset to the most recent tests.

This could be simply copying over the hosts file with the old version, or it could be using git to reset the entire tree and pulling new changes.

Using the bot

Now that you have the bot up and running, the following is a list of commands you can use the bot:

test	Runs a test on a host
stop	Stops a test on a host
hosts	Lists the available hosts
host	Details info for a particular host
running	Lists the currently running tests

Allow me to describe these in greater detail:

test

This command is the heart of the bot’s functionality… This will begin a test on the specified host, or the default host if none are specified.

Example:

<User> ,test
<Bot> Started. Results will populate at http://example.com/cucumber/staging.html
<User> ,test prod
<Bot> Started. Results will populate at http://example.com/cucumber/prod.html

stop

This command stops a test that is currently running.

Example:

<User> ,test prod
<Bot> Started. Results will populate at http://example.com/cucumber/prod.html
<User> Oops
<User> ,stop prod
<Bot> The test for prod is now complete... You may view results here:
      http://example.com/cucumber/prod.html
<Bot> prod stopped...

hosts

Lists the currently available test hosts.

Example:

<User> ,hosts
<Bot> Hosts: staging prod super-serious-business
<User> ,test WomboCombo
<Bot> What's a WomboCombo?
<User> ,test super-serious-business
<Bot> Started. Results will populate at http://example.com/cucumber/super-serious-business.html

host

Lists information about a particular host.

Example:

<User> ,hosts
<Bot> Hosts: staging prod super-serious-business
<User> ,host super-serious-business
<Bot> super-serious-business is at: http://serious.example.com with test
      results that populate at: http://example.com/cucumber/super-serious-business.html

running

Lists the currently running tests.

Example:

<User> ,test staging
<Bot> Started. Results will populate at http://example.com/cucumber/staging.html
<User> ,running
<Bot> Currently running tests: staging URL: http://example.com/cucumber/staging.html
<User> ,stop prod
<Bot> Cannot find an active process for prod
<User> ,test staging
<Bot> There is already a test for staging ya dingus! Results:
      http://example.com/cucumber/staging.html
<User> ,test prod
<Bot> Started. Results will populate at http://example.com/cucumber/prod.html
<User> ,running
<Bot> Currently running tests: staging URL: http://example.com/cucumber/staging.html
      prod URL: http://example.com/cucumber/prod.html
<User> ,stop prod
<Bot> The test for prod is now complete... You may view results here:
      http://example.com/cucumber/prod.html
<Bot> prod stopped...
<User> ,running
<Bot> Currently running tests: staging URL: http://example.com/cucumber/staging.html

Bugs? Suggestions?

If you find any bugs or have any suggestions for the runner, feel free to post an issue in the issue tracker.