Skip to content

Commit

Permalink
updated to version 0.3
Browse files Browse the repository at this point in the history
  • Loading branch information
@letsgetrandy
letsgetrandy committed Oct 17, 2013
1 parent a91c8ee commit 9fa22b5
Show file tree
Hide file tree
Showing 38 changed files with 7,303 additions and 912 deletions.
4 changes: 4 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
.DS_Store
.bowerrc
contrib
diffs
screenshots
10 changes: 4 additions & 6 deletions .specterrc
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
[paths]
baseline = screenshots
#testroot = demo
#diff = /tmp/specter/diff
#fail = /tmp/specter/fail
#path = "/path/to/specter/install/dir"
diffdir = diffs

[args]
#specter = "args to pass to specter/casper"
#phantom = "args to pass to phantomjs"

#debug
#notests
#verbose
39 changes: 20 additions & 19 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ srcdir = $(CURDIR)
found_location = $(shell which specter)
found_version = $(shell specter --version)
expected_location = $(bindir)/specter
expected_version = 0.2
expected_version = 0.3


all:
@echo "Nothing to do.\n"
@echo "Did you mean to run 'make install'?\n"
xul: bin/omni.ja bin/application.ini bin/specter
@echo "done."
@echo "run 'sudo make install' to install"

install: PHANTOMJS CASPERJS PYTHON $(bindir)/specter installcheck update
install: $(bindir)/specter installcheck update

installcheck: SPECTER-exists
@echo "Checking for specter in the path...\c"
Expand All @@ -24,17 +24,23 @@ else
@echo "Specter was found in the path, but not in the expected location"
@echo "Expected v$(expected_version) in $(expected_location)"
@echo "Found v$(found_version) in $(found_location)"
@echo "If you wish to upgrade, run 'sudo make upgrade'."
endif

uninstall:
rm $(srcdir)/specter
#rm $(srcdir)/specter
@rm $(bindir)/specter

update:
@echo "Updating .specterrc files"
@find ~ -type f -name '.specterrc' -print0 | xargs -0 python $(srcdir)/bin/update_rc_files.py

upgrade: uninstall install

clean:
@echo "Nothing to clean."
@rm bin/specter
@rm bin/omni.ja
@rm bin/application.ini

$(bindir)/specter:
@ln -s $(srcdir)/bin/specter $(bindir)/specter
Expand All @@ -45,20 +51,15 @@ SPECTER-exists:
@which specter > /dev/null
@echo "OK"

PHANTOMJS:
@echo "Checking for PhantomJS...\c"
@which phantomjs > /dev/null
@echo "OK"
bin/application.ini:
@cp src/application.ini bin/.

CASPERJS:
@echo "Checking for CasperJS...\c"
@which casperjs > /dev/null
@echo "OK"
bin/omni.ja:
@cd src; \
zip -r ../bin/omni.ja chrome defaults chrome.manifest

PYTHON:
@echo "Checking for Python...\c"
@which python > /dev/null
@echo "OK"
bin/specter:
@cp src/specter bin/.

.PHONY : all install uninstall installcheck clean SPECTER-exists \
PHANTOMJS CASPERJS PYTHON
132 changes: 51 additions & 81 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,21 +1,35 @@
Specter
=======

Automated visual regression testing.
Specter is the automated visual regression testing framework that you've been
wishing for.

Specter adds [Resemble.js](http://huddle.github.com/Resemble.js/) image comparison functionality to [CasperJS](http://github.com/n1k0/casperjs) testing and navigation framework for [PhantomJS](http://github.com/ariya/phantomjs/), in order to produce automated visual regression testing.
For most other forms of software development there exist unit testing
frameworks with which a developer can verify that new code does nothing to
break existing code, but for the front-end (UI) developer, there are very few
tools to test design and layout, and the ones that do exist are terribly
clumsy. This has always resulted in a huge need for heavy manual testing and
QA after changes are made, to ensure that a change in one place doesn't break
something somewhere else.

Specter aims to change that. After you've built a page and you know it's right
you can set up a Specter test to prove that page, and each of its individual
components are correctly rendered. Specter will capture screenshots of the
elements matching the selectors you specify, at all the screen dimensions you
desire. Then, just run your suite of Specter tests the next time you make a
change to ensure you haven't broken anything. If something _does_ break, you'll
get a nice screenshot with the differences highlighted in red.

### Why?

The problem with functional UI tests is that they make assertions on HTML markup, not the actual rendering. You can't know through automated tests if something has visually broke, too much margin, disabled state etc. This situation is exacerbated by the increasing use of CSS3 for visual state changes that were traditionally built with JavaScript and DOM manipulation, ':target' pseudoclass or keyframes for example. Read more on Huddle's Engineering blog: [CSS Regression Testing](http://tldr.huddle.com/blog/css-testing/).
### Requirements

Specter is a XUL application, so it needs Firefox or XULRunner in order to work.

## Installation

Specter requires [CasperJS](http://github.com/n1k0/casperjs), which can be installed on Mac via Homebrew (`brew install casperjs`) or installed manually on any other system by cloning the repo.
## Installation

To use Specter, just clone this repo and run `make install` to link the binary into your path and make it executable.
To use Specter, just clone this repo and run `make install` to link the binary
into your path and make it executable.

```
$ git clone https://github.com/letsgetrandy/specter.git
Expand All @@ -27,43 +41,21 @@ $ make install

Check out the [demo](http://github.com/letsgetrandy/specter/tree/master/demo) for a full working example (run `specter demo` from the command line).

### Getting screenshots

Specter adds a global `specter` object to your casper tests, with a `screenshot()` method which requires two parameters:
### Quick Start

```javascript
specter.screenshot(css_selector, file_name);

// css_selector: String. A selector to define what you want to capture
// file_name : String. A descriptor to use when saving the screenshot
```

Just write your [casper functional tests](http://casperjs.org/testing.html) as normal, but have specter capture screenshots as you go.
Create your test file(s). Eg:

```javascript
/* global casper:false */

casper
.start('http://localhost/', function () {
casper.viewport(1024, 600);
specter.turn_off_animations();
specter.screenshot("main > .content", "main");
this.test.assertExists('form#login #id_username', 'username field');
this.test.assertExists('form#login #id_password', 'password field');
})
.thenOpen('http://localhost/password-reset/', function () {
specter.screenshot("main > .content", "password-reset");
})
.run(function(){
this.test.done(0);
open("http://www.google.com/", function() {

test([960, 640, 320], function() {
capture("", "");
});
});
```

### Comparing

Specter will compare the after the functional tests have finished. Just invoke your tests by calling `specter` rather than `casperjs`.

Specter can run a single test file:
Then, invoke specter on your tests. Specter can run a single test file:

```bash
$ specter tests/test-login.js
Expand All @@ -82,22 +74,20 @@ $ specter tests
```


### Options

**--no-color** disables Casper's hideous command-line color scheme
### Baselines

**--ignore-failed-tests** forces Specter to run image comparisons even if the functional tests finished with errors.
The first time Specter captures a selector for a particular test, there is
nothing to compare it to, so that capture becomes the _baseline_. After that,
any time that selector is captured for that test, it will be compared to the
baseline.

**--skip-capture** skips the functional tests and image capture step, and only runs the comparison step against previously collected diffs.
When you make an intentional change, you'll wish to update the baseline. You
do this with the `--rebase` option.


## Workflow

* Define what screenshots you need in your regular tests
* Find the screenshot directory and check that they look as you expect. These images will be used as a baseline. Subsequent test runs will report if the latest screenshot is different to the baseline
* Commit/push these baseline images with your normal tests (presuming you're using a version control system)
* Run the tests again. New screenshots will be created to compare against the baseline image. These new images can be ignored, they will be replaced every test run. They don't need to be committed
* If there are test failures, image diffs will be generated.
```bash
# update the baseline for the homepage tests
specter --rebase tests/test-homepage.js
```


## Configuration
Expand All @@ -106,57 +96,37 @@ Specter follows standard .rc file behavior.

* Global defaults can be set in `/etc/specterrc`
* User-level settings can be set in `~/.specterrc`
* Project-level settings can be set within the project in a `.specterrc` file in the current working directory, or any directory above it.
* Project-level settings can be set within the project in a `.specterrc` file
in the current working directory, or any directory above it.

```bash
$ cat .specterrc

[paths]
testroot = static/styles/tests
baseline = static/styles/screenshots
diff = /tmp/specter/diff
fail = /tmp/specter/fail

[args]
specter = "--args --to --pass --to --specter/casper"
phantom = "--args --to --pass --to --phantomjs"
diffdir = /tmp/specter
```

**paths.testroot** is the folder that contains all the test files to run

**paths.baseline** is the directory in which baseline screen captures should be stored

**paths.diff** is where diff images are stored for comparison later
**testroot** is the folder that contains all the test files to run

**paths.fail** is where failure images are stored
**baseline** is the directory in which baseline screen captures should be stored

**args.specter** are command-line arguments to always pass to specter and/or casper
**diffdir** is where diff images are stored for comparison later

**args.phantom** are command-line arguments to always pass to phantomjs


_Please note that relative paths in .specterrc files are relative to that file._

_v-0.3 note: baselines made with v-0.2 are not compatible with v-0.3.
_
_v-0.2 note: the "specter" section in the rc files has been replaced by "paths" and "args" sections._


## Demo

Run all demo tests simply by passing the demo directory as an argument:

```
$ specter demo
```

Or, run a single demo test by passing just that file as an argument:

```
$ specter demo/test-facebook.js
```


--------------------------------------

Maintained by [Randy Hunt](http://github.com/letsgetrandy)
Created and maintained by [Randy Hunt](http://github.com/letsgetrandy)

Specter is based on PhantomCSS, originally created by [James Cryer](http://github.com/jamescryer) and the Huddle development team.
Portions of Specter were inspired by [PhantomCSS](https://github.com/Huddle/PhantomCSS),
and by [SlimerJS](https://github.com/laurentj/slimerjs).

0 comments on commit 9fa22b5

Please sign in to comment.