-
Notifications
You must be signed in to change notification settings - Fork 2
FAQ
As your site grows in size and complexity, your risk of regression dramatically increases. A tiny adjustment to a margin or background may fix the page you're working on but ends up breaking 5 other pages that you didn't even open to test... maybe you didn't even know they existed!
Checking every page is too much work, and greatly slows down a developer's productivity, so web developers have historically been content with leaving the work of finding these bugs in the hands of a QA team, because "it's their job to test everything." Worse, many companies don't even have a QA team, so these bugs find their way into the wild where they live far too long.
Automation changes that. When you have a test suite, you can simply run it after your change and have the confidence that your changes only affected the area(s) you expected it to. This can lift the burden from QA teams and improve developer work quality. And for development teams that have no QA department, this is an absolutely essential tool to have.
Earliest versions of Specter were indeed built with PhantomJS, but in the end it became necessary to move to a more reliable browser. PhantomJS has many rendering bugs where system controls are concerned. Radio buttons, checkboxes, and drop-down menus render in strange, unpredictable ways, often quite far from where one would expect to find them. The Phantom team has expressed no intention of fixing this, which is understandable, because PhantomJS was always intended to be a headless browser, so little oddities in rendering shouldn't matter. But for a visual testing tool, they matter very much!
The best place to start, then, is with a browser that is known to render correctly every time. None has ever been better than Firefox. Also, Mozilla provides several additional capabilities for XUL apps, so this made Firefox a very natural choice.
As a XUL app, Specter can be compiled into a tiny package, whose only external dependency is either Firefox or XULRunner, and which can be run on any platform.
Both Chrome and IE are very scriptable, so there's nothing stopping anyone from standing up a version of Specter for either of these browsers. If and when they do, I will be happy to merge that into this project, but I personally have no interest in doing that work. (In particular, I avoid any activity that makes it necessary for me to use Windows.) But if you are reading this and you have interest in working on adding support for other browsers, please contact me so that I can help you get started and we can co-ordinate making something that merges seamlessly into this project!
PhantomCSS (created by James Cryer) brings with it a lot of additional requirements and complexity. PhantomCSS requires CasperJS which in turn requires PhantomJS. As discussed above, PhantomJS turns out to be a bad choice for a visual regression testing tool.
But CasperJS also turns out to be a very heavy toolset when your goal is just to test visual regressions, as it is really meant to be a full-blown functional testing suite, so it's a lot bigger than what is needed for just comparing screenshots. It also imposes some incredibly verbose syntax on its users. Writing tests for a single page of a site in three different resolutions often required 50-80 lines of code with PhantomCSS/CasperJS, whereas in Specter I can now write that same test in less than 10 lines.