Basic automated smoke tests

[deathcap]

Perform a simple automated "smoke test" periodically, to ensure the project can be built and runs.

Ideally this test would run periodically even with no project code changes, since there are external dependencies which could break it, e.g., recently:

• New version of web browsers more strictly implementing WebGL: gl-shader: Error compling vertex shader: ERROR: 0:18: '=' : global variable initializers must be constant expressions voxel-shader#11 - test on latest each
• Dependent npm modules can be "unpublished": nonexistent dependency (esprima-six unpublished) #60 (esprima-six) - normally, npm versioning takes care of using older, compatible, versions of modules when newer versions are published, but the module owner can delete a module, making it unavailable indefinitely.
• Dependency changes - not usually a problem if projects follow semver and dependents adhere to API contract, but always a chance the interfaces could break down unexpectedly. (using "^x.y.z", as in most voxel projects, will pickup new minor and patch versions but not API-incompatible major versions).
• Other external changes?
• Code changes

[z3t0]

something like https://docs.travis-ci.com/user/languages/javascript-with-nodejs ?

[deathcap]

Sort of, but I'd want to test in each web browser. More like https://ci.testling.com/ but it says "Testling is currently not working" at the moment

[z3t0]

How about https://docs.travis-ci.com/user/gui-and-headless-browsers/ ?

[deathcap]

Thanks! https://docs.travis-ci.com/user/gui-and-headless-browsers/ looks to be just what I need.

• xvfb + "If you need a browser, Firefox is installed on all Travis CI environments." - only one browser, but it's preinstalled
• PhantomJS "headless WebKit with JavaScript API" - headless, but includes WebKit API
• Sauce Labs https://saucelabs.com/opensauce/ - free for open source projects, "170 different device/OS/browser combinations". Will sign up for an OSS account

[z3t0]

Awesome, I'd help but I'm not too familiar with travis (besides simpler implementations)

[deathcap]

Testing with Sauce Labs, OSS account voxelmetaverse, manual session:

Google Chrome dev on Windows 10 - no WebGL in their VMs? fails with "You need WebGL and Pointer Lock"
Google Chrome dev on Linux - same results
Google Chrome dev on OS X El Capitan - same results
Google Chrome 47 on OS X El Capitan - same results
Firefox dev on Linux - same results

https://twitter.com/saucelabs/status/418861018400313344 confirms lack of WebGL support is a known limitation of Sauce Labs:

@greweb @saucelabs any chance to have WebGL running in SauceLabs? doesn't seem to work https://saucelabs.com/tests/a064a932a9344fca8286e07fa246575e
@saucelabs @greweb we've added it as a feature request. stay tuned!

this was from January 2nd, 2014, seems its still not supported :/

Next up, see if Travis CI's installation of Firefox in a virtual X framebuffer (xvfb) can support WebGL

edit: looks like Chromium's with --use-gl=osmesa can run WebGL headless: http://stackoverflow.com/questions/13763325/is-there-a-way-to-run-unit-test-for-a-webgl-webapp-in-a-headless-browser