Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize Circle CI builds #1468

Open
shawnbot opened this issue Feb 4, 2016 · 1 comment
Open

Parallelize Circle CI builds #1468

shawnbot opened this issue Feb 4, 2016 · 1 comment

Comments

@shawnbot
Copy link
Contributor

shawnbot commented Feb 4, 2016

We can apparently parallelize some of our tests on Circle CI. If this is possible (and relatively straightforward), we should do it! At the very least our unit, browser, and a11y test suites could run in parallel and probably reduce the build time by a couple of minutes. This would in theory speed up the rate of development and help people catch bugs more quickly.

@shawnbot
Copy link
Contributor Author

shawnbot commented Feb 4, 2016

One thing that will be really tricky if we decide to do this is parallelizing the Sauce Connect setup. Right now spinning up an HTTP server on localhost and making it available to Sauce Labs via a secure tunnel, and each build gets a unique tunnel ID so that Sauce tests the right code. Multiple containers probably can't use the same tunnel ID, so we'd have to ensure that only one container runs the HTTP server and creates the tunnel, which the parallelized containers would use.

The alternative is an approach that I've also considered for EITI (cc @gemfarmer), in which we skip the Jekyll build and HTTP server on CI altogether and run our CI tests against the public Federalist preview URL for each branch. The only tricky thing about this approach is the timing: Federalist and Circle get the github webhook at the same time, and if Federalist takes longer to build the site than Circle does to spin up a build container, then the tests will run against an older (or non-existent, in the case of the a first branch push) version of the site.

Now that Federalist gives us commit info in the site object, though, we can have our CI build wait for Federalist to finish by requesting a file containing the commit SHA:

---
---
{{ site.commit.commit }}

against which we would compare the $CIRCLE_SHA1 environment variable and exit once they match.

Site builds (not test builds) might be slower on Federalist than they are on Circle, but this could reduce a lot of the CI complexity. cc @yozlet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants