-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove local build+serve step from CI tests #1474
Comments
There's one snag in this that I realized while trying to figure out why this build keeps failing is that Circle doesn't enable our custom environment variables for pull requests from a fork. (This would be a security risk because a malicious forker could modify the test script to just Fork PRs will need to be exempt from automated browser tests because, even if we remove Sauce Connect from the equation, Federalist still doesn't build a preview for forks, so there wouldn't be a public URL to test. In the case of PRs from open source contributors, we can either:
Either way, we can detect a fork PR in our test script with something like: if [ "${CIRCLE_PROJECT_USERNAME}" != "${CIRCLE_PR_USERNAME}" ]; then
exit 1
fi Thoughts, @ultrasaurus or @yozlet? |
I like the idea of running unit tests always, then limiting browser tests to branches, then always test locally or push a branch before merging... @yozlet ? |
main thing is that I want to make sure we never have a false positive: #1471 and simplicity of the tests is a big plus |
👍, plus add an |
@barkimedes this is ready to go. You can see all of the gory details in #1496. |
I'm taking your word for this one. If you say this is done and the tests are passing, then I'm cool with closing the issue whenever you're cool with closing it. 🇺🇸 |
Sweet, closing! |
This will go a long way in addressing #1468 and possibly #1471. Currently, our CI test script builds the site and serves it locally, then establishes a secure tunnel with Sauce Labs so that it can test the site from
localhost
. This not only takes more time per build (setting up the Ruby environment, building the site, and waiting for the Sauce Connect tunnel), but also introduces a level of complexity that makes CI failures (and, apparently, false positives) difficult to debug.Removing the build and serve steps, and removing Sauce Connect from the equation, dramatically simplifies our build process. I've already figured out how to coordinate the timing between Circle CI and Federalist builds, and this approach should be easy to integrate.
The text was updated successfully, but these errors were encountered: