building every revision

[mcr]

In order to facilitate git bisect, we have a requirement that every commit can be built.
It does not have to pass testing, but it has to compile. If it doesn't, then it's a problem.
We have a script that builds every revision since the last time we did a version tag, but it's unreasonably long winded, and just now, we ran out of compute time since we pushed a particularly long rebase to github.
This seems like something that travis could keep track of if we configured it to pay attention.
Maybe there is some way to do this that I've missed, or maybe it's a good feature request.

[BanzaiMan]

The only way to guarantee that "every commit is built" is to push every commit to GitHub. For example, if you push more than 3 tags simultaneously. https://developer.github.com/v3/activity/events/types/#createevent

[mcr]

Yes, so I could push every commit one at a time to github, but that won't give me any kind of real record that every single commit compiled. What would be ideal is if the build VM/container could wipe the source code and pull another commit... but remember which ones that it did, so that it never did it over and over again (which is what happens today).

[BanzaiMan]

…but that won't give me any kind of real record that every single commit compiled.

Why not? Each push will build that commit from scratch.

but remember which ones that it did, so that it never did it over and over again (which is what happens today).

What does this sentence mean? There are so many "it"'s here I am having a hard time understanding what you are saying.

[piraces]

I agree with @mcr. This could be an awesome feature for travis-ci in order to facilitate git bisect.

[mcr]

@BanzaiMan

1. If I push a number of commits, it won't build the intermediate commits. So I'd have to have some script that slowly pushed things to github (so as not to overwhelm travis). That would be annoying for collaboration.
   (expanding all the "it" references)
2. I would like travis be able to build a commit, record that the build succeeded or failed, and then go on to the next commit, and try that one. Travis should not attempt to build the same commit id twice: that is just wasted cycles.

In particular, failure to build every commit usually results in the developers either moving some code around, or re-ordering some commits with a rebase. Travis should not have to validate any of the commits that were before the rebase diverged.

[mcr]

What I'd need is some kind of API that a build could use to communicate with Travis from within the test environment, and could annotate commits, and also inquire as to whether the commit has been annotated.

[Traumflug]

Getting this isn't too complicated. This oneliner does it:

for C in {32..0..2}; do git push thirtybees testbranch~${C}:testbranch; sleep 360; done

• {32..0..2} creates a list, 32 30 28 26 24 24 22 20 18 16 14 12 10 8 6 4 2 0. For every single commit, use {32..0}.
• 32 is the length of the branch to push.
• thirtybees is the name of the remote repository. Often that's origin.
• sleep 360 waits the approximate build time. That's crucial. Builds not yet started when the next commit comes in get skipped, so the next commit should come in only after the previous one was at least started.
• git push <repo> testbranch~10:testbranch pushes the version 10 commits back from the local branch testbranch to the top of the remote branch testbranch. Rarely used, one can not only push branch tips, but arbitrary versions to the remote repository.
• As builds happen sequentially, this takes a while. 96 minutes in this case.
• Moving such a test branch to another/the main branch using fast-forward rebase/merge preserves all the intermediate builds. Only the tip will get built again, because Travis intentionally wants to build some versions twice, see Do not build the same SHA multiple times #2200

Caveat of this strategy: one can forget to use this oneliner. Obviously :-)