New lower-bounds check fails unexpectedly

[voodoos]

The new lower-bounds check (introduced in #727) fails on Merlin:
https://ci.ocamllabs.io/github/ocaml/merlin/commit/744801438d71718e828f00e8cee95140833c61aa/variant/(lint-lower-bounds)

This is due to the fact that ocaml-ci does not respect the build command provided in the .opam file which is:
["dune" "build" "-p" name "-j" jobs].

This command builds the project in release mode, and Merlin does not depend on Menhir when built in release mode.

But ocaml-ci runs:
dune build @install @check @runtest

Which does not build the project in release mode.

We can bump the menhir version to fix the issue, but it feels wrong for ocaml-ci to disrespect opam's build instruction.

This difference of behaviour between opam and ocaml-ci is also present for normal builds. What was the reason for this choice ?

[tmcgilchrist]

Thanks for the feedback @voodoos

The really short answer is we try to install a set of packages that satisfy all the .opam files within a git repository, rather than trying to find a set that satisfies some of the .opam files and we do not use the release mode when doing that. Why we do we not use release mode rather than dev? Because a dev build profile includes stricter warnings than release which is what I think you want in a lint step.

Trying to find a common set of packages has a known limitation (See #470) where depending on a newer compiler version like 5.0 stops ocaml-ci from finding package sets for earlier compilers. #470 is investigating how we can improve that.

As for which release mode that should be used in a lower bounds check lint, we err'd on including stricter checks but we can try using the release mode as per opam-repo-ci. The original motivation for the feature was to push the linting checks from opam-repo-ci earlier into the CI process for people.

[voodoos]

Thanks for your answer Tim!

I agree that building in dev mode for everyday CI is good because of the additional warnings.
But if the lower bound check aims at reproducing opam-repo-ci's check, (and that's a great idea to improve the quality of PRs to the opam repository), I think it would be best if it used the same deps and build mode as opam-repo-ci.

[benmandrew]

@voodoos thanks for the feedback, it's useful to see these cases.

While we wanted to model our check after that of opam-repo-ci for the reasons you mentioned, I think it’s better to maintain consistency with the rest of OCaml-CI and build in dev mode, as our goal is quite different from that of opam-repo-ci. OCaml-CI tries to aid the development process rather than being a check before deployment.