dune exec --no-build is slow

[Khady]

Expected Behavior

dune exec --no-build or dune exec when there's nothing to build should be almost as fast as launched the binary manually

Actual Behavior

Even with --no-build and a path to the binary (no need to look up for a name), dune can be 40 times slower than just launching the bin by hand

$ time dune exec ./bin/main.exe --no-build -- --version

real	0m0.655s
user	0m0.474s
sys	0m0.170s
$ time _build/default/bin/main.exe --version

real	0m0.015s
user	0m0.007s
sys	0m0.007s

The cost seems to grow higher as the binary and project gets bigger. strace shows A LOT of activity.

Reproduction

Can't open source the specific case, but any open source project can highlight this behavior

git clone https://github.com/ahrefs/atd.git
cd atd
# setup ocam libs
time dune exec --no-build atdgen/bin/ag_main.exe -- -version
time _build/default/atdgen/bin/ag_main.exe -version

Specifications

• Version of dune (output of dune --version): 3.15
• Version of ocaml (output of ocamlc --version): 4.14.0
• Operating system (distribution and version): debian bullseye

[rgrinberg]

I don't know if there's much we can do. We need to load the rules in a directory to see if the executable exists. That's probably where the overhead comes from.

[Khady]

Couldn’t we have some shortcuts, such as checking the presence of the binary in _build before to load all the rules?

[rgrinberg]

It wouldn't be a valid optimization. The rules can change between runs and the stale artifact would remain.

[Khady]

Which would happen with --no-build anyway?

[rgrinberg]

It would not. Re-loading the rules will clear stale artifacts.