Be able to append a new dependency module on our final target

[dinosaure]

The workflow is simple: Be able to produce a *.ml file and adds it as a dependency required to produce our unikernel. Currently, the generated dune file produce (for Solo5):

(executable
 (enabled_if (= %{context_name} "solo5"))
 (name main)
 (modes (native exe))
 (libraries ...)
 (link_flags :standard -w -70 -color always -cclib "-z solo5-abi=hvt")
 (modules (:standard \ config manifest))
 (foreign_stubs (language c) (names manifest))
)

It will be nice to append my_module.ml into the modules field and let dune to infer how to produce it via deps arguments available from foreign. Currently, I will try to lookup into ~files argument and see if we can introspect it and put them into the modules field.

[samoht]

I am not sure to understand what you mean:

(modules (:standard \ config manifest))

means that dune will take all top-level modules apart from config.ml and manifest.ml. Could you clarify at which stage you want to take my_module.ml into account? Do you have a repro case where adding it to the build stage doesn't work?

[dinosaure]

means that dune will take all top-level modules apart from config.ml and manifest.ml.

Currently, if a module is produced by a script like:

(rule
 (target my_module.ml)
 (mode promote)
 (rule (with-stdout-to %{target} (run ...))))

And if you want to use this module into your unikernel, you must add it into the (modules ...) field to let dune to infer that it must produce it to make then the unikernel. Even if you explain how to produce my_module.ml, without adding it into the (modules ...) field leads to an error where dune is not able to find My_module.

[samoht]

But it should already added to the modules field. That's what :standard means: all the top-level modules.

Does it fail to be added only for generated files?

[dinosaure]

But it should already added to the modules field. That's what :standard means: all the top-level modules.

It's unfortunately not the case, I can find a minimal example where such design does not work.

Does it fail to be added only for generated files?

As far as I can say, yes, only generated files are missed by dune as dependencies to build the unikernel. Note that I tried (mode promote) and even that does not work.

[dinosaure]

This is a minimal example:

open Mirage

let my_module =
  let dune _info =
    [ Dune.stanzaf {dune|(rule
  (enabled_if (= %%{context_name} "default"))
  (target my_module.ml)
  (action (with-stdout-to %%{target} (bash "echo \"let str = \\\"Hello\"\\\""))))|dune} ] in
  impl ~dune "My_module" job

let unikernel =
  foreign "Unikernel.Make"
    ~deps:[ dep my_module ]
    (console @-> job)

let () =
  register "unikernel"
    [ unikernel $ default_console ]

module Make (Console : Mirage_console.S) = struct
  let log console fmt = Fmt.kstr (fun str -> Console.log console str) fmt

  let start console _my_module =
    log console "My_module.str: %S" My_module.str
end

Note that without the (enabled_if ..., this workflow works (or if the target is unix) but in the case where we need the host compiler to produce my_module.ml, this field is required (see #1323). The error is:

File "unikernel.ml", line 5, characters 36-49:
5 |     log console "My_module.str: %S" My_module.str
                                        ^^^^^^^^^^^^^
Error: Unbound module My_module

EDIT: and if I add my_module into the (modules ...) fieds, everything works for any targets.

[samoht]

So that seems to be a bug with the interaction between :standard and enabled_if?

[dinosaure]

I'm not sure, I already share by thought and be able to share between different contexts some files such as some compiled via the default context but needed to build our unikernel into our cross-compiler context. Should we make an issue on ocaml/dune?

[samoht]

Should we make an issue on ocaml/dune?

Yes, please! That definitely looks like a bug in dune. (having a small repro that doesn't involve the mirage tool would be super helpful)

[dinosaure]

NOTE: With the help of @TheLortex, it seems that we can transfer an artifact from the default context to our cross-compiler context via %{exe:...} which seems our current solution.