reduce opam complexity: mirage-protocols and mirage-stack

[hannesm]

in our last meeting (//cc @mirage/core) we discussed that the mirage ecosystem and packages are slightly complex, since the interface and implementation -- even if we only have a single one -- are disconnected in separate opam packages (and git repositories). one path is to move everything to a monorepo. another path is to move the signatures back to the implementations and cut major releases (this will of course make the monorepo approach still doable).

To start this off, I spend some time on mirage-protocols and mirage-stack -- moving them back to the respective repositories. This turned out pretty nicely, and touches only some packages (this meta-issue is meant to track the PRs with the goal to merge and release them):

• include Mirage_protocols.ETHERNET directly in ethernet ethernet#8 moves Mirage_protocols.ETHERNET back into mirage/ethernet. [new release] ethernet (3.0.0) ocaml/opam-repository#20190
• No mirage protocols arp#27 (depends on ethernet) [new release] arp (3.0.0) ocaml/opam-repository#20199
• move Mirage_protocols and Mirage_stack into tcpip directly mirage-tcpip#463 (depends on arp and ethernet) [new release] tcpip (7.0.0) ocaml/opam-repository#20201
• avoid mirage-protocols charrua#116 (depends on arp, ethernet, tcpip) [new release] charrua, charrua-unix, charrua-server and charrua-client (1.5.0) ocaml/opam-repository#20254
• Deprecate mirage-protocols#30 [new release] mirage-protocols (8.0.0) ocaml/opam-repository#20255
• Deprecated mirage-stack#21 [new release] mirage-stack (4.0.0) ocaml/opam-repository#20257
• update to recent ethernet and tcpip mirage-nat#46 [new release] mirage-nat (2.2.5) ocaml/opam-repository#20258
• mirage 3: update arp, ethernet, tcpip versions #1259 (and update arp, ethernet, tcpip versions #1260) [new release] mirage, mirage-types, mirage-types-lwt and mirage-runtime (3.10.8) ocaml/opam-repository#20279
• mirage-skeleton adapt unikernels (ping6, dhcp) to new tcpip and ethernet interfaces mirage-skeleton#320 (and mirage-dev: adapt unikernels (ping6, dhcp) to new tcpip and ethernet interfaces mirage-skeleton#321)

[hannesm]

The plan for smooth transition is:

• get the code into place (remove dependencies on miage-protocols & mirage-stack in ethernet, arp, tcpip, charrua)
• release mirage-protocols and mirage-stack that are completely deprecated
• still keep the same module types around -- a unikernel that uses Mirage_stack.V4V6 should continue to work
• once we revise/break the API, conflict in tcpip (or other libraries) with mirage-protocols/mirage-stack

I'll continue to work on that and try to get the smooth migration done. This is also independent of MirageOS releases (and will work with both mirage3 and mirage4).

[hannesm]

ok, an update to compatibility:

• mirage-protocols and mirage-stack now depend on tcpip (and are deprecated), in Deprecated mirage-stack#21 and Deprecate mirage-protocols#30
• this allows mirage-skeleton to mostly work (including static_website_tls) out of the box
• application/dhcp (and application/pin) do not work since ethernet revised the interface (it is packed now, and Ethernet.Packet.sizeof_ethernet is the new name of Ethernet_wire.sizeof_ethernet) -- but the fix is straightforward
• there are some other API changes (such as Mirage_protocols.Keepalive.t is no longer defined, etc.) -- but I couldn't find a package that uses them.

The release plan would be:

• the above 4 packages (ethernet, arp, tcpip, charrua), and mirage-protocols and mirage-stack (where existing ethernet, arp, tcpip, charrua will get upper bounds for mirage-protocols and mirage-stack)
• we can smoothly update the unikernels with the newer packages
• at some point in the future (with mirage 4): there will be no mirage-protocols and mirage-stack around anymore.

Any objections? I could do the above package-by-package end of the upcoming week (dec 9 / 10).

For tcpip, eventually @dinosaure would like to cut a release before the changes above with the rst fixes?

[dinosaure]

I would like to fix an other bug on mirage-tcpip before a release, see mirage/mirage-tcpip#462 but we can integrate your change and the fix into the same release.

[hannesm]

everything is released (PRs for the main / 4 branch are pending) with mirage 3.10.8. I'm done here.