Memory cleanup at exit

[dustanddreams]

This is a work-in-progress support of "memory cleanup at exit" code (OCAMLRUNPARAM=c), see issue #10865.

This is implemented in two parts:

1. perform the equivalent of what domain_terminate does in the main thread upon exit.
2. invoke the finalizer, if any, for all remaining objects in the shared heap.

This appears to work when only one domain is used. However, when multiple domains are used, several tests show that threads are still running at the time caml_shutdown is running, which can cause, at best, some memory leaks (e.g. no caml_free_signal_stack invocation in the remaining threads), and at worse, segmentation fault due to races between threads.

This needs that some form of rendezvous or thread suspension is needed at caml termination.

Comments about the current state of this work and future directions are welcome.

[dustanddreams]

In order to make progress for what is probably most of the use cases of that feature, I've currently restricted the cleanup to single-domain operation.

In this state, there are no segmentation faults or other misbehaviours upon exit, only memory leaks (i.e. it's as bad as before in the worst case.)

[jmid]

Allow me summarize my understanding of this work (feel free to correct me):

• with this PR both single-domain programs and "well-behaved multi-domain programs"¹ will free the malloc'ed memory at exit
• this will only happen in "memory cleanup at exit" mode (OCAMLRUNPARAM=c)

This strikes me as a good first step to get an initial "memory cleanup at exit" mode working in OCaml 5.

Footnotes

1. multi-domain programs that Domain.join any child Domains before reaching program exit, meaning that they will be single-domain at that point ↩

[dustanddreams]

Your understanding is correct.

[NickBarnes]

Possibly I am misunderstanding something here. In the current runtime, when a domain terminates, its major heap (a.k.a. "shared heap") is "orphaned" by caml_teardown_shared_heap, and stashed away in the (misleadingly-named) pool_freelist until the next GC "cycle" when it is adopted by some other domain (in caml_cycle_heap). The memory is not freed, nor can it be as there may be pointers into it from the major heaps of other domains. Likewise, finalisers are not run because the blocks may still be alive. This change appears to break that, by freeing the shared heap of a domain in domain_terminate (via caml_teardown_domain calling caml_teardown_shared_heap which calls caml_finalise_heap, etc).
I think that, instead, we need to call caml_finalise_heap (which should do approximately what it does in this PR, i.e. iterate through the pool lists and large alloc linked list) only when the last domain terminates. When some other domain terminates, its heap should continue to be orphaned and then adopted, as at present; blocks in that heap which die due to the domain termination will be collected (and finalisers run) in the normal way at a subsequent collection.

[gasche]

(This PR is related to #13010 and I was not sure which one to comment in.)

Here is the behavior I would expect for caml_shutdown:

1. all mutator threads are stopped
2. all domains are terminated
3. all resources allocated by the runtime are released

In 4.x, the cleanup-at-exit mode would make caml_do_exit call caml_shutdown, and I see no harm in keeping the same behavior in 5.x. (There the use-case is not an OCaml plugin within a larger non-OCaml program, as we know the process will exit anyway, it is to use valgrind to detect resources leaks in our C code or something like this.) If we are not in cleanup-at-exit mode, we don't need to do anything.

I could see a use-case for another function that does a "soft shutdown" by raising the Thread.Exit exception in all mutator threads, and letting the mutator themselves cleanup their own resources. This is nice, it corresponds to @damiendoligez's suggestion, it is more work and could be dealt with separately.

[dustanddreams]

This new version incorporates @NickBarnes's feedback, but should not be considered ready yet. There are still issues when using the debug runtime. On the other hand it will be easier to review as the cleanup code paths are more visible now.