-
Notifications
You must be signed in to change notification settings - Fork 68
Domain Local Allocation Buffers
This wiki page will discuss the planned implementation for Domain-local allocation buffers in OCaml Multicore.
Currently, the minor heaps are contained in a contiguous memory area, reserved (not committed) during the runtime's initialization. (in caml_init_domains.)
We reserve enough address space for every domain, using the Minor_heap_max as the effective size of each minor heap segments.
Minor_heap_max * Max_domains is thus reserved.
Each dom_internal relevant fields (minor_heap_base, minor_heap_area and minor_heap_end) are then set to the right range from within the minor heaps area.
Additionally, each minor heap segments do not only contains a domain's private minor heap, but does as well store the tls_area before the minor_heap, each separated by a guard page.
When a domain is started, on first use of this domain's slot, the tls_area is allocated (by committing the tls_area range for this domain.)
The minor heap for this domain is then allocated by committing the minor_heap range.
Later on, when a domain reaches the end of its private minor heap, a minor collection is requested, to free up every minor heaps in every currently running domains.
We will replace this mechanism by using the shared minor_heap segment (tentatively now named global_minor_heap) and adding an allocation pointer (tentatively named global_minor_heap_ptr) to allow domains to request and extend their private minor heaps segments through a simple pointer bump mechanism.
Instead of each domains having a segment, contiguously cut from the global_minor_heap, each domain will instead request a new minor_heap under two conditions: either the domain is starting, or the domain is running out of minor heap space.
The domain will fetch a new address range from the global_minor_heap by effectively bumping the global_minor_heap_ptr by the fixed size of a single minor heap segment. (through a simple CAS on global_minor_heap_ptr)
Once the range is reclaimed by this domain, the address range can be committed for further use.
When a domain runs out of minor heap space, instead of triggering a minor collection, instead a new minor heap segment can be fetched from the global_minor_heap, until the global_minor_heap_ptr reaches the end of the global_minor_heap area.
Once we run out of space in the global_minor_heap (global_minor_heap_ptr > global_minor_heap_end), a minor collection needs to be triggered.
During minor collection, each domains will proceed to free up the minor heap space. global_minor_heap_ptr can be reset to global_minor_heap start, and every domain can subsequently reclaim a new minor heap segment from there.