Set mlockall

[mrks]

I just ran into a situation where Hyrise was paged out. This leads to really weird performance results. We can fix it by calling

  mlockall(MCL_CURRENT | MCL_FUTURE);

in Hyrise::Hyrise(). We should discuss what the expected behavior is. For VMs or personal machines, we might be fine with paging, especially during the generation of benchmark data. An alternative would be to only call mlockall once all data is generated. In that case, we should also call it in the server.

Note: This seems to blow up jemalloc somehow. For some reason, it causes it to never return memory and for the binary to crash. Might mean that we have to abstain from locking the pages. :/

[julianmenzler]

Reference: https://man7.org/linux/man-pages/man2/mlockall.2.html

mlockall() locks all pages mapped into the address space of the
calling process. This includes the pages of the code, data, and
stack segment, as well as shared libraries, user space kernel
data, shared memory, and memory-mapped files. All mapped pages
are guaranteed to be resident in RAM when the call returns
successfully; the pages are guaranteed to stay in RAM until later
unlocked.

If MCL_FUTURE has been specified, then a later system call (e.g.,
mmap(2), sbrk(2), malloc(3)), may fail if it would cause the
number of locked bytes to exceed the permitted maximum (see
below). In the same circumstances, stack growth may likewise
fail: the kernel will deny stack expansion and deliver a SIGSEGV
signal to the process.

Looks a little dangerous to me. Therefore, I would like to see this as an optional flag only.

[mrks]

Which part of this do you consider dangerous? The whole idea is "Never spill to disk, rather crash (sigsegv)"?

[julianmenzler]

This makes total sense for servers with hundreds of gigabytes of RAM. But, for all other machines, we will probably end up with failures across the board. Therefore, I would implement this behavior as an optional setting.

[benrobby]

@mrks is this related to the issues that we have in #2350?

[mrks]

Highly unlikely.