linking order

[oerdnj]

This is actually interesting problem - when jemalloc is linked to a binary, but it is not used directly, but via library, the runtime linker might pull the jemalloc too late and libc would be already loaded. This could create a mess later.

Adding something that references any symbol from jemalloc into main() helps linker with the correct order, but adding a line into every binary doesn't really scale, right?

A more detailed example can be seen here: https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/8478#note_415474

I've tried all possible combinations of -z initfirst and -z now on the shared library and couple of extra ideas, but none helped, I always end up with:

   2611980:	file=libc.so.6 [0];  needed by /home/ondrej/Projects/bind9/bin/check/.libs/named-checkconf [0]
   2611980:	file=libc.so.6 [0];  generating link map
   2611980:       dynamic: 0x00007fd0f6ff9b80  base: 0x00007fd0f6e0e000   size: 0x00000000001f1660
   2611980:         entry: 0x00007fd0f6e321c0  phdr: 0x00007fd0f6e0e040  phnum:                 14

before

   2611980:	file=libjemalloc.so.2 [0];  needed by /home/ondrej/Projects/bind9/lib/isc/.libs/libisc-9.18.20-dev.so [0]
   2611980:     file=libjemalloc.so.2 [0];  generating link map
   2611980:       dynamic: 0x00007fd0f670bd58  base: 0x00007fd0f6656000   size: 0x00000000002db130
   2611980:         entry: 0x00007fd0f665d900  phdr: 0x00007fd0f6656040  phnum:                 12

[guangli-dai]

Setting the LD_PRELOAD like this sounds like a scalable solution.

[oerdnj]

Setting the LD_PRELOAD like this sounds like a scalable solution.

I am confused.

Are you saying that we should say to every user of virtually any software that links directly with jemalloc to set LD_PRELOAD on top of linking directly with jemalloc?

That doesn't exactly sound scalable to me, so I want to double check that this is what you meant.

[guangli-dai]

Sorry for the confusion, I think I misunderstood the question: I thought we were discussing as the users of the binaries.

To solve the problem by setting compilation flags, is the flag -ljemalloc given at the beginning? I think malloc symbols will be resolved using the earliest available library linked.

[oerdnj]

To solve the problem by setting compilation flags, is the flag -ljemalloc given at the beginning? I think malloc symbols will be resolved using the earliest available library linked.

Yes, it is.

It's actually dynamic linker that is causing the problem - since we don't use any jemalloc symbols directly, but only via the external library, jemalloc gets loaded too late (after glibc).