Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Undefined references to nearly all TBB symbols when built with -fvisibility=hidden #779

Open
rob-p opened this issue Feb 20, 2022 · 8 comments · May be fixed by #1114 or #786
Open

Undefined references to nearly all TBB symbols when built with -fvisibility=hidden #779

rob-p opened this issue Feb 20, 2022 · 8 comments · May be fixed by #1114 or #786

Comments

@rob-p
Copy link

rob-p commented Feb 20, 2022

Hi,

First, thank you for your work on the excellent TBB library. We've been using it in some of the software in our lab for a number of years and it's a great resource. Specifically, we use TBB extensively in our tool salmon and we also use it in our tool pufferfish.

Recently, I have been trying to rid our code of the last deprecated remnants of the older TBB library, and to upgrade to the oneAPI TBB interfaces and idioms. While the code changes themselves were relatively painless, the challenges we have faced in building using the new library have been anything but.

We make use of holy build box to build widely distributable linux executables for our tools. This allows us to link against most of our libraries statically, and to have a very old libc requirement. This worked great up until the newer versions of oneAPI TBB. However, the build now breaks in the final linking step, finding undefined references to basically all symbols.

I tried everything I could think of, and then eventually discovered that these missing symbol issues occur when I'm not even trying to build our code, but rather, when I am even building the TBB examples in this environment itself.

To reproduce this issue, one can simply do the following:

  • Create a (Docker) image for holy build box
  • Enter an interactive environment in the above docker image (e.g. with docker run -it <image name> bash)
  • Execute the following commands:
    • fetch the latest TBB curl -LJO https://github.com/oneapi-src/oneTBB/archive/refs/tags/v2021.5.0.tar.gz
    • tar xzvf v2021.5.0.tar.gz
    • cd oneTBB-2021.5.0
    • mkdir build && cd build
    • source /hbb_shlib/activate (activate the development environment of holy build box)
    • cmake ..
    • make

This will build the shared libraries without a problem, but when it attempts to build the examples, they will all fail with linker errors. Here is an example of what I see:

[ 16%] Linking C executable ../gnu_9.3_cxx17_64_relwithdebinfo/test_malloc_pure_c
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/test_malloc_pure_c.dir/tbbmalloc/test_malloc_pure_c.c.o: in
 function `MyExit':
/oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:50: undefined reference to `scalable_malloc'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:52: undefined ref
erence to `scalable_free'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/test_malloc_pure_c.dir/tbbmalloc/test_malloc_pure_c.c.o: in
 function `main':
/oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:64: undefined reference to `scalable_malloc'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:68: undefined ref
erence to `scalable_free'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:64: undefined ref
erence to `scalable_malloc'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:68: undefined ref
erence to `scalable_free'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:75: undefined ref
erence to `scalable_free'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:72: undefined ref
erence to `scalable_malloc'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:92: undefined ref
erence to `scalable_free'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:93: undefined ref
erence to `scalable_malloc'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /oneTBB-2021.5.0/test/tbbmalloc/test_malloc_pure_c.c:80: undefined ref
erence to `scalable_free'

and on and on like this. Strangely, these symbols seem to be present in the shared libraries (inspected with strings and c++filt), but they are simply not found when linking against these libraries. At this point, I've wasted a few days trying to get this to build in the holy build box image — updating that image several times to the most recent version — to no avail. I will mention that, with the old versions of TBB (e.g. the 2019 or 2020 versions), the build works. Only starting with the new oneAPI TBB versions from 2021.1 forward does linking seem to fail.

I would appreciate any insight you might have into this issue, that is really blocking us from adopting an updated and non-deprecated version of TBB.

Thanks,
Rob

@FooBarWidget
Copy link

This is because Holy Build Box sets CFLAGS/CXXFLAGS to -g -O2 -fvisibility=hidden -I/hbb_shlib/include by default. Proper usage of -fvisibility=hidden requires help from the library in question: it has to explicitly mark exportable symbols as such (with __attribute__((visibility ("default"))). It appears OneTBB doesn't do that.

Removing -fvisbility=hidden from CFLAGS/CXXFLAGS solves the symbol linking problem. But of course, TBB still exports too many symbols.

@rob-p
Copy link
Author

rob-p commented Feb 20, 2022

Thanks for the quick response, @FooBarWidget! What is the recommended way to remove this flag in the HBB environment used for building?

@FooBarWidget
Copy link

Override CFLAGS/CXXFLAGS. They're just environment variables.

rob-p pushed a commit to COMBINE-lab/pufferfish that referenced this issue Feb 20, 2022
@alexey-katranov
Copy link
Contributor

oneTBB relies on .def files to export only required systems. It seems they are not used under your build environment. To avoid all symbols exporting, either .def files should be used or include/oneapi/tbb/detail/_export.h should be extended to specify the required visibility for symbols.

@anton-potapov anton-potapov linked a pull request Feb 22, 2022 that will close this issue
14 tasks
@anton-potapov
Copy link
Contributor

@FooBarWidget , @rob-p I was able to reproduce the link errors with -fvisibility=hidden flag applied.

@alexey-katranov it seems that .def files are not enough, as they only pass in already exported symbols, but does not make them exported

@FooBarWidget
Copy link

FooBarWidget commented Feb 22, 2022

Export.h should define _EXPORT as __attribute__((visibility("default"))) on Unix, instead of as nothing.

@alexey-katranov
Copy link
Contributor

Yes, #786 tries to addresses the issue.

@anton-potapov anton-potapov changed the title Undefined references to nearly all TBB symbols when built under certain environment Undefined references to nearly all TBB symbols when built with -fvisibility=hidden Feb 24, 2022
@anton-potapov anton-potapov changed the title Undefined references to nearly all TBB symbols when built with -fvisibility=hidden Undefined references to nearly all TBB symbols when built with -fvisibility=hidden Feb 24, 2022
@mika-fischer
Copy link

Please consider making exporting of symbols configurable or disable exporting when comping tbb as a static library.

@phprus phprus linked a pull request May 27, 2023 that will close this issue
14 tasks
@isaevil isaevil reopened this May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants