Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible data race reported by clang thread sanitizer #1290

Open
emmenlau opened this issue Jan 10, 2024 · 6 comments
Open

Possible data race reported by clang thread sanitizer #1290

emmenlau opened this issue Jan 10, 2024 · 6 comments
Assignees

Comments

@emmenlau
Copy link

emmenlau commented Jan 10, 2024

We have found a possible data race reported when using OpenCV resize method, and reported it in opencv/opencv#24063. It seems that the origin is in oneTBB, so copying the report here:

As far as we can see there is nothing specific in our build that would explain the issue. We have just reproduced the issue again with OpenCV 4.9.0 (current latest), oneTBB 2021.10.0 and clang 17.x on Linux. Previously the issue could be experienced with lower version numbers of all tools as well.

Here is what we did:

  • Build OpenCV against oneTBB, AFAIK without any super special parameters
  • Build a simple OpenCV executable that calls resize(), with clang on Linux in debug mode, and enable thread sanitizer in the build:
    • Add option -fsanitize=thread to compile and link arguments list of clang
  • Run the executable, and observe the below info printed

The executable will print:

[ INFO:0@0.000] global registry_parallel.impl.hpp:96 ParallelBackendRegistry core(parallel): Enabled backends(2, sorted by priority): TBB(1000); OPENMP(990)
[ INFO:0@0.000] global parallel_for.tbb.hpp:54 ParallelForBackend Initializing TBB parallel backend: TBB_INTERFACE_VERSION=12100
[ INFO:0@0.000] global parallel.cpp:77 createParallelForAPI core(parallel): using backend: TBB (priority=1000)
==================
WARNING: ThreadSanitizer: data race (pid=908834)
  Write of size 8 at 0x7b2000000000 by thread T4:
    #0 operator delete[](void*) <null> (OpenCVResizeDataRaceReproducer+0x4b00af) (BuildId: 2827f2d96c42af79)
    #1 tbb::detail::r1::destroy_process_mask() /home/user/Src/oneTBB/src/tbb/misc_ex.cpp:93:5 (libtbb_debug.so.12+0x7f2d5) (BuildId: c02118eec6345e66)

  Previous read of size 8 at 0x7b2000000000 by thread T2:
    #0 memcmp <null> (OpenCVResizeDataRaceReproducer+0x4336f0) (BuildId: 2827f2d96c42af79)
    #1 tbb::detail::r1::affinity_helper::protect_affinity_mask(bool) /home/user/Src/oneTBB/src/tbb/misc_ex.cpp:113:26 (libtbb_debug.so.12+0x7f49a) (BuildId: c02118eec6345e66)

  Thread T4 (tid=908839, running) created by thread T1 at:
    #0 pthread_create <null> (OpenCVResizeDataRaceReproducer+0x42786b) (BuildId: 2827f2d96c42af79)
    #1 tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /home/user/Src/oneTBB/src/tbb/rml_thread_monitor.h:208:15 (libtbb_debug.so.12+0x877c5) (BuildId: c02118eec6345e66)

  Thread T2 (tid=908838, finished) created by main thread at:
    #0 pthread_create <null> (OpenCVResizeDataRaceReproducer+0x42786b) (BuildId: 2827f2d96c42af79)
    #1 tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /home/user/Src/oneTBB/src/tbb/rml_thread_monitor.h:208:15 (libtbb_debug.so.12+0x877c5) (BuildId: c02118eec6345e66)
    #2 __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 (libc.so.6+0x29d8f) (BuildId: 203de0ae33b53fee1578b117cb4123e85d0534f0)

SUMMARY: ThreadSanitizer: data race (/data/user/Debug/OpenCVResizeDataRaceReproducer+0x4b00af) (BuildId: 2827f2d96c42af79) in operator delete[](void*)
==================
ThreadSanitizer: reported 1 warnings
@Alejandro-FA
Copy link

Alejandro-FA commented Feb 12, 2024

The following minimal example also produces a data race detected by ThreadSanitizer:

std::vector<std::uint64_t> v(1'000'000);
std::mt19937 gen {std::random_device{}()};
std::ranges::generate(v, gen);
std::sort(std::execution::par, v.begin(), v.end());

Error produced:

2:   Write of size 8 at 0x7f76295f7848 by thread T2:
2:     #0 __pstl::__tbb_backend::__func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >::execute(tbb::detail::d1::execution_data&) /usr/include/c++/13/pstl/parallel_backend_tbb.h:618 (main.out+0x443c9f) (BuildId: 5e0a4b105b08ad76466044248f6318145de6ec50)
2:     #1 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /localdisk/ci/runner010/intel-innersource/001/_work/libraries.threading.infrastructure.onetbb-ci/libraries.threading.infrastructure.onetbb-ci/onetbb_source_code/src/tbb/task_dispatcher.h:323 (libtbb_debug.so.12+0x1a365) (BuildId: 209272c75536bf0182a8275d42c8148347e6b5e5)
2: 
2:   Previous write of size 8 at 0x7f76295f7848 by main thread:
2:     #0 __pstl::__tbb_backend::__task::__task() /usr/include/c++/13/pstl/parallel_backend_tbb.h:515 (main.out+0x443848) (BuildId: 5e0a4b105b08ad76466044248f6318145de6ec50)
2:     #1 __pstl::__tbb_backend::__func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >::__func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >(__pstl::execution::v1::parallel_policy const&) /usr/include/c++/13/pstl/parallel_backend_tbb.h:663 (main.out+0x443848)
2:     #2 __pstl::__tbb_backend::__func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >* tbb::detail::d1::small_object_allocator::new_object<__pstl::__tbb_backend::__func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >, __pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >(tbb::detail::d1::execution_data&, __pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}>&&) /opt/intel/oneapi/tbb/2021.11/include/oneapi/tbb/detail/_small_object_pool.h:55 (main.out+0x443848)
2:     #3 __pstl::__tbb_backend::__task* __pstl::__tbb_backend::__task::allocate_func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >(__pstl::execution::v1::parallel_policy const&) /usr/include/c++/13/pstl/parallel_backend_tbb.h:531 (main.out+0x443848)
2:     #4 __pstl::__tbb_backend::__task* __pstl::__tbb_backend::__task::make_child_of<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >(__pstl::__tbb_backend::__task*, __pstl::execution::v1::parallel_policy const&) /usr/include/c++/13/pstl/parallel_backend_tbb.h:563 (main.out+0x443848)
2:     #5 __pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}>::operator()(__pstl::__tbb_backend::__task*) /usr/include/c++/13/pstl/parallel_backend_tbb.h:1145 (main.out+0x443848)
2:     #6 __pstl::__tbb_backend::__func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >::execute(tbb::detail::d1::execution_data&) /usr/include/c++/13/pstl/parallel_backend_tbb.h:620 (main.out+0x443cbb) (BuildId: 5e0a4b105b08ad76466044248f6318145de6ec50)
2:     #7 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /localdisk/ci/runner010/intel-innersource/001/_work/libraries.threading.infrastructure.onetbb-ci/libraries.threading.infrastructure.onetbb-ci/onetbb_source_code/src/tbb/task_dispatcher.h:323 (libtbb_debug.so.12+0x53c35) (BuildId: 209272c75536bf0182a8275d42c8148347e6b5e5)
2:     #8 tbb::detail::d1::task_arena_function<__pstl::__tbb_backend::__parallel_stable_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}>(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}, unsigned long)::{lambda()#1}, void>::operator()() const /opt/intel/oneapi/tbb/2021.11/include/oneapi/tbb/task_arena.h:68 (main.out+0x437cb6) (BuildId: 5e0a4b105b08ad76466044248f6318145de6ec50)
2:     #9 tbb::detail::r1::isolate_within_arena(tbb::detail::d1::delegate_base&, long)::{lambda()#1}::operator()() const /localdisk/ci/runner010/intel-innersource/001/_work/libraries.threading.infrastructure.onetbb-ci/libraries.threading.infrastructure.onetbb-ci/onetbb_source_code/src/tbb/arena.cpp:865 (libtbb_debug.so.12+0x113e6) (BuildId: 209272c75536bf0182a8275d42c8148347e6b5e5)
2:     #10 void __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>) /usr/include/c++/13/pstl/algorithm_impl.h:2118 (main.out+0x40e62a) (BuildId: 5e0a4b105b08ad76466044248f6318145de6ec50)
2:     #11 std::enable_if<__pstl::execution::v1::is_execution_policy<std::remove_cv<std::remove_reference<__pstl::execution::v1::parallel_policy const&>::type>::type>::value, void>::type std::sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>) /usr/include/c++/13/pstl/glue_algorithm_impl.h:633 (main.out+0x40e62a)
2:     #12 std::enable_if<__pstl::execution::v1::is_execution_policy<std::remove_cv<std::remove_reference<__pstl::execution::v1::parallel_policy const&>::type>::type>::value, void>::type std::sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > > >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >) /usr/include/c++/13/pstl/glue_algorithm_impl.h:645 (main.out+0x40e62a)
2:     #13 main /<edited>/src/main.cpp:131 (main.out+0x40e62a)
2: 
2:   Thread T2 (tid=40991, running) created by main thread at:
2:     #0 pthread_create <null> (libtsan.so.2+0x3e006) (BuildId: 732e44e7f1cd4f0f9ca7d27895a253bebdea6827)
2:     #1 tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /localdisk/ci/runner010/intel-innersource/001/_work/libraries.threading.infrastructure.onetbb-ci/libraries.threading.infrastructure.onetbb-ci/onetbb_source_code/src/tbb/rml_thread_monitor.h:208 (libtbb_debug.so.12+0x4a120) (BuildId: 209272c75536bf0182a8275d42c8148347e6b5e5)
2:     #2 __pstl::__tbb_backend::__func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >::execute(tbb::detail::d1::execution_data&) /usr/include/c++/13/pstl/parallel_backend_tbb.h:620 (main.out+0x443cbb) (BuildId: 5e0a4b105b08ad76466044248f6318145de6ec50)
2:     #3 tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter>(tbb::detail::d1::task*, tbb::detail::r1::external_waiter&) /localdisk/ci/runner010/intel-innersource/001/_work/libraries.threading.infrastructure.onetbb-ci/libraries.threading.infrastructure.onetbb-ci/onetbb_source_code/src/tbb/task_dispatcher.h:323 (libtbb_debug.so.12+0x53c35) (BuildId: 209272c75536bf0182a8275d42c8148347e6b5e5)
2:     #4 tbb::detail::d1::task_arena_function<__pstl::__tbb_backend::__parallel_stable_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}>(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}, unsigned long)::{lambda()#1}, void>::operator()() const /opt/intel/oneapi/tbb/2021.11/include/oneapi/tbb/task_arena.h:68 (main.out+0x437cb6) (BuildId: 5e0a4b105b08ad76466044248f6318145de6ec50)
2:     #5 tbb::detail::r1::isolate_within_arena(tbb::detail::d1::delegate_base&, long)::{lambda()#1}::operator()() const /localdisk/ci/runner010/intel-innersource/001/_work/libraries.threading.infrastructure.onetbb-ci/libraries.threading.infrastructure.onetbb-ci/onetbb_source_code/src/tbb/arena.cpp:865 (libtbb_debug.so.12+0x113e6) (BuildId: 209272c75536bf0182a8275d42c8148347e6b5e5)
2:     #6 void __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>) /usr/include/c++/13/pstl/algorithm_impl.h:2118 (main.out+0x40e62a) (BuildId: 5e0a4b105b08ad76466044248f6318145de6ec50)
2:     #7 std::enable_if<__pstl::execution::v1::is_execution_policy<std::remove_cv<std::remove_reference<__pstl::execution::v1::parallel_policy const&>::type>::type>::value, void>::type std::sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>) /usr/include/c++/13/pstl/glue_algorithm_impl.h:633 (main.out+0x40e62a)
2:     #8 std::enable_if<__pstl::execution::v1::is_execution_policy<std::remove_cv<std::remove_reference<__pstl::execution::v1::parallel_policy const&>::type>::type>::value, void>::type std::sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > > >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >) /usr/include/c++/13/pstl/glue_algorithm_impl.h:645 (main.out+0x40e62a)
2:     #9 main  /<edited>/src/main.cpp:131 (main.out+0x40e62a)
2: 
2: SUMMARY: ThreadSanitizer: data race /usr/include/c++/13/pstl/parallel_backend_tbb.h:618 in __pstl::__tbb_backend::__func_task<__pstl::__tbb_backend::__stable_sort_func<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long*, std::less<unsigned long>, __pstl::__internal::__pattern_sort<__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false> >(__pstl::execution::v1::parallel_policy const&, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>, std::integral_constant<bool, false>, std::integral_constant<bool, true>, std::integral_constant<bool, true>)::{lambda()#1}::operator()() const::{lambda(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, std::less<unsigned long>)#1}> >::execute(tbb::detail::d1::execution_data&)
2: ==================

I am using gcc:

❯ gcc -v    
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/13/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,objc,obj-c++,ada,go,d,m2,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-libstdcxx-zoneinfo=/usr/share/zoneinfo --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-13.2.1-20231205/obj-x86_64-redhat-linux/isl-install --enable-offload-targets=nvptx-none --without-cuda-driver --enable-offload-defaulted --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 13.2.1 20231205 (Red Hat 13.2.1-6) (GCC) 

@pavelkumbrasev
Copy link
Contributor

Hi @emmenlau, @Alejandro-FA, could you please try to rebuild TBB with ThreadSanitizer support .

@Alejandro-FA
Copy link

Alejandro-FA commented Feb 13, 2024

Hi @pavelkumbrasev, I have tried it but I get an error in the build process.

I have configured the project as follows:

cd /tmp
git clone https://github.com/oneapi-src/oneTBB.git
cd oneTBB
mkdir cmake-build && cd cmake-build
cmake -DCMAKE_INSTALL_PREFIX=/tmp/intel/oneTBB_tsan -DTBB_SANITIZE=thread ..

Here is the output of the configuration process:

-- The CXX compiler identification is GNU 13.2.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_BUILD_TYPE is not specified. Using default: RelWithDebInfo
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Checking for one of the modules 'hwloc'
-- Looking for getcontext
-- Looking for getcontext - found
-- Performing Test _fsanitize_thread
-- Performing Test _fsanitize_thread - Success
-- The C compiler identification is GNU 13.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The tbbbind_2_5 target will be configured using the HWLOC 2.10.0
-- Performing Test LINKER_HAS_NO_AS_NEEDED
-- Performing Test LINKER_HAS_NO_AS_NEEDED - Success
-- Configuring done (0.7s)
-- Generating done (0.1s)
-- Build files have been written to: /tmp/oneTBB/cmake-build

And when doing make -j I get the following error:

[ 15%] Building CXX object test/CMakeFiles/test_malloc_lib_unload.dir/tbbmalloc/test_malloc_lib_unload.cpp.o
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/concurrent_bounded_queue.cpp:17:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::prepare_wait(tbb::detail::r1::wait_node<Context>&) [with Context = long unsigned int]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:223:29,
    inlined from ‘bool tbb::detail::r1::concurrent_monitor_base<Context>::wait(Pred&&, NodeType&&) [with NodeType = tbb::detail::r1::sleep_node<long unsigned int>; Pred = tbb::detail::r1::wait_bounded_queue_monitor(concurrent_monitor*, std::size_t, std::ptrdiff_t, tbb::detail::d1::delegate_base&)::<lambda()>; Context = long unsigned int]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:260:21,
    inlined from ‘void tbb::detail::r1::wait_bounded_queue_monitor(concurrent_monitor*, std::size_t, std::ptrdiff_t, tbb::detail::d1::delegate_base&)’ at /tmp/oneTBB/src/tbb/concurrent_bounded_queue.cpp:57:53:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:132: src/tbb/CMakeFiles/tbb.dir/concurrent_bounded_queue.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/tbb_allocator.h:20,
                 from /tmp/oneTBB/src/tbb/task_group_context.cpp:18:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::prepare_wait(tbb::detail::r1::wait_node<Context>&) [with Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:223:29,
    inlined from ‘bool tbb::detail::r1::concurrent_monitor_base<Context>::wait(Pred&&, NodeType&&) [with NodeType = tbb::detail::r1::resume_node&; Pred = tbb::detail::r1::suspend_point_type::resume_task::execute(tbb::detail::d1::execution_data&)::<lambda()>; Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:260:21,
    inlined from ‘virtual tbb::detail::d1::task* tbb::detail::r1::suspend_point_type::resume_task::execute(tbb::detail::d1::execution_data&)’ at /tmp/oneTBB/src/tbb/task_dispatcher.h:77:27:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:440: src/tbb/CMakeFiles/tbb.dir/task_group_context.cpp.o] Error 1
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/arena_slot.h:21,
                 from /tmp/oneTBB/src/tbb/arena_slot.cpp:17:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::arena::advertise_new_work() [with new_work_type work_type = tbb::detail::r1::arena::wakeup]’ at /tmp/oneTBB/src/tbb/arena.h:408:29:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
[ 15%] Linking CXX shared library ../../gnu_13.2_cxx11_64_relwithdebinfo/libtbbbind_2_5.so
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:118: src/tbb/CMakeFiles/tbb.dir/arena_slot.cpp.o] Error 1
[ 15%] Built target tbbbind_2_5
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/misc.h:22,
                 from /tmp/oneTBB/src/tbb/governor.h:22,
                 from /tmp/oneTBB/src/tbb/governor.cpp:17:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::prepare_wait(tbb::detail::r1::wait_node<Context>&) [with Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:223:29,
    inlined from ‘bool tbb::detail::r1::concurrent_monitor_base<Context>::wait(Pred&&, NodeType&&) [with NodeType = tbb::detail::r1::resume_node&; Pred = tbb::detail::r1::suspend_point_type::resume_task::execute(tbb::detail::d1::execution_data&)::<lambda()>; Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:260:21,
    inlined from ‘virtual tbb::detail::d1::task* tbb::detail::r1::suspend_point_type::resume_task::execute(tbb::detail::d1::execution_data&)’ at /tmp/oneTBB/src/tbb/task_dispatcher.h:77:27:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:174: src/tbb/CMakeFiles/tbb.dir/governor.cpp.o] Error 1
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/address_waiter.cpp:17:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::prepare_wait(tbb::detail::r1::wait_node<Context>&) [with Context = tbb::detail::r1::address_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:223:29,
    inlined from ‘bool tbb::detail::r1::concurrent_monitor_base<Context>::wait(Pred&&, NodeType&&) [with NodeType = tbb::detail::r1::sleep_node<tbb::detail::r1::address_context>; Pred = tbb::detail::d1::delegate_base&; Context = tbb::detail::r1::address_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:260:21,
    inlined from ‘void tbb::detail::r1::wait_on_address(void*, tbb::detail::d1::delegate_base&, uintptr_t)’ at /tmp/oneTBB/src/tbb/address_waiter.cpp:71:48:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:76: src/tbb/CMakeFiles/tbb.dir/address_waiter.cpp.o] Error 1
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/cache_aligned_allocator.h:20,
                 from /tmp/oneTBB/src/tbb/small_object_pool.cpp:17:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::prepare_wait(tbb::detail::r1::wait_node<Context>&) [with Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:223:29,
    inlined from ‘bool tbb::detail::r1::concurrent_monitor_base<Context>::wait(Pred&&, NodeType&&) [with NodeType = tbb::detail::r1::resume_node&; Pred = tbb::detail::r1::suspend_point_type::resume_task::execute(tbb::detail::d1::execution_data&)::<lambda()>; Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:260:21,
    inlined from ‘virtual tbb::detail::d1::task* tbb::detail::r1::suspend_point_type::resume_task::execute(tbb::detail::d1::execution_data&)’ at /tmp/oneTBB/src/tbb/task_dispatcher.h:77:27:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:398: src/tbb/CMakeFiles/tbb.dir/small_object_pool.cpp.o] Error 1
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/task_dispatcher.h:20,
                 from /tmp/oneTBB/src/tbb/arena.cpp:17:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::notify(const P&) [with P = tbb::detail::r1::arena::request_workers(int, int, bool)::<lambda(tbb::detail::r1::market_context)>; Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:350:29,
    inlined from ‘void tbb::detail::r1::arena::request_workers(int, int, bool)’ at /tmp/oneTBB/src/tbb/arena.cpp:352:45:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:104: src/tbb/CMakeFiles/tbb.dir/arena.cpp.o] Error 1
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/scheduler_common.h:20,
                 from /tmp/oneTBB/src/tbb/task.cpp:18:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::notify(const P&) [with P = tbb::detail::r1::notify_waiters(uintptr_t)::<lambda(tbb::detail::r1::market_context)>; Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:350:29,
    inlined from ‘void tbb::detail::r1::notify_waiters(uintptr_t)’ at /tmp/oneTBB/src/tbb/task.cpp:221:80:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:412: src/tbb/CMakeFiles/tbb.dir/task.cpp.o] Error 1
[ 15%] Linking C shared library ../../gnu_13.2_cxx11_64_relwithdebinfo/libtbbmalloc.so
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/mutex.h:21,
                 from /tmp/oneTBB/src/tbb/threading_control.h:20,
                 from /tmp/oneTBB/src/tbb/threading_control.cpp:17:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::abort_all() [with Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:422:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::destroy() [with Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:456:24,
    inlined from ‘tbb::detail::r1::thread_control_monitor::~thread_control_monitor()’ at /tmp/oneTBB/src/tbb/thread_control_monitor.h:102:16,
    inlined from ‘void tbb::detail::r1::cache_aligned_deleter::operator()(T*) const [with T = tbb::detail::r1::thread_control_monitor]’ at /tmp/oneTBB/src/tbb/scheduler_common.h:75:16,
    inlined from ‘std::unique_ptr<_Tp, _Dp>::~unique_ptr() [with _Tp = tbb::detail::r1::thread_control_monitor; _Dp = tbb::detail::r1::cache_aligned_deleter]’ at /usr/include/c++/13/bits/unique_ptr.h:404:17,
    inlined from ‘tbb::detail::r1::threading_control_impl::~threading_control_impl()’ at /tmp/oneTBB/src/tbb/threading_control.h:42:7,
    inlined from ‘void tbb::detail::r1::cache_aligned_deleter::operator()(T*) const [with T = tbb::detail::r1::threading_control_impl]’ at /tmp/oneTBB/src/tbb/scheduler_common.h:75:16,
    inlined from ‘std::unique_ptr<_Tp, _Dp>::~unique_ptr() [with _Tp = tbb::detail::r1::threading_control_impl; _Dp = tbb::detail::r1::cache_aligned_deleter]’ at /usr/include/c++/13/bits/unique_ptr.h:404:17:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
[ 15%] Built target tbbmalloc
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:482: src/tbb/CMakeFiles/tbb.dir/threading_control.cpp.o] Error 1
[ 15%] Building CXX object test/CMakeFiles/_test_malloc_used_by_lib.dir/tbbmalloc/test_malloc_used_by_lib.cpp.o
[ 15%] Building CXX object test/CMakeFiles/test_malloc_init_shutdown.dir/tbbmalloc/test_malloc_init_shutdown.cpp.o
[ 15%] Building C object test/CMakeFiles/test_malloc_pure_c.dir/tbbmalloc/test_malloc_pure_c.c.o
[ 16%] Building CXX object src/tbbmalloc_proxy/CMakeFiles/tbbmalloc_proxy.dir/function_replacement.cpp.o
[ 16%] Building CXX object test/CMakeFiles/test_malloc_compliance.dir/tbbmalloc/test_malloc_compliance.cpp.o
[ 16%] Building CXX object src/tbbmalloc_proxy/CMakeFiles/tbbmalloc_proxy.dir/proxy.cpp.o
[ 16%] Building CXX object test/CMakeFiles/test_malloc_pools.dir/tbbmalloc/test_malloc_pools.cpp.o
[ 16%] Building CXX object test/CMakeFiles/test_scalable_allocator.dir/tbbmalloc/test_scalable_allocator.cpp.o
[ 17%] Building CXX object test/CMakeFiles/test_malloc_regression.dir/tbbmalloc/test_malloc_regression.cpp.o
In file included from /usr/include/c++/13/atomic:41,
                 from /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_utils.h:22,
                 from /tmp/oneTBB/src/tbb/task_dispatcher.h:20,
                 from /tmp/oneTBB/src/tbb/task_dispatcher.cpp:17:
In function ‘void std::atomic_thread_fence(memory_order)’,
    inlined from ‘void tbb::detail::d0::atomic_fence_seq_cst()’ at /tmp/oneTBB/src/tbb/../../include/oneapi/tbb/detail/_machine.h:87:29,
    inlined from ‘void tbb::detail::r1::concurrent_monitor_base<Context>::prepare_wait(tbb::detail::r1::wait_node<Context>&) [with Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:223:29,
    inlined from ‘bool tbb::detail::r1::concurrent_monitor_base<Context>::wait(Pred&&, NodeType&&) [with NodeType = tbb::detail::r1::sleep_node<tbb::detail::r1::market_context>; Pred = tbb::detail::r1::coroutine_waiter::pause(tbb::detail::r1::arena_slot&)::<lambda()>&; Context = tbb::detail::r1::market_context]’ at /tmp/oneTBB/src/tbb/concurrent_monitor.h:260:21,
    inlined from ‘void tbb::detail::r1::sleep_waiter::sleep(uintptr_t, Pred) [with Pred = tbb::detail::r1::coroutine_waiter::pause(tbb::detail::r1::arena_slot&)::<lambda()>]’ at /tmp/oneTBB/src/tbb/waiters.h:115:92,
    inlined from ‘void tbb::detail::r1::coroutine_waiter::pause(tbb::detail::r1::arena_slot&)’ at /tmp/oneTBB/src/tbb/waiters.h:178:14:
/usr/include/c++/13/bits/atomic_base.h:144:26: error: ‘atomic_thread_fence’ is not supported with ‘-fsanitize=thread’ [-Werror=tsan]
  144 |   { __atomic_thread_fence(int(__m)); }
      |     ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1plus: all warnings being treated as errors
make[2]: *** [src/tbb/CMakeFiles/tbb.dir/build.make:426: src/tbb/CMakeFiles/tbb.dir/task_dispatcher.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:457: src/tbb/CMakeFiles/tbb.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 17%] Linking C executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_malloc_pure_c
[ 17%] Built target test_malloc_pure_c
[ 18%] Linking CXX shared library ../gnu_13.2_cxx11_64_relwithdebinfo/lib_test_malloc_used_by_lib.so
[ 18%] Built target _test_malloc_used_by_lib
[ 18%] Linking CXX shared library ../../gnu_13.2_cxx11_64_relwithdebinfo/libtbbmalloc_proxy.so
[ 18%] Built target tbbmalloc_proxy
[ 18%] Linking CXX executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_handle_perror
[ 18%] Built target test_handle_perror
[ 18%] Linking CXX executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_malloc_lib_unload
[ 18%] Built target test_malloc_lib_unload
[ 18%] Linking CXX executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_malloc_init_shutdown
[ 18%] Built target test_malloc_init_shutdown
[ 18%] Linking CXX executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_malloc_regression
[ 18%] Built target test_malloc_regression
[ 19%] Linking CXX executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_malloc_compliance
[ 19%] Built target test_malloc_compliance
[ 20%] Linking CXX executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_malloc_pools
[ 20%] Built target test_malloc_pools
[ 20%] Linking CXX executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_malloc_whitebox
[ 21%] Linking CXX executable ../gnu_13.2_cxx11_64_relwithdebinfo/test_scalable_allocator
[ 21%] Built target test_malloc_whitebox
[ 21%] Built target test_scalable_allocator
make: *** [Makefile:166: all] Error 2

@Alejandro-FA
Copy link

From your comment, do I understand correctly that if I want to use Thread Sanitizer with my application, I need to build oneTBB with Thread Sanitizer support? At the moment I simply installed oneTBB following these instructions: https://www.intel.com/content/www/us/en/docs/onetbb/get-started-guide/2021-6/install-onetbb-on-linux-os.html

@phprus
Copy link
Contributor

phprus commented Feb 13, 2024

@pavelkumbrasev
Copy link
Contributor

@Alejandro-FA, it might lead to unexpected false positive reports that will lead into TBB. So to properly test application that use TBB it also should be built with TSan support.

The error you got probably related to GNU 13.2.1 (we didn't test TSan with this version yet) (problem). Could you please try to build it with different version (GNU < 11 should be fine).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants