Make TSAN tests pass with the GIL disabled in free-threaded builds

[mpage]

Feature or enhancement

We need to make the TSAN tests pass with the GIL disabled before we can disable the GIL by default in free-threaded builds.

This should proceed in two phases:

1. Add suppressions for existing TSAN warnings.
2. Triage, fix, and remove the suppressions for the warnings enumerated in (1).

How to run the TSAN tests

1. Build with TSAN:
   env CC=clang CXX=clang++ ./configure --disable-gil --with-thread-sanitizer --with-pydebug && make -j
2. Run tests:
   env TSAN_OPTIONS=suppressions=<repo_root>/Tools/tsan/suppressions_free_threading.txt ./python -m test --tsan -j4

Working on a race

1. Verify that the TSAN tests are passing using the steps from above.
2. Pick a suppression from the section below and assign it to yourself (edit it and add your username next to it). Some of them may be related. If you find that other suppressions are related to the race you're working on please assign them to yourself or contact their owner if they're already assigned.
3. Optionally, head over to the race index to see examples and which tests reproduce the race corresponding to the suppression.
4. Delete the suppression from <repo_root>/Tools/tsan/suppressions_free_threading.txt , run the TSAN tests, and verify that the race is reported by TSAN. You may need to comment out unrelated functions (notably, _PyEval_EvalFrameDefault) in order to reproduce the race.
5. Fix the race.

Suppressions

Tasks

Beta Give feedback

1. _add_to_weak_set
2. _in_weak_set
3. _mi_heap_delayed_free_partial
4. _PyEval_EvalFrameDefault
5. _PyFunction_SetVersion - @mpage
6. _PyImport_AcquireLock - @colesbury
7. _PyImport_ReleaseLock - @colesbury
8. _PyInterpreterState_SetNotRunningMain
9. _PyInterpreterState_IsRunningMain
10. _PyObject_GC_IS_SHARED - @mpage
11. _PyObject_GC_SET_SHARED - @mpage
12. _PyObject_GC_TRACK - @mpage
13. _PyType_HasFeature
14. _PyType_Lookup
15. assign_version_tag
16. compare_unicode_unicode
17. delitem_common
18. dictkeys_decref - @DinoV
19. dictkeys_incref - @DinoV
20. dictresize
21. gc_restore_tid
22. initialize_new_array - @SonicField
23. insertdict - @SonicField
24. lookup_tp_dict - @SonicField
25. mi_heap_visit_pages
26. PyMember_GetOne - @dpdani
27. PyMember_SetOne - @dpdani
28. new_reference
29. set_contains_key
30. set_inheritable
31. unicode_hash

Linked PRs

• gh-117657: Add TSAN suppressions for the free-threaded build #117736
• gh-117657: Quiet TSAN warning about a data race between start_the_world() and tstate_try_attach() #117828
• gh-117657: Quiet more TSAN warnings due to incorrect modeling of compare/exchange #117830
• gh-117657: Fix data races in the method cache in free-threaded builds #117954
• gh-117657: Quiet erroneous TSAN reports of data races in _PySeqLock #117955
• gh-117657: Fix small issues with instrumentation and TSAN #118064
• gh-117657: use relaxed loads for checking dict keys immortality #118067
• [gh-117657] _Py_MergeZeroLocalRefcount isn't loading ob_ref_shared with strong enough semantics #118111
• gh-117657: Quiet TSAN warnings about remaining non-atomic accesses of tstate->state #118165
• gh-117657: Fixes a few small TSAN issues in dictobject #118200
• [gh-117657] Fix some issues with TSAN in typeobject #118249
• gh-117657: Add a couple more TSAN suppressions #118256
• gh-117657: Fix race data race in _Py_IsOwnedByCurrentThread() #118258
• gh-117657: Fix TSAN list set failure #118260
• gh-117657: Fix data race in _Py_IsImmortal #118261
• gh-117657: Fix data races when writing / reading ob_gc_bits #118292
• gh-117657: Disable the function/code cache in free-threaded builds #118301
• gh-117657: TSAN fix race on gstate->young.count #118313
• gh-117657: Don't specialize RESUME_CHECK when specialization is disabled #118349
• gh-117657: Use PyMutex based recursive lock for import lock #118523
• gh-117657: Use clang 18 in TSAN builds #118533
• gh-117657: Replace TSAN suppresions with more specific rules #118722
• gh-117657: Log TSAN warnings to separate files and archive them #118747
• gh-117657: initialize_new_array #118769
• [3.13] gh-117657: Fix data races when writing / reading ob_gc_bits (GH-118292) #118796
• gh-117657: Acquire a critical section around SemLock.__{enter,exit}__ #118812
• gh-117657: initialize_new_array remove race #118842
• gh-117657: Fix QSBR race condition #118843
• [3.13] gh-117657: Acquire a critical section around SemLock.__{enter,exit}__ (GH-118812) #118856
• gh-117657: Fix data races reported by TSAN on interp->threads.main #118865
• [3.13] gh-117657: Replace TSAN suppresions with more specific rules (GH-118722) #118870
• [3.13] gh-117657: Fix data races reported by TSAN on interp->threads.main (GH-118865) #118904
• [3.13] gh-117657: Fix QSBR race condition (GH-118843) #118905
• [3.13] gh-117657: Log TSAN warnings to separate files and archive them (GH-118747) #118931
• gh-117657: Fix itertools.count thread safety #119267
• gh-117657: Fix itertools.count thread safety #119268
• gh-117657: Fix missing atomic in dict_resize #119312
• gh-117657 TSAN Fix race in PyMember_Get and PyMember_Set, for Py_T_OBJECT_EX type #119368
• [3.13] gh-117657: Fix missing atomic in dict_resize #119416
• [3.13] gh-117657: Fix missing atomic in dict_resize (GH-119312) #119417