BUG: numpy.lib.recfunctions.structured_to_unstructured is not thread-safe

[kohlerjl]

Describe the issue:

numpy.lib.recfunctions.structured_to_unstructured uses numpy.testing.suppress_warnings, which is inherently thread unsafe. However, structured_to_unstructured is not documented as thread unsafe, and would otherwise not be expected to alter global state.

If this function is used in a multi-threaded program, multiple suppress_warnings context managers may be entered and exited out of order, leaving the global warnings.show_warnings handler in an invalid state. Any subsequent code that issues any warning will trigger an AttributeError instead. This issue resulted in a crash of one of our production services. This error is related to #8413, and an example of this failure is constructed below.

From a quick search, this usage in structured_to_unstructured appears to be the only usage of suppress_warnings in the numpy library outside of testing routines. It would seem this suppress_warnings context manager is only suitable for testing purposes, and should not be used in library or production code.

Reproduce the code example:

import logging
import warnings
import threading

from numpy.testing import suppress_warnings

def thread_a():
    logging.info("Entering a")
    with suppress_warnings() as sup:
        logging.info("Entered a")
        entered_a.set()
        entered_b.wait()
        logging.info("Exiting a")
    logging.info("Exited a")
    exited_a.set()

def thread_b():
    logging.info("Entering b")
    entered_a.wait()
    with suppress_warnings() as sup:
        logging.info("Entered b")
        entered_b.set()
        exited_a.wait()
        logging.info("Exiting b")
    logging.info("Exited b")
    exited_b.set()

entered_a = threading.Event()
exited_a = threading.Event()
entered_b = threading.Event()
exited_b = threading.Event()
    
thread1 = threading.Thread(target=thread_a)
thread2 = threading.Thread(target=thread_b)

entered_a.clear()
exited_a.clear()
entered_b.clear()
exited_b.clear()

thread1.start()
thread2.start()

exited_b.wait()

warnings.warn('test')

Error message:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [11], in <cell line: 14>()
     10 thread2.start()
     12 exited_b.wait()
---> 14 warnings.warn('test')

File /usr/lib/python3.10/warnings.py:109, in _showwarnmsg(msg)
    105         if not callable(sw):
    106             raise TypeError("warnings.showwarning() must be set to a "
    107                             "function or method")
--> 109         sw(msg.message, msg.category, msg.filename, msg.lineno,
    110            msg.file, msg.line)
    111         return
    112 _showwarnmsg_impl(msg)

File /usr/lib/python3.10/site-packages/numpy/testing/_private/utils.py:2273, in suppress_warnings._showwarning(self, message, category, filename, lineno, use_warnmsg, *args, **kwargs)
   2271 if self._forwarding_rule == "always":
   2272     if use_warnmsg is None:
-> 2273         self._orig_show(message, category, filename, lineno,
   2274                         *args, **kwargs)
   2275     else:
   2276         self._orig_showmsg(use_warnmsg)

AttributeError: 'suppress_warnings' object has no attribute '_orig_show'

NumPy/Python version information:

1.22.3 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0]

[seberg]

Thanks for the report. Yes, suppress_warnings was mainly meant for testing and is outdated anyway. But warnings.catch_warnings isn't actually better, unfortunately warning contexts are never thread-safe as of now.

In any case, I suspect that with gh-12447 this should just be deleted in any case, so it is probably a nice easy thing to fix up.