"RuntimeError: main thread is not in main loop" when training multiple sklearn classifiers

[lise-brinck]

I am experiencing an error that I don't understand and I don't understand why my fix works.

I am training multiple sklearn classifiers with different feature sets and tracking them all as child runs of a parent run.

possible_models = [
    Perceptron(shuffle=False),
    LogisticRegression(),
    SGDClassifier(shuffle=False),
]

with mlflow.start_run():
    for model in possible_models:
        for feature_set in feature_sets:
            with mlflow.start_run(nested=True):
                mlflow.autolog(log_datasets=False)
                run_and_log(model, df, feature_set, test_datapath)

def run_and_log(
    model: BaseEstimator, df: pd.DataFrame, feature_set: List[str], test_datapath: str
):
    X_train, y_train = get_data_from_df(df, feature_set)
    model.fit(X_train, y_train)
    feature_names = model.feature_names_in_
    X_test, y_test = load_testset(test_datapath, feature_names)
    signature = mlflow.models.infer_signature(X_train, model.predict(X_test))
    mlflow.sklearn.log_model(
        model,
        artifact_path="model",
        signature=signature,
    )
    scores = score_model(model, X_test, y_test)
    mlflow.log_metrics(scores)

feature_sets is a list of lists of features, List[List[str]].
get_data_from_df() returns a tuple of a pd.DataFrame with only the features from feature_set and a pd.Series.
load_testset() loads a pd.DataFrame from a path to a csv-file and adds features from the classifier.
score_models() returns a dict with different metrics, i.e., precision, accuracy etc.

It works fine with just one classifier, but when it gets to the second one, I get the following error:

Exception ignored in: <function Image.__del__ at 0x7f2d91df8ae0>
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.11.4/lib/python3.11/tkinter/__init__.py", line 4082, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.__del__ at 0x7f2d98649ee0>
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.11.4/lib/python3.11/tkinter/__init__.py", line 410, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.__del__ at 0x7f2d98649ee0>
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.11.4/lib/python3.11/tkinter/__init__.py", line 410, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.__del__ at 0x7f2d98649ee0>
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.11.4/lib/python3.11/tkinter/__init__.py", line 410, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop
Exception ignored in: <function Variable.__del__ at 0x7f2d98649ee0>
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.11.4/lib/python3.11/tkinter/__init__.py", line 410, in __del__
    if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop
Tcl_AsyncDelete: async handler deleted by the wrong thread

It seems, the error occurs becasue of mlflow.log_metrics() and as far as I can tell, it has to do with matplotlib.
This Stackoverflow thread lead me to the solution of setting the backend for matplotlib. I don't understand why this solves it.

Furthermore, it seems simply importing matplotlib into the Python script that I'm running the training from also works. Even without setting the backend for matplotlib. This, I don't understand at all.

Can anyone enlighten me on why the error occurs and why importing matplotlib solves it?