Replies: 1 comment
-
I met the same problem when I am using DBSCAN in a thread. I tried
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
We are trying to upgrade scikit-learn version from 0.24.2 to 1.0.2 (or the latest stable version) Our code is running on EKS pods.
From time to time (it’s not always reproducible), when we call means.predict() the process hangs.
There is no high CPU or memory consumption and we’re not seeing any log messages. It looks like nothing really happens in the pod.
After we generated stack traces of all the threads every minute, we saw that the following thread hangs and the stack trace doesn’t change
The input to _make_controller_from_path that makes the process hang, changes every time. Paths that worked in one process (e.g. /usr/lib/x86_64-linux-gnu/libssl.so.1.1), caused the thread to hang in another process.
When we initialize the threadpoolctl for sklearn first thing in our process by setting
sklearn._sklearn_threadpool_controller = threadpoolctl.ThreadpoolController()
, this doesn’t happen (we’ve tested it many times)This is a possible, but not an elegant workaround, and we very much prefer to understand and fix the root cause.
We load the kmeans model via pickle this way:
pickle.load(io_bytes).predict(descriptors)
In light of the stack trace and the workaround above, we don’t think that this is the problem.Did anyone encounter such a problem and can help us understand the root cause?
Thank you
Beta Was this translation helpful? Give feedback.
All reactions