You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
and...
Process SpawnProcess-1:
Traceback (most recent call last):
File "/root/autodl-tmp/conda/envs/wslconda/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/root/autodl-tmp/conda/envs/wslconda/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/root/autodl-tmp/conda/envs/wslconda/lib/python3.10/site-packages/datadreamer/utils/background_utils.py", line 177, in _process_func_wrapper
restore_parent_process_context(parent_context=parent_context, env=env)
File "/root/autodl-tmp/conda/envs/wslconda/lib/python3.10/site-packages/datadreamer/utils/background_utils.py", line 155, in restore_parent_process_context
ctx.instance.enter()
AttributeError: 'UserDict' object has no attribute 'instance'
This package does not seem to implement parallel reasoning at all? According to my observation, it actually just repeats the same prompts to each model instead of sharding the prompts to the model, so there is no speed bonus for parallel reasoning at all.
from datadreamer.llms import ParallelLLM,VLLM,LLM,HFTransformers
VLLM(Mistral-7B-Instruct-v0.2", device=[1]).model
will rasie device error
File ~/autodl-tmp/conda/envs/wslconda/lib/python3.10/site-packages/datadreamer/llms/vllm.py:74, in VLLM.model(self)
72 env = os.environ.copy()
73 print(self.device)
---> 74 assert isinstance(self.device, list)
75 env.update(get_device_env_variables(self.device))
76 kwargs = self.kwargs.copy()
The text was updated successfully, but these errors were encountered: