Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]Failed to run execute(wait=False) in another thread #3229

Open
VessaliusOz opened this issue Aug 18, 2022 · 1 comment
Open

[BUG]Failed to run execute(wait=False) in another thread #3229

VessaliusOz opened this issue Aug 18, 2022 · 1 comment
Assignees
Labels
type: bug Something isn't working

Comments

@VessaliusOz
Copy link

Describe the bug
execute(wait=False) failed when running in a thread pool

import os
import time
import mars.tensor as mt
import mars.dataframe as md
import concurrent.futures


def func():
    df = md.DataFrame(
            mt.random.rand(4, 4, chunk_size=2),
            columns=list("abcd"),
        )
    df.execute()
    pid = os.getpid()
    info = df.apply(
        lambda s: s if os.getpid() == pid else (time.sleep(5) or s)
    ).execute(wait=False)
    result = info.result()
    return result


with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
    future = executor.submit(func)
    print(future.result())

To Reproduce
To help us reproducing this bug, please provide information below:

  1. Your Python version 3.9.13
  2. The version of Mars you use 0.9.0
  3. Versions of crucial packages, such as numpy, scipy and pandas
    i. numpy: 1.23.0
    ii. ray: 1.13.0
    iii. scipy: 1.8.1
    iv. pandas: 1.4.3
  4. Full stack of the error.
Traceback (most recent call last):
  File "/Users/vessalius/.local/share/virtualenvs/outer_mars-bUMhzz2w/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 101, in _ensure_future
    self._future_local.future
AttributeError: '_thread._local' object has no attribute 'future'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/vessalius/Desktop/ray_serving_test/problem.py", line 25, in <module>
    print(future.result())
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 446, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/vessalius/Desktop/ray_serving_test/problem.py", line 19, in func
    result = info.result()
  File "/Users/vessalius/.local/share/virtualenvs/outer_mars-bUMhzz2w/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 131, in result
    self._ensure_future()
  File "/Users/vessalius/.local/share/virtualenvs/outer_mars-bUMhzz2w/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 110, in _ensure_future
    self._future_local.aio_future = asyncio.wrap_future(fut)
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/futures.py", line 411, in wrap_future
    loop = events.get_event_loop()
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/events.py", line 642, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-1_0'.
  1. Minimized code to reproduce the error.

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

@fyrestone fyrestone self-assigned this Aug 18, 2022
@fyrestone
Copy link
Contributor

The ExecutionInfo.result() does not have to call asyncio.wrap_future. I will fix it.

@fyrestone fyrestone added the type: bug Something isn't working label Aug 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants