Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Abortable task support #888

Open
1 task done
jerome-benoit opened this issue Aug 20, 2023 · 0 comments · May be fixed by #1625
Open
1 task done

[FEATURE] Abortable task support #888

jerome-benoit opened this issue Aug 20, 2023 · 0 comments · May be fixed by #1625
Labels
enhancement New feature or request

Comments

@jerome-benoit
Copy link
Contributor

jerome-benoit commented Aug 20, 2023

Duplicate issue

  • I've searched for any related issues and avoided creating a duplicate issue.

Description

Node.js offers an abortable API that can be used to implement abortable task in poolifier.

The issue with this API is that abort signal can't be transferred to workers as a transferable object for worker_threads, and the abortable API does not seem to be implemented in cluster module.
And aborting a task at the cost of killing the worker running it is not acceptable:

  • other tasks will be also stopped, abortables or not
  • cluster or worker_threads worker restart, which one of the main source of performance degradation, can lead to OOM condition if several abort signals are sent, ... .

Definitely not a proper solution for a worker pool aiming at performance and fine grained controlling and signalling about tasks lifecycle. And a pool not offering pre-instantiated stable processus/threads/workers to execute code block can't be called a pool ;-)

Alternatives solutions:

  • abort signal 'abort' trigger a message sent to the worker node handling the task and the logic to cancel it: queued => removed, executing => code handling the cancelation.
  • abort signal 'abort' trigger a 'message' using the IPC infrastructure with atomicity to come. Avoid the known bottlenecks on MessageChannel (worker_threads). Cluster IPC for messaging does not seem to be hitten by the same bottlenecks.

The API can be similar to the existing Node.js modules using abortable signal (although I'm not very fond of it).

Attachments

No response

@jerome-benoit jerome-benoit added the enhancement New feature or request label Aug 20, 2023
@jerome-benoit jerome-benoit linked a pull request Nov 18, 2023 that will close this issue
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant