Replies: 7 comments
-
You can create dynamic workflows using this. |
Beta Was this translation helpful? Give feedback.
-
@WoaDmulL Where can I find example/documentation for |
Beta Was this translation helpful? Give feedback.
-
There's no documentation, yet...except of what I gave you before. So for example, you can build a chain of another tasks and replace the previous task with it.
The execution order will be Task A, B, C, D, ... . Everything will run in order ( asynchronously ). You can generate groups, chains or chords dynamically and replace them ( like an extension tree ). |
Beta Was this translation helpful? Give feedback.
-
@WoaDmulL Awesome thanks! Basically will end up creating a DAG, how do I get each of the tasks result or output or status ? Also im assuming if a task fails, the workflow will stop at failed and won't trigger tasks after failed ones right ? |
Beta Was this translation helpful? Give feedback.
-
For each task is assigned an UUID and each subtask has assigned a parent_id and a root_id. The root_id is always the task A ( from the example ), and the parent_id the id inherited from the previous task ( Task B parent_id is Task A id ). You can store them and use the signals or the on_success/on_failure methods to trace tasks' states. You could also use link_error option from the signature method and on_error method to do something else when a task fails. Another versions is to be sure that the workflow completely runs and each task finishes with success. |
Beta Was this translation helpful? Give feedback.
-
@WoaDmulL and @kavink, you may keep in mind #4299 and #4642 when working with canvas elements. |
Beta Was this translation helpful? Give feedback.
-
@jenstroeger Also both link_error/on_error are not working with class based tasks ( #4022 ). Also chords will fail in a workflow if the worker stops and you have to catch both WorkerLostError and WorkerShutdown and to retry the tasks. Also the way you create signatures has to be completely lazy, so a EDIT: You can't |
Beta Was this translation helpful? Give feedback.
-
Celery has very simple primitives for workflows, I want basically create dynamic workflows from config(which change very often). I did find Selinon which looks very close to what I want.
Wondering if there are any future plans for Celery to make the Workflows more extensible or should I try to integrate Selinon or What would you recommend if I need to basically dynamically create a DAG of dynamic tasks?
Beta Was this translation helpful? Give feedback.
All reactions