Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: Can i use airflow operators with CWL-Airflow? #63

Open
Amitg1 opened this issue Jul 8, 2021 · 1 comment
Open

Question: Can i use airflow operators with CWL-Airflow? #63

Amitg1 opened this issue Jul 8, 2021 · 1 comment

Comments

@Amitg1
Copy link

Amitg1 commented Jul 8, 2021

Can I use airflow operators with CWL-Airflow? or only cmdtools like in CWL?

if so, how I am expected to push new operators or cmdtools to Airflow?

Thanks! great job.

@michael-kotliar
Copy link
Member

michael-kotliar commented Jul 8, 2021

Hello,
CWL format is more suitable for running command-line tools. And if you are using Docker containers each step will be completely isolated from others, so it's not an easy task to run standard Airflow operator from inside CWL step. I would recommend using something like ExternalTaskSensor for that purpose.

Would you mind describing your use case? Which Airflow Operators are you planning to use and after which workflow steps? Perhaps, this is some common scenario that I'm not aware of and that is worth implementing in CWL-Airflow natively.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants