Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic task mapping - partial args are not check and only fail at trigger #39639

Open
1 of 2 tasks
raphaelauv opened this issue May 15, 2024 · 2 comments
Open
1 of 2 tasks
Assignees
Labels
area:dynamic-task-mapping AIP-42 kind:bug This is a clearly a bug

Comments

@raphaelauv
Copy link
Contributor

raphaelauv commented May 15, 2024

Apache Airflow version

2.9.1

What happened?

    PythonOperator(
        task_id="test_2",
        python_callable=a,
        pool="pool_1",
        pool_slots="a",
    )

airflow fail with

Screenshot from 2024-05-15 14-34-18

but

with

    PythonOperator.partial(
        task_id="test",
        python_callable=a,
        pool="pool_1",
        pool_slots="a",
    ).expand(show_return_value_in_logs=[True])

airflow do not fail to parse the file and fail at trigger of the pipeline

What you think should happen instead?

airflow should detect incorrect args of partial

How to reproduce

from airflow.operators.python import PythonOperator
from pendulum import today
from airflow import DAG

with DAG(
        dag_id="test_partial",
        schedule_interval=None,
        start_date=today("UTC").add(days=-1)):
    def a():
        return

    PythonOperator.partial(
        task_id="test",
        python_callable=a,
        pool="pool_1",
        pool_slots="a",
    ).expand(show_return_value_in_logs=[True])

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@raphaelauv raphaelauv added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels May 15, 2024
@RNHTTR RNHTTR added area:dynamic-task-mapping AIP-42 and removed area:core needs-triage label for new issues that we didn't triage yet labels May 16, 2024
@karenbraganz
Copy link

I would like to be assigned this issue.

@RNHTTR
Copy link
Collaborator

RNHTTR commented May 20, 2024

Done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dynamic-task-mapping AIP-42 kind:bug This is a clearly a bug
Projects
None yet
Development

No branches or pull requests

3 participants