[BUG] Pipelines being created with double the amount of blocks

[jgonzaz]

Mage version

v0.9.65

Describe the bug

Using a pipeline to orchestrate the firing of other pipelines. Triggered pipelines are being created with double the desired amount of steps.

Project is set up in AWS and using ecs executor (config below) and run_in_one_process = True in the triggered pipeline metadata.yaml file . Currently using ecs executor but this behavior has been seen using both ecs executor and local_python pipeline executors

To reproduce

Have not been able to successfully reproduce the issue

Expected behavior

Pipeline to be created with the number of blocks present in the metadata.yaml file for pipeline.

Screenshots

Operating system

No response

Additional context

No response

[wangxiaoyou1993]

@jgonzaz are you able to reproduce this issue in local Mage?

[jgonzaz]

@wangxiaoyou1993 Have not been able to reproduce in mage locally. Believe it was likely tied to ECS scheduler setup. Maxed out the memory on the scheduler which likely led to the duplication of steps. Increased resources for the scheduler snd have not seen this error duplicated