Description

find_pipelines is added since 0.18.3 as a pipeline autodiscovery feature. While this is useful, it is sometimes hard to debug because it raises warning but not error so the program will keep running. Cc @deepyaman

Context

For library developers, they may need strict control and want the pipeline fails explicitly. This features was designed for development, so any half-developed pipeline won't prevent other pipeline from running (incomplete pipeline, not installed library etc)

Some quotes from users:
User 1 (library developer)

We recently have users reporting issues when running tensorflow on M1. The error was hidden in some kedro pipelines and it took a while to dig out the root cause from the user. In general, it can be difficult to predict when and where we will encounter these issue. Therefore, we are planning to build some CI workflows for checking our packages and pipelines for M1 machines in general.

User 2 (general user)

I'm noticing that warnings like SyntaxErrors and type errors are considered "warnings" by Kedro, which continues to try to run the pipeline. Is there a setting to escalate these to Errors so they can abort the pipeline run?
15 days ago
Is there a way to mark a pipeline as a work in progress and then tell find_pipelines to error on warnings? Or is the answer to just not use pipeline autodiscovery

Possible Implementation

find_pipelines(raise_errors=False) - default unchanged, optionally fail eagerly.

Possible Alternatives