[Feature Request] Allow the target function in evaluate to be a local function

[pamelafox]

I currently get this error when passing a local function to evaluate:

promptflow.exceptions.UserErrorException: Failed to import <function run_evaluation..wrap_target at 0x318bc7ba0> from module <module 'scripts.evaluate' from '/Users/pamelafox/ai-rag-chat-evaluator/scripts/evaluate.py'>, please make sure it's a global function.

That is due to the usage of multiprocessing and its need to serialize the function.

Perhaps you could use asyncio.taskgroup or a different mechanism such that local functions are allowed. That would permit me to write much cleaner and more portable, testable code.

[D-W-]

Hi @pamelafox , thanks for the suggestion. We've already tracking this internally. The reason we did not support this in the first place was to maintain parity between local and cloud (it's hard to run a local function in cloud). However, we will prioritize support for local scenarios first.