Is there a way to search mlflow runs through kedro-mlflow?

[praneeth5222]

Hello
I am new to kedro-mlflow and I have 2 use cases that I wasn't able to figure out through kedro-mlflow

1. Search runs
   Is there an equivalent of this search runs api implementation in kedro-mlflow?

mlflow.search_runs(experiment_names=[ "test" ],filter_string=tags.pipeline_name='test_pipe'\
                                       and attributes.status = 'FINISHED',
                                       order_by=["start_time DESC"])

2. Log multiple artifacts at once-
   I have a use case where the final kedro node needs to log around 10 json files to MLflow artifacts and if it is using the MLflow API, I can use this log_artifacts to log all the artifacts at once. Is there an equivalent of log_artifacts in kedro-mlflow

[Galileo-Galilei]

Hi @praneeth5222 ,

1. kedro-mlflow is basically just about setting up mlflow configuration and automatic logging, but all mlflow commands still stand so your usual mlflow code should be working as is:

mlflow.search_runs(experiment_names=[ "test" ],filter_string=tags.pipeline_name='test_pipe'\
                                       and attributes.status = 'FINISHED',
                                       order_by=["start_time DESC"])

What error are you facing?

2. You can use PartitionedDataSet to log a dict to a folder, e.g.:

my_dataset: # this is supposed to be a dict of json
    type: kedro_mlflow.io.artifacts.MlflowArtifactDataSet
    data_set:
        type: PartitionedDataSet  
        path: /path/to/a/local/folder # the attribute is "path", and not "filepath"!
        dataset: "json.JSONDataset"

[praneeth5222]

Thank you @Galileo-Galilei
I was under the impression that even for my first use case, I can do something from the data catalog and retrieve the mlflow model based on my search criteria

[Galileo-Galilei]

Hum, what do you want to load from the catalog? The latest model? You can fetch a lot of things from the run(s), so this is quite ambiguous.

Actually I think the best way to perform experiment it is to save all your experiments in the registry with many stages under the same "model name", and just load the last one with MlflowModelRegistryDataSet.

[praneeth5222]

Thank you @Galileo-Galilei
Is there a way to load a specific date experiment instead of the recent one with MlflowModelRegistryDataSet?