Is it possible to set artifact location?

[fernandosm]

I run pipelines locally with kedro, I have PostgreSQL server as backend store and I would like to have an artifact store in a network folder. Is it possible to set artifact location using kedro-mlflow plugin?

[Galileo-Galilei]

Hi @fernandosm,

first of all, sorry for the late reply. It seems that mlflow itself does not support this option just by setting environment variables. You need to setup your mlflow server like described in scenario 4 of documentation . Once the server is up, you just have to specify you mlflow_tracking_uri inside mlflow.yml.

You can find more detailed explanations here: #15.

[fernandosm]

Hi, @Galileo-Galilei,

First of all, thank you for your time answering the question 👍
About that, I think I did not make myself clear. I will try to explain better

Suppose that I have the following configuration:

• mlflow_tracking_uri (inside mlflow.yml) set as sqlite:///X:/mlrun/ml_db.sqlite
• and the server running at --backend-store-uri sqlite:///X:/mlrun/ml_db.sqlite and --default-artifact-root Y:/mlrun/
• and inside a node I call log_figure()

When I type kedro run from C:/kedro-project/ it saves information correctly inside X:/mlrun/ml_db.sqlite and artifacts are stored in C:/kedro-project/mlrun/ folder.

#15 has a great explanation and If I understood correctly, do I need to call log_artifact. Is that right?

Or does kedro-mlflow has something to deal with that already?

I am sorry for the ill-formulated question and I hope now it is more clear.

[Galileo-Galilei]

So if I understand correctly, your should modify your function like this to return the plot object:

# nodes.py

def my_func(x):
    ...
    fig=matplotlib.plt.plot()
    # mlflow.log_figure(fig) # <- remove this line 
    ...
    return result, fig # return the figure

Then return it in your pipeline.py:

def create_pipeline(**kwargs):
    pipeline(
        ...
        node(my_func, inputs="x_in_catalog", outputs=["my_result", "my_fig"]),
        ...
    )

And finally persist the result in the catalog:

catalog.yml
my_fig:
    type: kedro_mlflow.io.artifacts.MlflowArtifactDataSet
    data_set:
        type:  matplotlib.MatplotlibWriter # or any valid kedro DataSet
        filepath: /path/to/a/LOCAL/destination/file.csv # must be a local file, wherever you want to log the data in the end

It is true that I don't have a dataset to wrap log_figure which was introduced in recent mlflow versions, but above way works perfectly and is the old way tp do this, before log_figure was introduced.

Does it help?