Importing metrics issue since there is not a way to pass the model path if stored locally

[Starignus]

If you want to import a metric or the metrics module like:

from langkit import toxicity,
from langkit import llm_metrics 

By default, it downloads the models from the Huggingfaces when you try to import the module. The issue is when your organisation blocks the connection for downloading big files, but the organisation hosts the models in a secure location. For your reference, see this issue on the Transformers page

I searched in Langkit documentation for a way that the user could indicate the path of the models, but I could not find anything. Besides, it is impossible to pass any variable to a module when importing it. The problem can be solved by letting the user provide a path in a configuration file (e.g. JSON) that could override the default path. For example, in the toxicity module, I can see that the option can be taken.

This can be a potential blocker if an organisation wants to try the package and cannot since it might have some security concerns. This will be a good enhancement.

[FelipeAdachi]

Hi @Starignus ,

Thank you for reporting this!
We are working on this and will let you know how this advances in this thread.

[Starignus]

@FelipeAdachi Many thanks; I would appreciate the update!

[Starignus]

@FelipeAdachi, do you think calling any other model hosted locally will also be possible? e.g. Llama Guard