Use arbitrary open-source llm

[TribeDH]

Hi everyone, is it possible (or will be in the future) to set the -model parameter to an arbitrary llm outside the models list? For example, this can be useful for extracting entities in documents written in a different language or using domain-specific fine-tuned llm. Thanks for your time.

[caufieldjh]

Thanks @TribeDH. There are definite plans to support this.

[TribeDH]

That's awesome! After some tests on an italian crime news dataset, I found that llama-2-7b-chat gets the better results in a foreign language, and adding some simple prompt engineering adjustments in the class' prompt section (for example "act as an italian speaker") really improves the results. I hope this can help the research meanwhile the llm upgrade is done.

[caufieldjh]

Fantastic! I've also seen some very exciting results with llama2-7b and mistral-7b so I want to be sure we support those.