Implementation of specific BERT models

[stefanoscotta]

Is your feature request related to a problem?
It is not really a problem but it would make the code working in a more efficiently way, in particular in other languages.

What is the expected behavior?
In the configuration file it seems that I can only choose a generic "HF BERT", I was wondering if there was some other specific BERT model already implemented and how would it be possible to set them up.

Additional context (if any):

[arielge]

hi @stefanoscotta,
Thanks for your interest in Label Sleuth!
Generally, while there are a handful of models already implemented in Label Sleuth (See here), it is quite simple to add new models, following the steps here.
More specifically, currently there is just one implementation of BERT which uses the bert-base-uncased model. In the way it is implemented at the moment, the choice of these specific model weights stems from a default value set here in the HFTransformers class.
As I see it, the best way to support multiple transformer models would be to keep the existing HFTransformer class but without specifying a default value for the pretrained model. Then, there can be multiple implementations that use this existing training and inference code but rely on different model weights from Hugging Face. This would be similar to the way SVMs are implemented in LS, with a generic model class that implements the common model logic, and classes for specific implementations (1, 2) that merely initialize this generic class with different parameter values.
If this is something that interests you, it would be great if you could contribute this small change, which would indeed ease adding and supporting additional transformer models.