Demonstrate model testing done right

[archena]

Is your feature request related to a problem? Please describe.
Since this repo represents an ideal machine learning project, as part of that we should show how to test models. ML model testing is a broad topic, so it's a point of discussion of how it should be implemented. As opposed to software testing, model testing will not give a definitive answer to "Is the model performing well/as expected?", but can be used to compare experiments and newly trained models to previously deployed ones.

Describe the solution you'd like
To be discussed further, but there are at least two types of testing to be included:

• Single metric performance (e.g. accuracy -- already included in the training script, precision, recall, ROC AUC, etc.)
• Behavioral testing -- testing against specifically crafted test examples, to see if certain desired behaviours are satisfied.

Additional context
Some interesting things to look at for behavioural testing:

• https://www.aclweb.org/anthology/2020.acl-main.442/ -- the original paper
• https://www.youtube.com/watch?v=Cse-3MM7mso -- the video summary of the paper
• https://github.com/marcotcr/checklist -- the library used (NLP specific)