Indicator of significance for the reports

[NickNtamp]

Not all reports have the same significance!

Below you can find an example of drift metrics but this also applies on performance.

Lets say that:

1. On Monday the client inserts an inference dataset with 1000 rows and a report of "Drift detected=FALSE" is produced.
2. On Tuesday the client inserts an inference dataset with 1000 rows and a report of "Drift detected=FALSE" is produced.
3. On Wednesday the client inserts an inference dataset with 35 rows and a report of "Drift detected=TRUE" is produced.

Even though we have a drift detection, seems that it is not important as there are much less datapoints for the specific day. We have to find a way to indicate if something is important or not (for instance for being important the drift have to be detected in a dataset with datapoints at least 50% of the average datapoints of the previous time periods - in this case for example 500 rows).

[momegas]

This should be covered my monitors at the moment, where the user can create a monitor and specify the severity of the alerts produced. I the monitors feature does not cover the full extend for this but should be ok for the current milestone. Keeping this as it a nice idea.