Show progress during unfairness mitigation

[romanlutz]

Is your feature request related to a problem? Please describe.

When running GridSearch or ExponentiatedGradient it may take a while to run all the iterations. It would be nice to have an idea about the progress.

Describe the solution you'd like

For GridSearch this could be something simple like completed 7/100 iterations, or perhaps something more sophisticated like completed 7/100 iterations, projected to finish in 4:32. For ExponentiatedGradient we can't know the number of iterations ahead of time, but we can tell the user how close we are to the stopping criterion as well as the current iteration number (in case we hit the maximum number of iterations).
We should perhaps also think about solutions other packages have for this.

[riedgar-ms]

As noted on the original email thread, setting the logger to 'debug' level will at least get regular output from the algorithms, so users know that something is happening.

[romanlutz]

Yeah, but that doesn't tell you much. You can sort of infer that each group of log messages is an iteration, but it's impossible to keep count...

[hildeweerts]

I haven't looked into fairlearn's GridSearch implementation so I don't know whether this is relevant, but I believe that in scikit-learn's GridSearchCV progress is printed through the joblib.Parallel dependency.

[romanlutz]

@hildeweerts it’s definitely relevant. @adrinjalali suggested using GridSearchCV for this a while back, so if we make such a change we could get that perhaps for free.

[riedgar-ms]

joblib.Parallel looks like the sort of thing we should be using to speed up GridSearch training. Would be a bit of an overkill just for adding a print statement for every iteration, though.

[arjsingh]

How about using something like tqdm?
https://tqdm.github.io/

[riedgar-ms]

That looks good as an option - please fork and PR :-)

[romanlutz]

The consensus in #517 was not to use tqdm to avoid introducing a new dependency. Instead, logging is preferable as @adrinjalali explained in #517 . If anyone wants to start with this task please comment below!