Add the ability to automatically select best batch size

[sarthakpati]

Is your feature request related to a problem? Please describe.
Currently, GaNDLF requires the user to select the optimal batch_size for the hardware they are training on, which is not very efficient in the case of hyper-parameter tuning.

Describe the solution you'd like
Adding the ability to choose the best batch_size automatically based on every other hyper-parameter would make the process much smoother.

Describe alternatives you've considered
N.A.

Additional context

• https://stackoverflow.com/a/66429367
• https://discuss.pytorch.org/t/how-to-determine-the-largest-batch-size-of-a-given-model-saturating-the-gpu/146075/2

[github-actions]

Stale issue message

[github-actions]

Stale issue message

[benmalef]

@sarthakpati you can assign this to me

[sarthakpati]

@sarthakpati you can assign this to me

Thanks! 👍🏽

[github-actions]

Stale issue message