pytorch support

[ifangcheng]

This is a nice package for HPO, but it seems that it is built only on TF.
Just wondering is there any version that supports pytorch?
I think that will make the package more popular for HPO reseachers!

[lucfra]

Thanks @ifangcheng !

In fact, there is a person that's working on it; but I don't know when it will be ready.
I presume that at least some part of this package (the reverse mode hypergradient) should be easier to implement in Pytorch since - as far as I know - Pytorch should support by default dynamic computational graphs. Unfortunately I don't have much experience with Pytorch myself.... but, of course, if you want to help and get involved, you'd be very welcomed! :-)

Best,
Luca

[AntoineHX]

Hi,

Do you have any news on the work for the PyTorch implementation ? I'm really interested because i'd like to use specifically the reverse mode HG in PyTorch ! :D
Thanks for your great work, it's still very helpful even only on TF !

Best,
Antoine

[lucfra]

Hi @AntoineHX

We are shortly releasing a PyTorch implementation of an algorithm related to RTHO (real time hyperparamter optimization) for optimizing learning rate schedules online.
Unfortunately at the moment we have no specific short term plans of releasing a general implementation/wrappers for reverse mode HG in PyTorch, but this may change in the future.
I'll keep you updated!

Cheers,
Luca

[lucfra]

Hi all,

just wanted to post the link of a new package for gradient-based hyperparameter tuning in PyTorch.

https://github.com/awslabs/adatune

Right now it only contains the code for running an enhanced version of RTHO, but it may grow in the future months.

Cheers,
Luca