Change in ELBO during training

[soumyasanyal]

Hi,

Thanks for sharing the code, this is quite helpful! One thing I noticed while training the model - I find that the mean validation ELBo almost never improves (i.e. the validation ELBo after 1st epoch is the lowest among all 10 epochs of training). But training ELBo reduces smoothly (as expected). In such a situation, how do we choose the best model? Is it the one after 1st epoch (but might not be sufficiently trained) or its generally found that training for longer epochs helps SentenceVAE models so we pick the last checkpoint? Also, curious to know if others also faced similar issues while training or I'm missing something here?

Thanks,
Soumya