Training code for VAE

[kl2004]

Thanks for the great work and the released code base! 💯

After checking current training code, I notice that the VAE is loading from the pretrained checkpoint vae_ch160v4096z32.pth.

As mentioned #5, the training code for VAE will be released at https://github.com/FoundationVision/vae-pro. Is there a target date for it? I'm very interested in training a VAE model on my custom dataset.

[luohao123]

Same request here, @FoundationVision any plan to release the VAE training code?

[keyu-tian]

@kl2004 @kl2004 we're actively cleaning the codes now. For a temporary reference you can refer to the VQVAE forward at https://github.com/FoundationVision/VAR/blob/main/models/vqvae.py#L56-L59.

[luohao123]

thanks,

def forward(self, inp, ret_usages=False):   # -> rec_B3HW, idx_N, loss
        VectorQuantizer2.forward
        h_BChw, usages, vq_loss, mean_entropy_loss = self.quantize(self.quant_conv(self.encoder(inp)), ret_usages=ret_usages)
        return self.decoder(self.post_quant_conv(h_BChw)), usages, vq_loss, mean_entropy_loss

does this runable code?

[RobertLuo1]

I would like to if the evaluation (e.g. FID LPIPS PSNR, etc.) code of VAE can be released?

[eanson023]

Still looking forward to it...

[ArmeriaWang]

Thanks for the great work @FoundationVision , still looking forward to the training code for multi-res VQVAE!

[FanqingM]

Still lokking forward

[StarCycle]

Still looking forward!

[Junda24]

Still looking forward!