Questions about the paper.

[pengzhangzhi]

Hi, thank you for sharing such impressive work! I am wondering if there is another path to derive a diffusion classifier. Diffusion model is an SDE, which has a closed ODE form. The ODE allows us to calculate the exact log-likelihood of an input X. The higher the likelihood, the more reliable the model believes the input X is from the distribution. So I think using the log-likelihood can be another strategy to implement all the functionalities in your paper, such as zero-shot classifier.

Best,
Zhangzhi

[alexlioralexli]

Hi Zhangzhi,

Thanks for bringing this up! I am a big fan of the score SDE and FFJORD papers. I did look into treating the diffusion model as a continuous normalizing flow, but this approach has some issues that I'm working through. The main problems are:

1. Batching is hard. Solving the probability flow is inherently sequential, whereas our Monte Carlo estimates of the ELBO are trivial to parallelize.
2. Calculating the log-likelihood via the ODE requires computing the divergence (trace of jacobian), which is difficult to estimate. Previous work typically uses Hutchison's trace estimator, but this requires many samples to get a precise value of the divergence. Tricks like the matched $\epsilon$ (that we used in our paper) will probably help with reducing variance, but there appears to be no way around random sampling.

We're actively working on getting a fast implementation and adding these results to the paper. Stay tuned!

[pengzhangzhi]

Yes! You did a great job on this paper. I am just curious about how effective the log-likelihood estimation could be since song claimed that the exact log-likelihood is SOTA. Looking forward to your updates!

[black0017]

Yes! You did a great job on this paper. I am just curious about how effective the log-likelihood estimation could be since song claimed that the exact log-likelihood is SOTA. Looking forward to your updates!

Hello @pengzhangzhi, which paper exactly of Song et al. are you referring to?