About optimize Pctxt

[shouwangzhe200]

According to above equation, is it only the learnable prompt Pctxt trained during finetune? Or the paramters of Unet are also finetuned at the same time?

[icsl-Jeon]

I am also very curious..

[lijiaxing0213]

Optimization should only be performed on the embeddings of the task prompt, while the parameters of the unet are frozen. You can refer to the implementation of the EmbeddingLayerWithFixes class at https://mmagic.readthedocs.io/en/latest/autoapi/mmagic/models/editors/disco_diffusion/clip_wrapper/index.html#mmagic.models.editors.disco_diffusion.clip_wrapper.EmbeddingLayerWithFixes.