Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use DoRA as diff to standardize with sdwebui #3298

Closed
wants to merge 1 commit into from

Conversation

ntc-ai
Copy link

@ntc-ai ntc-ai commented Apr 19, 2024

There is a discrepancy in the implementation of DoRA between sdwebui and comfyui.

sdwebui: Implements DoRA as a weight difference.
comfyui: Directly modifies the weight.

The DoRA paper does not include the strength. Setting lora strength to zero in comfyui modifies the output image which is unexpected.
Testing the two approaches in https://github.com/catid/dora results in the same final evaluation loss.

Testing:

  • lora work as expected
  • loha/lkr/glora paths not tested

Resources:

sdwebui diff code: https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/adadb4e3c7382bf3e4f7519126cd6c70f4f8557b/extensions-builtin/Lora/network.py#L189
DoRA trained with sdwebui diff approach: https://civitai.com/models/397590/looking-away-to-looking-at-viewer-slidersntcaixyz?modelVersionId=443385

@synchromach
Copy link

Try rebasing onto master once again, the dev probably won't reply until you do it.

comfyanonymous added a commit that referenced this pull request May 25, 2024
This is a different version of #3298 with more correct behavior.
@comfyanonymous
Copy link
Owner

This should be slightly more correct than this pull request:
ffc4b7c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants