feat(ui): protect against t2i adapters with incompatible image dimensions

[psychedelicious]

Summary

T2I Adapter requires images be multiples of 64. It's not clear if this is fixable or worth the effort to fix.

Pre-4.2.0, there was a little hack that set the image dimensions to the nearest multiple of 64 when selecting a T2I adapter. This was lost in the CL implementation. I don't like this solution - the user can change the image dimensions at any time and then they'll get a mysterious error when invoking.

To address the issue, an additional pre-Invoke check is added. You cannot click Invoke if you have a T2I layer and the image dimensions are not a multiple of 64.

Another issue apparent in the discord thread is that there is no way to differentiate between T2I adapter models and ControlNet models in the drop-down if the model names don't include "controlnet" or "t2i adapter" in them. It wasn't clear that the model in use was a T2I adapter model.

To address this, the model select now groups both by base model and model type:

While fixing up the pre-invoke check messages, I reworked them for Control Layers. It's a bit stricter now - you cannot invoke if an enabled RG layer has no mask, for example. The messages are more descriptive, too.

These changes are all CL-only - they do not apply to UC.

Related Issues / Discussions

https://discord.com/channels/1020123559063990373/1149506274971631688/1238333845648969789

Closes #5370

QA Instructions

Have a play with the pre-invoke checks, I believe that's the only change that may warrant some discussion.

Merge Plan

n/a

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• Documentation added / updated (if applicable)

[psychedelicious]

Also fixed pluralization on the tooltip for the generations count.

[Adreitz]

@psychedelicious In my experience, T2I-Adapters require a multiple of 32, not 64, for image resolution. I use T2I SDXL Canny for upscaling control in Nodes, and it works fine for controlling diffusion at 3840x2400 (not tiled). Am I misunderstanding the limitation?

[psychedelicious]

@Adreitz I found a note I had left myself some time ago saying that dimensions must be multiples of 64, but also I had read about it being 32. I did testing at lower resolutions then that and not all multiples of 32 worked, so I made it 64.

For example, 544 * 544 fails:

RuntimeError: The size of tensor a (68) must match the size of tensor b (64) at non-singleton dimension 3

64 works all the time. However, I realize now that I only tested on SD1.5. I just check and SDXL works with multiples of 32.

Digging deeper, we see why: https://github.com/invoke-ai/InvokeAI/blob/main/invokeai/app/invocations/latent.py#L762-L778

This confirms that SDXL works with multiples of 32 and SD1.5 works with multiples of 64. I can update the check to match these constraints.

@RyanJDick, can you advise if this is still a hard requirement for T2I Adapter?

[RyanJDick]

@RyanJDick, can you advise if this is still a hard requirement for T2I Adapter?

It's been a while since I looked at this, so my memory is a little fuzzy on the details. I made this PR on diffusers to reduce the requirements from "64 for SD1 and 32 for SDXL" to "8 for SD1 and 16 for SDXL". It sounds like we are still running into the old limits. I'd have to dig deeper to figure out why. What would say is the priority of this?

[hipsterusername]

Probably low.

[psychedelicious]

What would say is the priority of this?

IMO, we shouldn't spend any time on it. Nobody has complained about the constraints on image dimensions with T2I Adapter. The issue is that I've changed unwittingly changed constraints in the last release. I've addressed this #6366 by allowing images that are multiples of 32 when using SDXL T2I Adapter.

[Adreitz]

As far as I'm concerned, reduced restrictions are a nice-to-have but not necessary. Preventing a regression in the restrictions was my main point, and I think you've addressed that. Thanks.