Feature Request: Ability to specify different prompts/weights/params for different objects or areas of the image

[TiagoTiago]

Describe the feature you'd like to see:

edit: Since the title got adjusted, here's the original suggestion I had in the title itself:

Move parameters to the composition node itself, add button to create additional prompts, each prompt with a weight parameter, and also checkbox to inpaint alpha/mask

---

Something I've been wanting to try was to have multiple simultaneous AI passes, each masked to different objects on the scene and with their own prompts, with objects that will rotate during animation (or have the camera move around them) having multiple prompts specifying from which angle it's visible with the weights for those prompts controlled by drivers that calculate values based on the rotation of the objects relative to the camera; and then finally combine all of those together in a final AI pass.

ps: Lemme know if you need me to break it down into separate suggestions for each part

Additional information

No response

[benrugg]

Wow, that's a very cool idea. I think that's probably beyond the scope of what I'll be able to do, but I would love collaboration for this. It would be pretty incredible to be able to have different prompts for different objects/areas of the screen.

[lucaspedrajas]

can you just enable multiple AI render nodes at the same time ? that would be helpful to make compositions, also the ultimate workflow would be to configure the prompt and everything in the AI render Node so we can have different prompts per render layer or per scene, we can figure out things to mask out the result

thanks for the add-on!

[benrugg]

Yeah, that's a cool idea. Right now the AI Render node doesn't work like that, but it could potentially be configured like that. I'll give that some consideration for the future!

[LeonardMeagher2]

can you just enable multiple AI render nodes at the same time ? that would be helpful to make compositions, also the ultimate workflow would be to configure the prompt and everything in the AI render Node so we can have different prompts per render layer or per scene, we can figure out things to mask out the result

thanks for the add-on!

https://www.youtube.com/watch?v=TYrHBFQPMys
There is probably an extension for everything! Very cool.
Not sure if a blender addon could solve everything automatic1111 does without a ton of work.

[benrugg]

Good call on that Automatic1111 extension. If they add some kind of API access, then it would actually be possible to do something like vertex painting or textures in Blender that could hook into multidiffusion. Which would be very cool.