Pick transparent renderables

[grassydragon]

Closes #6614

[grassydragon]

Should I add a boolean material property to support excluding a material from picking?

[romainguy]

@pixelflinger can explain this in more detail but this function is in the deep inner loop of Filament and written to be branchless. You should be using long circuit operators instead of short circuit. I'm also not sure if we want to test for picking here.

[grassydragon]

You should be using long circuit operators instead of short circuit.

Then I can cast the boolean value to an integer to avoid the compilation error.

I'm also not sure if we want to test for picking here.

It is required to render the transparent objects in the picking pass.

[romainguy]

How are transparents handled with this change? Do they take precedence over opaques behind them?

[grassydragon]

How are transparents handled with this change? Do they take precedence over opaques behind them?

I believe that the whole scene is rendered to the picking buffer based on the fragment depth. Yes, a renderable is picked if it is the closest to the camera.

[pixelflinger]

Thanks for the PR I need to maul over it for a while. My areas of concerns:

• we have an extra pass (in the case of SSAO enabled, we would reuse the structure pass).
• the picking pass is now full resolution, so 4x more costly
• me might want a way to exclude the transparents (?)

Maybe a picking pass is the right way to do it, this would allow to control what goes into that pass. Maybe we need a way to select the resolution, because 1/4 res picking might still be useful at 1/4 the price.
Also wondering if there is a clever way to use the color/depth write modes to be able to generate the picking buffer in the same pass as the structure buffer as before.

Lots of things to think about...

[grassydragon]

me might want a way to exclude the transparents (?)

Yes, I think it would be great to use a boolean material property for that.

[grassydragon]

Hi @pixelflinger!
What are your thoughts on this pull request? I'm ready to attempt adding more changes if needed.

[pixelflinger]

Hi @pixelflinger! What are your thoughts on this pull request? I'm ready to attempt adding more changes if needed.

We're very sorry for the delay, we've been, and still are, very busy. This will take a bit longer.

[dandingol03]

@grassydragon @pixelflinger How is going about this pr. I also met the same problem which transparent cannot be picking in post-process

[grassydragon]

@grassydragon @pixelflinger How is going about this pr. I also met the same problem which transparent cannot be picking in post-process

I guess that everyone at Google is now preparing for I/O.

[hshapley]

With I/O behind us, any chance the team could take another look at this? We have a need for picking transparent renderables, so it'd be great to have a solution for this. Would supporting customization of the picking pass (ability to exclude transparents and specify resolution) be sufficient to get this approved? Or is the current approach sufficient for now? Thanks.

[grassydragon]

Hi @pixelflinger!
I'm ready to address any issues in the pull request, however, we need to agree on how this feature should be implemented. Does the Filament team have a complete vision on that?

[pixelflinger]

@grassydragon sorry for the long time getting back to this PR.

This is what I would like to see:

• have an option to include or not the transparents. I think there is value to not include the transparents in picking.
• reuse the structure pass in the case transparents are not included (each extra pass costs about the same as a shadow pass, it's not insignificant). Thanks to the framegraph design this can be done pretty easily: keep the old code, keep the new code and just select which "picking" resource you want to use based on wether transparents should be included -- the framegraph will take care of removing the pass that's not needed.