Read texture data

[Orctonk]

Hello, I am wondering whether there exists/are plans to implement a way to read back pixel data from a texture.

My current use case is to use filament to visualize physics simulations while also using filament cameras to simulate visual sensors. As such I would need a way to read back the depth buffer to simulate depth sensors.

Thanks in advance!

[bejado]

If you want to transfer the texture data to the CPU, we provide a readPixels API to do just that.

[Orctonk]

From what i can tell the readPixel API just reads the color data from the RenderTarget supplied. Is there something I'm missing or do I have to create a new RenderTarget with the Depth texture as color attachment to read the depth texture?

[bejado]

Ah, yes I don't think reading from the depth buffer is something we've tested recently. Glancing at the code, it should just work with the OpenGL backend if you provide PixelDataFormat::DEPTH_COMPONENT as the format argument to the PixelBufferDescriptor. This in turn will pass GL_DEPTH_COMPONENT to glReadPixels. Metal/Vulkan probably need some additional work to support this.

[Orctonk]

Sorry for the late reply, I was away last week!

I've now tried using PixelDataFormat::DEPTH_COMPONENT but it seems like the entire buffer is set to 0. It might be worth noting that I'm using the swapchain rendertarget. Do you have any other idea how this might be accomplished or a working example of reading the depth data?

[bejado]

I don't have any other suggestions off the top of my head, but when I get a chance I can try to spin up a quick demo and see what's going wrong.

[pixelflinger]

Reading the depth buffer is not supported in this way.

However, what you can do is use a custom rendertarget on a view and make sure you have post-processing disabled also make sure that the depth texture is marked as SAMPLEABLE. Once you've rendered that view, the depth texture should contain valid depth data.

BUT... I don't think this is the right way to do this -- this is because you can't easily interpret the content of the depth buffer. You don't know how we are encoding it. We could tell you (it's reversed Z with infinite far) but that could change.

So, I think a better solution is to use a special material for your scene object, then use a custom view/render target and have this material write the linear depth into the color buffer (e.g. in a R16F texture), then you can jsut use readpixels.

[gvasilev-walmart]

@pixelflinger @bejado
Hey guys
Are there any updates about depth buffer accessibility?