Coco label for "invisible" object

[tensarflow]

Describe the issue

Hello!

I am trying to generate images of baskets with products in them. I need to label specific areas of the basket and specify their fullness as classes. The labels must be bounding boxes, so I'm using the write_coco_annotations function to generate the labels.
My issue is that the area that must be labeled is actually just a virtual area. There is no physical object that limits this area.
What I tried is to put an invisible plane mesh on top of the area by assigning a material and turning the alpha channel value to zero. I found out that this way the labels are not being generated. When I then tuned the alpha channel value a tiny bit up, just enough that it can be seen, the labels are being generated. But this was a bad solution because even though it is transparent, it can be seen with the naked eye. Then I had the genius idea :) to put bars on the sides of the basket segments (see image, the colored bars along the basket are labelled). This has the problem that it generates wrong labels if one of the bars are not rendered (duh). See the right most label where only the bar is labelled instead of the basket.
So I'm looking for a solution that allows me to generate bounding boxes of a virtual segment of the basket.
My last approach will be to write my own label generator that scans the scene for label meshes and calculates the bounding boxes on the image.

Minimal code example

No response

Files required to run the code

No response

Expected behavior

A solution that allows me to generate bounding boxes of a virtual segment of the basket

BlenderProc version

main branch

[cornerfarmer]

Hey @tensarflow,

sorry for the late response. One easy solution would be the following: You use the plane mesh, but dont make it transparent. Then in the end you render twice, first using your usual RGB rendering without the plane meshes and after adding the plane meshes an additional rendering where you only use the segmentation outputs. To make the overhead as small as possible, you could also set bproc.renderer.set_max_amount_of_samples(1) for the second rendering.
Let me know if that works for you.