PointCloud generated with yarp::sig::utils::depthRgbToPC has points with depth 0

[ste93]

Hi all,

I've noticed that the generation of the pointcloud with yarp::sig::utils::depthRgbToPC has inside points with depth zero (that in fact are not valid). This can lead to both use a lot of network without the real need and also to filter later the pointcloud to discard these points. I was wondering if this behaviour is wanted or rather if we can change the generation to discard these points (maybe adding also a dense or something else parameter to avoid generating these points or simply discard them when depth is 0 if they are not used)

a simple fix can be:

size_t pointNumber = 0;
for (size_t u = 0; u < w; ++u) {
    for (size_t v = 0; v < h; ++v) {
        if (depthImage.pixel(u,v)){
            yarp::sig::DataXYZRGBA point;
            point.x = (u - intrinsics.principalPointX)/intrinsics.focalLengthX*depthImage.pixel(u,v);
            point.y = (v - intrinsics.principalPointY)/intrinsics.focalLengthY*depthImage.pixel(u,v);
            point.z = depthImage.pixel(u,v);

            point.r = colorImagePixelRGB.pixel(u,v).r;
            point.g = colorImagePixelRGB.pixel(u,v).g;
            point.b = colorImagePixelRGB.pixel(u,v).b;
            yarpCloud.push_back(point);
            pointNumber ++;
        }
    }
}
yarpCloud.resize(pointNumber);
PointCloud2Type& pc2Ros = rosPublisherPort_pointCloud.prepare();

[ste93]

cc @drdanz @randaz81 @Nicogene @elandini84 @morpheus1820

[PeterBowman]

Such a point cloud is said to be "organized" in PCL terminology. If you want to interface with PCL, there are different algorithms specifically designed for working with either organized or unorganized clouds. The former are usually much, much faster since it is possible to iterate over rows and columns pretty much like in a typical camera frame. At roboticslab-uc3m, we are currently exploiting this so that a RGBD frame acquisition is followed by a pinhole-model conversion into a point cloud, which we are able to convert then into a mesh in real time. Without a dense an organized structure, this is not possible.

Edit: this YT video shows an example of RT depth-frame-to-mesh conversion, simulated environment, real camera (context at roboticslab-uc3m/vision#102).

[Nicogene]

Yes as @PeterBowman mentioned it is a matter if you want to produce an organized or unorganized PointCloud, maybe we can add a flag for that(organized by default)

[PeterBowman]

The convention is that height = 1 corresponds to an unorganized cloud, which the YARP implementation already follows (see isOrganized()).

I need to amend my previous comment. "Dense" seems to indicate that there are no NaNs or Infs in the cloud (ref). Not to be confused with "organized".

[ste93]

Ok thanks, I will check it and add a flag

[ste93]

fixed in pr #2601