Pick vertex/facet/cell under the cursor

[sebmestrallet]

Hello,

I try to find a replacement for libigl's unproject_onto_mesh() (unproject a screen location to pick a facet).

None of Geogram's examples seems to use this kind of functionality (geogram_demo_Delaunay2d uses the cursor position, but there is no projection on a 3D mesh).

While looking into Graphite, I dug under the "probe attributes" tool, which find the facet under the cursor to display attributes. I discovered MeshGrobTool, RayPick and RenderingContext classes, but all of this seems independant of Geogram.

Is this functionality available in Geogram ?

[BrunoLevy]

Graphite's picking works as follows (and all basic functionalities are implemented in Geogram, I confirm):

1. draw the mesh with color-coded element ids (depending on whether you want to pick points/edges/facets/cells, the elements you want to pick are drawn with color-coded ids, all the rest is drawn in white / 0xffffff color).
2. read the id of the picked element using glReadPixels
3. unproject the picked point to get its 3D location

For 1), MeshGfx in geogram_gfx/mesh/mesh_gfx.h has a set_picking_mode function (to draw an image with element IDs instead of standard rendering), and you can chose the type of elements you want to pick. Just do that in the back buffer without calling SwapBuffers (and no one will notice !)

For 2), it is something like:

           Memory::byte pixel[4];
            glPixelStorei(GL_PACK_ALIGNMENT, 1);
            glPixelStorei(GL_PACK_ROW_LENGTH, 1);
            glReadPixels(
                GLint(x),GLint(y),1,1,GL_RGBA,GL_UNSIGNED_BYTE,pixel
            );

For 3), the GLUP library in geomgram_gfx re-implements the old fixed functionality pipeline in OpenGL3, so it has glupUnProject() that works just like its ancestor.

For a reference, see:

1. drawing the mesh with color-coded IDs: MeshGfx reference manual

2. reading the picked element: relevant lines in Graphite sources

3. un-projecting the point: relevant lines in Graphite sources

[sebmestrallet]

Thank you very much Bruno, I'm going to test this

[sebmestrallet]

Okay, I got it working!

Additional info:

• There's a lot of coordinates transformation in RenderingContext : screen_to_ndc(), ndc_to_screen() + get_picked_point() that flips the Y axis. In Geogram, SimpleApplication::cursor_pos_callback() also do some kind of normalization. For glReadPixels(), we only need pixels coordinates from the bottom-left corner (that is, arguments of cursor_pos_callback() as integers, Y axis flipped).
• draw_scene() must be called between set_picking_mode() and glReadPixels()

Here's my code (SimpleMeshApplication-based class) :

void cursor_pos_callback( double x, double y, int source ) override {
    cursor_pos_ = vec2(x,y); // store x and y in a private variable
    SimpleMeshApplication::cursor_pos_callback(x,y,source);
}

void mouse_button_callback(int button, int action, int mods, int source) override {
    if((action==EVENT_ACTION_DOWN) && (button == 0)) { // if left click
        index_t x = index_t(cursor_pos_.x), y = index_t(cursor_pos_.y); // double to integer conversion of current cursor position
        if(x >= get_width() || y >= get_height()) { // if cursor out of the window
            return;
        }
        y = get_height()-1-y;// change Y axis orientation
        mesh_gfx()->set_picking_mode(MESH_FACETS); // instead of rendering colors, mesh_gfx will render facet indices
        draw_scene(); // rendering
        // read the id of the picked facet using glReadPixels()
        Memory::byte picked_facet_id_as_pixel[4];
        glPixelStorei(GL_PACK_ALIGNMENT, 1);
        glPixelStorei(GL_PACK_ROW_LENGTH, 1);
        glReadPixels(
            GLint(x),GLint(y),1,1,GL_RGBA,GL_UNSIGNED_BYTE,picked_facet_id_as_pixel
        );
        mesh_gfx()->set_picking_mode(MESH_NONE); // go back to color rendering mode
        // decode the facet id from the pixel's color
        index_t picked_facet_id =
                 index_t(picked_facet_id_as_pixel[0])        |
                (index_t(picked_facet_id_as_pixel[1]) << 8)  |
                (index_t(picked_facet_id_as_pixel[2]) << 16) |
                (index_t(picked_facet_id_as_pixel[3]) << 24);
        if(picked_facet_id == index_t(-1)) { // if no facet under the cursor
            return;
        }
        // do what you want with picked_facet_id, like updating a facet attribute
    }
    SimpleMeshApplication::mouse_button_callback(button,action,mods,source);
}

[BrunoLevy]

Fantastic !
If you want to send a PR with a new example, I'll merge it (may help other people)

[sebmestrallet]

For reference, the new example has been merged in #102