You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was using a code from visp. They have a script where you can generate all the images (RGB, depth, normal etc) and a 2D bounding box for yolov7 export. Now there was 1 problem that I saw in their script that the pose estimation w.r.t camera frame seems wrong (cTo) but there bounding box was correct.
Now I want to extract the 8 corners and 1 centroid in 3D space as well as the 2D projection of those 9 vertices on camera plane (9 pixel cordinates). Is there a script you suggest that is already doing it the way visp is doing it (scriptable way) so that I can just incorporate the code block ??
Minimal code example
They have a bounding box code like this:
defbounding_box_2d_from_vertices(object: bproc.types.MeshObject, K: np.ndarray, camTworld: np.ndarray) ->Tuple[List[float], float, np.ndarray]:
''' Compute the 2D bounding from an object's vertices returns A tuple containing: [xmin, ymin, xmax, ymax] in pixels the proportion of visible vertices (that are not behind the camera, i.e., negative Z) The 2D points, in normalized units in camera space, in ViSP/OpenCV frame '''worldTobj=homogeneous_no_scaling(object)
obj=object.blender_objverts=np.ones(len(obj.data.vertices) *3)
obj.data.vertices.foreach_get("co", verts)
points=verts.reshape(-1, 3)
# verts = [v.co for v in obj.data.vertices]camTobj=camTworld @ worldTobjpoints_cam=camTobj @ np.concatenate((points, np.ones((len(points), 1))), axis=-1).Tpoints_cam=convert_points_to_visp_frame((points_cam[:3] /points_cam[3, None]).T)
visible_points=points_cam[points_cam[:, 2] >0]
visible_points_m_2d=visible_points[:, :2] /visible_points[:, 2, None]
visible_points_px_2d=K @ np.concatenate((visible_points_m_2d, np.ones((len(visible_points_m_2d), 1))), axis=-1).Tvisible_points_px_2d=visible_points_px_2d.T[:, :2] /visible_points_px_2d.T[:, 2, None]
mins=np.min(visible_points_px_2d, axis=0)
assertlen(mins) ==2maxes=np.max(visible_points_px_2d, axis=0)
return [mins[0], mins[1], maxes[0], maxes[1]], len(visible_points) /len(points_cam), visible_points_m_2d
I tried to add something similar to this:
defproject_bounding_box_to_image_plane(object, K, camTworld):
""" Project the 3D bounding box and centroid of an object onto the 2D image plane using BlenderProc. Parameters: - object: bproc.types.MeshObject, the object whose bounding box to project. - K: The camera intrinsic matrix. - camTworld: The camera-to-world transformation matrix. Returns: - List of 2D coordinates of the projected bounding box vertices and the centroid on the image plane. """# Get the 8 corners of the bounding box in world coordinatesbounding_box_world=np.array(object.get_bound_box())
# Calculate the centroid of the bounding boxcentroid_world=np.mean(bounding_box_world, axis=0)
# Combine bounding box corners with the centroidpoints_world=np.vstack([bounding_box_world, centroid_world])
# Transform points from world to camera coordinatesworldTcam=np.linalg.inv(camTworld)
points_world_homogeneous=np.hstack((points_world, np.ones((points_world.shape[0], 1))))
points_camera_homogeneous=worldTcam @ points_world_homogeneous.T# Project points onto image planepoints_image_homogeneous=K @ points_camera_homogeneous[:3, :]
points_image=points_image_homogeneous[:2, :] /points_image_homogeneous[2, :]
returnpoints_image.T.tolist()
But it seems that both the object dimension and keypoints didn't match.
Files required to run the code
No response
Expected behavior
Here was the output:
Both the keypoint location and the dimension is incorrect.
BlenderProc version
v2.7.0
The text was updated successfully, but these errors were encountered:
Describe the issue
I was using a code from visp. They have a script where you can generate all the images (RGB, depth, normal etc) and a 2D bounding box for yolov7 export. Now there was 1 problem that I saw in their script that the pose estimation w.r.t camera frame seems wrong (cTo) but there bounding box was correct.
Now I want to extract the 8 corners and 1 centroid in 3D space as well as the 2D projection of those 9 vertices on camera plane (9 pixel cordinates). Is there a script you suggest that is already doing it the way visp is doing it (scriptable way) so that I can just incorporate the code block ??
Minimal code example
They have a bounding box code like this:
I tried to add something similar to this:
But it seems that both the object dimension and keypoints didn't match.
Files required to run the code
No response
Expected behavior
Here was the output:
Both the keypoint location and the dimension is incorrect.
BlenderProc version
v2.7.0
The text was updated successfully, but these errors were encountered: