Extracting all images and segmentation masks for replica is a bit easy, but it's a bit complicated for matterport habitat 0.2, as they don't have a list of all the regions available, in their latest version (HM3D v0.2), which is here. They don't have much documentation for segmentation as well, so I decided to make my scripts public since it took a while for me to figure this out.

The script basically looks for heights from random unique navigable points, and uses them as "floors". It then extracts a topdown view of this floor and builds an "island" of navigable regions. It then creates various points on the island, separated by a certain distance, and looks around to take 360/turn_angle number of images. These points are sometimes not navigable, or it looks outside the island, giving a black image. The script removes these images by checking if there are more than 15% black(index 0) pixels in the segmentation mask that is generated by habitat. I think this extracts most of the images in the navigable regions. Any suggestions or questions are welcome.

• The dataset should be downloaded by following the instructions for HM3D 0.2 here:
• Make sure you have habitat installed. Conda worked for me, but other methods should work too. Follow the instructions here.
• The scripts were built by referring the demo here.