PRRendering Dataset Generator

Required Libraries & Setup

Most of the required libraries are automatically downloaded, compiled and installed correctly, however the requirements listed bellow need to be installed manually. On Linux, Clang-9 is needed to compile PhysX. For everything else GCC-9 is fine. On Windows, Visual Studio 19 has been tested and works, although earlier versions should be fine, too.

CMake 3.17

• Download & install at least version 3.15, preferrably 3.17
• Although 3.15 is supported, only 3.17 has been tested

Python 3.7

• Download & install according to the docs

Blender 2.82 as Python Module

• Setup the build pipeline accordingly for Windows or Linux
• Blender has to be built as described here, since there is no official release for the python module version
• An unofficial prebuilt version is provided & installed automatically
• The 2.82 folder needs to be put into the python install directory

Blenderseed 2.0

• Blenderseed just needs to be downloaded, as it is a Blender plugin
• Linux is currently not offically supported and needs to be built manually
• Appleseed is included in the Blenderseed plugin
• An unofficial prebuilt version is provided & installed automatically

Boost 1.73

• Most Linux distributions include boost
• On Windows it is necessary to install / build boost
• Boost is statically linked, build / install accordingly
• The required modules are System, Thread, Filesystem, Python (3.7, may have to be built manually!)

Boost build instructions

• Build bjam / b2 using bootstrap
• Create a user-config.jam in the same directory
• Adjust and add the following:

using python : 3.7 
  : Path-to-Python-dir\\python.exe
  : Path-to-Python-dir\\include
  : Path-to-Python-dir\\libs ;

• Execute b2 two times:

./b2 --with-python --with-filesystem --with-thread --with-system stage
./b2 --with-python --with-filesystem --with-thread --with-system install

• Potentially add and set BOOST_ROOT entry in cmake

3D Scan Dataset

• Download script is also provided in data/3Rscan. It requires python2 to run smoothly
• To prepare the dataset, run the unpack.sh script and set path to the downloaded data
• Set path of 3RScan in the config.json

3D Objects

• The 3D objects can be downloaded through the provided link
• Unpack the zip file into a folder and setup the config file
• Other meshes may be used instead, supported are Wavefront and glTF.

Building

Windows

• Create a new folder (e.g. build) and setup cmake
• Click generate & wait until all dependencies have been built
• Build the project using Visual Studio

Linux

• Make sure both Clang 9 & GCC 9 are installed
• Make sure boost & boost::python37 are available
• Make sure python 3.7 dev is available
• Create a new folder (e.g. build)
• Generate the project from root directory

cmake -S ./ -B ./build -DCMAKE_BUILD_TYPE=Release

• Compile the project in build folder

cd build && make -j 16

• Copy all PhysX libraries & the appleseed library from the build folder to /lib/

Configuration & Options

• The config.json file contains options & settings
• General settings are in the first block, available memory needs to be set
• The paths in the second block need to be set to folders & files
• Blurry image detection can be adjusted in the third block
• Simulation & render output can be controlled in the fourth block
• Object physics can be adjusted in the fifth block
• Optionally, custom intrinsics can be set in the sixth block
• Lastly, the objects that will be used in the simulation have to be defined

Scene & Meshes Layout

Meshes

• Meshes need to be of .obj, .ply or glTF 2.0 (.glb / .gltf) format
• Used meshes need to be listed in the corresponding array in the config file (render_objs)
• Each mesh has three properties but only the path is required
  • Each file path (mesh_path) can be either relative to the base directory (meshes_path) or an absolute path
  • The optional diffuse albedo texture (mesh_texture) needs to be in the same directory as the mesh
  • The classification (mesh_class) can be any arbitrary string
  • The scaling factor (mesh_unit) needs to be set so 1 unit = 1 meter
• If no texture path is given, the program attempts to use a default diffuse albedo texture (meshdir\objectname_color.png)
• The meshes are read with the assumption that Y=forward, Z=up

Scenes

• Each scene needs to have a rgbd folder and a mesh in .obj format
• The scene mesh needs to be named mesh.refined.obj, it may be scaled using the config file (scene_unit)
  • The mesh is read with the assumption that Y=forward, Z=up
  • If present, the diffuse albedo texture needs to be named mesh.refined_0.png
• The rgbd folder must contain a _info.txt, containing details about the used camera intrinsics
  • The camera width & height in pixels are named m_colorWidth = x & m_colorHeight = x
  • The intrinsics are a 4x4, rhs matrix named m_calibrationColorIntrinsic = x x ... (16 values)
  • Alternatively, the intrinsics may be set manually in the config file instead
• The frames have to end with framenum.color.jpg with corresponding framenum.pose.txt
• The pose file has to contain only the 4x4, rhs extrinsics matrix, one row per line, with Y=forward, Z=up