SPEAR: A Simulator for Photorealistic Embodied AI Research

Interactive simulators are becoming powerful tools for training embodied agents, but existing simulators suffer from limited content diversity, physical interactivity, and visual fidelity. We address these limitations by introducing SPEAR: A Simulator for Photorealistic Embodied AI Research. To create our simulator, we worked closely with a team of professional artists for over a year to construct 300 unique virtual indoor environments with 2,566 unique rooms and 17,234 unique objects that can be manipulated individually. Each of our environments features detailed geometry, photorealistic materials, and a unique floor plan and object layout designed by a professional artist, i.e., we do not rely on remixing existing layouts to create additional content. Our environments are implemented as Unreal Engine assets, and we provide an OpenAI Gym interface for interacting with the environments via Python.

The SPEAR code is released under an MIT License, and the SPEAR assets are released under various licenses that permit academic use.

Citation

If you find SPEAR useful in your research, please cite this repository as follows:

@misc{roberts:2022,
    author       = {Mike Roberts AND Quentin Leboutet AND Rachith Prakash AND Renhan Wang AND
                    Hailin Zhang AND Rui Tang AND Marti Ferragut AND Stefan Leutenegger AND
                    Stephan R. Richter AND Vladlen Koltun AND Matthias M{\"u}ller AND German Ros},
    title        = {{SPEAR}: {A} Simulator for Photorealistic Embodied AI Research},
    howpublished = {\url{http://github.com/isl-org/spear}},
    year         = {2022},
}

Getting Started

Minimum and recommended system specifications

Minimum and recommended system specifications for the Unreal Engine are given here. Note that some of the Unreal Engine's latest rendering features are not available on macOS.

Precompiled binaries

See our latest release notes for download links. The easiest way to start working with SPEAR is to download a precompiled binary for your platform. Our precompiled binaries come pre-packaged with the scene pictured above. You can start interactively navigating around this scene with the keyboard and mouse simply by running the downloaded binary with no additional arguments.

Documentation

• Our Getting Started tutorial explains how to interact with multiple scenes and our Python interface.
• Our Building SpearSim tutorial explains how to build from source.
• Our Contribution Guidelines document contains information on how to contribute effectively.

Licenses

Code

• The code in this repository is licensed under an MIT License.
• The licenses for all of our third-party code dependencies are given here.

Assets

• The apartment, debug, and warehouse scenes are licensed under a CC0 License.
• The OpenBot and Fetch assets in this repository are licensed under a CC0 License.
• The license for the kujiale scenes is given here.
• The license for the StarterContent assets referenced in the debug scenes is given here.