Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations

By Prarthana Bhattacharyya and Krzysztof Czarnecki.

We provide code support and configuration files to reproduce the results in the paper for "Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations" on KITTI 3D object detection. Our code is based on OpenPCDet, which is a clean open-sourced project for benchmarking 3D object detection methods.

Introduction

Learning Deformations in 3D Object Detection.

Deformable PV-RCNN is a high-performing point-cloud based 3D object detector. Currently, the proposal refinement methods used by the state-of-the-art two-stage detectors cannot adequately accommodate differing object scales, varying point-cloud density, part-deformation and clutter. We present a proposal refinement module inspired by 2D deformable convolution networks that can adaptively gather instance-specific features from locations where informative content exists. We also propose a simple context gating mechanism which allows the keypoints to select relevant context information for the refinement stage. We outperform previously published methods on the highly competitive KITTI 3D Object Detection benchmark on cars and cyclists, and are the leading method among point-cloud based approaches on the Orientation Estimation benchmark for pedestrians and cyclists.

Demo

Visualizing PV-RCNN and Deformable PV-RCNN results.

Usage

a. Clone the repo:

git clone --recursive https://github.com/AutoVision-cloud/Deformable-PV-RCNN

b. Copy Deformable PV-RCNN src into OpenPCDet:

sh ./init.sh

c. Install OpenPCDet and prepare KITTI data:

Please refer to INSTALL.md for installation and dataset preparation.

d. Run experiments with a specific configuration file:

Please refer to GETTING_STARTED.md to learn more about how to train and run inference on this detector.

Results and models

The 3D AP results on KITTI 3D Object Detection validation split of the moderate category are shown in the table below.

	Car	Pedestrian	Cyclist	download
PV-RCNN	83.69	54.84	69.47	model-PV-RCNN
Deformable PV-RCNN-v1	83.30	58.33	73.46	model-def-PV-RCNN
Deformable PV-RCNN-v2	83.31	59.31	74.18	model-def-PV-RCNN-all

Notes:

• Our models are trained with Pytorch 1.5, 8 GTX 1080Ti GPUs and are available for download.
• v1 denotes using Conv3 and Conv4 features for deformations.
• v2 denotes using all convolutional features for deformations. The current configurations are set for v2.
• To run inference with v1, remove x_conv1 and x_conv2 from MODEL.PFE.DEF_SOURCE in the config file.

The results on KITTI 3D Object Detection test split are shown in the table below.

	FPS	Car	Pedestrian	Cyclist	3D mAP
PV-RCNN	13	81.43	43.29	63.71	62.81
Deformable PV-RCNN	12	81.46	40.89	68.54	63.63

The results on KITTI Orientation Estimation benchmark are shown in the table below.

	Car	Pedestrian	Cyclist	mAOS
PV-RCNN	94.57	52.42	79.70	75.56
Deformable PV-RCNN	94.52	54.38	80.05	76.32

Citing Deformable PV-RCNN

@misc{bhattacharyya2020deformable,
    title={Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations},
    author={Prarthana Bhattacharyya and Krzysztof Czarnecki},
    year={2020},
    eprint={2008.08766},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

Acknowledgement

• second.pytorch
• PointRCNN
• PV-RCNN