Skip to content

Code and dataset from "RGB-D-E: Event Camera Calibration for Fast 6-DOF Object Tracking"

License

Notifications You must be signed in to change notification settings

lvsn/rgbde_tracking

Repository files navigation

RGB-D-E_tracking

Project page available here.

Evaluation code and dataset from "RGB-D-E: Event Camera Calibration for Fast 6-DOF Object Tracking " [arxiv paper]

Evaluation Dataset

Download the evaluation dataset here (12 GB).

The dataset contains multiple sequences each in different folder. Each sequence contains the following files:

  • camera.json: RGB-D sensor (Microsoft Kinect Azure) intrinsic calibration
  • dvs.json: Event based sensor (DAVIS346) intrinsic calibration
  • transfo_mat.npy: Extrinsic calibration
  • fevents.npz: Events data of shape 4xN (Timestamps, x, y, Polarity)
  • davis_frame.npz: Grayscale frame record from the event sensor
  • davis_frame_ts.npz: Timestamps associated to each grayscale frame
  • frames.npz: RGB-D frames from the Microsoft Kinect Azure
  • ts_frames.npz: Timestamps associated to each RGB-D frame
  • poses.npy: Ground truth 6DoF poses

Tracker

Download

Download and extract:

Running tracker

This repository used submodule and it should be initiated:

git submodule update --init --recursive

Update your PYTHONPATH:

export PYTHONPATH=$PYTHONPATH:./6DOF_tracking_evaluation

To run the tracker on the whole dataset and compute the tracking failures for both networks:

python tracking_event_6dof/inference/tracker_failure.py \
    -e ./model/event
    -f ./model/frame
    -d ./dataset
    -m ./dragon

To generate video result of each sequence:

python tracking_event_6dof/inference/tracker_failure.py \
    -e ./model/event
    -f ./model/frame
    -d ./dataset
    -m ./dragon
    -a /path/to/folder/to/save/videos

Note: Those examples suppose that the assets are extracted in the root folder

Citation

@misc{dubeau2020rgbde,
    title={RGB-D-E: Event Camera Calibration for Fast 6-DOF Object Tracking},
    author={Etienne Dubeau and Mathieu Garon and Benoit Debaque and Raoul de Charette and Jean-François Lalonde},
    year={2020},
    eprint={2006.05011},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

About

Code and dataset from "RGB-D-E: Event Camera Calibration for Fast 6-DOF Object Tracking"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages