This repo provides the evaluation codes used in our ICCV 2019 paper A Delay Metric for Video Object Detection: What Average Precision Fails to Tell, including:
- Mean Average Precision (mAP)
- Average Delay (AD)
- A redesigned NAB metric for the video object detection problem.
Download the groundtruth annotations and the sample detector outputs from Google Drive.
The groundtruth annotations of VIDT are stored in KITTI-format due to its simplicity and io-efficiency.
We provide the outputs of the following methods. The github repos that generate those outputs are also listed.
All the evaluation scripts are under ./experiments folder. For instance, to measure the mAP and AD of FGFA, run command:
python experiments/eval_map_ad.py examples/rfcn_fgfa_7 data/ILSVRC2015_KITTI_FORMAT
For every video sequence, output a file as <sequence_name>.txt. Each line in the file should be one single object in <frame_id> <class_id> <confidence> <xmin> <ymin> <xmax> <ymax> format.
This pure Python-based mAP evaluation code is refactored from Cartucho/mAP. It has been tested against the original matlab version.