April 2021
tl;dr: Propose a new task of open world object detection. Detect unknown objects, while able to continuous benefit from new labels on these unknown objects.
All existing object detectors have one strong assumption that all the classes are to be detected would be available at training phase. Although there are open set classification problem, the object detection is not studied. Object detectors are trained to detect unknown objects as background.
The open world object detector is required to detect unknown classes as unknown, and is able to detect these classes when the unknown instances are forwarded to a human annotator to label, without being trained from scratch.
- Contrastive loss (similar to push-pull loss) for each class and unknown class.
- Each class has a prototype vector (class specific feature anchor). This vector is updated after the burn-in stage and updated with momentum. --> looks like the method in MoCo
- With the features learned with contrastive loss, we can tell known from unknown, by the cls distribution of the logits.
- Autolabeling with RPN. Regions with high objectiveness score but do not have overlap with any GT are labeled as unknown.
-
Balanced finetuning
- How to learn new classes with a trained detector, data containing annotation of objects previously labeled as unknown.
- Train from scratch is impossible due to catestrophic forgetting. One way to alleviate forgetting is by storing few examples (
$N_{ex}$ = 50) and replaying, a proven effective method in few shot object detection. See FsDet.
- Evaluation metric on open world detection
- Wilderness Impact (WI): P (eval on known class) / P (eval on unknown class) - 1. This characterizes the impact of the introduction of new classes
- Absolute Open-Set Error (A-OSE): number of counts of unknown objects that get wrongly classified as known class.
- A sample qualitative result