Despite the great success of deep learning in stereo matching, recovering accurate disparity maps is still challenging. Currently, L1 and cross-entropy are the two most widely used losses for stereo network training. Compared with the former, the latter usually performs better thanks to its probability modeling and direct supervision to the cost volume. However, how to accurately model the stereo ground-truth for cross-entropy loss remains largely under-explored. Existing works simply assume that the ground-truth distributions are uni-modal, which ignores the fact that most of the edge pixels can be multi-modal. In this paper, a novel adaptive multi-modal cross-entropy loss (ADL) is proposed to guide the networks to learn different distribution patterns for each pixel. Moreover, we optimize the disparity estimator to further alleviate the bleeding or misalignment artifacts in inference. Extensive experimental results on public datasets show that our method is general and can help classic stereo networks regain state-of-the-art performance. In particular, GANet with our method ranks
- python == 3.9.12
- pytorch == 1.11.0
- torchvision == 0.12.0
- numpy == 1.21.5
- apex == 0.1
Download the datasets, and change the datapath
args. in ./scripts/sceneflow.sh
or ./scripts/kitti.sh
.
We use the distributed data parallel (DDP) to train the model.
Please execute the bash shell in ./scripts/
, as:
/bin/bash ./scripts/sceneflow.sh
/bin/bash ./scripts/kitti.sh
Training logs are saved in ./log/
.
Change loss_func
args. for different losses:
- SL1: smooth L1 loss
- ADL: ADaptive multi-modal cross-entropy Loss
If you want to train the GANet, please install the NVIDIA-Apex package and compile the GANet libs.
Please uncomment and execute val.py
.
EPE
, 1px
, 2px
, 3px
, D1
, 4px
, speed
are reported.
Change estimator
args. for different disparity estimators:
- softargmax: soft-argmax
- argmax: argmax
- SME: Single-Modal disparity Estimator
- DME: Dominant-Modal disparity Estimator