This is an implementation of HCCAN.
Some updates and corrections, please see update.md
Clone this repository.
git clone https://github.com/cutexin66/HCCAN
Please refer to GETTING_STARGTED.md to learn how to prepare the datasets and pretrained checkpoints.
The models with ResNet-50 backbone is available in [Gdrive]
RefCOCO | RefCOCO+ | RefCOCOg | Flickr30k | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
val | testA | testB | val | testA | testB | g-val | u-val | u-test | test | ||
R-50 | 83.61 | 86.66 | 78.57 | 69.90 | 74.96 | 59.33 | 71.53 | 71.49 | 71.84 | 79.47 |
-
Training
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 --use_env train.py --batch_size 8 --lr_bert 0.00001 --aug_crop --aug_scale --aug_translate --backbone resnet50 --detr_model ./checkpoints/detr-r50-unc.pth --bert_enc_num 12 --detr_enc_num 6 --dataset unc --max_query_len 20 --output_dir outputs/unc_r50 --epochs 90 --lr_drop 60
We recommend to set --max_query_len 40 for RefCOCOg, and --max_query_len 20 for other datasets.
We recommend to set --epochs 180 (--lr_drop 120 acoordingly) for RefCOCO+, and --epochs 90 (--lr_drop 60 acoordingly) for other datasets.
-
Evaluation
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch --nproc_per_node=8 --use_env eval.py --batch_size 8 --num_workers 4 --bert_enc_num 12 --detr_enc_num 6 --backbone resnet50 --dataset unc --max_query_len 20 --eval_set val --eval_model ./outputs/unc_r50/best_checkpoint.pth --output_dir ./outputs/unc_r50