SeqMatchNet

Code for the CoRL 2021 Oral paper "SeqMatchNet: Contrastive Learning with Sequence Matching for Place Recognition and Relocalization"

[OpenReview] [PDF] [CoRL 2021 YouTube Video]

SeqMatchNet: Contrastive Learning with Sequence Matching.

Setup

Conda

conda create -n seqnet numpy pytorch=1.8.0 torchvision tqdm scikit-learn faiss tensorboardx h5py wandb -c pytorch -c conda-forge

Download

Run bash download.sh to download single image NetVLAD descriptors (3.4 GB) for the Nordland-clean dataset ^[a] and the Oxford dataset (0.3 GB) ^[b].

You can download model trained on Oxford from here.

Run

Train

To train SeqMatchNet on the Oxford dataset with both the loss and negative mining based on sequence matching:

python main.py --mode train --seqL 5 --pooling --dataset oxford-v1.0 --loss_trip_method meanOfPairs --neg_trip_method meanOfPairs --expName ox10_MoP_negMoP

For the Nordland dataset:

python main.py --mode train --seqL 5 --pooling --dataset nordland-sw --loss_trip_method meanOfPairs --neg_trip_method meanOfPairs --expName nord-sw_MoP_negMoP

To train without sequence matching:

python main.py --mode train --seqL 5 --pooling --dataset oxford-v1.0 --loss_trip_method centerOnly --neg_trip_method centerOnly --expName ox10_CO_negCO

Test

python main.py --mode test --seqL 5 --pooling -dataset oxford-v1.0 --split test --resume ./data/runs/<name_of_the_model_file>

Acknowledgement

The code in this repository is based on oravus/seqNet and Nanne/pytorch-NetVlad.

Citation

@inproceedings{garg2021seqmatchnet,
  title={SeqMatchNet: Contrastive Learning with Sequence Matching for Place Recognition \& Relocalization},
  author={Garg, Sourav and Vankadari, Madhu and Milford, Michael},
  booktitle={5th Annual Conference on Robot Learning},
  year={2021}
}

Other Related Projects

SeqNet; Delta Descriptors (2020); Patch-NetVLAD (2021); CoarseHash (2020); seq2single (2019); LoST (2018)

[a] This is the clean version of the dataset that excludes images from the tunnels and red lights and can be downloaded from here.

[b] These will automatically save to ./data/, you can modify this path in download.sh and get_datasets.py to specify your workdir.