SpineNet-Pytorch

SpineNet is a scale-permuted backbone for object detection, proposed by Google Brain in CVPR 2020. This project is a kind of implementation of SpineNet using mmdetection.

It is highly based on the

• lucifer443/SpineNet-Pytorch
• The paper SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization
• Official TensorFlow Implementation

Models

COCO Object Detection Baselines

RetinaNet (Trained from scratch)

Backbone	Resolution	box AP	Params	FLOPs	box AP (paper)	Params (paper)	FLOPs (paper)	Download
SpineNet-49S	640x640	39.2	11.15M	30.04B	39.9	12.0M	33.8B	model
SpineNet-49	640x640	42.1	28.31M	83.7B	42.8	28.5M	85.4B	model
SpineNet-49	896x896	44.9	28.31M	164.05B	45.3	28.5M	167.4B	model
SpineNet-96	1024x1024	46.9	42.74M	261.35B	47.1	43.0M	265.4B	model
SpineNet-143	1280x1280	49.2	66.73M	518.32B	48.1	66.9M	524.4B	model
SpineNet-190	1280x1280	——	163.17M	1870.89B	52.1 (C)	163.6M	1885B	Training

Instance Segmentation Baselines

Mask R-CNN (Trained from scratch)

Backbone	Resolution	box AP	mask AP	Params	FLOPs	box mAP (paper)	mask mAP (paper)	Params (paper)	FLOPs (paper)	Download
SpineNet-49S	640x640	39.7	34.9	13.92M	63.77B	39.3	34.8	13.9M	60.2B	model
SpineNet-49	640x640	43.3	37.8	40.69M	231.17B	42.9	38.1	40.8M	216.1B	model
SpineNet-96	1024x1024	47.0	41.2	55.12M	330.72B	47.2	41.5	55.2M	315.0B	model
SpineNet-143	1280x1280	48.3	41.3	79.11M	515.58B	48.8	42.7	79.2M	498.8B	model

Note: The parameters and FLOPs are a little different from paper. More information about models can see in MODEL_DETAILS.md

Installation

1. Install mmdetection

This implementation is based on mmdetection(v1.1.0+8732ed9).

Please refer to INSTALL.md for more information.

a. Create a conda virtual environment and activate it.

conda create -n mmlab python=3.7 -y
conda activate mmlab

b. Install PyTorch and torchvision following the official instructions, e.g.,

conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.1 -c pytorch

c. Install mmcv

pip install mmcv==0.4.3

d. Clone the mmdetection repository.

git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection
git checkout 8732ed9

e. Install build requirements and then install mmdetection. (We install pycocotools via the github repo instead of pypi because the pypi version is old and not compatible with the latest numpy.)

pip install -r requirements/build.txt
pip install "git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI"
pip install -v -e .  # or "python setup.py develop"

Getting started

1. Copy the codes to mmdetection directory

git clone https://github.com/yan-roo/SpineNet-Pytorch.git
cp -r mmdet/ mmdetection/
cp -r configs/ mmdetection/

2. Prepare dataset (COCO)

wget http://images.cocodataset.org/zips/train2017.zip
wget http://images.cocodataset.org/zips/val2017.zip
wget http://images.cocodataset.org/zips/test2017.zip
wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip
unzip ${train/val/test}2017.zip
mv ${train/val/test}2017  ${mmdetection/data/coco/}

The directories should be arranged like this:

 >   mmdetection
 >     ├── mmdet
 >     ├── tools
 >     ├── configs
 >     ├── data
 >     │   ├── coco
 >     │   │   ├── annotations
 >     │   │   ├── train2017
 >     │   │   ├── val2017
 >     │   │   ├── test2017

3. Train a model

*Important*: The default learning rate in SpineNet-49S config files is for 8 GPUs and 32 img/gpu (batch size = 8*32 = 256).

According to the Linear Scaling Rule, you need to set the learning rate proportional to the batch size if you use different GPUs or images per GPU.

e.g., lr=0.28 for 8 GPUs * 32 img/gpu and lr=0.07 for 8 GPUs * 8 img/gpu.

You also can set the warm-up iterations.

e.g., warmup_iters=2000 for batch size 256 and warmup_iters=8000 for batch size 64

Train with a single GPU

Modify config/spinenet/model.py Line5 type='SyncBN' -> type='BN'

CONFIG_FILE=configs/spinenet/spinenet_49S_B_8gpu.py
python tools/train.py ${CONFIG_FILE} [optional arguments]

[optional arguments]: --resume_from ${epoch_.pth} / --validate

Train with multiple GPUs

CONFIG_FILE=configs/spinenet/spinenet_49S_B_8gpu.py
bash tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments]

4. Calculate parameters and FLOPs

python tools/get_flops.py ${CONFIG_FILE} --shape $SIZE $SIZE

5. Evaluation

python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --out  ${OUTPUT_FILE} --eval bbox
python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --out  ${OUTPUT_FILE} --eval bbox segm

More usages can reference GETTING_STARTED.md or MMDetection documentation.

Issues & FAQ

1. ModuleNotFoundError: No module named 'mmcv.cnn.weight_init'

   pip install mmcv==0.4.3	
   

2. ImportError: libtorch_cpu.so: cannot open shared object file: No such file or directory

   rm -r build
   python setup.py develop
   

3. AssertionError: Default process group is not initialized

   Modify config/spinenet/model.py Line5 type='SyncBN' -> type='BN'