refactor with open-mmlab://mmdetection

Author: xvjiarui

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,44 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+Thanks for your bug report and we appreciate that a lot.
+
+**Checklist**
+- [ ] I have searched related issues but could not get the expected help.
+- [ ] The bug has not been fixed in the latest version.
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+If there are any related issues or upstream bugs, please also refer to them.
+
+**Error traceback**
+1. What command or script did you run?
+```
+A placeholder for the command.
+```
+2. If applicable, paste the error trackback here using code blocks.
+```
+A placeholder for trackback.
+```
+
+**Reproduction details**
+1. Did you make any modifications on the code? Did you understand what you have modified?
+2. What dataset did you use?
+
+**Environment**
+ - OS: [e.g., Ubuntu 16.04.6]
+ - GCC [e.g., 5.4.0]
+ - PyTorch version [e.g., 1.1.0]
+- How you installed PyTorch [e.g., pip, conda, source]
+- GPU model [e.g., 1080Ti, V100]
+- CUDA and CUDNN version
+- [optional] Other information that may be related (such as `$PATH`, `$LD_LIBRARY_PATH`, `$PYTHONPATH`, etc.)
+
+**Bug fix**
+If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,22 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the feature**
+
+**Motivation**
+A clear and concise description of the motivation of the feature.
+Ex1. It is inconvenient when [....].
+Ex2. There is a recent paper [....], which is very helpful for [....].
+
+**Related resources**
+If there is an official code release or third-party implementations, please also provide the information here, which would be very helpful.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
+If you would like to implement the feature and create a PR, please leave a comment here and that would be much appreciated.

.github/ISSUE_TEMPLATE/general_questions.md

@@ -0,0 +1,10 @@
+---
+name: General questions
+about: Ask general questions to get help
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+

.gitignore

@@ -109,4 +109,13 @@ mmdet/version.py
 data
 .vscode
 .idea
+<<<<<<< HEAD
 work_dirs
+=======
+
+# custom
+*.pkl
+*.pkl.json
+*.log.json
+work_dirs/
+>>>>>>> upstream/master

.style.yapf

@@ -0,0 +1,4 @@
+[style]
+BASED_ON_STYLE = pep8
+BLANK_LINE_BEFORE_NESTED_CLASS_OR_DEF = true
+SPLIT_BEFORE_EXPRESSION_AFTER_OPENING_PAREN = true

.travis.yml

@@ -0,0 +1,13 @@
+dist: xenial
+language: python
+
+install:
+  - pip install flake8
+
+python:
+  - "3.5"
+  - "3.6"
+  - "3.7"
+
+script:
+  - flake8

LICENSE

@@ -1,3 +1,5 @@
+Copyright 2018-2019 Open-MMLab. All rights reserved.
+
                                  Apache License
                            Version 2.0, January 2004
                         http://www.apache.org/licenses/
@@ -186,7 +188,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright [yyyy] [name of copyright owner]
+   Copyright 2018-2019 Open-MMLab.
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -3,7 +3,20 @@
 By [Yue Cao](http://yue-cao.me), [Jiarui Xu](http://jerryxu.net), [Stephen Lin](https://scholar.google.com/citations?user=c3PYmxUAAAAJ&hl=en), Fangyun Wei, [Han Hu](https://sites.google.com/site/hanhushomepage/).
 
 This repo is a official implementation of ["GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond"](https://arxiv.org/abs/1904.11492) on COCO object detection based on open-mmlab's [mmdetection](https://github.com/open-mmlab/mmdetection).
-Many thanks to mmdetection for their simple and clean framework.
+Many thanks to mmdetection for their simple and clean framework. 
+
+*Update on 2019/07/01*
+
+The code is refactored. 
+More results are provided and all configs could be found in `configs/gcnet`.
+
+**Notes**: Both PyTorch official SyncBN and Apex SyncBN have some stability issues. 
+During training, mAP may drops to zero and back to normal during last few epochs. 
+
+*Update on 2019/06/03*
+
+GCNet is supported by the official mmdetection repo [here](https://github.com/open-mmlab/mmdetection/tree/master/configs/gcnet). 
+Thanks again for open-mmlab's work on open source projects.
 
 ## Introduction
 
@@ -22,41 +35,56 @@ Many thanks to mmdetection for their simple and clean framework.
 
 ## Main Results
 
-| Backbone  |      Model       |    Context     | Lr schd | box AP | mask AP | Download  |
-| :-------: | :--------------: | :------------: | :-----: | :----: | :-----: | :-------: |
-| R-50-FPN  |       Mask       |       -        |   1x    |  37.2  |  33.8   | [model](https://1drv.ms/f/s!AkEXj14LxwVpcEkPyk8YWZVm9fo) |
-|           |                  | GC(c3-c5, r16) |   1x    |  39.4  |  35.7   | [model](https://1drv.ms/f/s!AkEXj14LxwVpbSe8MYcjXToB090) |
-|           |                  | GC(c3-c5, r4)  |   1x    |  39.9  |  36.2   | [model](https://1drv.ms/f/s!AkEXj14LxwVpaiEOLB67my2og3c) |
-| R-101-FPN |       Mask       |       -        |   1x    |  39.8  |  36.0   | [model](https://1drv.ms/f/s!AkEXj14LxwVpcTx0JnBi1tuK4KU) |
-|           |                  | GC(c3-c5, r16) |   1x    |  41.1  |  37.4   | [model](https://1drv.ms/f/s!AkEXj14LxwVpbqlp5GGBiZsoVaU) |
-|           |                  | GC(c3-c5, r4)  |   1x    |  41.7  |  37.6   | [model](https://1drv.ms/f/s!AkEXj14LxwVpa-WTFi9kcyuWB8I) |
-| X-101-FPN |       Mask       |       -        |   1x    |  41.2  |  37.3   | [model](https://1drv.ms/f/s!AkEXj14LxwVpcgwQlf8ok6IZahc) |
-|           |                  | GC(c3-c5, r16) |   1x    |  42.4  |  38.0   | [model](https://1drv.ms/f/s!AkEXj14LxwVpb_4H8Q_bsmHZP7c) |
-|           |                  | GC(c3-c5, r4)  |   1x    |  42.9  |  38.5   | [model](https://1drv.ms/f/s!AkEXj14LxwVpbLZR1Wu2JjMNyW0) |
-| X-101-FPN |   Cascade Mask   |       -        |   1x    |  44.7  |  38.3   | [model](https://1drv.ms/f/s!Ak5O6Kz9hYOghnGJIdfM2fWlBsMR) |
-|           |                  | GC(c3-c5, r16) |   1x    |  45.9  |  39.3   | [model](https://1drv.ms/f/s!Ak5O6Kz9hYOghm7gwd5yd4ZX-Dub) |
-|           |                  | GC(c3-c5, r4)  |   1x    |  46.5  |  39.7   | [model](https://1drv.ms/f/s!Ak5O6Kz9hYOghk9roKJSNNpJGGsE) |
-| X-101-FPN | DCN Cascade Mask |       -        |   1x    |  47.1  |  40.4   | [model](https://1drv.ms/f/s!Ak5O6Kz9hYOghmw3k3L6SbKl84Ex) |
-|           |                  | GC(c3-c5, r16) |   1x    |  47.9  |  40.9   | [model](https://1drv.ms/f/s!Ak5O6Kz9hYOghm98OS64BAt0ZuOo) |
-|           |                  | GC(c3-c5, r4)  |   1x    |  47.9  |  40.8   | [model](https://1drv.ms/f/s!Ak5O6Kz9hYOghlHmLvwSeF43ZISq) |
-
+|  Backbone |       Model      | Backbone Norm |       Heads      |     Context    | Lr schd | Mem (GB) | Train time (s/iter) | Inf time (fps) | box AP | mask AP | Download |
+|:---------:|:----------------:|:-------------:|:----------------:|:--------------:|:-------:|:--------:|:-------------------:|:--------------:|:------:|:-------:|:--------:|
+|  R-50-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   |        -       |    1x   |    3.9   |        0.453        |      10.6      |  37.3  |   34.2  | [model]()|
+|  R-50-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   | GC(c3-c5, r16) |    1x   |    4.5   |        0.533        |      10.1      |  38.5  |   35.1  | [model]()|
+|  R-50-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   |  GC(c3-c5, r4) |    1x   |    4.6   |        0.533        |       9.9      |  38.9  |   35.5  | [model]()|
+|  R-50-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   |        -       |    2x   |     -    |          -          |        -       |  38.2  |   34.9  | [model]()|
+|  R-50-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   | GC(c3-c5, r16) |    2x   |     -    |          -          |        -       |  39.7  |   36.1  | [model]()|
+|  R-50-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   |  GC(c3-c5, r4) |    2x   |     -    |          -          |        -       |  40.0  |   36.2  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   |        -       |    1x   |    3.9   |        0.543        |      10.2      |  37.2  |   33.8  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   | GC(c3-c5, r16) |    1x   |    4.5   |        0.547        |       9.9      |  39.4  |   35.7  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   |  GC(c3-c5, r4) |    1x   |    4.6   |        0.603        |       9.4      |  39.9  |   36.2  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   |        -       |    2x   |    3.9   |        0.543        |      10.2      |  37.7  |   34.3  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   | GC(c3-c5, r16) |    2x   |    4.5   |        0.547        |       9.9      |  39.7  |   36.0  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   |  GC(c3-c5, r4) |    2x   |    4.6   |        0.603        |       9.4      |  40.2  |   36.3  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    | 4conv1fc(SyncBN) |        -       |    1x   |     -    |          -          |        -       |  38.8  |   34.6  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    | 4conv1fc(SyncBN) | GC(c3-c5, r16) |    1x   |     -    |          -          |        -       |  41.0  |   36.5  | [model]()|
+|  R-50-FPN |       Mask       |     SyncBN    | 4conv1fc(SyncBN) |  GC(c3-c5, r4) |    1x   |     -    |          -          |        -       |  41.4  |   37.0  | [model]()|
+| R-101-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   |        -       |    1x   |    5.8   |        0.571        |       9.5      |  39.4  |   35.9  | [model]()|
+| R-101-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   | GC(c3-c5, r16) |    1x   |    7.0   |        0.731        |       8.6      |  40.8  |   37.0  | [model]()|
+| R-101-FPN |       Mask       |     fixBN     |    2fc(w/o BN)   |  GC(c3-c5, r4) |    1x   |    7.1   |        0.747        |       8.6      |  40.8  |   36.9  | [model]()|
+| R-101-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   |        -       |    1x   |    5.8   |        0.665        |       9.2      |  39.8  |   36.0  | [model]()|
+| R-101-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   | GC(c3-c5, r16) |    1x   |    7.0   |        0.778        |       9.0      |  41.1  |   37.4  | [model]()|
+| R-101-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   |  GC(c3-c5, r4) |    1x   |    7.1   |        0.786        |       8.9      |  41.7  |   37.6  | [model]()|
+| X-101-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   |        -       |    1x   |    7.1   |        0.912        |       8.5      |  41.2  |   37.3  | [model]()|
+| X-101-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   | GC(c3-c5, r16) |    1x   |    8.2   |        1.055        |       7.7      |  42.4  |   38.0  | [model]()|
+| X-101-FPN |       Mask       |     SyncBN    |    2fc(w/o BN)   |  GC(c3-c5, r4) |    1x   |    8.3   |        1.037        |       7.6      |  42.9  |   38.5  | [model]()|
+| X-101-FPN |   Cascade Mask   |     SyncBN    |    2fc(w/o BN)   |        -       |    1x   |     -    |          -          |        -       |  44.7  |   38.3  | [model]()|
+| X-101-FPN |   Cascade Mask   |     SyncBN    |    2fc(w/o BN)   | GC(c3-c5, r16) |    1x   |     -    |          -          |        -       |  45.9  |   39.3  | [model]()|
+| X-101-FPN |   Cascade Mask   |     SyncBN    |    2fc(w/o BN)   |  GC(c3-c5, r4) |    1x   |     -    |          -          |        -       |  46.5  |   39.7  | [model]()|
+| X-101-FPN | DCN Cascade Mask |     SyncBN    |    2fc(w/o BN)   |        -       |    1x   |     -    |          -          |        -       |  47.1  |   40.4  | [model]()|
+| X-101-FPN | DCN Cascade Mask |     SyncBN    |    2fc(w/o BN)   | GC(c3-c5, r16) |    1x   |     -    |          -          |        -       |  47.9  |   40.9  | [model]()|
+| X-101-FPN | DCN Cascade Mask |     SyncBN    |    2fc(w/o BN)   |  GC(c3-c5, r4) |    1x   |     -    |          -          |        -       |  47.9  |   40.8  | [model]()|
 **Notes:**
 
 - `GC` denotes Global Context (GC) block is inserted after 1x1 conv of backbone. 
 - `DCN` denotes replace 3x3 conv with 3x3 Deformable Convolution in `c3-c5` stages of backbone.
 - `r4` and `r16` denote ratio 4 and ratio 16 in GC block respectively. 
+- Some of models are trained on 4 GPUs with 4 images on each GPU.
 
 ## Requirements
 
 - Linux(tested on Ubuntu 16.04)
 - Python 3.6+
-- PyTorch 1.0.0
+- PyTorch 1.1.0
 - Cython
 - [apex](https://github.com/NVIDIA/apex) (Sync BN)
 
 ## Install
 
-a. Install PyTorch 1.0 and torchvision following the [official instructions](https://pytorch.org/).
+a. Install PyTorch 1.1 and torchvision following the [official instructions](https://pytorch.org/).
 
 b. Install latest apex with CUDA and C++ extensions following this [instructions](https://github.com/NVIDIA/apex#quick-start). 
 The [Sync BN](https://nvidia.github.io/apex/parallel.html#apex.parallel.SyncBatchNorm) implemented by apex is required.
@@ -97,7 +125,7 @@ Please refer to mmdetection install [instruction](https://github.com/open-mmlab/
 ### Software environment
 
 - Python 3.6.7
-- PyTorch 1.0.0
+- PyTorch 1.1.0
 - CUDA 9.0
 - CUDNN 7.0
 - NCCL 2.3.5

compile.sh

@@ -36,3 +36,10 @@ if [ -d "build" ]; then
     rm -r build
 fi
 $PYTHON setup.py build_ext --inplace
+
+echo "Building masked conv op..."
+cd ../masked_conv
+if [ -d "build" ]; then
+    rm -r build
+fi
+$PYTHON setup.py build_ext --inplace

configs/cascade_mask_rcnn_r101_fpn_1x.py

@@ -24,7 +24,9 @@
         anchor_strides=[4, 8, 16, 32, 64],
         target_means=[.0, .0, .0, .0],
         target_stds=[1.0, 1.0, 1.0, 1.0],
-        use_sigmoid_cls=True),
+        loss_cls=dict(
+            type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+        loss_bbox=dict(type='SmoothL1Loss', beta=1.0 / 9.0, loss_weight=1.0)),
     bbox_roi_extractor=dict(
         type='SingleRoIExtractor',
         roi_layer=dict(type='RoIAlign', out_size=7, sample_num=2),
@@ -40,7 +42,15 @@
             num_classes=81,
             target_means=[0., 0., 0., 0.],
             target_stds=[0.1, 0.1, 0.2, 0.2],
-            reg_class_agnostic=True),
+            reg_class_agnostic=True,
+            loss_cls=dict(
+                type='CrossEntropyLoss',
+                use_sigmoid=False,
+                loss_weight=1.0),
+            loss_bbox=dict(
+                type='SmoothL1Loss',
+                beta=1.0,
+                loss_weight=1.0)),
         dict(
             type='SharedFCBBoxHead',
             num_fcs=2,
@@ -50,7 +60,15 @@
             num_classes=81,
             target_means=[0., 0., 0., 0.],
             target_stds=[0.05, 0.05, 0.1, 0.1],
-            reg_class_agnostic=True),
+            reg_class_agnostic=True,
+            loss_cls=dict(
+                type='CrossEntropyLoss',
+                use_sigmoid=False,
+                loss_weight=1.0),
+            loss_bbox=dict(
+                type='SmoothL1Loss',
+                beta=1.0,
+                loss_weight=1.0)),
         dict(
             type='SharedFCBBoxHead',
             num_fcs=2,
@@ -60,7 +78,15 @@
             num_classes=81,
             target_means=[0., 0., 0., 0.],
             target_stds=[0.033, 0.033, 0.067, 0.067],
-            reg_class_agnostic=True)
+            reg_class_agnostic=True,
+            loss_cls=dict(
+                type='CrossEntropyLoss',
+                use_sigmoid=False,
+                loss_weight=1.0),
+            loss_bbox=dict(
+                type='SmoothL1Loss',
+                beta=1.0,
+                loss_weight=1.0))
     ],
     mask_roi_extractor=dict(
         type='SingleRoIExtractor',
@@ -72,7 +98,9 @@
         num_convs=4,
         in_channels=256,
         conv_out_channels=256,
-        num_classes=81))
+        num_classes=81,
+        loss_mask=dict(
+            type='CrossEntropyLoss', use_mask=True, loss_weight=1.0)))
 # model training and testing settings
 train_cfg = dict(
     rpn=dict(
@@ -90,8 +118,14 @@
             add_gt_as_proposals=False),
         allowed_border=0,
         pos_weight=-1,
-        smoothl1_beta=1 / 9.0,
         debug=False),
+    rpn_proposal=dict(
+        nms_across_levels=False,
+        nms_pre=2000,
+        nms_post=2000,
+        max_num=2000,
+        nms_thr=0.7,
+        min_bbox_size=0),
     rcnn=[
         dict(
             assigner=dict(
@@ -146,9 +180,9 @@
 test_cfg = dict(
     rpn=dict(
         nms_across_levels=False,
-        nms_pre=2000,
-        nms_post=2000,
-        max_num=2000,
+        nms_pre=1000,
+        nms_post=1000,
+        max_num=1000,
         nms_thr=0.7,
         min_bbox_size=0),
     rcnn=dict(