ShipRSImagaeNet: A Large-scale Fine-Grained Dataset for Ship Detection in High-Resolution Optical Remote Sensing Images

Description

ShipRSImageNet is a large-scale fine-grainted dataset for ship detection in high-resolution optical remote sensing images. The dataset contains 3,435 images from various sensors, satellite platforms, locations, and seasons. Each image is around 930×930 pixels and contains ships with different scales, orientations, and aspect ratios. The images are annotated by experts in satellite image interpretation, categorized into 50 object categories images. The fully annotated ShipRSImageNet contains 17,573 ship instances. There are five critical contributions of the proposed ShipRSImageNet dataset compared with other existing remote sensing image datasets.

• Images are collected from various remote sensors cover- ing multiple ports worldwide and have large variations in size, spatial resolution, image quality, orientation, and environment.

• Ships are hierarchically classified into four levels and 50 ship categories.

• The number of images, ship instances, and ship cate- gories is larger than that in other publicly available ship datasets. Besides, the number is still increasing.

• We simultaneously use both horizontal and oriented bounding boxes, and polygons to annotate images, providing detailed information about direction, background, sea environment, and location of targets.

• We have benchmarked several state-of-the-art object detection algorithms on ShipRSImageNet, which can be used as a baseline for future ship detection methods.

Examples of Annotated Images

Image Source and Usage License

The ShipRSImageNet dataset collects images from a variety of sensor platforms and datasets, in particular:

• Images of the xView dataset are collected from WorldView-3 satellites with 0.3m ground resolution. Images in xView are pulled from a wide range of geographic locations. We only extract images with ship targets from them. Since the image in xView is huge for training, we slice them into 930×930 pixels with 150 pixels overlap to produce 532 images and relabeled them with both horizontal bounding box and oriented bounding box.

• We also collect 1,057 images from HRSC2016 and 1,846 images from FGSD datasets, corrected the mislabeled and relabeled missed small ship targets.

• 21 images from the Airbus Ship Detection Challenge.

• 17 images from Chinese satellites suchas GaoFen-2 and JiLin-1.

Use of the Google Earth images must respect the "Google Earth" terms of use.

All images and their associated annotations in ShipRSImageNet can be used for academic purposes only, but any commercial use is prohibited.

Object Category

The ship classification tree of proposed ShipRSImageNet is shown in the following figure. Level 0 distinguish whether the object is a ship, namely Class. Level 1 further classifies the ship object category, named as Category. Level 2 further subdivides the categories based on Level 1. Level 3 is the specific type of ship, named as Type.

At Level 3, ship objects are divided into 50 types. For brevity, we use the following abbreviations: DD for Destroyer, FF for Frigate, LL for Landing, AS for Auxiliary Ship, LSD for Landing Ship Dock, LHA for Landing Heli- copter Assault Ship, AOE for Fast Combat Support Ship, EPF for Expeditionary Fast Transport Ship, and RoRo for Roll- on Roll-off Ship. These 50 object classes are Other Ship, Other Warship, Submarine, Other Aircraft Carrier, Enterprise, Nimitz, Midway, Ticonderoga, Other Destroyer, Atago DD, Arleigh Burke DD, Hatsuyuki DD, Hyuga DD, Asagiri DD, Other Frigate, Perry FF, Patrol, Other Landing, YuTing LL, YuDeng LL, YuDao LL, YuZhao LL, Austin LL, Osumi LL, Wasp LL, LSD 41 LL, LHA LL, Commander, Other Auxiliary Ship, Medical Ship, Test Ship, Training Ship, AOE, Masyuu AS, Sanantonio AS, EPF, Other Merchant, Container Ship, RoRo, Cargo, Barge, Tugboat, Ferry, Yacht, Sailboat, Fishing Vessel, Oil Tanker, Hovercraft, Motorboat, and Dock.

Dataset Download

• Baidu Drive (Extraction code:h2qk):
  • ShipRSImageNet
• Google Drive:
  • ShipRSImageNet

Benchmark Code Installation

We keep all the experiment settings and hyper-parameters the same as depicted in MMDetection(v2.11.0) config files except for the number of categories and parameters. MMDe- tection is an open-source object detection toolbox based on PyTorch. It is a part of the OpenMMLab project developed by Multimedia Laboratory, CUHK.

This project is based on MMdetection(v2.11.0). MMDetection is an open source object detection toolbox based on PyTorch. It is a part of the OpenMMLab project.

Prerequisites

• Linux or macOS (Windows is in experimental support)
• Python 3.6+
• PyTorch 1.3+
• CUDA 9.2+ (If you build PyTorch from source, CUDA 9.0 is also compatible)
• GCC 5+
• MMCV

Installation

• Install MMdetection following the instructions. We are noting that our code is checked in mmdetection V2.11.0 and pytorch V1.7.1.

  • Create a conda virtual environment and activate it.

    conda create -n open-mmlab python=3.7 -y
    conda activate open-mmlab

  • Install PyTorch and torchvision following the official instructions, e.g.,

    conda install pytorch torchvision -c pytorch

    Note: Make sure that your compilation CUDA version and runtime CUDA version match. You can check the supported CUDA version for precompiled packages on the PyTorch website.

  • Install mmcv-full, we recommend you to install the pre-build package as below.

    pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html

    Please replace {cu_version} and {torch_version} in the url to your desired one. For example, to install the latest mmcv-full with CUDA 11 and PyTorch 1.7.1, use the following command:

    pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.1/index.html

  • Download this benchmark code.

    git clone https://github.com/open-mmlab/mmdetection.git
    cd mmdetection2.11-ShipRSImageNet

  • Install build requirements and then install MMDetection.

    pip install -r requirements/build.txt
    pip install -v -e .  # or "python setup.py develop"

Train with ShipRSImageNet

• Download the ShipRSImageNet dataset. It is recommended to symlink the ShipRSImageNet dataset root to $mmdetection2.11-ShipRSImageNet/data:

  ln -s $dataset/ShipRSImageNet/ $mmdetection2.11-ShipRSImageNet/data/

• If your folder structure is different, you may need to change the corresponding paths in config files.

• mmdetection2.11-ShipRSImageNet
  ├── mmdet
  ├── tools
  ├── configs
  ├── data
  │   ├── ShipRSImageNet
  │   │   ├── COCO_Format
  │   │   ├── masks
  │   │   ├── VOC_Format
  │   │   │   ├── annotations
  │   │   │   ├── ImageSets
  │   │   │   ├── JPEGImages

• Prepare a config file:

  • The benchamark config file of ShipRSImageNet already in the following:

    • $mmdetection2.11-ShipRSImageNet/configs/ShipRSImageNet/

• Example of train a model with ShipRSImageNet:

  • python tools/train.py configs/ShipRSImageNet/faster_rcnn/faster_rcnn_r50_fpn_100e_ShipRSImageNet_Level0.py

Models trained on ShipRSImageNet

We introduce two tasks: detection with horizontal bounding boxes (HBB for short) and segmentation with oriented bounding boxes (SBB for short). HBB aims at extracting bounding boxes with the same orientation of the image, it is an Object Detection task. SBB aims at semantically segmenting the image, it is a Semantic Segmentation task.

The evaluation protocol follows the same mAP and mAR of area small/medium/large and mAP(@IoU=0.50:0.95) calculation used by MS-COCO.

Level 0

Model	Backbone	Style	HBB mAP	SBB mAP	Extraction code	Download
Faster RCNN with FPN	R-50	Pytorch	0.550		2vrm	model
Faster RCNN with FPN	R-101	Pytorch	0.546		f362	model
Mask RCNN with FPN	R-50	Pytorch	0.566	0.440	24eq	model
Mask RCNN with FPN	R-101	Pytorch	0.557	0.436	lbcb	model
Cascade Mask RCNN with FPN	R-50	Pytorch	0.568	0.430	et6m	model
SSD	VGG16	Pytorch	0.464		qabf	model
Retinanet with FPN	R-50	Pytorch	0.418		7qdw	model
Retinanet with FPN	R-101	Pytorch	0.419		vdiq	model
FoveaBox	R-101	Pytorch	0.453		urbf	model
FCOS with FPN	R-101	Pytorch	0.333		94ub	model

Level 1

Model	Backbone	Style	HBB mAP	SBB mAP	Extraction code	Download
Faster RCNN with FPN	R-50	Pytorch	0.366	-	5i5a	model
Faster RCNN with FPN	R-101	Pytorch	0.461	-	6ts7	model
Mask RCNN with FPN	R-50	Pytorch	0.456	0.347	9gnt	model
Mask RCNN with FPN	R-101	Pytorch	0.472	0.371	wc62	model
Cascade Mask RCNN with FPN	R-50	Pytorch	0.485	0.365	a8bl	model
SSD	VGG16	Pytorch	0.397	-	uffe	model
Retinanet with FPN	R-50	Pytorch	0.368	-	lfio	model
Retinanet with FPN	R-101	Pytorch	0.359	-	p1rd	model
FoveaBox	R-101	Pytorch	0.389	-	kwiq	model
FCOS with FPN	R-101	Pytorch	0.351	-	1djo	model

Level 2

Model	Backbone	Style	HBB mAP	SBB mAP	Extraction code	Download
Faster RCNN with FPN	R-50	Pytorch	0.345	-	924l	model
Faster RCNN with FPN	R-101	Pytorch	0.479	-	fb1b	model
Mask RCNN with FPN	R-50	Pytorch	0.468	0.377	so8j	model
Mask RCNN with FPN	R-101	Pytorch	0.488	0.398	7q1g	model
Cascade Mask RCNN with FPN	R-50	Pytorch	0.492	0.389	t9gr	model
SSD	VGG16	Pytorch	0.423	-	t1ma	model
Retinanet with FPN	R-50	Pytorch	0.369	-	4h0o	model
Retinanet with FPN	R-101	Pytorch	0.411	-	g9ca	model
FoveaBox	R-101	Pytorch	0.427	-	8e12	model
FCOS with FPN	R-101	Pytorch	0.431	-	0hl0	model

Level 3

Model	Backbone	Style	HBB mAP	SBB mAP	Extraction code	Download
Faster RCNN with FPN	R-50	Pytorch	0.375	-	7qmo	model
Faster RCNN with FPN	R-101	Pytorch	0.543	-	bmla	model
Mask RCNN with FPN	R-50	Pytorch	0.545	0.450	a73h	model
Mask RCNN with FPN	R-101	Pytorch	0.564	0.472	7k9i	model
Cascade Mask RCNN with FPN	R-50	Pytorch	0.593	0.483	ebga	model
SSD	VGG16	Pytorch	0.483	-	otu5	model
Retinanet with FPN	R-50	Pytorch	0.326	-	tu5a	model
Retinanet with FPN	R-101	Pytorch	0.483	-	ptv0	model
FoveaBox	R-101	Pytorch	0.459	-	1acn	model
FCOS with FPN	R-101	Pytorch	0.498	-	40a8	model

Development kit

The ShipRSImageNet Development kit is based on DOTA Development kit and provides the following function

• Load and image, and show the bounding box on it.

• Covert VOC format label to COCO format label.

Citation

If you make use of the ShipRSImageNet dataset, please cite our following paper:

Z. Zhang, L. Zhang, Y. Wang, P. Feng and R. He, "ShipRSImageNet: A Large-Scale Fine-Grained Dataset for Ship Detection in High-Resolution Optical Remote Sensing Images," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 8458-8472, 2021, doi: 10.1109/JSTARS.2021.3104230.

Contact

If you have any the problem or feedback in using ShipRSImageNet, please contact:

• Zhengning Zhang at 23880666@qq.com

License

ShipRSImageNet is released under the Apache 2.0 license. Please see the LICENSE file for more information.