diff --git a/docs/zh_cn/advanced_guides/contribute_dataset.md b/docs/zh_cn/advanced_guides/contribute_dataset.md
new file mode 100644
index 0000000000..4222de32a6
--- /dev/null
+++ b/docs/zh_cn/advanced_guides/contribute_dataset.md
@@ -0,0 +1,461 @@
+# 在 mmsegmentation projects 中贡献一个标准格式的数据集
+
+- 在开始您的贡献流程前，请先阅读[《OpenMMLab 贡献代码指南》](https://mmcv.readthedocs.io/zh_CN/latest/community/contributing.html)，以详细的了解 OpenMMLab 代码库的代码贡献流程。
+- 该教程以 [Gaofen Image Dataset (GID)](https://www.sciencedirect.com/science/article/pii/S0034425719303414) 高分 2 号卫星所拍摄的遥感图像语义分割数据集作为样例，来演示在 mmsegmentation 中的数据集贡献流程。
+
+## 步骤 1： 配置 mmsegmentation 开发所需必要环境
+
+- 开发所必需的环境安装请参考[中文快速入门指南](https://github.com/open-mmlab/mmsegmentation/blob/main/docs/zh_cn/get_started.md)或[英文 get_started](https://github.com/open-mmlab/mmsegmentation/blob/main/docs/en/get_started.md)。
+
+- 如果您已安装了最新版的 pytorch、mmcv、mmengine，那么您可以跳过步骤 1 至[步骤 2](<#[步骤-2](#%E6%AD%A5%E9%AA%A4-2%E4%BB%A3%E7%A0%81%E8%B4%A1%E7%8C%AE%E5%89%8D%E7%9A%84%E5%87%86%E5%A4%87%E5%B7%A5%E4%BD%9C)>)。
+
+- **注：** 在此处无需安装 mmsegmentation，只需安装开发 mmsegmentation 所必需的 pytorch、mmcv、mmengine 等即可。
+
+**新建虚拟环境（如已有合适的开发环境，可跳过）**
+
+- 从[官方网站](https://docs.conda.io/en/latest/miniconda.html)下载并安装 Miniconda
+- 创建一个 conda 环境，并激活
+
+```shell
+conda create --name openmmlab python=3.8 -y
+conda activate openmmlab
+```
+
+**安装 pytorch （如环境下已安装 pytorch，可跳过）**
+
+- 参考 [official instructions](https://pytorch.org/get-started/locally/) 安装 **PyTorch**
+
+**使用 mim 安装 mmcv、mmengine**
+
+- 使用 [MIM](https://github.com/open-mmlab/mim) 安装 [MMCV](https://github.com/open-mmlab/mmcv)
+
+```shell
+pip install -U openmim
+mim install mmengine
+mim install "mmcv>=2.0.0"
+```
+
+## 步骤 2：代码贡献前的准备工作
+
+### 2.1 Fork mmsegmentation 仓库
+
+- 通过浏览器打开[mmsegmentation 官方仓库](https://github.com/open-mmlab/mmsegmentation/tree/main)。
+- 登录您的 GitHub 账户，以下步骤均需在 GitHub 登录的情况下进行。
+- Fork mmsegmentation 仓库
+ 
+- Fork 之后，mmsegmentation 仓库将会出现在您的个人仓库中。
+
+### 2.2 在您的代码编写软件中 git clone mmsegmentation
+
+这里以 VSCODE 为例
+
+- 打开 VSCODE，新建终端窗口并激活您在[步骤 1 ](#%E6%AD%A5%E9%AA%A4-1-%E9%85%8D%E7%BD%AE-mmsegmentation-%E5%BC%80%E5%8F%91%E6%89%80%E9%9C%80%E5%BF%85%E8%A6%81%E7%8E%AF%E5%A2%83)中所安装的虚拟环境。
+- 在您 GitHub 的个人仓库中找到您 Fork 的 mmsegmentation 仓库，复制其链接。
+ 
+- 在终端中执行命令
+ ```bash
+ git clone {您所复制的个人仓库的链接}
+ ```
+ 
+ **注：** 如提示以下信息，请在 GitHub 中添加 [SSH 秘钥](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent)
+ 
+- 进入 mmsegmentation 目录（之后的操作均在 mmsegmentation 目录下）。
+ ```bash
+ cd mmsegmentation
+ ```
+- 在终端中执行以下命令，添加官方仓库为上游仓库。
+ ```bash
+ git remote add upstream git@github.com:open-mmlab/mmsegmentation.git
+ ```
+- 使用以下命令检查 remote 是否添加成功。
+ ```bash
+ git remote -v
+ ```
+ 
+
+### 2.3 切换目录至 mmsegmentation 并从源码安装mmsegmentation
+
+在`mmsegmentation`目录下执行`pip install -v -e .`，通过源码构建方式安装 mmsegmentaion 库。
+安装完成后，您将能看到如下图所示的文件树。
+
+
+### 2.4 切换分支为 dev-1.x
+
+正如您在[ mmsegmentation 官网](https://github.com/open-mmlab/mmsegmentation/tree/main)所见，该仓库有许多分支，默认分支`main`为稳定的发行版本，以及用于贡献者进行开发的`dev-1.x`分支。`dev-1.x`分支是贡献者们用来提交创意和 PR 的分支，`dev-1.x`分支的内容会被周期性的合入到`main`分支。
+
+
+回到 VSCODE 中，在终端执行命令
+
+```bash
+git checkout dev-1.x
+```
+
+### 2.5 创新属于自己的新分支
+
+在基于`dev-1.x`分支下，使用如下命令，创建属于您自己的分支。
+
+```bash
+# git checkout -b 您的GitHubID/您的分支想要实现的功能的名字
+# git checkout -b AI-Tianlong/support_GID_dataset
+git checkout -b {您的GitHubID/您的分支想要实现的功能的名字}
+```
+
+### 2.6 配置 pre-commit
+
+OpenMMLab 仓库对代码质量有着较高的要求，所有提交的 PR 必须要通过代码格式检查。pre-commit 详细配置参阅[配置 pre-commit](https://mmcv.readthedocs.io/zh_CN/latest/community/contributing.html#pre-commit)。
+
+## 步骤 3：在`mmsegmentation/projects`下贡献您的代码
+
+**先对 GID 数据集进行分析**
+
+这里以贡献高分 2 号遥感图像语义分割数据集 GID 为例，GID 数据集是由我国自主研发的高分 2 号卫星所拍摄的光学遥感图像所创建，经图像预处理后共提供了 150 张 6800x7200 像素的 RGB 三通道遥感图像。并提供了两种不同类别数的数据标注，一种是包含 5 类有效物体的 RGB 标签，另一种是包含 15 类有效物体的 RGB 标签。本教程将针对 5 类标签进行数据集贡献流程讲解。
+
+GID 的 5 类有效标签分别为：0-背景-\[0,0,0\](mask 标签值-标签名称-RGB 标签值)、1-建筑-\[255,0,0\]、2-农田-\[0,255,0\]、3-森林-\[0,0,255\]、4-草地-\[255,255,0\]、5-水-\[0,0,255\]。在语义分割任务中，标签是与原图尺寸一致的单通道图像，标签图像中的像素值为真实样本图像中对应像素所包含的物体的类别。GID 数据集提供的是具有 RGB 三通道的彩色标签，为了模型的训练需要将 RGB 标签转换为 mask 标签。并且由于图像尺寸为 6800x7200 像素，对于神经网络的训练来有些过大，所以将每张图像裁切成了没有重叠的 512x512 的图像以便进行训练。
+
+
+### 3.1 在`mmsegmentation/projects`下创建新的项目文件夹
+
+在`mmsegmentation/projects`下创建文件夹`gid_dataset`
+
+
+### 3.2 贡献您的数据集代码
+
+为了最终能将您在 projects 中贡献的代码更加顺畅的移入核心库中（对代码要求质量更高），非常建议按照核心库的目录来编辑您的数据集文件。
+关于数据集有 4 个必要的文件：
+
+- **1** `mmseg/datasets/gid.py` 定义了数据集的尾缀、CLASSES、PALETTE、reduce_zero_label等
+- **2** `configs/_base_/gid.py` GID 数据集的配置文件，定义了数据集的`dataset_type`（数据集类型，`mmseg/datasets/gid.py`中注册的数据集的类名）、`data_root`(数据集所在的根目录，建议将数据集通过软连接的方式将数据集放至`mmsegmentation/data`)、`train_pipline`(训练的数据流)、`test_pipline`(测试和验证时的数据流)、`img_rations`(多尺度预测时的多尺度配置)、`tta_pipeline`（多尺度预测）、`train_dataloader`(训练集的数据加载器)、`val_dataloader`(验证集的数据加载器)、`test_dataloader`(测试集的数据加载器)、`val_evaluator`(验证集的评估器)、`test_evaluator`(测试集的评估器)。
+- **3** 使用了 GID 数据集的模型训练配置文件
+ 这个是可选的，但是强烈建议您添加。在核心库中，所贡献的数据集需要和参考文献中所提出的结果精度对齐，为了后期将您贡献的代码合并入核心库。如您的算力充足，最好能提供对应的模型配置文件在您贡献的数据集上所验证的结果以及相应的权重文件，并撰写较为详细的README.md文档。[示例参考结果](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/deeplabv3plus#mapillary-vistas-v12)
+ 
+- **4** 使用如下命令格式： 撰写`docs/zh_cn/user_guides/2_dataset_prepare.md`来添加您的数据集介绍，包括但不限于数据集的下载方式，数据集目录结构、数据集生成等一些必要性的文字性描述和运行命令。以更好地帮助用户能更快的实现数据集的准备工作。
+
+### 3.3 贡献`tools/dataset_converters/gid.py`
+
+由于 GID 数据集是由未经过切分的 6800x7200 图像所构成的数据集，并且没有划分训练集、验证集与测试集。以及其标签为 RGB 彩色标签，需要将标签转换为单通道的 mask label。为了方便训练，首先将 GID 数据集进行裁切和标签转换，并进行数据集划分，构建为 mmsegmentation 所支持的格式。
+
+```python
+# tools/dataset_converters/gid.py
+import argparse
+import glob
+import math
+import os
+import os.path as osp
+from PIL import Image
+
+import mmcv
+import numpy as np
+from mmengine.utils import ProgressBar, mkdir_or_exist
+
+def parse_args():
+ parser = argparse.ArgumentParser(
+ description='Convert GID dataset to mmsegmentation format')
+ parser.add_argument('dataset_img_path', help='GID images folder path')
+ parser.add_argument('dataset_label_path', help='GID labels folder path')
+ parser.add_argument('--tmp_dir', help='path of the temporary directory')
+ parser.add_argument('-o', '--out_dir', help='output path', default='data/gid')
+ parser.add_argument(
+ '--clip_size',
+ type=int,
+ help='clipped size of image after preparation',
+ default=256)
+ parser.add_argument(
+ '--stride_size',
+ type=int,
+ help='stride of clipping original images',
+ default=256)
+ args = parser.parse_args()
+ return args
+
+GID_COLORMAP = dict(
+ Background=(0, 0, 0), #0-背景-黑色
+ Building=(255, 0, 0), #1-建筑-红色
+ Farmland=(0, 255, 0), #2-农田-绿色
+ Forest=(0, 0, 255), #3-森林-蓝色
+ Meadow=(255, 255, 0),#4-草地-黄色
+ Water=(0, 0, 255)#5-水-蓝色
+)
+palette = list(GID_COLORMAP.values())
+classes = list(GID_COLORMAP.keys())
+
+#############用列表来存一个 RGB 和一个类别的对应################
+def colormap2label(palette):
+ colormap2label_list = np.zeros(256**3, dtype = np.longlong)
+ for i, colormap in enumerate(palette):
+ colormap2label_list[(colormap[0] * 256 + colormap[1])*256+colormap[2]] = i
+ return colormap2label_list
+
+#############给定那个列表，和vis_png然后生成masks_png################
+def label_indices(RGB_label, colormap2label_list):
+ RGB_label = RGB_label.astype('int32')
+ idx = (RGB_label[:, :, 0] * 256 + RGB_label[:, :, 1]) * 256 + RGB_label[:, :, 2]
+ # print(idx.shape)
+ return colormap2label_list[idx]
+
+def RGB2mask(RGB_label, colormap2label_list):
+ # RGB_label = np.array(Image.open(RGB_label).convert('RGB')) #打开RGB_png
+ mask_label = label_indices(RGB_label, colormap2label_list) # .numpy()
+ return mask_label
+
+colormap2label_list = colormap2label(palette)
+
+def clip_big_image(image_path, clip_save_dir, args, to_label=False):
+ """
+ Original image of GID dataset is very large, thus pre-processing
+ of them is adopted. Given fixed clip size and stride size to generate
+ clipped image, the intersection of width and height is determined.
+ For example, given one 6800 x 7200 original image, the clip size is
+ 256 and stride size is 256, thus it would generate 29 x 27 = 783 images
+ whose size are all 256 x 256.
+
+ """
+
+ image = mmcv.imread(image_path, channel_order='rgb')
+ # image = mmcv.bgr2gray(image)
+
+ h, w, c = image.shape
+ clip_size = args.clip_size
+ stride_size = args.stride_size
+
+ num_rows = math.ceil((h - clip_size) / stride_size) if math.ceil(
+ (h - clip_size) /
+ stride_size) * stride_size + clip_size >= h else math.ceil(
+ (h - clip_size) / stride_size) + 1
+ num_cols = math.ceil((w - clip_size) / stride_size) if math.ceil(
+ (w - clip_size) /
+ stride_size) * stride_size + clip_size >= w else math.ceil(
+ (w - clip_size) / stride_size) + 1
+
+ x, y = np.meshgrid(np.arange(num_cols + 1), np.arange(num_rows + 1))
+ xmin = x * clip_size
+ ymin = y * clip_size
+
+ xmin = xmin.ravel()
+ ymin = ymin.ravel()
+ xmin_offset = np.where(xmin + clip_size > w, w - xmin - clip_size,
+ np.zeros_like(xmin))
+ ymin_offset = np.where(ymin + clip_size > h, h - ymin - clip_size,
+ np.zeros_like(ymin))
+ boxes = np.stack([
+ xmin + xmin_offset, ymin + ymin_offset,
+ np.minimum(xmin + clip_size, w),
+ np.minimum(ymin + clip_size, h)
+ ], axis=1)
+
+ if to_label:
+ image = RGB2mask(image, colormap2label_list) #这里得改一下
+
+ for count, box in enumerate(boxes):
+ start_x, start_y, end_x, end_y = box
+ clipped_image = image[start_y:end_y,
+ start_x:end_x] if to_label else image[
+ start_y:end_y, start_x:end_x, :]
+ img_name = osp.basename(image_path).replace('.tif', '')
+ img_name = img_name.replace('_label', '')
+ if count % 3 == 0:
+ mmcv.imwrite(
+ clipped_image.astype(np.uint8),
+ osp.join(
+ clip_save_dir.replace('train', 'val'),
+ f'{img_name}_{start_x}_{start_y}_{end_x}_{end_y}.png'))
+ else:
+ mmcv.imwrite(
+ clipped_image.astype(np.uint8),
+ osp.join(
+ clip_save_dir,
+ f'{img_name}_{start_x}_{start_y}_{end_x}_{end_y}.png'))
+ count += 1
+
+def main():
+ args = parse_args()
+
+ """
+ According to this paper: https://ieeexplore.ieee.org/document/9343296/
+ select 15 images contained in GID, , which cover the whole six
+ categories, to generate train set and validation set.
+
+ According to Paper: https://ieeexplore.ieee.org/document/9343296/
+
+ """
+
+ if args.out_dir is None:
+ out_dir = osp.join('data', 'gid')
+ else:
+ out_dir = args.out_dir
+
+ print('Making directories...')
+ mkdir_or_exist(osp.join(out_dir, 'img_dir', 'train'))
+ mkdir_or_exist(osp.join(out_dir, 'img_dir', 'val'))
+ mkdir_or_exist(osp.join(out_dir, 'ann_dir', 'train'))
+ mkdir_or_exist(osp.join(out_dir, 'ann_dir', 'val'))
+
+ src_path_list = glob.glob(os.path.join(args.dataset_img_path, '*.tif'))
+ print(f'Find {len(src_path_list)} pictures')
+
+ prog_bar = ProgressBar(len(src_path_list))
+
+ dst_img_dir = osp.join(out_dir, 'img_dir', 'train')
+ dst_label_dir = osp.join(out_dir, 'ann_dir', 'train')
+
+ for i, img_path in enumerate(src_path_list):
+ label_path = osp.join(args.dataset_label_path, osp.basename(img_path.replace('.tif', '_label.tif')))
+
+ clip_big_image(img_path, dst_img_dir, args, to_label=False)
+ clip_big_image(label_path, dst_label_dir, args, to_label=True)
+ prog_bar.update()
+
+ print('Done!')
+
+if __name__ == '__main__':
+ main()
+```
+
+### 3.4 贡献`mmseg/datasets/gid.py`
+
+可参考[`projects/mapillary_dataset/mmseg/datasets/mapillary.py`](https://github.com/open-mmlab/mmsegmentation/blob/main/projects/mapillary_dataset/mmseg/datasets/mapillary.py)并在此基础上修改相应变量以适配您的数据集。
+
+```python
+# mmseg/datasets/gid.py
+# Copyright (c) OpenMMLab. All rights reserved.
+from mmseg.datasets.basesegdataset import BaseSegDataset
+from mmseg.registry import DATASETS
+
+# 注册数据集类
+@DATASETS.register_module()
+class GID_Dataset(BaseSegDataset):
+ """Gaofen Image Dataset (GID)
+
+ Dataset paper link:
+ https://www.sciencedirect.com/science/article/pii/S0034425719303414
+ https://x-ytong.github.io/project/GID.html
+
+ GID 6 classes: background(others), built-up, farmland, forest, meadow, water
+
+ In This example, select 10 images from GID dataset as training set,
+ and select 5 images as validation set.
+ The selected images are listed as follows:
+
+ GF2_PMS1__L1A0000647767-MSS1
+ GF2_PMS1__L1A0001064454-MSS1
+ GF2_PMS1__L1A0001348919-MSS1
+ GF2_PMS1__L1A0001680851-MSS1
+ GF2_PMS1__L1A0001680853-MSS1
+ GF2_PMS1__L1A0001680857-MSS1
+ GF2_PMS1__L1A0001757429-MSS1
+ GF2_PMS2__L1A0000607681-MSS2
+ GF2_PMS2__L1A0000635115-MSS2
+ GF2_PMS2__L1A0000658637-MSS2
+ GF2_PMS2__L1A0001206072-MSS2
+ GF2_PMS2__L1A0001471436-MSS2
+ GF2_PMS2__L1A0001642620-MSS2
+ GF2_PMS2__L1A0001787089-MSS2
+ GF2_PMS2__L1A0001838560-MSS2
+
+ The ``img_suffix`` is fixed to '.tif' and ``seg_map_suffix`` is
+ fixed to '.tif' for GID.
+ """
+ METAINFO = dict(
+ classes=('Others', 'Built-up', 'Farmland', 'Forest',
+ 'Meadow', 'Water'),
+
+ palette=[[0, 0, 0], [255, 0, 0], [0, 255, 0], [0, 255, 255],
+ [255, 255, 0], [0, 0, 255]])
+
+ def __init__(self,
+ img_suffix='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=None,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
+```
+
+### 3.5 贡献使用 GID 的训练 config file
+
+```python
+_base_ = [
+ '../../../configs/_base_/models/deeplabv3plus_r50-d8.py',
+ './_base_/datasets/gid.py',
+ '../../../configs/_base_/default_runtime.py',
+ '../../../configs/_base_/schedules/schedule_240k.py'
+]
+custom_imports = dict(
+ imports=['projects.gid_dataset.mmseg.datasets.gid'])
+
+crop_size = (256, 256)
+data_preprocessor = dict(size=crop_size)
+model = dict(
+ data_preprocessor=data_preprocessor,
+ pretrained='open-mmlab://resnet101_v1c',
+ backbone=dict(depth=101),
+ decode_head=dict(num_classes=6),
+ auxiliary_head=dict(num_classes=6))
+
+```
+
+### 3.6 撰写`docs/zh_cn/user_guides/2_dataset_prepare.md`
+
+**Gaofen Image Dataset (GID)**
+
+- GID 数据集可在[此处](https://x-ytong.github.io/project/Five-Billion-Pixels.html)进行下载。
+- GID 数据集包含 150 张 6800x7200 的大尺寸图像，标签为 RGB 标签。
+- 此处选择 15 张图像生成训练集和验证集，该 15 张图像包含了所有六类信息。所选的图像名称如下：
+
+```None
+ GF2_PMS1__L1A0000647767-MSS1
+ GF2_PMS1__L1A0001064454-MSS1
+ GF2_PMS1__L1A0001348919-MSS1
+ GF2_PMS1__L1A0001680851-MSS1
+ GF2_PMS1__L1A0001680853-MSS1
+ GF2_PMS1__L1A0001680857-MSS1
+ GF2_PMS1__L1A0001757429-MSS1
+ GF2_PMS2__L1A0000607681-MSS2
+ GF2_PMS2__L1A0000635115-MSS2
+ GF2_PMS2__L1A0000658637-MSS2
+ GF2_PMS2__L1A0001206072-MSS2
+ GF2_PMS2__L1A0001471436-MSS2
+ GF2_PMS2__L1A0001642620-MSS2
+ GF2_PMS2__L1A0001787089-MSS2
+ GF2_PMS2__L1A0001838560-MSS2
+```
+
+执行以下命令进行裁切及标签的转换，需要修改为您所存储 15 张图像及标签的路径。
+
+```
+python projects/gid_dataset/tools/dataset_converters/gid.py [15 张图像的路径] [15 张标签的路径]
+```
+
+完成裁切后的 GID 数据结构如下：
+
+```none
+mmsegmentation
+├── mmseg
+├── tools
+├── configs
+├── data
+│ ├── gid
+│ │ ├── ann_dir
+| │ │ │ ├── train
+| │ │ │ ├── val
+│ │ ├── img_dir
+| │ │ │ ├── train
+| │ │ │ ├── val
+
+```
+
+### 3.7 贡献的代码及文档通过`pre-commit`检查
+
+使用命令
+
+```bash
+git add .
+git commit -m "添加描述"
+git push
+```
+
+### 3.8 在 GitHub 中向 mmsegmentation 提交 PR
+
+具体步骤可见[《OpenMMLab 贡献代码指南》](https://mmcv.readthedocs.io/zh_CN/latest/community/contributing.html)
diff --git a/docs/zh_cn/advanced_guides/models.md b/docs/zh_cn/advanced_guides/models.md
index 408a57863c..6eb22517a4 100644
--- a/docs/zh_cn/advanced_guides/models.md
+++ b/docs/zh_cn/advanced_guides/models.md
@@ -30,17 +30,17 @@
## 基本接口
-MMSegmentation 封装 `BaseModel` 并实现了 [BaseSegmenter](https://github.com/open-mmlab/mmsegmentation/blob/1.x/mmseg/models/segmentors/base.py#L15) 类，主要提供 `forward`、`train_step`、`val_step` 和 `test_step` 接口。接下来将详细介绍这些接口。
+MMSegmentation 封装 `BaseModel` 并实现了 [BaseSegmentor](https://github.com/open-mmlab/mmsegmentation/blob/1.x/mmseg/models/segmentors/base.py#L15) 类，主要提供 `forward`、`train_step`、`val_step` 和 `test_step` 接口。接下来将详细介绍这些接口。
### forward
- 
+ 
编码器解码器数据流
- 
+ 
级联编码器解码器数据流
@@ -115,7 +115,7 @@ MMSegmentation 封装 `BaseModel` 并实现了 [BaseSegmenter](https://github.co
-Dict\[str, `torch.Tensor`\]：用于记录日志的张量的`字典`。
- 
+ 
train_step 数据流
@@ -132,7 +132,7 @@ MMSegmentation 封装 `BaseModel` 并实现了 [BaseSegmenter](https://github.co
- `list` - 给定数据的预测结果。
- 
+ 
test_step/val_step 数据流
diff --git a/docs/zh_cn/get_started.md b/docs/zh_cn/get_started.md
index fc26389a4b..b92ab33490 100644
--- a/docs/zh_cn/get_started.md
+++ b/docs/zh_cn/get_started.md
@@ -4,7 +4,7 @@
本教程中，我们将会演示如何使用 PyTorch 准备环境。
-MMSegmentation 可以在 Linux, Windows 和 macOS 系统上运行，并且需要安装 Python 3.6+, CUDA 9.2+ 和 PyTorch 1.5+
+MMSegmentation 可以在 Linux, Windows 和 macOS 系统上运行，并且需要安装 Python 3.7+, CUDA 10.2+ 和 PyTorch 1.8+
**注意:**
如果您已经安装了 PyTorch, 可以跳过该部分，直接到[下一小节](##安装)。否则，您可以按照以下步骤操作。
@@ -194,6 +194,16 @@ docker build -t mmsegmentation docker/
docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmsegmentation/data mmsegmentation
```
+### 可选依赖
+
+#### 安装 GDAL
+
+[GDAL](https://gdal.org/) 是一个用于栅格和矢量地理空间数据格式的转换库。安装 GDAL 可以读取复杂格式和极大的遥感图像。
+
+```shell
+conda install GDAL
+```
+
## 问题解答
如果您在安装过程中遇到了其他问题，请第一时间查阅 [FAQ](notes/faq.md) 文件。如果没有找到答案，您也可以在 GitHub 上提出 [issue](https://github.com/open-mmlab/mmsegmentation/issues/new/choose)
diff --git a/docs/zh_cn/notes/faq.md b/docs/zh_cn/notes/faq.md
index e081acc16f..83bcb22e42 100644
--- a/docs/zh_cn/notes/faq.md
+++ b/docs/zh_cn/notes/faq.md
@@ -8,8 +8,9 @@
| MMSegmentation version | MMCV version | MMEngine version | MMClassification (optional) version | MMDetection (optional) version |
| :--------------------: | :----------------------------: | :---------------: | :---------------------------------: | :----------------------------: |
-| dev-1.x branch | mmcv >= 2.0.0rc4 | MMEngine >= 0.7.1 | mmcls==1.0.0rc6 | mmdet >= 3.0.0 |
-| main branch | mmcv >= 2.0.0rc4 | MMEngine >= 0.7.1 | mmcls==1.0.0rc6 | mmdet >= 3.0.0 |
+| dev-1.x branch | mmcv >= 2.0.0 | MMEngine >= 0.7.4 | mmpretrain>=1.0.0rc7 | mmdet >= 3.0.0 |
+| main branch | mmcv >= 2.0.0 | MMEngine >= 0.7.4 | mmpretrain>=1.0.0rc7 | mmdet >= 3.0.0 |
+| 1.1.0 | mmcv >= 2.0.0 | MMEngine >= 0.7.4 | mmpretrain>=1.0.0rc7 | mmdet >= 3.0.0 |
| 1.0.0 | mmcv >= 2.0.0rc4 | MMEngine >= 0.7.1 | mmcls==1.0.0rc6 | mmdet >= 3.0.0 |
| 1.0.0rc6 | mmcv >= 2.0.0rc4 | MMEngine >= 0.5.0 | mmcls>=1.0.0rc0 | mmdet >= 3.0.0rc6 |
| 1.0.0rc5 | mmcv >= 2.0.0rc4 | MMEngine >= 0.2.0 | mmcls>=1.0.0rc0 | mmdet>=3.0.0rc6 |
diff --git a/docs/zh_cn/user_guides/2_dataset_prepare.md b/docs/zh_cn/user_guides/2_dataset_prepare.md
index e5e6c76c47..630b72762e 100644
--- a/docs/zh_cn/user_guides/2_dataset_prepare.md
+++ b/docs/zh_cn/user_guides/2_dataset_prepare.md
@@ -2,6 +2,7 @@
我们建议将数据集根目录符号链接到 `$MMSEGMENTATION/data`。
如果您的目录结构不同，您可能需要更改配置文件中相应的路径。
+对于中国境内的用户，我们也推荐通过开源数据平台 [OpenDataLab](https://opendatalab.com/) 来下载dsdl标准数据，以获得更好的下载和使用体验，这里有一个下载dsdl数据集并进行训练的案例[DSDLReadme](../../../configs/dsdl/README.md)，欢迎尝试。
```none
mmsegmentation
@@ -345,9 +346,10 @@ python tools/dataset_converters/loveda.py /path/to/loveDA
## ISPRS Potsdam
-[Potsdam](https://www2.isprs.org/commissions/comm2/wg4/benchmark/2d-sem-label-potsdam/) 城市语义分割数据集用于 2D 语义分割竞赛 —— Potsdam。
+[Potsdam](https://www.isprs.org/education/benchmarks/UrbanSemLab/2d-sem-label-potsdam.aspx) 城市语义分割数据集用于 2D 语义分割竞赛 —— Potsdam。
-数据集可以在竞赛[主页](https://www2.isprs.org/commissions/comm2/wg4/benchmark/data-request-form/)上请求获得。
+数据集可以在竞赛[主页](https://www.isprs.org/education/benchmarks/UrbanSemLab/default.aspx)上请求获得。
+这里也提供了[BaiduNetdisk](https://pan.baidu.com/s/1K-cLVZnd1X7d8c26FQ-nGg?pwd=mseg)，提取码：mseg、 [Google Drive](https://drive.google.com/drive/folders/1w3EJuyUGet6_qmLwGAWZ9vw5ogeG0zLz?usp=sharing)以及[OpenDataLab](https://opendatalab.com/ISPRS_Potsdam/download)。
实验中需要下载 '2_Ortho_RGB.zip' 和 '5_Labels_all_noBoundary.zip'。
对于 Potsdam 数据集，请运行以下命令调整数据集目录。
@@ -360,9 +362,10 @@ python tools/dataset_converters/potsdam.py /path/to/potsdam
## ISPRS Vaihingen
-[Vaihingen](https://www2.isprs.org/commissions/comm2/wg4/benchmark/2d-sem-label-vaihingen/) 城市语义分割数据集用于 2D 语义分割竞赛 —— Vaihingen。
+[Vaihingen](https://www.isprs.org/education/benchmarks/UrbanSemLab/2d-sem-label-vaihingen.aspx) 城市语义分割数据集用于 2D 语义分割竞赛 —— Vaihingen。
-数据集可以在竞赛[主页](https://www2.isprs.org/commissions/comm2/wg4/benchmark/data-request-form/)上请求获得。
+数据集可以在竞赛[主页](https://www.isprs.org/education/benchmarks/UrbanSemLab/default.aspx)上请求获得。
+这里也提供了[BaiduNetdisk](https://pan.baidu.com/s/109D3WLrLafsuYtLeerLiiA?pwd=mseg)，提取码：mseg 、 [Google Drive](https://drive.google.com/drive/folders/1w3NhvLVA2myVZqOn2pbiDXngNC7NTP_t?usp=sharing)。
实验中需要下载 'ISPRS_semantic_labeling_Vaihingen.zip' 和 'ISPRS_semantic_labeling_Vaihingen_ground_truth_eroded_COMPLETE.zip'。
对于 Vaihingen 数据集，请运行以下命令调整数据集目录。
@@ -616,3 +619,33 @@ python tools/convert_datasets/refuge.py --raw_data_root=/path/to/refuge/REFUGE2/
- 您可以在配置中使用 `MapillaryDataset_v1` 和 `Mapillary Dataset_v2` 设置数据集版本。
在此处 [V1.2](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/_base_/datasets/mapillary_v1.py) 和 [V2.0](https://github.com/open-mmlab/mmsegmentation/blob/main/configs/_base_/datasets/mapillary_v2.py) 查看 Mapillary Vistas 数据集配置文件
+
+## LEVIR-CD
+
+[LEVIR-CD](https://justchenhao.github.io/LEVIR/) 大规模遥感建筑变化检测数据集。
+
+数据集可以在[主页](https://justchenhao.github.io/LEVIR/)上请求获得。
+
+数据集的补充版本可以在[主页](https://github.com/S2Looking/Dataset)上请求获得。
+
+请下载数据集的补充版本，然后解压 `LEVIR-CD+.zip`，数据集的内容包括：
+
+```none
+│ ├── LEVIR-CD+
+│ │ ├── train
+│ │ │ ├── A
+│ │ │ ├── B
+│ │ │ ├── label
+│ │ ├── test
+│ │ │ ├── A
+│ │ │ ├── B
+│ │ │ ├── label
+```
+
+对于 LEVIR-CD 数据集，请运行以下命令无重叠裁剪影像：
+
+```shell
+python tools/dataset_converters/levircd.py --dataset-path /path/to/LEVIR-CD+ --out_dir /path/to/LEVIR-CD
+```
+
+裁剪后的影像大小为256x256，与原论文保持一致。
diff --git a/docs/zh_cn/user_guides/5_deployment.md b/docs/zh_cn/user_guides/5_deployment.md
new file mode 100644
index 0000000000..6a2fc69aec
--- /dev/null
+++ b/docs/zh_cn/user_guides/5_deployment.md
@@ -0,0 +1,242 @@
+# 教程5：模型部署
+
+# MMSegmentation 模型部署
+
+- [教程5：模型部署](#教程5模型部署)
+- [MMSegmentation 模型部署](#mmsegmentation-模型部署)
+ - [安装](#安装)
+ - [安装 mmseg](#安装-mmseg)
+ - [安装 mmdeploy](#安装-mmdeploy)
+ - [模型转换](#模型转换)
+ - [模型规范](#模型规范)
+ - [模型推理](#模型推理)
+ - [后端模型推理](#后端模型推理)
+ - [SDK 模型推理](#sdk-模型推理)
+ - [模型支持列表](#模型支持列表)
+ - [注意事项](#注意事项)
+
+______________________________________________________________________
+
+[MMSegmentation](https://github.com/open-mmlab/mmsegmentation/tree/main) 又称`mmseg`，是一个基于 PyTorch 的开源对象分割工具箱。它是 [OpenMMLab](https://openmmlab.com/) 项目的一部分。
+
+## 安装
+
+### 安装 mmseg
+
+请参考[官网安装指南](https://mmsegmentation.readthedocs.io/en/latest/get_started.html)。
+
+### 安装 mmdeploy
+
+mmdeploy 有以下几种安装方式:
+
+**方式一：** 安装预编译包
+
+请参考[安装概述](https://mmdeploy.readthedocs.io/zh_CN/latest/get_started.html#mmdeploy)
+
+**方式二：** 一键式脚本安装
+
+如果部署平台是 **Ubuntu 18.04 及以上版本**， 请参考[脚本安装说明](../01-how-to-build/build_from_script.md)，完成安装过程。
+比如，以下命令可以安装 mmdeploy 以及配套的推理引擎——`ONNX Runtime`.
+
+```shell
+git clone --recursive -b main https://github.com/open-mmlab/mmdeploy.git
+cd mmdeploy
+python3 tools/scripts/build_ubuntu_x64_ort.py $(nproc)
+export PYTHONPATH=$(pwd)/build/lib:$PYTHONPATH
+export LD_LIBRARY_PATH=$(pwd)/../mmdeploy-dep/onnxruntime-linux-x64-1.8.1/lib/:$LD_LIBRARY_PATH
+```
+
+**说明**:
+
+- 把 `$(pwd)/build/lib` 添加到 `PYTHONPATH`，目的是为了加载 mmdeploy SDK python 包 `mmdeploy_runtime`，在章节 [SDK模型推理](#sdk模型推理)中讲述其用法。
+- 在[使用 ONNX Runtime推理后端模型](#后端模型推理)时，需要加载自定义算子库，需要把 ONNX Runtime 库的路径加入环境变量 `LD_LIBRARY_PATH`中。
+ **方式三：** 源码安装
+
+在方式一、二都满足不了的情况下，请参考[源码安装说明](../01-how-to-build/build_from_source.md) 安装 mmdeploy 以及所需推理引擎。
+
+## 模型转换
+
+你可以使用 [tools/deploy.py](https://github.com/open-mmlab/mmdeploy/tree/main/tools/deploy.py) 把 mmseg 模型一键式转换为推理后端模型。
+该工具的详细使用说明请参考[这里](https://github.com/open-mmlab/mmdeploy/tree/main/docs/en/02-how-to-run/convert_model.md#usage).
+
+以下，我们将演示如何把 `unet` 转换为 onnx 模型。
+
+```shell
+cd mmdeploy
+
+# download unet model from mmseg model zoo
+mim download mmsegmentation --config unet-s5-d16_fcn_4xb4-160k_cityscapes-512x1024 --dest .
+
+# convert mmseg model to onnxruntime model with dynamic shape
+python tools/deploy.py \
+ configs/mmseg/segmentation_onnxruntime_dynamic.py \
+ unet-s5-d16_fcn_4xb4-160k_cityscapes-512x1024.py \
+ fcn_unet_s5-d16_4x4_512x1024_160k_cityscapes_20211210_145204-6860854e.pth \
+ demo/resources/cityscapes.png \
+ --work-dir mmdeploy_models/mmseg/ort \
+ --device cpu \
+ --show \
+ --dump-info
+```
+
+转换的关键之一是使用正确的配置文件。项目中已内置了各后端部署[配置文件](https://github.com/open-mmlab/mmdeploy/tree/main/configs/mmseg)。
+文件的命名模式是：
+
+```
+segmentation_{backend}-{precision}_{static | dynamic}_{shape}.py
+```
+
+其中：
+
+- **{backend}:** 推理后端名称。比如，onnxruntime、tensorrt、pplnn、ncnn、openvino、coreml 等等
+- **{precision}:** 推理精度。比如，fp16、int8。不填表示 fp32
+- **{static | dynamic}:** 动态、静态 shape
+- **{shape}:** 模型输入的 shape 或者 shape 范围
+
+在上例中，你也可以把 `unet` 转为其他后端模型。比如使用`segmentation_tensorrt-fp16_dynamic-512x1024-2048x2048.py`，把模型转为 tensorrt-fp16 模型。
+
+```{tip}
+当转 tensorrt 模型时, --device 需要被设置为 "cuda"
+```
+
+## 模型规范
+
+在使用转换后的模型进行推理之前，有必要了解转换结果的结构。 它存放在 `--work-dir` 指定的路路径下。
+
+上例中的`mmdeploy_models/mmseg/ort`，结构如下：
+
+```
+mmdeploy_models/mmseg/ort
+├── deploy.json
+├── detail.json
+├── end2end.onnx
+└── pipeline.json
+```
+
+重要的是：
+
+- **end2end.onnx**: 推理引擎文件。可用 ONNX Runtime 推理
+- \***.json**: mmdeploy SDK 推理所需的 meta 信息
+
+整个文件夹被定义为**mmdeploy SDK model**。换言之，**mmdeploy SDK model**既包括推理引擎，也包括推理 meta 信息。
+
+## 模型推理
+
+### 后端模型推理
+
+以上述模型转换后的 `end2end.onnx` 为例，你可以使用如下代码进行推理：
+
+```python
+from mmdeploy.apis.utils import build_task_processor
+from mmdeploy.utils import get_input_shape, load_config
+import torch
+
+deploy_cfg = 'configs/mmseg/segmentation_onnxruntime_dynamic.py'
+model_cfg = './unet-s5-d16_fcn_4xb4-160k_cityscapes-512x1024.py'
+device = 'cpu'
+backend_model = ['./mmdeploy_models/mmseg/ort/end2end.onnx']
+image = './demo/resources/cityscapes.png'
+
+# read deploy_cfg and model_cfg
+deploy_cfg, model_cfg = load_config(deploy_cfg, model_cfg)
+
+# build task and backend model
+task_processor = build_task_processor(model_cfg, deploy_cfg, device)
+model = task_processor.build_backend_model(backend_model)
+
+# process input image
+input_shape = get_input_shape(deploy_cfg)
+model_inputs, _ = task_processor.create_input(image, input_shape)
+
+# do model inference
+with torch.no_grad():
+ result = model.test_step(model_inputs)
+
+# visualize results
+task_processor.visualize(
+ image=image,
+ model=model,
+ result=result[0],
+ window_name='visualize',
+ output_file='./output_segmentation.png')
+```
+
+### SDK 模型推理
+
+你也可以参考如下代码，对 SDK model 进行推理：
+
+```python
+from mmdeploy_runtime import Segmentor
+import cv2
+import numpy as np
+
+img = cv2.imread('./demo/resources/cityscapes.png')
+
+# create a classifier
+segmentor = Segmentor(model_path='./mmdeploy_models/mmseg/ort', device_name='cpu', device_id=0)
+# perform inference
+seg = segmentor(img)
+
+# visualize inference result
+## random a palette with size 256x3
+palette = np.random.randint(0, 256, size=(256, 3))
+color_seg = np.zeros((seg.shape[0], seg.shape[1], 3), dtype=np.uint8)
+for label, color in enumerate(palette):
+ color_seg[seg == label, :] = color
+# convert to BGR
+color_seg = color_seg[..., ::-1]
+img = img * 0.5 + color_seg * 0.5
+img = img.astype(np.uint8)
+cv2.imwrite('output_segmentation.png', img)
+```
+
+除了python API，mmdeploy SDK 还提供了诸如 C、C++、C#、Java等多语言接口。
+你可以参考[样例](https://github.com/open-mmlab/mmdeploy/tree/main/demo)学习其他语言接口的使用方法。
+
+## 模型支持列表
+
+| Model | TorchScript | OnnxRuntime | TensorRT | ncnn | PPLNN | OpenVino |
+| :-------------------------------------------------------------------------------------------------------- | :---------: | :---------: | :------: | :--: | :---: | :------: |
+| [FCN](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/fcn) | Y | Y | Y | Y | Y | Y |
+| [PSPNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/pspnet)[\*](#static_shape) | Y | Y | Y | Y | Y | Y |
+| [DeepLabV3](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/deeplabv3) | Y | Y | Y | Y | Y | Y |
+| [DeepLabV3+](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/deeplabv3plus) | Y | Y | Y | Y | Y | Y |
+| [Fast-SCNN](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/fastscnn)[\*](#static_shape) | Y | Y | Y | N | Y | Y |
+| [UNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/unet) | Y | Y | Y | Y | Y | Y |
+| [ANN](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/ann)[\*](#static_shape) | Y | Y | Y | N | N | N |
+| [APCNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/apcnet) | Y | Y | Y | Y | N | N |
+| [BiSeNetV1](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/bisenetv1) | Y | Y | Y | Y | N | Y |
+| [BiSeNetV2](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/bisenetv2) | Y | Y | Y | Y | N | Y |
+| [CGNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/cgnet) | Y | Y | Y | Y | N | Y |
+| [DMNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/dmnet) | ? | Y | N | N | N | N |
+| [DNLNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/dnlnet) | ? | Y | Y | Y | N | Y |
+| [EMANet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/emanet) | Y | Y | Y | N | N | Y |
+| [EncNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/encnet) | Y | Y | Y | N | N | Y |
+| [ERFNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/erfnet) | Y | Y | Y | Y | N | Y |
+| [FastFCN](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/fastfcn) | Y | Y | Y | Y | N | Y |
+| [GCNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/gcnet) | Y | Y | Y | N | N | N |
+| [ICNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/icnet)[\*](#static_shape) | Y | Y | Y | N | N | Y |
+| [ISANet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/isanet)[\*](#static_shape) | N | Y | Y | N | N | Y |
+| [NonLocal Net](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/nonlocal_net) | ? | Y | Y | Y | N | Y |
+| [OCRNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/ocrnet) | Y | Y | Y | Y | N | Y |
+| [PointRend](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/point_rend)[\*](#static_shape) | Y | Y | Y | N | N | N |
+| [Semantic FPN](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/sem_fpn) | Y | Y | Y | Y | N | Y |
+| [STDC](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/stdc) | Y | Y | Y | Y | N | Y |
+| [UPerNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/upernet)[\*](#static_shape) | N | Y | Y | N | N | N |
+| [DANet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/danet) | ? | Y | Y | N | N | Y |
+| [Segmenter](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/segmenter)[\*](#static_shape) | N | Y | Y | Y | N | Y |
+| [SegFormer](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/segformer)[\*](#static_shape) | ? | Y | Y | N | N | Y |
+| [SETR](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/setr) | ? | Y | N | N | N | Y |
+| [CCNet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/ccnet) | ? | N | N | N | N | N |
+| [PSANet](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/psanet) | ? | N | N | N | N | N |
+| [DPT](https://github.com/open-mmlab/mmsegmentation/tree/main/configs/dpt) | ? | N | N | N | N | N |
+
+## 注意事项
+
+- 所有 mmseg 模型仅支持 "whole" 推理模式。
+
+-
PSPNet，Fast-SCNN 仅支持静态输入，因为多数推理框架的 [nn.AdaptiveAvgPool2d](https://github.com/open-mmlab/mmsegmentation/blob/0c87f7a0c9099844eff8e90fa3db5b0d0ca02fee/mmseg/models/decode_heads/psp_head.py#L38) 不支持动态输入。
+
+- 对于仅支持静态形状的模型，应使用静态形状的部署配置文件，例如 `configs/mmseg/segmentation_tensorrt_static-1024x2048.py`
+
+- 对于喜欢部署模型生成概率特征图的用户，将 `codebase_config = dict(with_argmax=False)` 放在部署配置中就足够了。
diff --git a/docs/zh_cn/user_guides/deployment.md b/docs/zh_cn/user_guides/deployment.md
deleted file mode 100644
index f98110c8b5..0000000000
--- a/docs/zh_cn/user_guides/deployment.md
+++ /dev/null
@@ -1 +0,0 @@
-# 模型部署
diff --git a/mmseg/apis/inference.py b/mmseg/apis/inference.py
index 4aadffc798..81cd17d798 100644
--- a/mmseg/apis/inference.py
+++ b/mmseg/apis/inference.py
@@ -187,7 +187,7 @@ def show_result_pyplot(model: BaseSegmentor,
if hasattr(model, 'module'):
model = model.module
if isinstance(img, str):
- image = mmcv.imread(img)
+ image = mmcv.imread(img, channel_order='rgb')
else:
image = img
if save_dir is not None:
diff --git a/mmseg/datasets/__init__.py b/mmseg/datasets/__init__.py
index a90d53c88e..587385bf26 100644
--- a/mmseg/datasets/__init__.py
+++ b/mmseg/datasets/__init__.py
@@ -1,7 +1,7 @@
# Copyright (c) OpenMMLab. All rights reserved.
# yapf: disable
from .ade import ADE20KDataset
-from .basesegdataset import BaseSegDataset
+from .basesegdataset import BaseCDDataset, BaseSegDataset
from .chase_db1 import ChaseDB1Dataset
from .cityscapes import CityscapesDataset
from .coco_stuff import COCOStuffDataset
@@ -9,9 +9,11 @@
from .dataset_wrappers import MultiImageMixDataset
from .decathlon import DecathlonDataset
from .drive import DRIVEDataset
+from .dsdl import DSDLSegDataset
from .hrf import HRFDataset
from .isaid import iSAIDDataset
from .isprs import ISPRSDataset
+from .levir import LEVIRCDDataset
from .lip import LIPDataset
from .loveda import LoveDADataset
from .mapillary import MapillaryDataset_v1, MapillaryDataset_v2
@@ -22,16 +24,18 @@
from .stare import STAREDataset
from .synapse import SynapseDataset
# yapf: disable
-from .transforms import (CLAHE, AdjustGamma, BioMedical3DPad,
+from .transforms import (CLAHE, AdjustGamma, Albu, BioMedical3DPad,
BioMedical3DRandomCrop, BioMedical3DRandomFlip,
BioMedicalGaussianBlur, BioMedicalGaussianNoise,
- BioMedicalRandomGamma, GenerateEdge, LoadAnnotations,
- LoadBiomedicalAnnotation, LoadBiomedicalData,
- LoadBiomedicalImageFromFile, LoadImageFromNDArray,
- PackSegInputs, PhotoMetricDistortion, RandomCrop,
- RandomCutOut, RandomMosaic, RandomRotate,
- RandomRotFlip, Rerange, ResizeShortestEdge,
- ResizeToMultiple, RGB2Gray, SegRescale)
+ BioMedicalRandomGamma, ConcatCDInput, GenerateEdge,
+ LoadAnnotations, LoadBiomedicalAnnotation,
+ LoadBiomedicalData, LoadBiomedicalImageFromFile,
+ LoadImageFromNDArray, LoadMultipleRSImageFromFile,
+ LoadSingleRSImageFromFile, PackSegInputs,
+ PhotoMetricDistortion, RandomCrop, RandomCutOut,
+ RandomMosaic, RandomRotate, RandomRotFlip, Rerange,
+ ResizeShortestEdge, ResizeToMultiple, RGB2Gray,
+ SegRescale)
from .voc import PascalVOCDataset
# yapf: enable
@@ -51,5 +55,7 @@
'BioMedicalGaussianNoise', 'BioMedicalGaussianBlur',
'BioMedicalRandomGamma', 'BioMedical3DPad', 'RandomRotFlip',
'SynapseDataset', 'REFUGEDataset', 'MapillaryDataset_v1',
- 'MapillaryDataset_v2'
+ 'MapillaryDataset_v2', 'Albu', 'LEVIRCDDataset',
+ 'LoadMultipleRSImageFromFile', 'LoadSingleRSImageFromFile',
+ 'ConcatCDInput', 'BaseCDDataset', 'DSDLSegDataset'
]
diff --git a/mmseg/datasets/basesegdataset.py b/mmseg/datasets/basesegdataset.py
index ddf476bae9..9c4668c1f5 100644
--- a/mmseg/datasets/basesegdataset.py
+++ b/mmseg/datasets/basesegdataset.py
@@ -235,7 +235,9 @@ def load_data_list(self) -> List[dict]:
data_list = []
img_dir = self.data_prefix.get('img_path', None)
ann_dir = self.data_prefix.get('seg_map_path', None)
- if osp.isfile(self.ann_file):
+ if not osp.isdir(self.ann_file) and self.ann_file:
+ assert osp.isfile(self.ann_file), \
+ f'Failed to load `ann_file` {self.ann_file}'
lines = mmengine.list_from_file(
self.ann_file, backend_args=self.backend_args)
for line in lines:
@@ -250,6 +252,7 @@ def load_data_list(self) -> List[dict]:
data_info['seg_fields'] = []
data_list.append(data_info)
else:
+ _suffix_len = len(self.img_suffix)
for img in fileio.list_dir_or_file(
dir_path=img_dir,
list_dir=False,
@@ -258,7 +261,288 @@ def load_data_list(self) -> List[dict]:
backend_args=self.backend_args):
data_info = dict(img_path=osp.join(img_dir, img))
if ann_dir is not None:
- seg_map = img.replace(self.img_suffix, self.seg_map_suffix)
+ seg_map = img[:-_suffix_len] + self.seg_map_suffix
+ data_info['seg_map_path'] = osp.join(ann_dir, seg_map)
+ data_info['label_map'] = self.label_map
+ data_info['reduce_zero_label'] = self.reduce_zero_label
+ data_info['seg_fields'] = []
+ data_list.append(data_info)
+ data_list = sorted(data_list, key=lambda x: x['img_path'])
+ return data_list
+
+
+@DATASETS.register_module()
+class BaseCDDataset(BaseDataset):
+ """Custom dataset for change detection. An example of file structure is as
+ followed.
+
+ .. code-block:: none
+
+ ├── data
+ │ ├── my_dataset
+ │ │ ├── img_dir
+ │ │ │ ├── train
+ │ │ │ │ ├── xxx{img_suffix}
+ │ │ │ │ ├── yyy{img_suffix}
+ │ │ │ │ ├── zzz{img_suffix}
+ │ │ │ ├── val
+ │ │ ├── img_dir2
+ │ │ │ ├── train
+ │ │ │ │ ├── xxx{img_suffix}
+ │ │ │ │ ├── yyy{img_suffix}
+ │ │ │ │ ├── zzz{img_suffix}
+ │ │ │ ├── val
+ │ │ ├── ann_dir
+ │ │ │ ├── train
+ │ │ │ │ ├── xxx{seg_map_suffix}
+ │ │ │ │ ├── yyy{seg_map_suffix}
+ │ │ │ │ ├── zzz{seg_map_suffix}
+ │ │ │ ├── val
+
+ The image names in img_dir and img_dir2 should be consistent.
+ The img/gt_semantic_seg pair of BaseSegDataset should be of the same
+ except suffix. A valid img/gt_semantic_seg filename pair should be like
+ ``xxx{img_suffix}`` and ``xxx{seg_map_suffix}`` (extension is also included
+ in the suffix). If split is given, then ``xxx`` is specified in txt file.
+ Otherwise, all files in ``img_dir/``and ``ann_dir`` will be loaded.
+ Please refer to ``docs/en/tutorials/new_dataset.md`` for more details.
+
+
+ Args:
+ ann_file (str): Annotation file path. Defaults to ''.
+ metainfo (dict, optional): Meta information for dataset, such as
+ specify classes to load. Defaults to None.
+ data_root (str, optional): The root directory for ``data_prefix`` and
+ ``ann_file``. Defaults to None.
+ data_prefix (dict, optional): Prefix for training data. Defaults to
+ dict(img_path=None, img_path2=None, seg_map_path=None).
+ img_suffix (str): Suffix of images. Default: '.jpg'
+ img_suffix2 (str): Suffix of images. Default: '.jpg'
+ seg_map_suffix (str): Suffix of segmentation maps. Default: '.png'
+ filter_cfg (dict, optional): Config for filter data. Defaults to None.
+ indices (int or Sequence[int], optional): Support using first few
+ data in annotation file to facilitate training/testing on a smaller
+ dataset. Defaults to None which means using all ``data_infos``.
+ serialize_data (bool, optional): Whether to hold memory using
+ serialized objects, when enabled, data loader workers can use
+ shared RAM from master process instead of making a copy. Defaults
+ to True.
+ pipeline (list, optional): Processing pipeline. Defaults to [].
+ test_mode (bool, optional): ``test_mode=True`` means in test phase.
+ Defaults to False.
+ lazy_init (bool, optional): Whether to load annotation during
+ instantiation. In some cases, such as visualization, only the meta
+ information of the dataset is needed, which is not necessary to
+ load annotation file. ``Basedataset`` can skip load annotations to
+ save time by set ``lazy_init=True``. Defaults to False.
+ max_refetch (int, optional): If ``Basedataset.prepare_data`` get a
+ None img. The maximum extra number of cycles to get a valid
+ image. Defaults to 1000.
+ ignore_index (int): The label index to be ignored. Default: 255
+ reduce_zero_label (bool): Whether to mark label zero as ignored.
+ Default to False.
+ backend_args (dict, Optional): Arguments to instantiate a file backend.
+ See https://mmengine.readthedocs.io/en/latest/api/fileio.htm
+ for details. Defaults to None.
+ Notes: mmcv>=2.0.0rc4, mmengine>=0.2.0 required.
+ """
+ METAINFO: dict = dict()
+
+ def __init__(self,
+ ann_file: str = '',
+ img_suffix='.jpg',
+ img_suffix2='.jpg',
+ seg_map_suffix='.png',
+ metainfo: Optional[dict] = None,
+ data_root: Optional[str] = None,
+ data_prefix: dict = dict(
+ img_path='', img_path2='', seg_map_path=''),
+ filter_cfg: Optional[dict] = None,
+ indices: Optional[Union[int, Sequence[int]]] = None,
+ serialize_data: bool = True,
+ pipeline: List[Union[dict, Callable]] = [],
+ test_mode: bool = False,
+ lazy_init: bool = False,
+ max_refetch: int = 1000,
+ ignore_index: int = 255,
+ reduce_zero_label: bool = False,
+ backend_args: Optional[dict] = None) -> None:
+
+ self.img_suffix = img_suffix
+ self.img_suffix2 = img_suffix2
+ self.seg_map_suffix = seg_map_suffix
+ self.ignore_index = ignore_index
+ self.reduce_zero_label = reduce_zero_label
+ self.backend_args = backend_args.copy() if backend_args else None
+
+ self.data_root = data_root
+ self.data_prefix = copy.copy(data_prefix)
+ self.ann_file = ann_file
+ self.filter_cfg = copy.deepcopy(filter_cfg)
+ self._indices = indices
+ self.serialize_data = serialize_data
+ self.test_mode = test_mode
+ self.max_refetch = max_refetch
+ self.data_list: List[dict] = []
+ self.data_bytes: np.ndarray
+
+ # Set meta information.
+ self._metainfo = self._load_metainfo(copy.deepcopy(metainfo))
+
+ # Get label map for custom classes
+ new_classes = self._metainfo.get('classes', None)
+ self.label_map = self.get_label_map(new_classes)
+ self._metainfo.update(
+ dict(
+ label_map=self.label_map,
+ reduce_zero_label=self.reduce_zero_label))
+
+ # Update palette based on label map or generate palette
+ # if it is not defined
+ updated_palette = self._update_palette()
+ self._metainfo.update(dict(palette=updated_palette))
+
+ # Join paths.
+ if self.data_root is not None:
+ self._join_prefix()
+
+ # Build pipeline.
+ self.pipeline = Compose(pipeline)
+ # Full initialize the dataset.
+ if not lazy_init:
+ self.full_init()
+
+ if test_mode:
+ assert self._metainfo.get('classes') is not None, \
+ 'dataset metainfo `classes` should be specified when testing'
+
+ @classmethod
+ def get_label_map(cls,
+ new_classes: Optional[Sequence] = None
+ ) -> Union[Dict, None]:
+ """Require label mapping.
+
+ The ``label_map`` is a dictionary, its keys are the old label ids and
+ its values are the new label ids, and is used for changing pixel
+ labels in load_annotations. If and only if old classes in cls.METAINFO
+ is not equal to new classes in self._metainfo and nether of them is not
+ None, `label_map` is not None.
+
+ Args:
+ new_classes (list, tuple, optional): The new classes name from
+ metainfo. Default to None.
+
+
+ Returns:
+ dict, optional: The mapping from old classes in cls.METAINFO to
+ new classes in self._metainfo
+ """
+ old_classes = cls.METAINFO.get('classes', None)
+ if (new_classes is not None and old_classes is not None
+ and list(new_classes) != list(old_classes)):
+
+ label_map = {}
+ if not set(new_classes).issubset(cls.METAINFO['classes']):
+ raise ValueError(
+ f'new classes {new_classes} is not a '
+ f'subset of classes {old_classes} in METAINFO.')
+ for i, c in enumerate(old_classes):
+ if c not in new_classes:
+ label_map[i] = 255
+ else:
+ label_map[i] = new_classes.index(c)
+ return label_map
+ else:
+ return None
+
+ def _update_palette(self) -> list:
+ """Update palette after loading metainfo.
+
+ If length of palette is equal to classes, just return the palette.
+ If palette is not defined, it will randomly generate a palette.
+ If classes is updated by customer, it will return the subset of
+ palette.
+
+ Returns:
+ Sequence: Palette for current dataset.
+ """
+ palette = self._metainfo.get('palette', [])
+ classes = self._metainfo.get('classes', [])
+ # palette does match classes
+ if len(palette) == len(classes):
+ return palette
+
+ if len(palette) == 0:
+ # Get random state before set seed, and restore
+ # random state later.
+ # It will prevent loss of randomness, as the palette
+ # may be different in each iteration if not specified.
+ # See: https://github.com/open-mmlab/mmdetection/issues/5844
+ state = np.random.get_state()
+ np.random.seed(42)
+ # random palette
+ new_palette = np.random.randint(
+ 0, 255, size=(len(classes), 3)).tolist()
+ np.random.set_state(state)
+ elif len(palette) >= len(classes) and self.label_map is not None:
+ new_palette = []
+ # return subset of palette
+ for old_id, new_id in sorted(
+ self.label_map.items(), key=lambda x: x[1]):
+ if new_id != 255:
+ new_palette.append(palette[old_id])
+ new_palette = type(palette)(new_palette)
+ else:
+ raise ValueError('palette does not match classes '
+ f'as metainfo is {self._metainfo}.')
+ return new_palette
+
+ def load_data_list(self) -> List[dict]:
+ """Load annotation from directory or annotation file.
+
+ Returns:
+ list[dict]: All data info of dataset.
+ """
+ data_list = []
+ img_dir = self.data_prefix.get('img_path', None)
+ img_dir2 = self.data_prefix.get('img_path2', None)
+ ann_dir = self.data_prefix.get('seg_map_path', None)
+ if osp.isfile(self.ann_file):
+ lines = mmengine.list_from_file(
+ self.ann_file, backend_args=self.backend_args)
+ for line in lines:
+ img_name = line.strip()
+ if '.' in osp.basename(img_name):
+ img_name, img_ext = osp.splitext(img_name)
+ self.img_suffix = img_ext
+ self.img_suffix2 = img_ext
+ data_info = dict(
+ img_path=osp.join(img_dir, img_name + self.img_suffix),
+ img_path2=osp.join(img_dir2, img_name + self.img_suffix2))
+
+ if ann_dir is not None:
+ seg_map = img_name + self.seg_map_suffix
+ data_info['seg_map_path'] = osp.join(ann_dir, seg_map)
+ data_info['label_map'] = self.label_map
+ data_info['reduce_zero_label'] = self.reduce_zero_label
+ data_info['seg_fields'] = []
+ data_list.append(data_info)
+ else:
+ for img in fileio.list_dir_or_file(
+ dir_path=img_dir,
+ list_dir=False,
+ suffix=self.img_suffix,
+ recursive=True,
+ backend_args=self.backend_args):
+ if '.' in osp.basename(img):
+ img, img_ext = osp.splitext(img)
+ self.img_suffix = img_ext
+ self.img_suffix2 = img_ext
+ data_info = dict(
+ img_path=osp.join(img_dir, img + self.img_suffix),
+ img_path2=osp.join(img_dir2, img + self.img_suffix2))
+ if ann_dir is not None:
+ seg_map = img + self.seg_map_suffix
data_info['seg_map_path'] = osp.join(ann_dir, seg_map)
data_info['label_map'] = self.label_map
data_info['reduce_zero_label'] = self.reduce_zero_label
diff --git a/mmseg/datasets/chase_db1.py b/mmseg/datasets/chase_db1.py
index 5cc1fc5677..626ddf75e9 100644
--- a/mmseg/datasets/chase_db1.py
+++ b/mmseg/datasets/chase_db1.py
@@ -1,4 +1,5 @@
# Copyright (c) OpenMMLab. All rights reserved.
+import mmengine.fileio as fileio
from mmseg.registry import DATASETS
from .basesegdataset import BaseSegDataset
@@ -27,4 +28,5 @@ def __init__(self,
seg_map_suffix=seg_map_suffix,
reduce_zero_label=reduce_zero_label,
**kwargs)
- assert self.file_client.exists(self.data_prefix['img_path'])
+ assert fileio.exists(
+ self.data_prefix['img_path'], backend_args=self.backend_args)
diff --git a/mmseg/datasets/drive.py b/mmseg/datasets/drive.py
index c42e18e711..76c0160a6b 100644
--- a/mmseg/datasets/drive.py
+++ b/mmseg/datasets/drive.py
@@ -1,4 +1,5 @@
# Copyright (c) OpenMMLab. All rights reserved.
+import mmengine.fileio as fileio
from mmseg.registry import DATASETS
from .basesegdataset import BaseSegDataset
@@ -27,4 +28,5 @@ def __init__(self,
seg_map_suffix=seg_map_suffix,
reduce_zero_label=reduce_zero_label,
**kwargs)
- assert self.file_client.exists(self.data_prefix['img_path'])
+ assert fileio.exists(
+ self.data_prefix['img_path'], backend_args=self.backend_args)
diff --git a/mmseg/datasets/dsdl.py b/mmseg/datasets/dsdl.py
new file mode 100644
index 0000000000..bf7e4e61b5
--- /dev/null
+++ b/mmseg/datasets/dsdl.py
@@ -0,0 +1,116 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import os
+from typing import Dict, List, Optional, Sequence, Union
+
+from mmseg.registry import DATASETS
+from .basesegdataset import BaseSegDataset
+
+try:
+ from dsdl.dataset import DSDLDataset
+except ImportError:
+ DSDLDataset = None
+
+
+@DATASETS.register_module()
+class DSDLSegDataset(BaseSegDataset):
+ """Dataset for dsdl segmentation.
+
+ Args:
+ specific_key_path(dict): Path of specific key which can not
+ be loaded by it's field name.
+ pre_transform(dict): pre-transform functions before loading.
+ used_labels(sequence): list of actual used classes in train steps,
+ this must be subset of class domain.
+ """
+
+ METAINFO = {}
+
+ def __init__(self,
+ specific_key_path: Dict = {},
+ pre_transform: Dict = {},
+ used_labels: Optional[Sequence] = None,
+ **kwargs) -> None:
+
+ if DSDLDataset is None:
+ raise RuntimeError(
+ 'Package dsdl is not installed. Please run "pip install dsdl".'
+ )
+ self.used_labels = used_labels
+
+ loc_config = dict(type='LocalFileReader', working_dir='')
+ if kwargs.get('data_root'):
+ kwargs['ann_file'] = os.path.join(kwargs['data_root'],
+ kwargs['ann_file'])
+ required_fields = ['Image', 'LabelMap']
+
+ self.dsdldataset = DSDLDataset(
+ dsdl_yaml=kwargs['ann_file'],
+ location_config=loc_config,
+ required_fields=required_fields,
+ specific_key_path=specific_key_path,
+ transform=pre_transform,
+ )
+ BaseSegDataset.__init__(self, **kwargs)
+
+ def load_data_list(self) -> List[Dict]:
+ """Load data info from a dsdl yaml file named as ``self.ann_file``
+
+ Returns:
+ List[dict]: A list of data list.
+ """
+
+ if self.used_labels:
+ self._metainfo['classes'] = tuple(self.used_labels)
+ self.label_map = self.get_label_map(self.used_labels)
+ else:
+ self._metainfo['classes'] = tuple(['background'] +
+ self.dsdldataset.class_names)
+ data_list = []
+
+ for i, data in enumerate(self.dsdldataset):
+ datainfo = dict(
+ img_path=os.path.join(self.data_prefix['img_path'],
+ data['Image'][0].location),
+ seg_map_path=os.path.join(self.data_prefix['seg_map_path'],
+ data['LabelMap'][0].location),
+ label_map=self.label_map,
+ reduce_zero_label=self.reduce_zero_label,
+ seg_fields=[],
+ )
+ data_list.append(datainfo)
+
+ return data_list
+
+ def get_label_map(self,
+ new_classes: Optional[Sequence] = None
+ ) -> Union[Dict, None]:
+ """Require label mapping.
+
+ The ``label_map`` is a dictionary, its keys are the old label ids and
+ its values are the new label ids, and is used for changing pixel
+ labels in load_annotations. If and only if old classes in class_dom
+ is not equal to new classes in args and nether of them is not
+ None, `label_map` is not None.
+ Args:
+ new_classes (list, tuple, optional): The new classes name from
+ metainfo. Default to None.
+ Returns:
+ dict, optional: The mapping from old classes to new classes.
+ """
+ old_classes = ['background'] + self.dsdldataset.class_names
+ if (new_classes is not None and old_classes is not None
+ and list(new_classes) != list(old_classes)):
+
+ label_map = {}
+ if not set(new_classes).issubset(old_classes):
+ raise ValueError(
+ f'new classes {new_classes} is not a '
+ f'subset of classes {old_classes} in class_dom.')
+ for i, c in enumerate(old_classes):
+ if c not in new_classes:
+ label_map[i] = 255
+ else:
+ label_map[i] = new_classes.index(c)
+ return label_map
+ else:
+ return None
diff --git a/mmseg/datasets/hrf.py b/mmseg/datasets/hrf.py
index 0df6ccc49c..fd669cce26 100644
--- a/mmseg/datasets/hrf.py
+++ b/mmseg/datasets/hrf.py
@@ -1,4 +1,5 @@
# Copyright (c) OpenMMLab. All rights reserved.
+import mmengine.fileio as fileio
from mmseg.registry import DATASETS
from .basesegdataset import BaseSegDataset
@@ -27,4 +28,5 @@ def __init__(self,
seg_map_suffix=seg_map_suffix,
reduce_zero_label=reduce_zero_label,
**kwargs)
- assert self.file_client.exists(self.data_prefix['img_path'])
+ assert fileio.exists(
+ self.data_prefix['img_path'], backend_args=self.backend_args)
diff --git a/mmseg/datasets/levir.py b/mmseg/datasets/levir.py
new file mode 100644
index 0000000000..f467481bad
--- /dev/null
+++ b/mmseg/datasets/levir.py
@@ -0,0 +1,31 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+
+from mmseg.registry import DATASETS
+from .basesegdataset import BaseCDDataset
+
+
+@DATASETS.register_module()
+class LEVIRCDDataset(BaseCDDataset):
+ """ISPRS dataset.
+
+ In segmentation map annotation for ISPRS, 0 is to ignore index.
+ ``reduce_zero_label`` should be set to True. The ``img_suffix`` and
+ ``seg_map_suffix`` are both fixed to '.png'.
+ """
+
+ METAINFO = dict(
+ classes=('background', 'changed'),
+ palette=[[0, 0, 0], [255, 255, 255]])
+
+ def __init__(self,
+ img_suffix='.png',
+ img_suffix2='.png',
+ seg_map_suffix='.png',
+ reduce_zero_label=False,
+ **kwargs) -> None:
+ super().__init__(
+ img_suffix=img_suffix,
+ img_suffix2=img_suffix2,
+ seg_map_suffix=seg_map_suffix,
+ reduce_zero_label=reduce_zero_label,
+ **kwargs)
diff --git a/mmseg/datasets/stare.py b/mmseg/datasets/stare.py
index 2bfce23449..1b997bb785 100644
--- a/mmseg/datasets/stare.py
+++ b/mmseg/datasets/stare.py
@@ -1,4 +1,6 @@
# Copyright (c) OpenMMLab. All rights reserved.
+import mmengine.fileio as fileio
+
from mmseg.registry import DATASETS
from .basesegdataset import BaseSegDataset
@@ -26,4 +28,5 @@ def __init__(self,
seg_map_suffix=seg_map_suffix,
reduce_zero_label=reduce_zero_label,
**kwargs)
- assert self.file_client.exists(self.data_prefix['img_path'])
+ assert fileio.exists(
+ self.data_prefix['img_path'], backend_args=self.backend_args)
diff --git a/mmseg/datasets/transforms/__init__.py b/mmseg/datasets/transforms/__init__.py
index 25f4ee4a98..fe60285471 100644
--- a/mmseg/datasets/transforms/__init__.py
+++ b/mmseg/datasets/transforms/__init__.py
@@ -2,12 +2,13 @@
from .formatting import PackSegInputs
from .loading import (LoadAnnotations, LoadBiomedicalAnnotation,
LoadBiomedicalData, LoadBiomedicalImageFromFile,
- LoadImageFromNDArray)
+ LoadImageFromNDArray, LoadMultipleRSImageFromFile,
+ LoadSingleRSImageFromFile)
# yapf: disable
-from .transforms import (CLAHE, AdjustGamma, BioMedical3DPad,
+from .transforms import (CLAHE, AdjustGamma, Albu, BioMedical3DPad,
BioMedical3DRandomCrop, BioMedical3DRandomFlip,
BioMedicalGaussianBlur, BioMedicalGaussianNoise,
- BioMedicalRandomGamma, GenerateEdge,
+ BioMedicalRandomGamma, ConcatCDInput, GenerateEdge,
PhotoMetricDistortion, RandomCrop, RandomCutOut,
RandomMosaic, RandomRotate, RandomRotFlip, Rerange,
ResizeShortestEdge, ResizeToMultiple, RGB2Gray,
@@ -22,5 +23,6 @@
'LoadBiomedicalAnnotation', 'LoadBiomedicalData', 'GenerateEdge',
'ResizeShortestEdge', 'BioMedicalGaussianNoise', 'BioMedicalGaussianBlur',
'BioMedical3DRandomFlip', 'BioMedicalRandomGamma', 'BioMedical3DPad',
- 'RandomRotFlip'
+ 'RandomRotFlip', 'Albu', 'LoadSingleRSImageFromFile', 'ConcatCDInput',
+ 'LoadMultipleRSImageFromFile'
]
diff --git a/mmseg/datasets/transforms/loading.py b/mmseg/datasets/transforms/loading.py
index d2e93b1abb..001810e817 100644
--- a/mmseg/datasets/transforms/loading.py
+++ b/mmseg/datasets/transforms/loading.py
@@ -12,6 +12,11 @@
from mmseg.registry import TRANSFORMS
from mmseg.utils import datafrombytes
+try:
+ from osgeo import gdal
+except ImportError:
+ gdal = None
+
@TRANSFORMS.register_module()
class LoadAnnotations(MMCV_LoadAnnotations):
@@ -493,3 +498,130 @@ def transform(self, single_input: Union[str, np.ndarray, dict]) -> dict:
if 'img' in inputs:
return self.from_ndarray(inputs)
return self.from_file(inputs)
+
+
+@TRANSFORMS.register_module()
+class LoadSingleRSImageFromFile(BaseTransform):
+ """Load a Remote Sensing mage from file.
+
+ Required Keys:
+
+ - img_path
+
+ Modified Keys:
+
+ - img
+ - img_shape
+ - ori_shape
+
+ Args:
+ to_float32 (bool): Whether to convert the loaded image to a float32
+ numpy array. If set to False, the loaded image is a float64 array.
+ Defaults to True.
+ """
+
+ def __init__(self, to_float32: bool = True):
+ self.to_float32 = to_float32
+
+ if gdal is None:
+ raise RuntimeError('gdal is not installed')
+
+ def transform(self, results: Dict) -> Dict:
+ """Functions to load image.
+
+ Args:
+ results (dict): Result dict from :obj:``mmcv.BaseDataset``.
+
+ Returns:
+ dict: The dict contains loaded image and meta information.
+ """
+
+ filename = results['img_path']
+ ds = gdal.Open(filename)
+ if ds is None:
+ raise Exception(f'Unable to open file: {filename}')
+ img = np.einsum('ijk->jki', ds.ReadAsArray())
+
+ if self.to_float32:
+ img = img.astype(np.float32)
+
+ results['img'] = img
+ results['img_shape'] = img.shape[:2]
+ results['ori_shape'] = img.shape[:2]
+ return results
+
+ def __repr__(self):
+ repr_str = (f'{self.__class__.__name__}('
+ f'to_float32={self.to_float32})')
+ return repr_str
+
+
+@TRANSFORMS.register_module()
+class LoadMultipleRSImageFromFile(BaseTransform):
+ """Load two Remote Sensing mage from file.
+
+ Required Keys:
+
+ - img_path
+ - img_path2
+
+ Modified Keys:
+
+ - img
+ - img2
+ - img_shape
+ - ori_shape
+
+ Args:
+ to_float32 (bool): Whether to convert the loaded image to a float32
+ numpy array. If set to False, the loaded image is a float64 array.
+ Defaults to True.
+ """
+
+ def __init__(self, to_float32: bool = True):
+ if gdal is None:
+ raise RuntimeError('gdal is not installed')
+ self.to_float32 = to_float32
+
+ def transform(self, results: Dict) -> Dict:
+ """Functions to load image.
+
+ Args:
+ results (dict): Result dict from :obj:``mmcv.BaseDataset``.
+
+ Returns:
+ dict: The dict contains loaded image and meta information.
+ """
+
+ filename = results['img_path']
+ filename2 = results['img_path2']
+
+ ds = gdal.Open(filename)
+ ds2 = gdal.Open(filename2)
+
+ if ds is None:
+ raise Exception(f'Unable to open file: {filename}')
+ if ds2 is None:
+ raise Exception(f'Unable to open file: {filename2}')
+
+ img = np.einsum('ijk->jki', ds.ReadAsArray())
+ img2 = np.einsum('ijk->jki', ds2.ReadAsArray())
+
+ if self.to_float32:
+ img = img.astype(np.float32)
+ img2 = img2.astype(np.float32)
+
+ if img.shape != img2.shape:
+ raise Exception(f'Image shapes do not match:'
+ f' {img.shape} vs {img2.shape}')
+
+ results['img'] = img
+ results['img2'] = img2
+ results['img_shape'] = img.shape[:2]
+ results['ori_shape'] = img.shape[:2]
+ return results
+
+ def __repr__(self):
+ repr_str = (f'{self.__class__.__name__}('
+ f'to_float32={self.to_float32})')
+ return repr_str
diff --git a/mmseg/datasets/transforms/transforms.py b/mmseg/datasets/transforms/transforms.py
index fb7e2a0e66..da3bd5f936 100644
--- a/mmseg/datasets/transforms/transforms.py
+++ b/mmseg/datasets/transforms/transforms.py
@@ -1,10 +1,12 @@
# Copyright (c) OpenMMLab. All rights reserved.
import copy
+import inspect
import warnings
from typing import Dict, List, Optional, Sequence, Tuple, Union
import cv2
import mmcv
+import mmengine
import numpy as np
from mmcv.transforms.base import BaseTransform
from mmcv.transforms.utils import cache_randomness
@@ -15,6 +17,15 @@
from mmseg.datasets.dataset_wrappers import MultiImageMixDataset
from mmseg.registry import TRANSFORMS
+try:
+ import albumentations
+ from albumentations import Compose
+ ALBU_INSTALLED = True
+except ImportError:
+ albumentations = None
+ Compose = None
+ ALBU_INSTALLED = False
+
@TRANSFORMS.register_module()
class ResizeToMultiple(BaseTransform):
@@ -186,7 +197,7 @@ def transform(self, results: dict) -> dict:
def __repr__(self):
repr_str = self.__class__.__name__
- repr_str += f'(clip_limit={self.clip_limit}, '\
+ repr_str += f'(clip_limit={self.clip_limit}, ' \
f'tile_grid_size={self.tile_grid_size})'
return repr_str
@@ -1151,8 +1162,8 @@ def _mosaic_transform_seg(self, results: dict) -> dict:
x1_c, y1_c, x2_c, y2_c = crop_coord
# crop and paste image
- mosaic_seg[y1_p:y2_p, x1_p:x2_p] = gt_seg_i[y1_c:y2_c,
- x1_c:x2_c]
+ mosaic_seg[y1_p:y2_p, x1_p:x2_p] = \
+ gt_seg_i[y1_c:y2_c, x1_c:x2_c]
results[key] = mosaic_seg
@@ -1760,9 +1771,9 @@ def __repr__(self):
repr_str += f'(prob={self.prob}, '
repr_str += f'prob_per_channel={self.prob_per_channel}, '
repr_str += f'sigma_range={self.sigma_range}, '
- repr_str += 'different_sigma_per_channel='\
+ repr_str += 'different_sigma_per_channel=' \
f'{self.different_sigma_per_channel}, '
- repr_str += 'different_sigma_per_axis='\
+ repr_str += 'different_sigma_per_axis=' \
f'{self.different_sigma_per_axis})'
return repr_str
@@ -2135,3 +2146,178 @@ def __repr__(self):
repr_str += f'(prob={self.prob}, axes={self.axes}, ' \
f'swap_label_pairs={self.swap_label_pairs})'
return repr_str
+
+
+@TRANSFORMS.register_module()
+class Albu(BaseTransform):
+ """Albumentation augmentation. Adds custom transformations from
+ Albumentations library. Please, visit
+ `https://albumentations.readthedocs.io` to get more information. An example
+ of ``transforms`` is as followed:
+
+ .. code-block::
+ [
+ dict(
+ type='ShiftScaleRotate',
+ shift_limit=0.0625,
+ scale_limit=0.0,
+ rotate_limit=0,
+ interpolation=1,
+ p=0.5),
+ dict(
+ type='RandomBrightnessContrast',
+ brightness_limit=[0.1, 0.3],
+ contrast_limit=[0.1, 0.3],
+ p=0.2),
+ dict(type='ChannelShuffle', p=0.1),
+ dict(
+ type='OneOf',
+ transforms=[
+ dict(type='Blur', blur_limit=3, p=1.0),
+ dict(type='MedianBlur', blur_limit=3, p=1.0)
+ ],
+ p=0.1),
+ ]
+ Args:
+ transforms (list[dict]): A list of albu transformations
+ keymap (dict): Contains {'input key':'albumentation-style key'}
+ update_pad_shape (bool): Whether to update padding shape according to \
+ the output shape of the last transform
+ """
+
+ def __init__(self,
+ transforms: List[dict],
+ keymap: Optional[dict] = None,
+ update_pad_shape: bool = False):
+ if not ALBU_INSTALLED:
+ raise ImportError(
+ 'albumentations is not installed, '
+ 'we suggest install albumentation by '
+ '"pip install albumentations>=0.3.2 --no-binary qudida,albumentations"' # noqa
+ )
+
+ # Args will be modified later, copying it will be safer
+ transforms = copy.deepcopy(transforms)
+
+ self.transforms = transforms
+ self.keymap = keymap
+ self.update_pad_shape = update_pad_shape
+
+ self.aug = Compose([self.albu_builder(t) for t in self.transforms])
+
+ if not keymap:
+ self.keymap_to_albu = {
+ 'img': 'image',
+ 'gt_masks': 'masks',
+ }
+ else:
+ self.keymap_to_albu = copy.deepcopy(keymap)
+ self.keymap_back = {v: k for k, v in self.keymap_to_albu.items()}
+
+ def albu_builder(self, cfg: dict) -> object:
+ """Build a callable object from a dict containing albu arguments.
+
+ Args:
+ cfg (dict): Config dict. It should at least contain the key "type".
+
+ Returns:
+ Callable: A callable object.
+ """
+
+ assert isinstance(cfg, dict) and 'type' in cfg
+ args = cfg.copy()
+
+ obj_type = args.pop('type')
+ if mmengine.is_str(obj_type):
+ if not ALBU_INSTALLED:
+ raise ImportError(
+ 'albumentations is not installed, '
+ 'we suggest install albumentation by '
+ '"pip install albumentations>=0.3.2 --no-binary qudida,albumentations"' # noqa
+ )
+ obj_cls = getattr(albumentations, obj_type)
+ elif inspect.isclass(obj_type):
+ obj_cls = obj_type
+ else:
+ raise TypeError(
+ f'type must be a valid type or str, but got {type(obj_type)}')
+
+ if 'transforms' in args:
+ args['transforms'] = [
+ self.albu_builder(t) for t in args['transforms']
+ ]
+
+ return obj_cls(**args)
+
+ @staticmethod
+ def mapper(d: dict, keymap: dict):
+ """Dictionary mapper.
+
+ Renames keys according to keymap provided.
+ Args:
+ d (dict): old dict
+ keymap (dict): {'old_key':'new_key'}
+ Returns:
+ dict: new dict.
+ """
+
+ updated_dict = {}
+ for k, _ in zip(d.keys(), d.values()):
+ new_k = keymap.get(k, k)
+ updated_dict[new_k] = d[k]
+ return updated_dict
+
+ def transform(self, results):
+ # dict to albumentations format
+ results = self.mapper(results, self.keymap_to_albu)
+
+ # Convert to RGB since Albumentations works with RGB images
+ results['image'] = cv2.cvtColor(results['image'], cv2.COLOR_BGR2RGB)
+
+ results = self.aug(**results)
+
+ # Convert back to BGR
+ results['image'] = cv2.cvtColor(results['image'], cv2.COLOR_RGB2BGR)
+
+ # back to the original format
+ results = self.mapper(results, self.keymap_back)
+
+ # update final shape
+ if self.update_pad_shape:
+ results['pad_shape'] = results['img'].shape
+
+ return results
+
+ def __repr__(self):
+ repr_str = self.__class__.__name__ + f'(transforms={self.transforms})'
+ return repr_str
+
+
+@TRANSFORMS.register_module()
+class ConcatCDInput(BaseTransform):
+ """Concat images for change detection.
+
+ Required Keys:
+
+ - img
+ - img2
+
+ Args:
+ input_keys (tuple): Input image keys for change detection.
+ Default: ('img', 'img2').
+ """
+
+ def __init__(self, input_keys=('img', 'img2')):
+ self.input_keys = input_keys
+
+ def transform(self, results: dict) -> dict:
+ img = []
+ for input_key in self.input_keys:
+ img.append(results.pop(input_key))
+ results['img'] = np.concatenate(img, axis=2)
+ return results
+
+ def __repr__(self):
+ repr_str = self.__class__.__name__
+ repr_str += f'(input_keys={self.input_keys}, '
+ return repr_str
diff --git a/mmseg/models/backbones/__init__.py b/mmseg/models/backbones/__init__.py
index e3107306ea..d9228a500b 100644
--- a/mmseg/models/backbones/__init__.py
+++ b/mmseg/models/backbones/__init__.py
@@ -3,6 +3,7 @@
from .bisenetv1 import BiSeNetV1
from .bisenetv2 import BiSeNetV2
from .cgnet import CGNet
+from .ddrnet import DDRNet
from .erfnet import ERFNet
from .fast_scnn import FastSCNN
from .hrnet import HRNet
@@ -28,5 +29,6 @@
'ResNeSt', 'MobileNetV2', 'UNet', 'CGNet', 'MobileNetV3',
'VisionTransformer', 'SwinTransformer', 'MixVisionTransformer',
'BiSeNetV1', 'BiSeNetV2', 'ICNet', 'TIMMBackbone', 'ERFNet', 'PCPVT',
- 'SVT', 'STDCNet', 'STDCContextPathNet', 'BEiT', 'MAE', 'PIDNet', 'MSCAN'
+ 'SVT', 'STDCNet', 'STDCContextPathNet', 'BEiT', 'MAE', 'PIDNet', 'MSCAN',
+ 'DDRNet'
]
diff --git a/mmseg/models/backbones/ddrnet.py b/mmseg/models/backbones/ddrnet.py
new file mode 100644
index 0000000000..4508aade82
--- /dev/null
+++ b/mmseg/models/backbones/ddrnet.py
@@ -0,0 +1,222 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+import torch.nn as nn
+from mmcv.cnn import ConvModule, build_norm_layer
+from mmengine.model import BaseModule
+
+from mmseg.models.utils import DAPPM, BasicBlock, Bottleneck, resize
+from mmseg.registry import MODELS
+from mmseg.utils import OptConfigType
+
+
+@MODELS.register_module()
+class DDRNet(BaseModule):
+ """DDRNet backbone.
+
+ This backbone is the implementation of `Deep Dual-resolution Networks for
+ Real-time and Accurate Semantic Segmentation of Road Scenes
+
`_.
+ Modified from https://github.com/ydhongHIT/DDRNet.
+
+ Args:
+ in_channels (int): Number of input image channels. Default: 3.
+ channels: (int): The base channels of DDRNet. Default: 32.
+ ppm_channels (int): The channels of PPM module. Default: 128.
+ align_corners (bool): align_corners argument of F.interpolate.
+ Default: False.
+ norm_cfg (dict): Config dict to build norm layer.
+ Default: dict(type='BN', requires_grad=True).
+ act_cfg (dict): Config dict for activation layer.
+ Default: dict(type='ReLU', inplace=True).
+ init_cfg (dict, optional): Initialization config dict.
+ Default: None.
+ """
+
+ def __init__(self,
+ in_channels: int = 3,
+ channels: int = 32,
+ ppm_channels: int = 128,
+ align_corners: bool = False,
+ norm_cfg: OptConfigType = dict(type='BN', requires_grad=True),
+ act_cfg: OptConfigType = dict(type='ReLU', inplace=True),
+ init_cfg: OptConfigType = None):
+ super().__init__(init_cfg)
+
+ self.in_channels = in_channels
+ self.ppm_channels = ppm_channels
+
+ self.norm_cfg = norm_cfg
+ self.act_cfg = act_cfg
+ self.align_corners = align_corners
+
+ # stage 0-2
+ self.stem = self._make_stem_layer(in_channels, channels, num_blocks=2)
+ self.relu = nn.ReLU()
+
+ # low resolution(context) branch
+ self.context_branch_layers = nn.ModuleList()
+ for i in range(3):
+ self.context_branch_layers.append(
+ self._make_layer(
+ block=BasicBlock if i < 2 else Bottleneck,
+ inplanes=channels * 2**(i + 1),
+ planes=channels * 8 if i > 0 else channels * 4,
+ num_blocks=2 if i < 2 else 1,
+ stride=2))
+
+ # bilateral fusion
+ self.compression_1 = ConvModule(
+ channels * 4,
+ channels * 2,
+ kernel_size=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None)
+ self.down_1 = ConvModule(
+ channels * 2,
+ channels * 4,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None)
+
+ self.compression_2 = ConvModule(
+ channels * 8,
+ channels * 2,
+ kernel_size=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None)
+ self.down_2 = nn.Sequential(
+ ConvModule(
+ channels * 2,
+ channels * 4,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg),
+ ConvModule(
+ channels * 4,
+ channels * 8,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg=None))
+
+ # high resolution(spatial) branch
+ self.spatial_branch_layers = nn.ModuleList()
+ for i in range(3):
+ self.spatial_branch_layers.append(
+ self._make_layer(
+ block=BasicBlock if i < 2 else Bottleneck,
+ inplanes=channels * 2,
+ planes=channels * 2,
+ num_blocks=2 if i < 2 else 1,
+ ))
+
+ self.spp = DAPPM(
+ channels * 16, ppm_channels, channels * 4, num_scales=5)
+
+ def _make_stem_layer(self, in_channels, channels, num_blocks):
+ layers = [
+ ConvModule(
+ in_channels,
+ channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg),
+ ConvModule(
+ channels,
+ channels,
+ kernel_size=3,
+ stride=2,
+ padding=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg)
+ ]
+
+ layers.extend([
+ self._make_layer(BasicBlock, channels, channels, num_blocks),
+ nn.ReLU(),
+ self._make_layer(
+ BasicBlock, channels, channels * 2, num_blocks, stride=2),
+ nn.ReLU(),
+ ])
+
+ return nn.Sequential(*layers)
+
+ def _make_layer(self, block, inplanes, planes, num_blocks, stride=1):
+ downsample = None
+ if stride != 1 or inplanes != planes * block.expansion:
+ downsample = nn.Sequential(
+ nn.Conv2d(
+ inplanes,
+ planes * block.expansion,
+ kernel_size=1,
+ stride=stride,
+ bias=False),
+ build_norm_layer(self.norm_cfg, planes * block.expansion)[1])
+
+ layers = [
+ block(
+ in_channels=inplanes,
+ channels=planes,
+ stride=stride,
+ downsample=downsample)
+ ]
+ inplanes = planes * block.expansion
+ for i in range(1, num_blocks):
+ layers.append(
+ block(
+ in_channels=inplanes,
+ channels=planes,
+ stride=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg_out=None if i == num_blocks - 1 else self.act_cfg))
+
+ return nn.Sequential(*layers)
+
+ def forward(self, x):
+ """Forward function."""
+ out_size = (x.shape[-2] // 8, x.shape[-1] // 8)
+
+ # stage 0-2
+ x = self.stem(x)
+
+ # stage3
+ x_c = self.context_branch_layers[0](x)
+ x_s = self.spatial_branch_layers[0](x)
+ comp_c = self.compression_1(self.relu(x_c))
+ x_c += self.down_1(self.relu(x_s))
+ x_s += resize(
+ comp_c,
+ size=out_size,
+ mode='bilinear',
+ align_corners=self.align_corners)
+ if self.training:
+ temp_context = x_s.clone()
+
+ # stage4
+ x_c = self.context_branch_layers[1](self.relu(x_c))
+ x_s = self.spatial_branch_layers[1](self.relu(x_s))
+ comp_c = self.compression_2(self.relu(x_c))
+ x_c += self.down_2(self.relu(x_s))
+ x_s += resize(
+ comp_c,
+ size=out_size,
+ mode='bilinear',
+ align_corners=self.align_corners)
+
+ # stage5
+ x_s = self.spatial_branch_layers[2](self.relu(x_s))
+ x_c = self.context_branch_layers[2](self.relu(x_c))
+ x_c = self.spp(x_c)
+ x_c = resize(
+ x_c,
+ size=out_size,
+ mode='bilinear',
+ align_corners=self.align_corners)
+
+ return (temp_context, x_s + x_c) if self.training else x_s + x_c
diff --git a/mmseg/models/backbones/swin.py b/mmseg/models/backbones/swin.py
index c0ace3c139..67b28a96e1 100644
--- a/mmseg/models/backbones/swin.py
+++ b/mmseg/models/backbones/swin.py
@@ -716,20 +716,22 @@ def init_weights(self):
]
for table_key in relative_position_bias_table_keys:
table_pretrained = state_dict[table_key]
- table_current = self.state_dict()[table_key]
- L1, nH1 = table_pretrained.size()
- L2, nH2 = table_current.size()
- if nH1 != nH2:
- print_log(f'Error in loading {table_key}, pass')
- elif L1 != L2:
- S1 = int(L1**0.5)
- S2 = int(L2**0.5)
- table_pretrained_resized = F.interpolate(
- table_pretrained.permute(1, 0).reshape(1, nH1, S1, S1),
- size=(S2, S2),
- mode='bicubic')
- state_dict[table_key] = table_pretrained_resized.view(
- nH2, L2).permute(1, 0).contiguous()
+ if table_key in self.state_dict():
+ table_current = self.state_dict()[table_key]
+ L1, nH1 = table_pretrained.size()
+ L2, nH2 = table_current.size()
+ if nH1 != nH2:
+ print_log(f'Error in loading {table_key}, pass')
+ elif L1 != L2:
+ S1 = int(L1**0.5)
+ S2 = int(L2**0.5)
+ table_pretrained_resized = F.interpolate(
+ table_pretrained.permute(1, 0).reshape(
+ 1, nH1, S1, S1),
+ size=(S2, S2),
+ mode='bicubic')
+ state_dict[table_key] = table_pretrained_resized.view(
+ nH2, L2).permute(1, 0).contiguous()
# load state_dict
self.load_state_dict(state_dict, strict=False)
diff --git a/mmseg/models/decode_heads/__init__.py b/mmseg/models/decode_heads/__init__.py
index 18235456bc..36c37ec2dd 100644
--- a/mmseg/models/decode_heads/__init__.py
+++ b/mmseg/models/decode_heads/__init__.py
@@ -4,6 +4,7 @@
from .aspp_head import ASPPHead
from .cc_head import CCHead
from .da_head import DAHead
+from .ddr_head import DDRHead
from .dm_head import DMHead
from .dnl_head import DNLHead
from .dpt_head import DPTHead
@@ -41,5 +42,5 @@
'SETRMLAHead', 'DPTHead', 'SETRMLAHead', 'SegmenterMaskTransformerHead',
'SegformerHead', 'ISAHead', 'STDCHead', 'IterativeDecodeHead',
'KernelUpdateHead', 'KernelUpdator', 'MaskFormerHead', 'Mask2FormerHead',
- 'LightHamHead', 'PIDHead'
+ 'LightHamHead', 'PIDHead', 'DDRHead'
]
diff --git a/mmseg/models/decode_heads/ddr_head.py b/mmseg/models/decode_heads/ddr_head.py
new file mode 100644
index 0000000000..ba26d6503c
--- /dev/null
+++ b/mmseg/models/decode_heads/ddr_head.py
@@ -0,0 +1,116 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+from typing import Tuple, Union
+
+import torch.nn as nn
+from mmcv.cnn import ConvModule, build_activation_layer, build_norm_layer
+from torch import Tensor
+
+from mmseg.models.decode_heads.decode_head import BaseDecodeHead
+from mmseg.models.losses import accuracy
+from mmseg.models.utils import resize
+from mmseg.registry import MODELS
+from mmseg.utils import OptConfigType, SampleList
+
+
+@MODELS.register_module()
+class DDRHead(BaseDecodeHead):
+ """Decode head for DDRNet.
+
+ Args:
+ in_channels (int): Number of input channels.
+ channels (int): Number of output channels.
+ num_classes (int): Number of classes.
+ norm_cfg (dict, optional): Config dict for normalization layer.
+ Default: dict(type='BN').
+ act_cfg (dict, optional): Config dict for activation layer.
+ Default: dict(type='ReLU', inplace=True).
+ """
+
+ def __init__(self,
+ in_channels: int,
+ channels: int,
+ num_classes: int,
+ norm_cfg: OptConfigType = dict(type='BN'),
+ act_cfg: OptConfigType = dict(type='ReLU', inplace=True),
+ **kwargs):
+ super().__init__(
+ in_channels,
+ channels,
+ num_classes=num_classes,
+ norm_cfg=norm_cfg,
+ act_cfg=act_cfg,
+ **kwargs)
+
+ self.head = self._make_base_head(self.in_channels, self.channels)
+ self.aux_head = self._make_base_head(self.in_channels // 2,
+ self.channels)
+ self.aux_cls_seg = nn.Conv2d(
+ self.channels, self.out_channels, kernel_size=1)
+
+ def init_weights(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ nn.init.kaiming_normal_(
+ m.weight, mode='fan_out', nonlinearity='relu')
+ elif isinstance(m, nn.BatchNorm2d):
+ nn.init.constant_(m.weight, 1)
+ nn.init.constant_(m.bias, 0)
+
+ def forward(
+ self,
+ inputs: Union[Tensor,
+ Tuple[Tensor]]) -> Union[Tensor, Tuple[Tensor]]:
+ if self.training:
+ c3_feat, c5_feat = inputs
+ x_c = self.head(c5_feat)
+ x_c = self.cls_seg(x_c)
+ x_s = self.aux_head(c3_feat)
+ x_s = self.aux_cls_seg(x_s)
+
+ return x_c, x_s
+ else:
+ x_c = self.head(inputs)
+ x_c = self.cls_seg(x_c)
+ return x_c
+
+ def _make_base_head(self, in_channels: int,
+ channels: int) -> nn.Sequential:
+ layers = [
+ ConvModule(
+ in_channels,
+ channels,
+ kernel_size=3,
+ padding=1,
+ norm_cfg=self.norm_cfg,
+ act_cfg=self.act_cfg,
+ order=('norm', 'act', 'conv')),
+ build_norm_layer(self.norm_cfg, channels)[1],
+ build_activation_layer(self.act_cfg),
+ ]
+
+ return nn.Sequential(*layers)
+
+ def loss_by_feat(self, seg_logits: Tuple[Tensor],
+ batch_data_samples: SampleList) -> dict:
+ loss = dict()
+ context_logit, spatial_logit = seg_logits
+ seg_label = self._stack_batch_gt(batch_data_samples)
+
+ context_logit = resize(
+ context_logit,
+ size=seg_label.shape[2:],
+ mode='bilinear',
+ align_corners=self.align_corners)
+ spatial_logit = resize(
+ spatial_logit,
+ size=seg_label.shape[2:],
+ mode='bilinear',
+ align_corners=self.align_corners)
+ seg_label = seg_label.squeeze(1)
+
+ loss['loss_context'] = self.loss_decode[0](context_logit, seg_label)
+ loss['loss_spatial'] = self.loss_decode[1](spatial_logit, seg_label)
+ loss['acc_seg'] = accuracy(
+ context_logit, seg_label, ignore_index=self.ignore_index)
+
+ return loss
diff --git a/mmseg/models/decode_heads/decode_head.py b/mmseg/models/decode_heads/decode_head.py
index 8bdbb24a1c..4faf54559d 100644
--- a/mmseg/models/decode_heads/decode_head.py
+++ b/mmseg/models/decode_heads/decode_head.py
@@ -45,7 +45,7 @@ class BaseDecodeHead(BaseModule, metaclass=ABCMeta):
in_channels (int|Sequence[int]): Input channels.
channels (int): Channels after modules, before conv_seg.
num_classes (int): Number of classes.
- out_channels (int): Output channels of conv_seg.
+ out_channels (int): Output channels of conv_seg. Default: None.
threshold (float): Threshold for binary segmentation in the case of
`num_classes==1`. Default: None.
dropout_ratio (float): Ratio of dropout layer. Default: 0.1.
diff --git a/mmseg/models/losses/__init__.py b/mmseg/models/losses/__init__.py
index 2f7e39cb28..9af5e40f23 100644
--- a/mmseg/models/losses/__init__.py
+++ b/mmseg/models/losses/__init__.py
@@ -5,6 +5,7 @@
cross_entropy, mask_cross_entropy)
from .dice_loss import DiceLoss
from .focal_loss import FocalLoss
+from .huasdorff_distance_loss import HuasdorffDisstanceLoss
from .lovasz_loss import LovaszLoss
from .ohem_cross_entropy_loss import OhemCrossEntropy
from .tversky_loss import TverskyLoss
@@ -14,5 +15,6 @@
'accuracy', 'Accuracy', 'cross_entropy', 'binary_cross_entropy',
'mask_cross_entropy', 'CrossEntropyLoss', 'reduce_loss',
'weight_reduce_loss', 'weighted_loss', 'LovaszLoss', 'DiceLoss',
- 'FocalLoss', 'TverskyLoss', 'OhemCrossEntropy', 'BoundaryLoss'
+ 'FocalLoss', 'TverskyLoss', 'OhemCrossEntropy', 'BoundaryLoss',
+ 'HuasdorffDisstanceLoss'
]
diff --git a/mmseg/models/losses/cross_entropy_loss.py b/mmseg/models/losses/cross_entropy_loss.py
index 770b997486..caad344109 100644
--- a/mmseg/models/losses/cross_entropy_loss.py
+++ b/mmseg/models/losses/cross_entropy_loss.py
@@ -124,7 +124,7 @@ def binary_cross_entropy(pred,
assert label[label != ignore_index].max() <= 1, \
'For pred with shape [N, 1, H, W], its label must have at ' \
'most 2 classes'
- pred = pred.squeeze()
+ pred = pred.squeeze(1)
if pred.dim() != label.dim():
assert (pred.dim() == 2 and label.dim() == 1) or (
pred.dim() == 4 and label.dim() == 3), \
diff --git a/mmseg/models/losses/dice_loss.py b/mmseg/models/losses/dice_loss.py
index 2ee89a81f4..2b7c9d4cf1 100644
--- a/mmseg/models/losses/dice_loss.py
+++ b/mmseg/models/losses/dice_loss.py
@@ -1,125 +1,160 @@
# Copyright (c) OpenMMLab. All rights reserved.
-"""Modified from https://github.com/LikeLy-Journey/SegmenTron/blob/master/
-segmentron/solver/loss.py (Apache-2.0 License)"""
import torch
import torch.nn as nn
-import torch.nn.functional as F
from mmseg.registry import MODELS
-from .utils import get_class_weight, weighted_loss
-
-
-@weighted_loss
-def dice_loss(pred,
- target,
- valid_mask,
- smooth=1,
- exponent=2,
- class_weight=None,
- ignore_index=255):
- assert pred.shape[0] == target.shape[0]
- total_loss = 0
- num_classes = pred.shape[1]
- for i in range(num_classes):
- if i != ignore_index:
- dice_loss = binary_dice_loss(
- pred[:, i],
- target[..., i],
- valid_mask=valid_mask,
- smooth=smooth,
- exponent=exponent)
- if class_weight is not None:
- dice_loss *= class_weight[i]
- total_loss += dice_loss
- return total_loss / num_classes
-
-
-@weighted_loss
-def binary_dice_loss(pred, target, valid_mask, smooth=1, exponent=2, **kwards):
- assert pred.shape[0] == target.shape[0]
- pred = pred.reshape(pred.shape[0], -1)
- target = target.reshape(target.shape[0], -1)
- valid_mask = valid_mask.reshape(valid_mask.shape[0], -1)
-
- num = torch.sum(torch.mul(pred, target) * valid_mask, dim=1) * 2 + smooth
- den = torch.sum(pred.pow(exponent) + target.pow(exponent), dim=1) + smooth
-
- return 1 - num / den
+from .utils import weight_reduce_loss
+
+
+def dice_loss(
+ pred,
+ target,
+ weight,
+ eps=1e-3,
+ reduction='mean',
+ naive_dice=False,
+ avg_factor=None,
+):
+ """Calculate dice loss, there are two forms of dice loss is supported:
+
+ - the one proposed in `V-Net: Fully Convolutional Neural
+ Networks for Volumetric Medical Image Segmentation
+ `_.
+ - the dice loss in which the power of the number in the
+ denominator is the first power instead of the second
+ power.
+ Args:
+ pred (torch.Tensor): The prediction, has a shape (n, *)
+ target (torch.Tensor): The learning label of the prediction,
+ shape (n, *), same shape of pred.
+ weight (torch.Tensor, optional): The weight of loss for each
+ prediction, has a shape (n,). Defaults to None.
+ eps (float): Avoid dividing by zero. Default: 1e-3.
+ reduction (str, optional): The method used to reduce the loss into
+ a scalar. Defaults to 'mean'.
+ Options are "none", "mean" and "sum".
+ naive_dice (bool, optional): If false, use the dice
+ loss defined in the V-Net paper, otherwise, use the
+ naive dice loss in which the power of the number in the
+ denominator is the first power instead of the second
+ power.Defaults to False.
+ avg_factor (int, optional): Average factor that is used to average
+ the loss. Defaults to None.
+ """
-@MODELS.register_module()
-class DiceLoss(nn.Module):
- """DiceLoss.
+ input = pred.flatten(1)
+ target = target.flatten(1).float()
- This loss is proposed in `V-Net: Fully Convolutional Neural Networks for
- Volumetric Medical Image Segmentation `_.
+ a = torch.sum(input * target, 1)
+ if naive_dice:
+ b = torch.sum(input, 1)
+ c = torch.sum(target, 1)
+ d = (2 * a + eps) / (b + c + eps)
+ else:
+ b = torch.sum(input * input, 1) + eps
+ c = torch.sum(target * target, 1) + eps
+ d = (2 * a) / (b + c)
- Args:
- smooth (float): A float number to smooth loss, and avoid NaN error.
- Default: 1
- exponent (float): An float number to calculate denominator
- value: \\sum{x^exponent} + \\sum{y^exponent}. Default: 2.
- reduction (str, optional): The method used to reduce the loss. Options
- are "none", "mean" and "sum". This parameter only works when
- per_image is True. Default: 'mean'.
- class_weight (list[float] | str, optional): Weight of each class. If in
- str format, read them from a file. Defaults to None.
- loss_weight (float, optional): Weight of the loss. Default to 1.0.
- ignore_index (int | None): The label index to be ignored. Default: 255.
- loss_name (str, optional): Name of the loss item. If you want this loss
- item to be included into the backward graph, `loss_` must be the
- prefix of the name. Defaults to 'loss_dice'.
- """
+ loss = 1 - d
+ if weight is not None:
+ assert weight.ndim == loss.ndim
+ assert len(weight) == len(pred)
+ loss = weight_reduce_loss(loss, weight, reduction, avg_factor)
+ return loss
+
+
+@MODELS.register_module()
+class DiceLoss(nn.Module):
def __init__(self,
- smooth=1,
- exponent=2,
+ use_sigmoid=True,
+ activate=True,
reduction='mean',
- class_weight=None,
+ naive_dice=False,
loss_weight=1.0,
ignore_index=255,
- loss_name='loss_dice',
- **kwards):
+ eps=1e-3,
+ loss_name='loss_dice'):
+ """Compute dice loss.
+
+ Args:
+ use_sigmoid (bool, optional): Whether to the prediction is
+ used for sigmoid or softmax. Defaults to True.
+ activate (bool): Whether to activate the predictions inside,
+ this will disable the inside sigmoid operation.
+ Defaults to True.
+ reduction (str, optional): The method used
+ to reduce the loss. Options are "none",
+ "mean" and "sum". Defaults to 'mean'.
+ naive_dice (bool, optional): If false, use the dice
+ loss defined in the V-Net paper, otherwise, use the
+ naive dice loss in which the power of the number in the
+ denominator is the first power instead of the second
+ power. Defaults to False.
+ loss_weight (float, optional): Weight of loss. Defaults to 1.0.
+ ignore_index (int | None): The label index to be ignored.
+ Default: 255.
+ eps (float): Avoid dividing by zero. Defaults to 1e-3.
+ loss_name (str, optional): Name of the loss item. If you want this
+ loss item to be included into the backward graph, `loss_` must
+ be the prefix of the name. Defaults to 'loss_dice'.
+ """
+
super().__init__()
- self.smooth = smooth
- self.exponent = exponent
+ self.use_sigmoid = use_sigmoid
self.reduction = reduction
- self.class_weight = get_class_weight(class_weight)
+ self.naive_dice = naive_dice
self.loss_weight = loss_weight
+ self.eps = eps
+ self.activate = activate
self.ignore_index = ignore_index
self._loss_name = loss_name
def forward(self,
pred,
target,
+ weight=None,
avg_factor=None,
- reduction_override=None,
- **kwards):
+ reduction_override=None):
+ """Forward function.
+
+ Args:
+ pred (torch.Tensor): The prediction, has a shape (n, *).
+ target (torch.Tensor): The label of the prediction,
+ shape (n, *), same shape of pred.
+ weight (torch.Tensor, optional): The weight of loss for each
+ prediction, has a shape (n,). Defaults to None.
+ avg_factor (int, optional): Average factor that is used to average
+ the loss. Defaults to None.
+ reduction_override (str, optional): The reduction method used to
+ override the original reduction method of the loss.
+ Options are "none", "mean" and "sum".
+
+ Returns:
+ torch.Tensor: The calculated loss
+ """
+
assert reduction_override in (None, 'none', 'mean', 'sum')
reduction = (
reduction_override if reduction_override else self.reduction)
- if self.class_weight is not None:
- class_weight = pred.new_tensor(self.class_weight)
- else:
- class_weight = None
-
- pred = F.softmax(pred, dim=1)
- num_classes = pred.shape[1]
- one_hot_target = F.one_hot(
- torch.clamp(target.long(), 0, num_classes - 1),
- num_classes=num_classes)
- valid_mask = (target != self.ignore_index).long()
+
+ if self.activate:
+ if self.use_sigmoid:
+ pred = pred.sigmoid()
+ else:
+ raise NotImplementedError
loss = self.loss_weight * dice_loss(
pred,
- one_hot_target,
- valid_mask=valid_mask,
+ target,
+ weight,
+ eps=self.eps,
reduction=reduction,
+ naive_dice=self.naive_dice,
avg_factor=avg_factor,
- smooth=self.smooth,
- exponent=self.exponent,
- class_weight=class_weight,
- ignore_index=self.ignore_index)
+ )
+
return loss
@property
diff --git a/mmseg/models/losses/focal_loss.py b/mmseg/models/losses/focal_loss.py
index 104d6602c8..6507ed7a91 100644
--- a/mmseg/models/losses/focal_loss.py
+++ b/mmseg/models/losses/focal_loss.py
@@ -271,7 +271,13 @@ def forward(self,
num_classes = pred.size(1)
if torch.cuda.is_available() and pred.is_cuda:
if target.dim() == 1:
- one_hot_target = F.one_hot(target, num_classes=num_classes)
+ one_hot_target = F.one_hot(
+ target, num_classes=num_classes + 1)
+ if num_classes == 1:
+ one_hot_target = one_hot_target[:, 1]
+ target = 1 - target
+ else:
+ one_hot_target = one_hot_target[:, :num_classes]
else:
one_hot_target = target
target = target.argmax(dim=1)
@@ -280,7 +286,11 @@ def forward(self,
else:
one_hot_target = None
if target.dim() == 1:
- target = F.one_hot(target, num_classes=num_classes)
+ target = F.one_hot(target, num_classes=num_classes + 1)
+ if num_classes == 1:
+ target = target[:, 1]
+ else:
+ target = target[:, num_classes]
else:
valid_mask = (target.argmax(dim=1) != ignore_index).view(
-1, 1)
diff --git a/mmseg/models/losses/huasdorff_distance_loss.py b/mmseg/models/losses/huasdorff_distance_loss.py
new file mode 100644
index 0000000000..d950ba728f
--- /dev/null
+++ b/mmseg/models/losses/huasdorff_distance_loss.py
@@ -0,0 +1,160 @@
+# Copyright (c) OpenMMLab. All rights reserved.
+"""Modified from https://github.com/JunMa11/SegWithDistMap/blob/
+master/code/train_LA_HD.py (Apache-2.0 License)"""
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from scipy.ndimage import distance_transform_edt as distance
+from torch import Tensor
+
+from mmseg.registry import MODELS
+from .utils import get_class_weight, weighted_loss
+
+
+def compute_dtm(img_gt: Tensor, pred: Tensor) -> Tensor:
+ """
+ compute the distance transform map of foreground in mask
+ Args:
+ img_gt: Ground truth of the image, (b, h, w)
+ pred: Predictions of the segmentation head after softmax, (b, c, h, w)
+
+ Returns:
+ output: the foreground Distance Map (SDM)
+ dtm(x) = 0; x in segmentation boundary
+ inf|x-y|; x in segmentation
+ """
+
+ fg_dtm = torch.zeros_like(pred)
+ out_shape = pred.shape
+ for b in range(out_shape[0]): # batch size
+ for c in range(1, out_shape[1]): # default 0 channel is background
+ posmask = img_gt[b].byte()
+ if posmask.any():
+ posdis = distance(posmask)
+ fg_dtm[b][c] = torch.from_numpy(posdis)
+
+ return fg_dtm
+
+
+@weighted_loss
+def hd_loss(seg_soft: Tensor,
+ gt: Tensor,
+ seg_dtm: Tensor,
+ gt_dtm: Tensor,
+ class_weight=None,
+ ignore_index=255) -> Tensor:
+ """
+ compute huasdorff distance loss for segmentation
+ Args:
+ seg_soft: softmax results, shape=(b,c,x,y)
+ gt: ground truth, shape=(b,x,y)
+ seg_dtm: segmentation distance transform map, shape=(b,c,x,y)
+ gt_dtm: ground truth distance transform map, shape=(b,c,x,y)
+
+ Returns:
+ output: hd_loss
+ """
+ assert seg_soft.shape[0] == gt.shape[0]
+ total_loss = 0
+ num_class = seg_soft.shape[1]
+ if class_weight is not None:
+ assert class_weight.ndim == num_class
+ for i in range(1, num_class):
+ if i != ignore_index:
+ delta_s = (seg_soft[:, i, ...] - gt.float())**2
+ s_dtm = seg_dtm[:, i, ...]**2
+ g_dtm = gt_dtm[:, i, ...]**2
+ dtm = s_dtm + g_dtm
+ multiplied = torch.einsum('bxy, bxy->bxy', delta_s, dtm)
+ hd_loss = multiplied.mean()
+ if class_weight is not None:
+ hd_loss *= class_weight[i]
+ total_loss += hd_loss
+
+ return total_loss / num_class
+
+
+@MODELS.register_module()
+class HuasdorffDisstanceLoss(nn.Module):
+ """HuasdorffDisstanceLoss. This loss is proposed in `How Distance Transform
+ Maps Boost Segmentation CNNs: An Empirical Study.
+
+ `_.
+ Args:
+ reduction (str, optional): The method used to reduce the loss into
+ a scalar. Defaults to 'mean'.
+ class_weight (list[float] | str, optional): Weight of each class. If in
+ str format, read them from a file. Defaults to None.
+ loss_weight (float): Weight of the loss. Defaults to 1.0.
+ ignore_index (int | None): The label index to be ignored. Default: 255.
+ loss_name (str): Name of the loss item. If you want this loss
+ item to be included into the backward graph, `loss_` must be the
+ prefix of the name. Defaults to 'loss_boundary'.
+ """
+
+ def __init__(self,
+ reduction='mean',
+ class_weight=None,
+ loss_weight=1.0,
+ ignore_index=255,
+ loss_name='loss_huasdorff_disstance',
+ **kwargs):
+ super().__init__()
+ self.reduction = reduction
+ self.loss_weight = loss_weight
+ self.class_weight = get_class_weight(class_weight)
+ self._loss_name = loss_name
+ self.ignore_index = ignore_index
+
+ def forward(self,
+ pred: Tensor,
+ target: Tensor,
+ avg_factor=None,
+ reduction_override=None,
+ **kwargs) -> Tensor:
+ """Forward function.
+
+ Args:
+ pred (Tensor): Predictions of the segmentation head. (B, C, H, W)
+ target (Tensor): Ground truth of the image. (B, H, W)
+ avg_factor (int, optional): Average factor that is used to
+ average the loss. Defaults to None.
+ reduction_override (str, optional): The reduction method used
+ to override the original reduction method of the loss.
+ Options are "none", "mean" and "sum".
+ Returns:
+ Tensor: Loss tensor.
+ """
+ assert reduction_override in (None, 'none', 'mean', 'sum')
+ reduction = (
+ reduction_override if reduction_override else self.reduction)
+ if self.class_weight is not None:
+ class_weight = pred.new_tensor(self.class_weight)
+ else:
+ class_weight = None
+
+ pred_soft = F.softmax(pred, dim=1)
+ valid_mask = (target != self.ignore_index).long()
+ target = target * valid_mask
+
+ with torch.no_grad():
+ gt_dtm = compute_dtm(target.cpu(), pred_soft)
+ gt_dtm = gt_dtm.float()
+ seg_dtm2 = compute_dtm(
+ pred_soft.argmax(dim=1, keepdim=False).cpu(), pred_soft)
+ seg_dtm2 = seg_dtm2.float()
+
+ loss_hd = self.loss_weight * hd_loss(
+ pred_soft,
+ target,
+ seg_dtm=seg_dtm2,
+ gt_dtm=gt_dtm,
+ reduction=reduction,
+ avg_factor=avg_factor,
+ class_weight=class_weight,
+ ignore_index=self.ignore_index)
+ return loss_hd
+
+ @property
+ def loss_name(self):
+ return self._loss_name
diff --git a/mmseg/models/losses/utils.py b/mmseg/models/losses/utils.py
index f74efcf35c..0478034733 100644
--- a/mmseg/models/losses/utils.py
+++ b/mmseg/models/losses/utils.py
@@ -25,7 +25,7 @@ def get_class_weight(class_weight):
return class_weight
-def reduce_loss(loss, reduction):
+def reduce_loss(loss, reduction) -> torch.Tensor:
"""Reduce loss as specified.
Args:
@@ -45,7 +45,10 @@ def reduce_loss(loss, reduction):
return loss.sum()
-def weight_reduce_loss(loss, weight=None, reduction='mean', avg_factor=None):
+def weight_reduce_loss(loss,
+ weight=None,
+ reduction='mean',
+ avg_factor=None) -> torch.Tensor:
"""Apply element-wise weight and reduce loss.
Args:
diff --git a/mmseg/models/segmentors/encoder_decoder.py b/mmseg/models/segmentors/encoder_decoder.py
index 370d0305fd..b68dab1e9c 100644
--- a/mmseg/models/segmentors/encoder_decoder.py
+++ b/mmseg/models/segmentors/encoder_decoder.py
@@ -33,7 +33,7 @@ class EncoderDecoder(BaseSegmentor):
2. The ``predict`` method is used to predict segmentation results,
which includes two steps: (1) Run inference function to obtain the list of
seg_logits (2) Call post-processing function to obtain list of
- ``SegDataSampel`` including ``pred_sem_seg`` and ``seg_logits``.
+ ``SegDataSample`` including ``pred_sem_seg`` and ``seg_logits``.
.. code:: text
@@ -326,8 +326,9 @@ def inference(self, inputs: Tensor, batch_img_metas: List[dict]) -> Tensor:
Tensor: The segmentation results, seg_logits from model of each
input image.
"""
-
- assert self.test_cfg.mode in ['slide', 'whole']
+ assert self.test_cfg.get('mode', 'whole') in ['slide', 'whole'], \
+ f'Only "slide" or "whole" test mode are supported, but got ' \
+ f'{self.test_cfg["mode"]}.'
ori_shape = batch_img_metas[0]['ori_shape']
assert all(_['ori_shape'] == ori_shape for _ in batch_img_metas)
if self.test_cfg.mode == 'slide':
diff --git a/mmseg/models/segmentors/seg_tta.py b/mmseg/models/segmentors/seg_tta.py
index eacb6c00a9..5b8eb1d7b0 100644
--- a/mmseg/models/segmentors/seg_tta.py
+++ b/mmseg/models/segmentors/seg_tta.py
@@ -44,5 +44,6 @@ def merge_preds(self, data_samples_list: List[SampleList]) -> SampleList:
'pred_sem_seg': PixelData(data=seg_pred),
'gt_sem_seg': data_samples[0].gt_sem_seg
})
+ data_sample.set_metainfo({'img_path': data_samples[0].img_path})
predictions.append(data_sample)
return predictions
diff --git a/mmseg/version.py b/mmseg/version.py
index 748e4cb495..3a632a31e8 100644
--- a/mmseg/version.py
+++ b/mmseg/version.py
@@ -1,6 +1,6 @@
# Copyright (c) Open-MMLab. All rights reserved.
-__version__ = '1.0.0'
+__version__ = '1.1.0'
def parse_version_info(version_str):
diff --git a/mmseg/visualization/local_visualizer.py b/mmseg/visualization/local_visualizer.py
index 504004dfcb..0d693e5820 100644
--- a/mmseg/visualization/local_visualizer.py
+++ b/mmseg/visualization/local_visualizer.py
@@ -108,14 +108,14 @@ def _draw_sem_seg(self, image: np.ndarray, sem_seg: PixelData,
colors = [palette[label] for label in labels]
- self.set_image(image)
-
- # draw semantic masks
+ mask = np.zeros_like(image, dtype=np.uint8)
for label, color in zip(labels, colors):
- self.draw_binary_masks(
- sem_seg == label, colors=[color], alphas=self.alpha)
+ mask[sem_seg[0] == label, :] = color
- return self.get_image()
+ color_seg = (image * (1 - self.alpha) + mask * self.alpha).astype(
+ np.uint8)
+ self.set_image(color_seg)
+ return color_seg
def set_dataset_meta(self,
classes: Optional[List] = None,
@@ -226,6 +226,6 @@ def add_datasample(
self.show(drawn_img, win_name=name, wait_time=wait_time)
if out_file is not None:
- mmcv.imwrite(mmcv.bgr2rgb(drawn_img), out_file)
+ mmcv.imwrite(mmcv.rgb2bgr(drawn_img), out_file)
else:
self.add_image(name, drawn_img, step)
diff --git a/model-index.yml b/model-index.yml
index 5e87c386dd..3fe6a2e03d 100644
--- a/model-index.yml
+++ b/model-index.yml
@@ -8,11 +8,13 @@ Import:
- configs/cgnet/metafile.yaml
- configs/convnext/metafile.yaml
- configs/danet/metafile.yaml
+- configs/ddrnet/metafile.yaml
- configs/deeplabv3/metafile.yaml
- configs/deeplabv3plus/metafile.yaml
- configs/dmnet/metafile.yaml
- configs/dnlnet/metafile.yaml
- configs/dpt/metafile.yaml
+- configs/dsdl/metafile.yaml
- configs/emanet/metafile.yaml
- configs/encnet/metafile.yaml
- configs/erfnet/metafile.yaml
diff --git a/projects/XDecoder/README.md b/projects/XDecoder/README.md
new file mode 100644
index 0000000000..3d55575c6b
--- /dev/null
+++ b/projects/XDecoder/README.md
@@ -0,0 +1,17 @@
+# X-Decoder
+
+> [X-Decoder: Generalized Decoding for Pixel, Image, and Language](https://arxiv.org/pdf/2212.11270.pdf)
+
+
+
+## Abstract
+
+We present X-Decoder, a generalized decoding model that can predict pixel-level segmentation and language tokens seamlessly. X-Decodert takes as input two types of queries: (i) generic non-semantic queries and (ii) semantic queries induced from text inputs, to decode different pixel-level and token-level outputs in the same semantic space. With such a novel design, X-Decoder is the first work that provides a unified way to support all types of image segmentation and a variety of vision-language (VL) tasks. Further, our design enables seamless interactions across tasks at different granularities and brings mutual benefits by learning a common and rich pixel-level visual-semantic understanding space, without any pseudo-labeling. After pretraining on a mixed set of a limited amount of segmentation data and millions of image-text pairs, X-Decoder exhibits strong transferability to a wide range of downstream tasks in both zero-shot and finetuning settings. Notably, it achieves (1) state-of-the-art results on open-vocabulary segmentation and referring segmentation on eight datasets; (2) better or competitive finetuned performance to other generalist and specialist models on segmentation and VL tasks; and (3) flexibility for efficient finetuning and novel task composition (e.g., referring captioning and image editing).
+
+
+
+
+