C #3620
Open
C #3620
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Bump 1.0.0rc2 * typo
Merge MMSegmentation 1.x development branch dev-1.x to main branch 1.x for v1.0.0rc2
* [Doc]Translate the 1_config.md and modify a wrong statement in 1_config.md * Translate the 1_config.md and modify a wrong statement in 1_config.md * Modify some expressions * Apply suggestions from code review
…orial (2355) doc modify part of content changed parts of content modified Update docs/zh_cn/user_guides/4_train_test.md Co-authored-by: 谢昕辰 <xiexinch@outlook.com>
…en-mmlab#2397) * CodeCamp open-mmlab#1562 [Doc] update `overview.md` * Update overview.md * Update docs/zh_cn/overview.md Co-authored-by: 谢昕辰 <xiexinch@outlook.com> * Update docs/zh_cn/overview.md Co-authored-by: 谢昕辰 <xiexinch@outlook.com>
* Add torch1.13 in CI * use mim install mm packages * install all requirements * install wheel * add ref
* [Doc] Add ZN datasets.md in dev-1.x * fix typo * fix * fix
…ted.md (open-mmlab#2417) * DOC Update docs/zh_cn/get_started.md Co-authored-by: 谢昕辰 <xiexinch@outlook.com>
…n-mmlab#2412) add example project add ci ignore add version limits
* draft * update loss * update * add runner * add steps * update
…en-mmlab#2341) * [Doc] Change some content from customize_runtime to engine ZN doc * fix comments * add customize runtime setting zn doc * move optimizer content into one section * fix * fix * fix * fix
* tta init * use mmcv transform * test city * add multiscale * fix merge * add softmax to post process * add ut * add tta pipeline to other datasets * remove softmax * add encoder_decoder_tta ut * add encoder_decoder_tta ut * rename * rename file * rename config * rm aug_test * move flip to post process * fix channel
…ET_Mask2FormerHead` (open-mmlab#2448) ## Motivation Calling `mmseg.utils.register_all_modules` will import `MaskFormerHead` and `Mask2FormerHead`, it will crash if mmdet is not installed as `None` cannot be initialized. ## Modification - Modify `MMDET_MaskFormerHead=BaseModule` and `MMDET_Mask2FormerHead = BaseModule` when cannot import from mmdet
## Motivation
The conflict is caused by:
The user requested torch==1.12.1+cpu
torchvision 0.13.0+cpu depends on torch==1.12.0
## Modification
modify the torch==1.12.0+cpu
## Motivation To release 1.0.0rc3 ## Modification 1. Modify mmseg version 2. Add change log 3. Modify README 4. Modify faq 5. Revise docker file
## Motivation implement nnUNet-style Gaussian Noise and Blur
Add the random gamma correction transform for biomedical images, which follows the design of the nnUNet.
## Motivation Add the 3d pad transform for biomedical images, which follows the design of the nnUNet.
## Motivation Support ISNet. paper link: [ISNet: Integrate Image-Level and Semantic-Level Context for Semantic Segmentation](https://openaccess.thecvf.com/content/ICCV2021/papers/Jin_ISNet_Integrate_Image-Level_and_Semantic-Level_Context_for_Semantic_Segmentation_ICCV_2021_paper.pdf) ## Modification Add ISNet decoder head. Add ISNet config.
…augmentation in dev-1.x. (open-mmlab#2432) ## Motivation Add Synapse dataset in MMSegmentation. Old PR: open-mmlab#2372.
## Motivation As title. ## Modification - projects/isnet/README.md
## Motivation as title ## Modification 1. .circleci/test.yml 2. .github/workflows/merge_stage_test.yml
…mmlab#2469) ## Motivation Fix incorrect `img_shape` value assignment. ## Modification - mmseg/datasets/transforms/transforms.py
## Motivation - Add Chinese version of Synapse dataset preparation. - Modify all `，` and `。` to `,` and `.` in `docs/zh_cn/user_guides/2_dataset_prepare.md`.
## Motivation The motivation of this PR is to add `gt_edge_map` field to support boundary loss. ## Modification - GenerateEdge Modify `gt_edge` field to `gt_edge_map`. - PackSegInputs Add `gt_edge_map` to data_sample. - stack_batch Pad `gt_edge_map` to max_shape. ## BC-breaking (Optional) No ## Use cases (Optional) Reference `GenerateEdge`.
…-mmlab#2444) ## Support `HieraSeg` interface on `cityscapes` ## Motivation Support `HieraSeg` interface on cityscapes dataset Paper link : https://ieeexplore.ieee.org/document/9878466/ ``` @Article{li2022deep, title={Deep Hierarchical Semantic Segmentation}, author={Li, Liulei and Zhou, Tianfei and Wang, Wenguan and Li, Jianwu and Yang, Yi}, journal={CVPR}, year={2022} } ``` ## Modification Add `HieraSeg_Projects` on `projects/` Add `sep_aspp_contrast_head` decoder head. Add `HieraSeg` config. Add `hiera_loss`, `hiera_triplet_loss_cityscape`, `tree_triplet_loss`
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation Support inference and visualization of VPD ## Modification 1. add a new VPD model that does not generate black border in predictions 2. update `SegLocalVisualizer` to support depth visualization 3. update `MMSegInferencer` to support save predictions of depth estimation in method `postprocess` ## BC-breaking (Optional) Does the modification introduce changes that break the backward-compatibility of the downstream repos? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR. ## Use cases (Optional) Run inference with VPD using the this command ```sh python demo/image_demo_with_inferencer.py demo/classroom__rgb_00283.jpg vpd_depth --out-dir vis_results ``` The following image will be saved under `vis_results/vis`  ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 4. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 5. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D. 6. The documentation has been modified accordingly, like docstring or example tutorials.
open-mmlab#3324) Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation Use the pytorch-grad-cam tool to visualize Class Activation Maps (CAM). ## Modification Use the pytorch-grad-cam tool to visualize Class Activation Maps (CAM). requirement: pip install grad-cam run commad: python tools/analysis_tools/visualization_cam.py ## BC-breaking (Optional) Does the modification introduce changes that break the backward-compatibility of the downstream repos? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR. ## Use cases (Optional) If this PR introduces a new feature, it is better to list some use cases here, and update the documentation. ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 3. The documentation has been modified accordingly, like docstring or example tutorials.
## Motivation Support SAN for Open-Vocabulary Semantic Segmentation Paper: [Side Adapter Network for Open-Vocabulary Semantic Segmentation](https://arxiv.org/abs/2302.12242) official Code: [SAN](https://github.com/MendelXu/SAN) ## Modification - Added the parameters of backbone vit for implementing the image encoder of CLIP. - Added text encoder code. - Added segmentor multimodel encoder-decoder code for open-vocabulary semantic segmentation. - Added SideAdapterNetwork decode head code. - Added config files for train and inference. - Added tools for converting pretrained models. - Added loss implementation for mask classification model, such as SAN, Maskformer and remove dependency on mmdetection. - Added test units for text encoder, multimodel encoder-decoder, san decode head and hungarian_assigner. ## Use cases ### Convert Models **pretrained SAN model** The official pretrained model can be downloaded from [san_clip_vit_b_16.pth](https://huggingface.co/Mendel192/san/blob/main/san_vit_b_16.pth) and [san_clip_vit_large_14.pth](https://huggingface.co/Mendel192/san/blob/main/san_vit_large_14.pth). Use tools/model_converters/san2mmseg.py to convert offcial model into mmseg style. `python tools/model_converters/san2mmseg.py <MODEL_PATH> <OUTPUT_PATH>` **pretrained CLIP model** Use the CLIP model provided by openai to train SAN. The CLIP model can be download from [ViT-B-16.pt](https://openaipublic.azureedge.net/clip/models/5806e77cd80f8b59890b7e101eabd078d9fb84e6937f9e85e4ecb61988df416f/ViT-B-16.pt) and [ViT-L-14-336px.pt](https://openaipublic.azureedge.net/clip/models/3035c92b350959924f9f00213499208652fc7ea050643e8b385c2dac08641f02/ViT-L-14-336px.pt). Use tools/model_converters/clip2mmseg.py to convert model into mmseg style. `python tools/model_converters/clip2mmseg.py <MODEL_PATH> <OUTPUT_PATH>` ### Inference test san_vit-base-16 model on coco-stuff164k dataset `python tools/test.py ./configs/san/san-vit-b16_coco-stuff164k-640x640.py <TRAINED_MODEL_PATH>` ### Train test san_vit-base-16 model on coco-stuff164k dataset `python tools/train.py ./configs/san/san-vit-b16_coco-stuff164k-640x640.py --cfg-options model.pretrained=<PRETRAINED_MODEL_PATH>` ## Comparision Results ### Train on COCO-Stuff164k | | | mIoU | mAcc | pAcc | | --------------- | ----- | ----- | ----- | ----- | | san-vit-base16 | official | 41.93 | 56.73 | 67.69 | | | mmseg | 41.93 | 56.84 | 67.84 | | san-vit-large14 | official | 45.57 | 59.52 | 69.76 | | | mmseg | 45.78 | 59.61 | 69.21 | ### Evaluate on Pascal Context | | | mIoU | mAcc | pAcc | | --------------- | ----- | ----- | ----- | ----- | | san-vit-base16 | official | 54.05 | 72.96 | 77.77 | | | mmseg | 54.04 | 73.74 | 77.71 | | san-vit-large14 | official | 57.53 | 77.56 | 78.89 | | | mmseg | 56.89 | 76.96 | 78.74 | ### Evaluate on Voc12Aug | | | mIoU | mAcc | pAcc | | --------------- | ----- | ----- | ----- | ----- | | san-vit-base16 | official | 93.86 | 96.61 | 97.11 | | | mmseg | 94.58 | 97.01 | 97.38 | | san-vit-large14 | official | 95.17 | 97.61 | 97.63 | | | mmseg | 95.58 | 97.75 | 97.79 | --------- Co-authored-by: CastleDream <35064479+CastleDream@users.noreply.github.com> Co-authored-by: yeedrag <46050186+yeedrag@users.noreply.github.com> Co-authored-by: Yang-ChangHui <71805205+Yang-Changhui@users.noreply.github.com> Co-authored-by: Xu CAO <49406546+SheffieldCao@users.noreply.github.com> Co-authored-by: xiexinch <xiexinch@outlook.com> Co-authored-by: 小飞猪 <106524776+ooooo-create@users.noreply.github.com>
…or NVIDIA Jetson (open-mmlab#3372) Fine tune ONNX Models (MMSegemetation) Inference for NVIDIA Jetson
## Motivation open-mmlab#3383 ## Modification - Add bpe_simple_vocab_16e6.txt.gz to `MANIFEST.in`
## Motivation open-mmlab#3384 ## Modification - mmseg/apis/inference.py
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation Fixes open-mmlab#3412 ## Modification We just need to replace tensor creation using torch.stack() instead of torch.tensor(). ## BC-breaking (Optional) Does the modification introduce changes that break the backward-compatibility of the downstream repos? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR. ## Use cases (Optional) If this PR introduces a new feature, it is better to list some use cases here, and update the documentation. ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D. 4. The documentation has been modified accordingly, like docstring or example tutorials.
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation Current Visualization Hook can only get instances of `SegLocalVisualizer`. This makes impossible to use any other custom implementation. ## Modification This PR just allows to instantiate a different visualizer (following mmdetection implementation): https://github.com/open-mmlab/mmdetection/blob/main/mmdet/engine/hooks/visualization_hook.py#L58
## Motivation
When using `test_cfg` for `data_preprocessor`, `predict_by_feat` resizes
to the original size, not the padded size.
```
data_preprocessor = dict(
type="SegDataPreProcessor",
#type="SegDataPreProcessorWithPad",
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
bgr_to_rgb=True,
pad_val=0,
seg_pad_val=255,
test_cfg=dict(size=(128, 128)))
```
Refar to:
https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/decode_heads/san_head.py#L589-L592
## Checklist
1. Pre-commit or other linting tools are used to fix the potential lint
issues.
2. The modification is covered by complete unit tests. If not, please
add more unit test to ensure the correctness.
3. If the modification has potential influence on downstream projects,
this PR should be tested with downstream projects, like MMDet or
MMDet3D.
4. The documentation has been modified accordingly, like docstring or
example tutorials.
…t labels (open-mmlab#3466) Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers. ## Motivation It is difficult to visualize without "labels" when using the inferencer. - While using the `MMSegInferencer`, the visualized prediction contains labels on the mask, but it is difficult to pass `withLabels=False` without rewriting the config (which is harder to do when you initialize the inferencer with a model name rather than the config). - I thought it would be easier to just pass `withLabels=False` to `inferencer.__call__()` since you can also pass `opacity` and other parameters anyway. ## Modification Please briefly describe what modification is made in this PR. - Added `with_labels` to `visualize_kwargs` inside `MMSegInferencer`. - Modified to `visualize()` function. ## BC-breaking (Optional) Does the modification introduce changes that break the backward-compatibility of the downstream repos? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR. ## Use cases (Optional) If this PR introduces a new feature, it is better to list some use cases here, and update the documentation. ## Checklist 1. Pre-commit or other linting tools are used to fix the potential lint issues. 2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness. 3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D. 4. The documentation has been modified accordingly, like docstring or example tutorials. --------- Co-authored-by: xiexinch <xiexinch@outlook.com>
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
Please describe the motivation of this PR and the goal you want to achieve through this PR.
Modification
Please briefly describe what modification is made in this PR.
BC-breaking (Optional)
Does the modification introduce changes that break the backward-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
Use cases (Optional)
If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.
Checklist