μ Detection Detailer

This is yet another detection detailer named μ-Detection Detailer (shortly μ-DDetailer) forked from DDetailer as a postprocess extension for Stable Diffusion web UI.

Features

1. bbox/segment object detection using MMDetection, MMYOLO, mediapipe and ultralytics
2. Inpainting helper added to support dynamically generated masks/segments viewer using js canvas to select detections easily.
3. Support importing ADetailer/DDetailer infotext settings
4. both mmdet 2.x / mmdet 3.x are supported
5. Support Load/Save presets

Screenshot

Live preview with sd-webui

015.mp4

Introduction

It uses the MMDetection library to detect objects, mask them especially human faces, and then partially redraw them to bring out the details. Optionally It uses MMYOLO, mediapipe and ultralytics libraries to detect objects.

Original	Masked Preview	Inpaint

Segmentation

Default models enable person and face instance segmentation.

Detailing

With full-resolution inpainting, the extension is handy for improving faces without the hassle of manual masking.

Installation

1. Use git clone https://github.com/wkpark/uddetailer.git from your SD web UI /extensions folder. Alternatively, install from the extensions tab with url https://github.com/wkpark/uddetailer
2. Press the Apply and restart button or restart SD web UI.

The models and dependencies should download automatically. To install them manually, follow the official instructions for installing mmdet. The models can be downloaded here and should be placed in /models/mmdet/bbox for bounding box (anime-face_yolov3) or /models/mmdet/segm for instance segmentation models (dd-person_mask2former). See the MMDetection docs for guidance on training your own models. For using official MMDetection pretrained models see here, these are trained for photorealism. See Troubleshooting if you encounter issues during installation.

It also supports additional ultralytics detection models and you can install YoloV8 models manually. ultralytics's models should be placed in the models/yolo dir.

Usage

To use μ Detection Detailer in SD web UI, enable μ Detection Detailer first, and select optional parameters, then click 'Generate'. Here are some tips:

• anime-face_yolov3 can detect the bounding box of faces as the primary model while dd-person_mask2former isolates the head's silhouette as the secondary model by using the bitwise AND option. Refer to this example.
• The dilation factor expands the mask, while the x & y offsets move the mask around.
• You can change the default prompt, negative prompt, and even checkpoint model.
• The extension is available in the img2img tab as well and can improve the quality of your 10 pulls with moderate settings (low denoise).

Troubleshooting

If you get the message ERROR: 'Failed building wheel for pycocotools' follow these steps.

For any other issues installing, open an issue.

Models

Model	Target
anime-face_yolov3.pt	2D / anime face
dd-person_mask2former.pt 1	coco class segmentation
yolov5_ins_n.pth 2	coco class instance segmentaion
yolov5_ins_s.pth	coco class instance segmentaion
face_yolov8n.pt 3	2D / realistic face
face_yolov8s.pt 3	2D / realistic face
hand_yolov8n.pt 3	2D / realistic hand
hand_yolov8s.pt 3	2D / realistic hand
mediapipe_face_full 4	realistic face
mediapipe_face_short	realistic face
mediapipe_face_mesh	realistic face

Optional Models

Optional models could be downloaded using Model Download Helper

Model	License	Target
nsfwrecog_v1.onnx	License: APL	NSFW detection
person_yolov8n-seg.pt	License: AGPL	2D / realistic person
person_yolov8s-seg.pt	License: AGPL	2D / realistic person

Credits

dustysys/DDetailer - Author of the original Detection Detailer.

bingsu/ADetailer - Author of the ADetailer and creator of face_yolov8*pt, hand_yolov8*.pt and more.

hysts/anime-face-detector - Creator of anime-face_yolov3, which has impressive performance on a variety of art styles.

skytnt/anime-segmentation - Synthetic dataset used to train dd-person_mask2former.

jerryli27/AniSeg - Annotated dataset used to train dd-person_mask2former.

open-mmlab/MMDetection - Object detection toolset. dd-person_mask2former was trained via transfer learning using their R-50 Mask2Former instance segmentation model as a base.

open-mmlab/MMYOLO - MMYOLO is an open-source toolbox for YOLO series algorithms based on PyTorch and MMDetection.

google/mediapipe - Mediapipe is an open-source framework for building real-time perceptual computing applications.

ultralytics/ultralytics - Ultralytics is an open-source framework to support object detection and tracking, instance seg, classification and pose estimation framework.

AUTOMATIC1111/stable-diffusion-webui - Web UI for Stable Diffusion, base application for this extension.

padmalcom/nsfwrecog - NSFW Recog used for NSFW Consor helper

Footnotes

1. from MMDetection Instance Segmentation model ↩

2. from MMYolo Instance Segmentation model Yolov5 ↩

3. converted from ADetailer yolov8 models. See also https://huggingface.co/Bingsu/adetailer ↩ ↩² ↩³ ↩⁴

4. Mediapipe models ↩