Skip to content

Latest commit

 

History

History
130 lines (97 loc) · 7.69 KB

README_EN.md

File metadata and controls

130 lines (97 loc) · 7.69 KB
Raw

Real Cascade U-Nets for Anime Image Super Resolution

🔥 Real-CUGAN🔥 is an AI super resolution model for anime images, trained in a million scale anime dataset, using the same architecture as Waifu2x-CUNet. It supports 2x\3x\4x super resolving. For different enhancement strength, now 2x Real-CUGAN supports 5 model weights, 3x/4x Real-CUGAN supports 3 model weights.

Real-CUGAN package an executable environment for windows users. GUI will be supported in the future.

1. Comparasion

all._x264.mp4
  • visual effect comparasion
    texture chanllenge case compare1 line chanllenge case compare2 heavy artifact case compare3 bokeh effect case compare4
  • Detailed comparasion
Waifu2x(CUNet) Real-ESRGAN(Anime6B) Real-CUGAN
traning set Private anime training set, unknown magnitude and quality Private anime training set, unknown magnitude and quality Million scale high quality patch training set
speed(times) Baseline 2.2x 1x
visual effect can't deblur; artifacts are not completely removed The sharpening strength is the largest. The painting style may be changed. The lines may be incorrectly reconstructed. The bokeh effect region may be forced to be clear. Sharper line style than waifu2x. Bokeh effect region maintained better. Texture preserved better.
compatibility numbers of existing windows-apps，
Caffe，PyTorch，NCNN, VapourSynth		 PyTorch，VapourSynth，NCNN the same as waifu2x
enhancement supports multiple denoise strength only support default enhancement strength 5 different enhancement strength for 2x, 3 for 3x, 3 for 4x now
SR resolution 1x+2x 4x 2x+3x+4x are supported now, and 1x model is training

2. For windows users

modify config.py, and double click go.bat to execute Real-CUGAN.

  • Executable file：

    BaiduDrive(extract code:ds2a) 🔗｜GithubRelease 🔗｜HecaiDrive(extract code:tX4O) 🔗｜ GoogleDrive 🔗

  • System environment：

    • ✔️ Tested in windows10 64bit.
    • ✔️ Light version: cuda >= 10.0. 【Heavy version: cuda >= 11.1】
    • ❗ Note that 30 series nvidia GPU only supports heavy version.

  • config file：

    a. common settings

    • mode: image/video;
    • model_path: the path of model weights(2x model has 4 weight files, 3x/4x model only has 1 weight file);
    • device: cuda device number. If you want to use multiple GPUs to super resolve images, it's recommanded to manually divide the tasks into different folders and fill in different cuda device numbers;
    • you need fill in the input and output dir path for image task, and the input and output video file path for video task.

    b. settings for video task

    • nt: the threading number of each GPU. If the video memory is enough, >=2 is recommended (for faster inference).
    • n_gpu: the number of GPUs you will use.
    • encode_params: if you don't know how to use ffmpeg, you shouldn't change it.
    • half: FP16 inference or FP32 inference. 'True' is recommended.
    • tile: 0~4 is supported. The bigger the number, the less video memory is needed, and the lower inference speed it is.

3. For waifu2x-caffe users

We support two weights for waifu2x-caffe users now:

🔥 Real-CUGAN2x standard version and 🔥 Real-CUGAN2x no crop line version
BaiduDrive(extract code:ds2a) 🔗｜GithubRelease 🔗｜HecaiDrive(extract code:tX4O) 🔗｜ GoogleDrive 🔗
Users can replace the original weights with the new ones (remember to backup the original weights if you want to reuse them), and use the original setting to super resolve images.

❗❗❗ due to the crop mechanism of waifu2x-caffe, for standard version，big crop_size is recommanded, or the crop line artifact may be caused. If crop line edge artifact is found, please use our windows package or the 'no crop line' version. The tile_mode of windows package version is lossless. The 'no edge artifact' version waifu2x-caffe weights may cause more texture loss.

For developers, it is recommended to use the whole image as input. Pytorch version (tile mode) is recommended if you want the program to require less video memory.

4. Python environment dependence

✅ torch>=1.0.0
✅ numpy
✅ opencv-python
✅ moviepy

upcunet_v3.py:model file and image inference script
inference_video.py: a simple script for inferencing anime videos using Real-CUGAN

5. For VapourSynth users

Please see Readme

6.:european_castle: Model Zoo

You can download the weights from netdisk links.

1x 2x 3x/4x
denoise strength only no denoise version supported, training... no denoise/1x/2x/3x are supported now no denoise/3x are supported now, 1x/2x:training...
conservative version training... supported now
fast model under investigation

7. TODO：

  • Lightweight/fast version
  • Adjustable denoise, deblock, deblur, sharpening strength
  • Super resolve the image to specified resolution end to end
  • Optimize texture retention and reduce AI processing artifacts
  • Simple GUI

8. Acknowledgement

The training code is from but not limited to: 🌟 RealESRGAN🌟 .

The original waifu2x-cunet architecture is from: 🌟 CUNet🌟 .