Skip to content

Automatic line art colorization using various types of hint or without hint

License

Notifications You must be signed in to change notification settings

SerialLain3170/adeleine

Repository files navigation

adeleine

Update

  • 03/20/2023: Change repository's name from Colorization to adeleine

Overview

This repository is about the deep-learing based automatic line art colorization with or without hints. There are mainly three types of hints.

  • Atari: Colorization with hints that include some lines in desired color (ex. PaintsChainer)
  • Tag: Colorization with tags (ex. Tag2Pix)
  • Reference: Colorization with reference images (ex. style2paints V1)

Line extraction method

There are many kinds of line extraction methods, such as XDoG or SketchKeras. If we train the model with only single type of the line art, trained model tends to overfit and cannot colorize other types of line art properly (ref Tag2Pix). Therefore, three types of line art are mainly used.

  • XDoG: Line extraction using two Gaussian distributions difference to standard deviations
  • SketchKeras: Line extraction using UNet. Lines obtained by SketchKeras are like pencil drawings.
  • Sketch Simplification: Line extraction using Fully-Convolutional Networks. Lines obtained by Sketch Simplification are like digital drawings.

An example obtained by these line extraction methods is as follows.

Moreover, I usually add two types of data augmenation to line arts to avoid overfitting.

  • Randomly morphology transformation to take various thicks of lines
  • Randomly RGB values of lines to incorporate various depths of lines

Experiments without hints

Methods

  • pix2pix
  • pix2pixHD
  • bicyclegan

Results

Method Result
pix2pix & pix2pixHD
bicyclegan

Experiment with atari

Methods

  • userhint
  • userhint v2
  • whitebox
  • spade

Results

Method Result
userhint here
userhint v2
whitebox
spade

Experiment with reference

Methods

  • adain
  • scft
  • video

Results

Method Result
adain here
scft
video

GUI application

LICENSE

MIT License on only Adeleine. The license is not effective on other implementations.

References

About

Automatic line art colorization using various types of hint or without hint

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published