AnimeGAN

A Tensorflow implementation of AnimeGAN for fast photo animation ! 日本語
The paper can be accessed here or on the website.

Online access: Be grateful to @TonyLianLong for developing an online access project, you can implement photo animation through a browser without installing anything, click here to have a try.

Good news: tensorflow-1.15.0 is compatible with the code of this repository. In this version, you can run this code without any modification. The premise is that the CUDA and cudnn corresponding to the tf version are correctly installed. Maybe the versions between tf-1.8.0 and tf-1.15.0 are also supported and compatible with this repository, but I didn’t make too many extra attempts.

This is the Open source of the paper , which uses the GAN framwork to transform real-world photos into anime images.

Some suggestions:
1. since the real photos in the training set are all landscape photos, if you want to stylize the photos with people as the main body, you may as well add at least 3000 photos of people in the training set and retrain to obtain a new model.
2. In order to obtain a better face animation effect, when using 2 images as data pairs for training, it is suggested that the faces in the photos and the faces in the anime style data should be consistent in terms of gender as much as possible.
3. The generated stylized images will be affected by the overall brightness and tone of the style data, so try not to select the anime images of night as the style data, and it is necessary to make an exposure compensation for the overall style data to promote the consistency of brightness and darkness of the entire style data.

News:
AnimeGAN has been open source for almost a year. Now, it has 2 significant problems:
1. It is difficult to get the effect reported in the paper directly through code training.
2. The generated image is prone to high-frequency artifacts.

Therefore, I will release a new version of AnimeGAN's pre-trained model in the near future. And provide the corresponding hyperparameter settings, and these settings may be a little different from those mentioned in the paper.

The improvement directions of AnimeGAN+ mainly include the following 4 points:
1. Solve the problem of high-frequency artifacts in the generated image.
2. It is easy to train and directly achieve the effects in the paper.
3. Further reduce the number of parameters of the generator network.
4. Use new high-quality style data, which come from BD movies as much as possible.

AnimeGAN+ will be expected to be released this summer.(TBD)

Requirements

• python 3.6
• tensorflow-gpu
• tensorflow-gpu 1.8.0 (ubuntu, GPU 1080Ti or Titan xp, cuda 9.0, cudnn 7.1.3)
• tensorflow-gpu 1.15.0 (ubuntu, GPU 2080Ti, cuda 10.0.130, cudnn 7.6.0)
• opencv
• tqdm
• numpy
• glob
• argparse

Usage

1. Download vgg19 or Pretrained model

vgg19.npy

Pretrained model

2. Download dataset

Link

3. Do edge_smooth

eg. python edge_smooth.py --dataset Hayao --img_size 256

3. Train

eg. python main.py --phase train --dataset Hayao --epoch 101 --init_epoch 1

4. Test

eg. python main.py --phase test --dataset Hayao
or python test.py --checkpoint_dir checkpoint/AnimeGAN_Hayao_lsgan_300_300_1_3_10 --test_dir dataset/test/real --style_name H

Results

:blush: pictures from the paper 'AnimeGAN: a novel lightweight GAN for photo animation'

:heart_eyes: Photo to Hayao Style

Acknowledgment

This code is based on the CartoonGAN-Tensorflow and Anime-Sketch-Coloring-with-Swish-Gated-Residual-UNet. Thanks to the contributors of this project.