End-to-end, automatic face swapping pipeline

The Joker (Heath Ledger) swapped using our method onto very different subjects and images.

Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, and Gerard Medioni.

Overview

Code for the automatic, image-to-image face swapping method described in the paper:

Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, Gerard Medioni, "On Face Segmentation, Face Swapping, and Face Perception", IEEE Conference on Automatic Face and Gesture Recognition (FG), Xi'an, China on May 2018

Please see project page for more details, more resources and updates on this project.

If you find this code useful, please make sure to cite our paper in your work.

News

Version 1.0 is released: - Better performance and results. - Python binding. - Easier to build: removed many of the dependencies. - Software renderer: OpenGL is no longer required, the project can now be built on gui-less servers without any problems. - Added low resolution segmentation model for GPUs with low memory.

Installation

Both Linux and Windows are supported. - Ubuntu installation guide - Windows installation guide

Usage

• For Python follow the Python guide
• For running the applications follow the applications guide.
• For using the library's C++ interface, please take a look at the Doxygen generated documentation.

Citation

Please cite our paper with the following bibtex if you use our face renderer:

@inproceedings{nirkin2018_faceswap,
      title={On Face Segmentation, Face Swapping, and Face Perception},
      booktitle = {IEEE Conference on Automatic Face and Gesture Recognition},
      author={Nirkin, Yuval and Masi, Iacopo and Tran, Anh Tuan and Hassner, Tal and Medioni, G\'{e}rard},
      year={2018},
    }

Related projects

• Deep face segmentation, used to segment face regions in the face swapping pipeline.
• Interactive system for fast face segmentation ground truth labeling, used to produce the training set for our deep face segmentation.
• CNN3DMM, used to estimate 3D face shapes from single images.
• ResFace101, deep face recognition used in the paper to test face swapping capabilities.
• Hand augmentations, used to generate the hand augmentations for training the face segmentation.

Copyright

Copyright 2018, Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, and Gerard Medioni

The SOFTWARE provided in this page is provided "as is", without any guarantee made as to its suitability or fitness for any particular use. It may contain bugs, so use of this tool is at your own risk. We take no responsibility for any damage of any sort that may unintentionally be caused through its use.

License

This project is released under the GPLv3 license.

For a more permitting license please contact the author.