face.evoLVe: High-Performance Face Recognition Library based on PaddlePaddle & PyTorch

• Evolve to be more comprehensive, effective and efficient for face related analytics \& applications! (WeChat News)
• About the name:
• "face" means this repo is dedicated for face related analytics \& applications.
• "evolve" means unleash your greatness to be better and better. "LV" are capitalized to acknowledge the nurturing of Learning and Vision (LV) group, Nation University of Singapore (NUS).
• This work was done during Jian Zhao served as a short-term "Texpert" Research Scientist at Tencent FiT DeepSea AI Lab, Shenzhen, China.

Author	Jian Zhao
Homepage	https://zhaoj9014.github.io

---

License

The code of face.evoLVe is released under the MIT License.

---

News

:white_check_mark: CLOSED 02 September 2021: ~~Baidu PaddlePaddle officially merged face.evoLVe to faciliate researches and applications on face-related analytics (Official Announcement).~~

:white_check_mark: CLOSED 03 July 2021: ~~Provides training code for the paddlepaddle framework.~~

:white_check_mark: CLOSED 04 July 2019: ~~We will share several publicly available datasets on face anti-spoofing/liveness detection to facilitate related research and analytics.~~

:white_check_mark: CLOSED 07 June 2019: ~~We are training a better-performing IR-152 model on MS-Celeb-1M_Align_112x112, and will release the model soon.~~

:white_check_mark: CLOSED 23 May 2019: ~~We share three publicly available datasets to facilitate research on heterogeneous face recognition and analytics. Please refer to Sec. Data Zoo for details.~~

:white_check_mark: CLOSED 23 Jan 2019: ~~We share the name lists and pair-wise overlapping lists of several widely-used face recognition datasets to help researchers/engineers quickly remove the overlapping parts between their own private datasets and the public datasets. Please refer to Sec. Data Zoo for details.~~

:white_check_mark: CLOSED 23 Jan 2019: ~~The current distributed training schema with multi-GPUs under PyTorch and other mainstream platforms parallels the backbone across multi-GPUs while relying on a single master to compute the final bottleneck (fully-connected/softmax) layer. This is not an issue for conventional face recognition with moderate number of identities. However, it struggles with large-scale face recognition, which requires recognizing millions of identities in the real world. The master can hardly hold the oversized final layer while the slaves still have redundant computation resource, leading to small-batch training or even failed training. To address this problem, we are developing a highly-elegant, effective and efficient distributed training schema with multi-GPUs under PyTorch, supporting not only the backbone, but also the head with the fully-connected (softmax) layer, to facilitate high-performance large-scale face recognition. We will added this support into our repo.~~

:white_check_mark: CLOSED 22 Jan 2019: ~~We have released two feature extraction APIs for extracting features from pre-trained models, implemented with PyTorch build-in functions and OpenCV, respectively. Please check ./util/extract_feature_v1.py and ./util/extract_feature_v2.py.~~

:white_check_mark: CLOSED 22 Jan 2019: ~~We are fine-tuning our released IR-50 model on our private Asia face data, which will be released soon to facilitate high-performance Asia face recognition.~~

:white_check_mark: CLOSED 21 Jan 2019: ~~We are training a better-performing IR-50 model on MS-Celeb-1M_Align_112x112, and will replace the current model soon.~~

---

Contents

• Introduction
• Pre-Requisites
• Usage
• Face Alignment
• Data Processing
• Training and Validation
• Data Zoo
• Model Zoo
• Achievement
• Acknowledgement
• Donation
• Citation

---

face.evoLVe for High-Performance Face Recognition

Introduction

:information_desk_person:

• This repo provides a comprehensive face recognition library for face related analytics \& applications, including face alignment (detection, landmark localization, affine transformation, etc.), data processing (e.g., augmentation, data balancing, normalization, etc.), various backbones (e.g., ResNet, IR, IR-SE, ResNeXt, SE-ResNeXt, DenseNet, LightCNN, MobileNet, ShuffleNet, DPN, etc.), various losses (e.g., Softmax, Focal, Center, SphereFace, CosFace, AmSoftmax, ArcFace, Triplet, etc.) and bags of tricks for improving performance (e.g., training refinements, model tweaks, knowledge distillation, etc.).
• The current distributed training schema with multi-GPUs under PyTorch and other mainstream platforms parallels the backbone across multi-GPUs while relying on a single master to compute the final bottleneck (fully-connected/softmax) layer. This is not an issue for conventional face recognition with moderate number of identities. However, it struggles with large-scale face recognition, which requires recognizing millions of identities in the real world. The master can hardly hold the oversized final layer while the slaves still have redundant computation resource, leading to small-batch training or even failed training. To address this problem, this repo provides a highly-elegant, effective and efficient distributed training schema with multi-GPUs under PyTorch, supporting not only the backbone, but also the head with the fully-connected (softmax) layer, to facilitate high-performance large-scale face recognition.
• All data before \& after alignment, source codes and trained models are provided.
• This repo can help researchers/engineers develop high-performance deep face recognition models and algorithms quickly for practical use and deployment.

---

Pre-Requisites

:cake:

• Linux or macOS
• Python 3.7 (for training \& validation) and Python 2.7 (for visualization w/ tensorboardX)
• PyTorch 1.0 (for traininig \& validation, install w/ pip install torch torchvision)
• MXNet 1.3.1 (optional, for data processing, install w/ pip install mxnet-cu90)
• TensorFlow 1.12 (optional, for visualization, install w/ pip install tensorflow-gpu)
• tensorboardX 1.6 (optional, for visualization, install w/ pip install tensorboardX)
• OpenCV 3.4.5 (install w/ pip install opencv-python)
• bcolz 1.2.0 (install w/ pip install bcolz)

While not required, for optimal performance it is highly recommended to run the code using a CUDA enabled GPU. We used 4-8 NVIDIA Tesla P40 in parallel.

---

Usage

:orange_book:

• Clone the repo: git clone https://github.com/ZhaoJ9014/face.evoLVe.PyTorch.git.
• mkdir data checkpoint log at appropriate directory to store your train/val/test data, checkpoints and training logs.
• Prepare your train/val/test data (refer to Sec. Data Zoo for publicly available face related databases), and ensure each database folder has the following structure: ./data/db_name/ -> id1/ -> 1.jpg -> ... -> id2/ -> 1.jpg -> ... -> ... -> ... -> ...
• Refer to the codes of corresponding sections for specific purposes.

---

Face Alignment

:triangular_ruler:

• This section is based on the work of MTCNN.
• Folder: ./align
• Face detection, landmark localization APIs and visualization toy example with ipython notebook: ```python from PIL import Image from detector import detect_faces from visualization_utils import show_results

img = Image.open('some_img.jpg') # modify the image path to yours bounding_boxes, landmarks = detect_faces(img) # detect bboxes and landmarks for all faces in the image show_results(img, bounding_boxes, landmarks) # visualize the results * Face alignment API (perform face detection, landmark localization and alignment with affine transformations on a whole database foldersource_rootwith the directory structure as demonstrated in Sec. [Usage](#Usage), and store the aligned results to a new folderdest_rootwith the same directory structure): python face_align.py -source_root [source_root] -dest_root [dest_root] -crop_size [crop_size]

# python face_align.py -source_root './data/test' -dest_root './data/test_Aligned' -crop_size 112 * For macOS users, there is no need to worry about*.DS_Storefiles which may ruin your data, since they will be automatically removed when you run the scripts. * Keynotes for customed use: 1) specify the arguments ofsource_root,dest_rootandcrop_sizeto your own values when you runface_align.py; 2) pass your customedmin_face_size,thresholdsandnms_thresholdsvalues to thedetect_facesfunction ofdetector.pyto match your practical requirements; 3) if you find the speed using face alignment API is a bit slow, you can call face resize API to firstly resize the image whose smaller size is larger than a threshold (specify the arguments ofsource_root,dest_rootandmin_sideto your own values) before calling the face alignment API: python face_resize.py ```

---

Data Processing

:bar_chart:

• Folder: ./balance
• Remove low-shot data API (remove the low-shot classes with less than min_num samples in the training set root with the directory structure as demonstrated in Sec. Usage for data balance and effective model training): ``` python remove_lowshot.py -root [root] -min_num [min_num]

# python remove_lowshot.py -root './data/train' -min_num 10 * Keynotes for customed use: specify the arguments ofrootandmin_numto your own values when you runremove_lowshot.py```. * We prefer to include other data processing tricks, e.g., augmentation (flip horizontally, scale hue/satuation/brightness with coefficients uniformly drawn from [0.6,1.4], add PCA noise with a coefficient sampled from a normal distribution N(0,0.1), etc.), weighted random sampling, normalization, etc. to the main training script in Sec. Training and Validation to be self-contained.

---

Training and Validation

:coffee:

• Folder: ./
• Configuration API (configurate your overall settings for training \& validation) config.py: ```python import torch

configurations = { 1: dict( SEED = 1337, # random seed for reproduce results

      DATA_ROOT = '/media/pc/6T/jasonjzhao/data/faces_emore', # the parent root where your train/val/test data are stored
      MODEL_ROOT = '/media/pc/6T/jasonjzhao/buffer/model', # the root to buffer your checkpoints
      LOG_ROOT = '/media/pc/6T/jasonjzhao/buffer/log', # the root to log your train/val status
      BACKBONE_RESUME_ROOT = './', # the root to resume training from a saved checkpoint
      HEAD_RESUME_ROOT = './', # the root to resume training from a saved checkpoint

      BACKBONE_NAME = 'IR_SE_50', # support: ['ResNet_50', 'ResNet_101', 'ResNet_152', 'IR_50', 'IR_101', 'IR_152', 'IR_SE_50', 'IR_SE_101', 'IR_SE_152']
      HEAD_NAME = 'ArcFace', # support:  ['Softmax', 'ArcFace', 'CosFace', 'SphereFace', 'Am_softmax']
      LOSS_NAME = 'Focal', # support: ['Focal', 'Softmax']

      INPUT_SIZE = [112, 112], # support: [112, 112] and [224, 224]
      RGB_MEAN = [0.5, 0.5, 0.5], # for normalize inputs to [-1, 1]
      RGB_STD = [0.5, 0.5, 0.5],
      EMBEDDING_SIZE = 512, # feature dimension
      BATCH_SIZE = 512,
      DROP_LAST = True, # whether drop the last batch to ensure consistent batch_norm statistics
      LR = 0.1, # initial LR
      NUM_EPOCH = 125, # total epoch number (use the firt 1/25 epochs to warm up)
      WEIGHT_DECAY = 5e-4, # do not apply to batch_norm parameters
      MOMENTUM = 0.9,
      STAGES = [35, 65, 95], # epoch stages to decay learning rate

      DEVICE = torch.device("cuda:0" if torch.cuda.is_available() else "cpu"),
      MULTI_GPU = True, # flag to use multiple GPUs; if you choose to train with single GPU, you should first run "export CUDA_VISILE_DEVICES=device_id" to specify the GPU card you want to use
      GPU_ID = [0, 1, 2, 3], # specify your GPU ids
      PIN_MEMORY = True,
      NUM_WORKERS = 0,

), } ``` * Train \& validation API (all folks about training \& validation, i.e., import