github.com/thuml/Transfer-Learning-Library @v0.4 sqlite

Updates

2022.8

We release v0.4 of TLlib. Previous versions of TLlib can be found at here. In v0.4, we add implementations of the following methods: - Domain Adaptation for Object Detection [Code] [API] - Pre-trained Model Selection [Code] [API] - Semi-supervised Learning for Classification [Code] [API]

Besides, we maintain a collection of awesome papers in Transfer Learning in another repo A Roadmap for Transfer Learning.

2022.2

We adjusted our API following our survey Transferablity in Deep Learning.

API is divided by methods. Specifically, - dalib.adaptation => tllib.aligment (domain alignment methods) - dalib.translation => tllib.translation (domain translation methods) - dalib.modules, common.modules => tllib.modules - talib => tllib.regularization (regularization methods) - common.vision => tllib.vision - common.utils => tllib.utils

We also add some new method division, - tllib.normalization (normalization-based methods) - tllib.self_training (self-training methods) - tllib.ranking (model ranking/selection methods) - tllib.reweight (data reweighting/resampling methods)

examples is still divided by learning setup.

Introduction

TLlib is an open-source and well-documented library for Transfer Learning. It is based on pure PyTorch with high performance and friendly API. Our code is pythonic, and the design is consistent with torchvision. You can easily develop new algorithms, or readily apply existing algorithms.

The currently supported algorithms include:

Domain Adaptation for Classification [Code]

• DANN - Unsupervised Domain Adaptation by Backpropagation [ICML 2015] [Code]
• DAN - Learning Transferable Features with Deep Adaptation Networks [ICML 2015] [Code]
• JAN - Deep Transfer Learning with Joint Adaptation Networks [ICML 2017] [Code]
• ADDA - Adversarial Discriminative Domain Adaptation [CVPR 2017] [Code]
• CDAN - Conditional Adversarial Domain Adaptation [NIPS 2018] [Code]
• MCD - Maximum Classifier Discrepancy for Unsupervised Domain Adaptation [CVPR 2018] [Code]
• MDD - Bridging Theory and Algorithm for Domain Adaptation [ICML 2019] [Code]
• BSP - Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation [ICML 2019] [Code]
• MCC - Minimum Class Confusion for Versatile Domain Adaptation [ECCV 2020] [Code]

Domain Adaptation for Object Detection [Code]

• CycleGAN - Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks [ICCV 2017] [Code]
• D-adapt - Decoupled Adaptation for Cross-Domain Object Detection [ICLR 2022] [Code]

Domain Adaptation for Semantic Segmentation [Code]

• CycleGAN - Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks [ICCV 2017] [Code]
• CyCADA - Cycle-Consistent Adversarial Domain Adaptation [ICML 2018] [Code]
• ADVENT - Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation [CVPR 2019] [Code]
• FDA - Fourier Domain Adaptation for Semantic Segmentation [CVPR 2020] [Code]

Domain Adaptation for Keypoint Detection [Code]

• RegDA - Regressive Domain Adaptation for Unsupervised Keypoint Detection [CVPR 2021] [Code]

Domain Adaptation for Person Re-identification [Code]

• IBN-Net - Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net [ECCV 2018]
• MMT - Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification [ICLR 2020] [Code]
• SPGAN - Similarity Preserving Generative Adversarial Network [CVPR 2018] [Code]

Partial Domain Adaptation [Code]

• IWAN - Importance Weighted Adversarial Nets for Partial Domain Adaptation[CVPR 2018] [Code]
• AFN - Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation [ICCV 2019] [Code]

Open-set Domain Adaptation [Code]

• OSBP - Open Set Domain Adaptation by Backpropagation [ECCV 2018] [Code]

Domain Generalization for Classification [Code]

• IBN-Net - Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net [ECCV 2018]
• MixStyle - Domain Generalization with MixStyle [ICLR 2021] [Code]
• MLDG - Learning to Generalize: Meta-Learning for Domain Generalization [AAAI 2018] [Code]
• IRM - Invariant Risk Minimization [ArXiv] [Code]
• VREx - Out-of-Distribution Generalization via Risk Extrapolation [ICML 2021] [Code]
• GroupDRO - Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization [ArXiv] [Code]
• Deep CORAL - Correlation Alignment for Deep Domain Adaptation [ECCV 2016] [Code]

Domain Generalization for Person Re-identification [Code]

• IBN-Net - Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net [ECCV 2018]
• MixStyle - Domain Generalization with MixStyle [ICLR 2021] [Code]

Task Adaptation (Fine-Tuning) for Image Classification [Code]

• L2-SP - Explicit inductive bias for transfer learning with convolutional networks [ICML 2018] [Code]
• BSS - Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning [NIPS 2019] [Code]
• DELTA - DEep Learning Transfer using Fea- ture Map with Attention for convolutional networks [ICLR 2019] [Code]
• Co-Tuning - Co-Tuning for Transfer Learning [NIPS 2020] [Code]
• StochNorm - Stochastic Normalization [NIPS 2020] [Code]
• LWF - Learning Without Forgetting [ECCV 2016] [Code]
• Bi-Tuning - Bi-tuning of Pre-trained Representations [ArXiv] [Code]

Pre-trained Model Selection [Code]

• H-Score - An Information-theoretic Approach to Transferability in Task Transfer Learning [ICIP 2019] [Code]
• NCE - Negative Conditional Entropy in `Transferability and Hardness of Supervised Classification Tasks [ICCV 2019] [Code]
• LEEP - LEEP: A New Measure to Evaluate Transferability of Learned Representations [ICML 2020] [Code]
• LogME - Log Maximum Evidence in `LogME: Practical Assessment of Pre-trained Models for Transfer Learning [ICML 2021] [Code]

Semi-Supervised Learning for Classification [Code]

• Pseudo Label - Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks [ICML 2013] [Code]
• Pi Model - Temporal Ensembling for Semi-Supervised Learning [ICLR 2017] [Code]
• Mean Teacher - Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results [NIPS 2017] [Code]
• Noisy Student - Self-Training With Noisy Student Improves ImageNet Classification [CVPR 2020] [Code]
• UDA - Unsupervised Data Augmentation for Consistency Training [NIPS 2020] [Code]
• FixMatch - Simplifying Semi-Supervised Learning with Consistency and Confidence [NIPS 2020] [Code]
• Self-Tuning - Self-Tuning for Data-Efficient Deep Learning [[ICML 2021]](http://ise.thss.tsinghua.edu.cn/~mlong/doc/S