Pay Attention to Features, Transfer Learn Faster CNNs https://openreview.net/pdf?id=ryxyCeHtPB

Contrastive Representation Distillation https://openreview.net/pdf?id=SkgpBJrtvS includes two symmetric parts: (a) using teacher as anchor, choo

contrastive loss, corresponding to Eq.(18)

A Comprehensive Overhaul of Feature Distillation http://openaccess.thecvf.com/content_ICCV_2019/papers/ Heo_A_Comprehensive_Overhaul_of_Feature_Di

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons https://arxiv.org/pdf/1811.03233.pdf

Paying More Attention to Attention: Improving the Performance of Convolutional Neural Netkworks wia Attention Transfer https://arxiv.org/pdf/1612.

Knowledge Distillation with Adversarial Samples Supporting Decision Boundary https://arxiv.org/pdf/1805.05532.pdf

Correlation Congruence for Knowledge Distillation http://openaccess.thecvf.com/content_ICCV_2019/papers/ Peng_Correlation_Congruence_for_Knowledge

Deep Mutual Learning https://zpascal.net/cvpr2018/Zhang_Deep_Mutual_Learning_CVPR_2018_paper.pdf

A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning http://openaccess.thecvf.com/content_cvpr_2017/pa

araphrasing Complex Network: Network Compression via Factor Transfer http://papers.nips.cc/paper/7541-paraphrasing-complex-network-network-compress

FitNets: Hints for Thin Deep Nets https://arxiv.org/pdf/1412.6550.pdf

Knowledge Distillation via Instance Relationship Graph http://openaccess.thecvf.com/content_CVPR_2019/papers/ Liu_Knowledge_Distillation_via_Insta

Do Deep Nets Really Need to be Deep? http://papers.nips.cc/paper/5484-do-deep-nets-really-need-to-be-deep.pdf

Learning without Memorizing https://arxiv.org/pdf/1811.08051.pdf

Like What You Like: Knowledge Distill via Neuron Selectivity Transfer https://arxiv.org/pdf/1707.01219.pdf

Learning Deep Representations with Probabilistic Knowledge Transfer http://openaccess.thecvf.com/content_ECCV_2018/papers/Nikolaos_Passalis_Learnin

Relational Knowledge Distillation https://arxiv.org/pdf/1904.05068.pdf

Similarity-Preserving Knowledge Distillation https://arxiv.org/pdf/1907.09682.pdf

Sobolev Training for Neural Networks https://arxiv.org/pdf/1706.04859.pdf Knowledge Transfer with Jacobian Matching http://de.arxiv.org/pdf

Distilling the Knowledge in a Neural Network https://arxiv.org/pdf/1503.02531.pdf

Variational Information Distillation for Knowledge Transfer https://zpascal.net/cvpr2019/Ahn_Variational_Information_Distillation_for_Knowledge_T