Computes and stores the average and current value

Create two crops of the same image

backbone + projection head

Linear classifier

encoder + classifier

Supervised Contrastive Learning: https://arxiv.org/pdf/2004.11362.pdf. It also supports the unsupervised contrastive loss in SimCLR

Implements BatchNorm1d by BatchNorm2d, for SyncBN purpose