Args: logits: a list of logits, each with a contrastive task target: contrastive learning target criterion: typ

Args: x: feat, shape [bsz, n_dim] w: softmax weight, shape [bsz, self.K + 1, n_dim]

Args: q: query/anchor feature k: key feature queue: memory buffer

Args: memory: memory buffer x: features y: index of updating position

With 50% prob, negate the value

one epoch training

Args: k: key feature queue: memory buffer

Computes the accuracy over the k top predictions for the specified values of k

1x1 convolution

3x3 convolution with padding

Create a RandAugment transform :param config_str: String defining configuration of random augmentation. Consists of multiple sections separat

3x3 convolution with padding

Draw N samples from multinomial :param N: number of samples :return: samples

Args: gpu: current gpu id ngpus_per_node: num of process/gpus per node

logging to tensorboard Args: epoch: training epoch logs: loss and accuracy lr: learning rate train:

model_ema = m * model_ema + (1 - m) model

Shuffle BN implementation Args: x: input image on each GPU/process model_ema: momentum encoder on each GPU/process

Training based on memory bank mechanism. Only one forward pass.

MoCo encoder style training. This needs two forward passes, one for normal encoder, and one for moco encoder

Synchronize memory buffers Args: contrast: memory.

build loaders for contrastive training

build loaders for linear evaluation

load pre-trained weights for encoder Args: model: pretrained encoder, should be frozen

logging to tensorboard Args: epoch: training epoch logs: loss and accuracy lr: learning rate

rand augment ops for RGB images

rand augment ops for CMC images (removing color ops)

r"""ResNet-50 model from `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_ Args: pretrained (bo

load checkpoint

load classifier checkpoint

save model to checkpoint

save classifier to checkpoint

Wrap up models with apex and DDP Args: model: model model_ema: momentum encoder optimizer: optimizer

Wrap up models with DDP Args: model: pretrained encoder, should be frozen classifier: linear classifier

Args: index (int): index Returns: tuple: (image, index, ...)

Global average pooling over the input's spatial dimensions