decay learning rate every decay_epoch

compute the loss function for a single batch

compute the loss function for several batches

compute model-specific metrics and put it to metrics dict

Performs all_gather operation on the provided tensors. *** Warning ***: torch.distributed.all_gather has no gradient.

Utility function to convert the object id in the metadata to barebones object id found in the class to object mapping.

load a model and its optimizer

embed previous actions

take a list of frames tensors, pad it, apply dropout and extract embeddings

take a list of annotation tokens and extract embeddings with EncoderLang

add positional encodings, apply layernorm and dropout

add encodings (positional, token and so on)

compute encodings for all tokens using a normal flat encoder

Run per image evaluation on given images and store results (a list of dict) in self.evalImgs :return: None

forward the model for multiple time-steps (used for training)

forward the model for multiple time-steps (used for training)

Args: depth (int): Depth of caller conext, use 0 for caller depth. Default value: 0. Returns: str: module name of th