use lr=diff_lr for modules named found in diff_lr_names, otherwise use lr=default_lr Args: named_param_tuples_or_model: List([name, p

Strategy: suppose that the models that we will create will have prefixes appended to each of its keys, for example due to an extra level of n

Computes area of boxes. Args: boxlist: BoxList holding N boxes Returns: a numpy array with shape [N*1] representing box areas

Perfrom brightness jittering on the input images. The channels of images should be in order BGR. Args: var (float): jitter ratio

build text encoder. Args: model_config (dict): model config. pretrain (bool): Whether to do pretrain or finetuning. check

build text decoder the same as the multimodal encoder. Args: model_config (dict): model config. pretrain (bool): Whether to do pr

each text may have multiple ground_truth image, e.g., ssv2

each image may have multiple ground_truth text， e.g., COCO and Flickr30K

Arguments: cfg: config object for the experiment. dataset_list (list[str]): Contains the names of the datasets, i.e.,

build text_encoder and possiblly video-to-text multimodal fusion encoder. Returns: nn.Module. The text encoder

build vision encoder Returns: (vision_encoder, clip_teacher). Each is a `nn.Module`.

Clears the state to prepare for a fresh evaluation.

Seems only used during pre-training

Evaluates detections as being tp, fp or ignored from a single image. The evaluation is done in two stages: 1. All detections are matched to

Concatenate list of BoxLists. This op concatenates a list of input BoxLists into a larger BoxList. It also handles concatenation of BoxList fiel

Perfrom contrast jittering on the input images. The channels of images should be in order BGR. Args: var (float): jitter ratio fo

Loads label map proto and returns categories list compatible with eval. This function loads a label map and returns a list of dicts, each of which

Creates dictionary of COCO compatible categories keyed by category id. Args: categories: a list of dicts, each of which has the following keys:

named_param_tuples_with_lr: List([name, param, weight_decay, lr])

encode image / videos as features. Args: image (torch.Tensor): The input images. Returns: tuple. - mask (tor

encode image / videos as features. Args: image (torch.Tensor): The input images. test (bool): Whether testing.

eval values of dict leaf. Args: d (dict): The dict to eval. Returns: dict.

For open-ended QA that has a single answer, accuracy is computed based on exact match

automatically evaluate string to corresponding types. For example: not a string -> return the original input '0' -> 0 '

evaluate dataset using different methods based on dataset type. Args: dataset: Dataset object predictions(list[BoxList]): each ite

Build config from file. Supported filetypes: `.py`,`.yaml`,`.json`. Args: filepath (str): The config file path. Returns:

Run gather on arbitrary picklable data (not necessarily tensors). Args: data: any picklable object dst (int): destination ran

obtain the annotation for one media (video or image) Args: index (int): The media index. Returns: dict. - "i

get a `Config` instance. Args: default_config (dict): The default config. `default_config` will be overrided by c

Get corner coordinates of boxes. Returns: a list of 4 1-d numpy arrays [y_min, x_min, y_max, x_max]

Modified from https://github.com/huggingface/transformers/blob/v4.15.0/src/transformers/optimization.py Create a schedule with a learning ra

Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (

obtain the args for all_gather Returns: dict.

Get parameters for ``crop`` for a random sized crop. Args: img (PIL Image): Image to be cropped. scale (tuple): range