Gets predictions from an OWL-ViT model for a whole TFDS dataset. These predictions can then be evaluated using the COCO/LVIS APIs. Args: con

Generates query, key/valye, input mask and logging input mask based on ac_config.

Returns a random bounding box for Cutmix. Based on the implementation in timm: https://github.com/rwightman/pytorch-image-models/blob/master/timm

Returns a random bounding box for Cutmix. Based on the implementation in timm: https://github.com/rwightman/pytorch-image-models/blob/master/timm

Returns a list of shards, which may be postprocessed. Args: data_path: Path to the data, either sharded or a single file. fraction_data: Fr

Output (h, w) such that smallest side in image_size resizes to size. This function makes sure that the longest side is not larger than max_size,

Get streaming features. Args: features: (video_batch_size, num_tot_tokens, dim) train: bool Returns: streaming_features: (v

Get inputs to the text decoder. In evaluation, we create the zero-padded text-token with the first token being BOS. In training, we handle mu

Get inputs to the text decoder. In evaluation, we create the zero-padded text-token with the first token being BOS. In training, the vi

Constructs training preprocess string.

Constructs training preprocess string.

Constructs training preprocess string.

Constructs training preprocess string.

Constructs training preprocess string.

Runs a single step of training. Given the state of the training and a batch of data, computes the loss and updates the parameters of the model.

Runs a single step of training. Given the state of the training and a batch of data, computes the loss and updates the parameters of the model.

Runs a single step of training. Given the state of the training and a batch of data, computes the loss and updates the parameters of the model.

Runs a single step of training. Given the state of the training and a batch of data, the train step computes the loss and updates the parameters

Returns trainer given its name.

Returns trainer given its name.

Gets the trainer matching the given name.

Returns the trainer to use.

Gets the trainer matching the given name.

Returns trainer given its name.

Returns trainer function given config.

Returns trainer function given config.

Transforms a [in_min,in_max] image to [vmin,vmax] range. Input ranges in_min/in_max can be equal-size lists to rescale the invidudal channels ind

Returns the visual genome train source.

Returns the visual genome train source.

Returns the visual genome train source.

Returns the visual genome train source.

Returns the visual genome train source.

Returns the visual genome train source.

Returns the visual genome train source.

Returns the visual genome train source.

Returns config for ViViT.

Computes the genotype concordance. Genotype concordance is the fraction of predicted genotypes that exactly match the call set genotype. Args:

Compute heatmap loss. Args: heatmaps: a single array in shape B x m x C. m = sum_l hl * wl. B is the batch size, C is the number of

Flip an image, boxes [xyxy un-normalized] (, and masks) horizontally.

Flip an image, boxes [xyxy un-normalized] (, and masks) horizontally.

Hungarian matcher for a single example.

Constructs a permutation matrix from the column and row indices of ones.

Embeds images into feature maps. Args: images: images of shape (batch, input_size, input_size, 3), scaled to the input range define

Vision task with val and test splits.

Random crop input image.

Random crop input image.

Generate detections from model outputs. Args: outputs: dict of list of arrays. The keys should be 'heatmaps' and 'box_regs'. Both s

The main evaluation loop. Run evaluation on the whole validation set. Args: flax_model: Flax model (an instance of nn.Module). train_state:

The main evaluation loop. Run evaluation on the whole validation set. Args: flax_model: Flax model (an instance of nn.Module). train_state:

Initialize bottleneck tokens from a pretrained model.

Initialize class embedding. The class embedding of the current model has a shape of (num_frames, channels). Args: to_params: Params of the

Initialize the first 3D conv parameters. Initializes the first 3D conv layer from an image model. Args: from_conv1: The 2D conv weights from

Initialize input embedding.

Initialize input embedding.

Initialize input embedding. Args: to_params: PyTree of model parameters that will be updated. This argument is modified by the function.

Initialize encoder_block_parameters.

Initialize encoder_block_parameters.

Initialize encoder_block_parameters.

Updates the train_state with data from restored_train_state (AViT).

Initialize train state from a MTV checkpoint.

Updates the train_state with data from restored_train_state (PolyViT).

Updates the train_state with data from restored_train_state. This function is writen to be used for 'fine-tuning' experiments. Here, we do so

Updates the train_state with data from restored_train_state.

Updates the train_state with data from restored_train_state. This function is writen to be used for 'fine-tuning' experiments. Here, we do so

Updates the train_state with data from restored_train_state. This function is writen to be used for 'fine-tuning' experiments. Here, we do so

Updates the train_state with data from restored_train_state.

Updates the train_state with data from restored_train_state.

Updates the train_state with data from restored_train_state.

Updates the train_state with data from restored_train_state. This function is written to be used for 'fine-tuning' experiments. Here, we do s

Updates the train_state with data from restored_train_state. This function is written to be used for 'fine-tuning' experiments. Here, we do s

Updates the train_state with data from restored_train_state. This function is writen to be used for 'fine-tuning' experiments. Here, we do so

Updates the train_state with data from restored_train_state. This function is writen to be used for 'fine-tuning' experiments. The input embe