Runs evaluations against a single train_state/step. Args: train_state: The train state being evaluated. step: The global step corresp

Creates a split from the S3dis dataset using TensorFlow Datasets. For the training set, we drop the last partial batch. This is fine to do becaus

Samples a bounding box in the [cx, cy, w, h] in [0, 1] range format.

Samples a bounding box in the [cx, cy, w, h] in [0, 1] range format.

Samples a bounding box in the [cx, cy, w, h] in [0, 1] range format.

Samples a bounding box in the [cx, cy, w, h] in [0, 1] range format.

Sample rotated bounding boxes [cx, cy, w, h, a (radians)].

Generate a random RGB vector (in [0,255]). Args: key: random key Returns: tuple: RGB color

Uniform frame sampling. Sample frames based on uniform sampling following TSN (Wang et al., 2019) used by Tong et al. in VideoMAE. The stride is

mfp sample_sequence_uniformly with state.

Uniform sampling two non-overlapping sequences. Sample frames based on uniform sampling following TSN (Wang et al., 2019) used by Tong et al. in

Implements random shuffling and partitioning necessary for MAE. Args: n_batch: The batch size of the sequence to generate. n_tokens: The nu

Sinusoidal stochastic positional encoding. Args: rng_key: A PRNGKey. key_shape: The shape of keys and queries. num_realizations: The nu

Computes sparse weighted softmax cross entropy give logits and targets. Args: logits: Logits of shape [batch_size, length, vocab_size]. lab

Stochastic Positional Encoding gating mechanism. Args: rng_key: A PRNGKey. spe_code: the code of the stochastic positional encoding mechani

Encode video for feeding into ViT.

Encode video for feeding into ViT.

Applies a function to a single image or each image in a batch of them. Args: fn: the function to apply, receives an image, returns an image.

Tokenizes timestamps. Args: start: Start times of the events. end: End times of the events. duration: Duration of the video. abs_ti

Returns only the token indexes to keep in a sequence of tokens.

Given the train_state, trains the model on fake batches.

Given the train_state, trains the model on fake batches.

Transform boxes given the state of image crop.

Populates tree_flatten with leaf names. This function populates output of tree_flatten with leaf names, using a custom traversal that produces na

Find the points where the circle intersects with the triangle.

Unflattens the first, flat batch*beam dimension of a non-scalar array.

Function to update stale embedding as memory. Args: pmap_train_state: pmaped train state. bsz: Global batch size. retr_k: numbe

Resize video to a fixed max-size.

Computes weighted softmax cross entropy give logits and targets. This computes sum_(x,y) softmax-ce(x, y) for a single, potentially padded miniba

Write to the given writer and training step. After writing, the state gets reset. Args: writer: The summary will be written with this

Writes predictions to file in JSON format. Args: path: Path to write the prediction annotation JSON file. fname_app: Optional string

Writes predictions to file in JSON format. Args: path: Path to write the prediction annotation JSON file. is_groundtruth: bool; if th

Writes predictions to file in JSON format. Args: path: Path to write the prediction annotation JSON file. is_groundtruth: bool; if th

Initialize Detection model. Args: config: Configurations of the model. dataset_meta_data: Dataset meta data specifies `target_is_oneh

Initializes the instance of MomentRetrievalDatasetFactory. Initializes a data-loader using DeepMind Video Reader (DMVR) pre-processing (https

Prepends CLS token. Args: x: A 3D float tensor of shape (batch, sequence_len, channels) representing the tokens. Returns:

Prepends CLS token. Args: x: A 3D float tensor of shape (batch, sequence_len, channels) representing the tokens. name: Parame

Prepends CLS token for all frames. Args: x: A 4D float tensor of shape (batch, time, sequence_len, channels) representing the token

Prepends CLS tokens for all views. Args: tokens: Tokens from all views. Each one has a shape of (batch, time, sequence_len, channel

Append random/uniformly sampled token from full_x to x. Args: full_x: batch x num_tokens x hidden_dim num_context_tokens: number of c

Adds frame labels and displacements for one pyramid level. Args: segment_start_indices: A int tensor of shape (max_num_segments,) containing

Adds frame labels and displacements for one pyramid level.

Adds positional embeddings.

Adds positional embedding.

Adds tokenized text to the input pipeline.

Reshapes tokens from each view so they have the same temporal dim.

Applies cross view attention.

Apply `func` to image w/ `args` as input with probability `prob`.

Applies MLP block.

Apply depthwise convolutions and layer norm.

Applies self attentions for each view.

Apply the i-th transformation.

A wrapper that picks assertion based on tensor dtype.

At least `x` of `a` and `b` `Tensors` are equal.

Averages prediction from different crops and prompts.

Aggregate batch statistics accross devices. Args: x: Inputs to compute batch statistics on. axis: Reduction axes for the stats. p_agg:

Formats an RGB image (range [0.0, 1.0], shape [H, W, 3]) for bokeh.

Computes box regression loss.

Reshapes a batch of per-tile boxes into boxes for a mosaic.

Broadcast and concatenate two tensors (_broadcat in original). Args: x: (b1, 1, d) y: (1, b2, d) Returns: ret: (b1, b2, d * 2)

Pass information from top level to bottom in a feature pyramid.

Builds the encoder with bottlenecks.

Builds the encoder with bottlenecks.

Ceil division of two JAX arrays.

Checks configuration errors.

Call close on all writers on exit.

Computes classification loss.

Combine sequence results via sum.

Combine sequence results via weighted average.

Computes the unnormalized per-example classification loss and denom.

Computes the unnormalized per-example classification loss and denom.

Convert attention parameters.

Convert attention parameters.

Converts the format of detections for evaluation. Args: result_lists: A dictionary holding lists of detections. num_results: Length of the

Converts the format of groundtruth segments for evaluation. Args: result_lists: A dictionary holding lists of groundtruth. num_results: Len

Convert yxyx [0, 1] normalized boxes to xyxy unnormalized format.

Convert torch parameters to flax parameters.

Convert torch parameters to flax parameters.

Dataset function.

Dataset function.

Dataset function.