Sample frames at specified fps interval. Trick: generate timestamp grids based on sampling_fps, then use _filter_frames_to_video to a

Sample frames at specified fps interval. Trick: generate timestamp grids based on sampling_fps, then use _filter_frames_to_video to a

Sample a single initial point from the first visible frame for the given point_id. Used for finding initial query point to track for

Sample a single initial point from the first visible frame for the given point_id. Used for finding initial query point to track for

Add additional cached-path clients, the enables files with the `weka://` prefix

Decodes RLE masks to binary mask

Fully shard this model using `fully_shard`

Collators for tensor the preprocessor produces

Build coordinates for a single image

Compute cosine similarities between pooled patches. Args: x (torch.Tensor): Input tensor of shape (B*T, N, D). pooled_patches_idx

Converts old monolithic model configs into the new format This also include some old backwards-compatibility fixes that are no longer needed

Evaluate predictions with both spatial metrics (P/R/F1) and tracking metrics (HOTA).

Find sharded checkpoints within a directory.

Format input timestamp as text

Format output timestamp as text For output timestamps, the model should always use a consistent format

TODO [QUESTION]: Merge with `format_timestamps` in DataFormatter? Format time value for model input/output. Args: time_value: Ti

Remove all pruned tokens found with the specified heuristic and rearrange them to reduce total batch size. Default heuristic

Get all dataset classes

Return the subset of `video_fps` factors that remain multiples of `sampling_fps`. Examples: >>> get_candidate_target_fps(video_fps=6

Run a query through an OpenAI model

Load the metadata from a checkpoint. :param dir: The path/URL to the checkpoint.

Get the size in bytes of element of the given PyTorch dtype.

Gets the default evaluator for task `name`

Convert video point text to coordinates Args: text: The text containing video points video_w: The width of the video vide

Get the response from the LLM model. Args: prompt (str): The prompt to send to the model. openai_api_key (str): OpenAI API key.

Get a dictionary of model names to their classes

Build a list of strings of what a user might type in to the model for the given example, and its responses, by applying a prompt template to t

Build a list of strings of what a user might type in to the model for the given example, and its responses, by applying a prompt template to t

Sample `Gumbel(phi)` conditioned on the maximum value being equal to `T`. `phi` should have shape `(batch_size, num_classes)` and `T

Initialize the right client for the given remote resource. This is helpful to avoid threading issues with boto3.

Interpolate frame scores to a target number of frames using bilinear interpolation. Args: scores: Original scores array, sampled at

Check if in a distributed context.

Check if the point (t, x, y) is within the region defined by the boolean mask. Parameters: - point (tuple of floats): (t, x, y) coordina

Get all binary masks for frame from HF gt_masks. Args: gt_masks: {'mask_id': } frame_idx: Frame index to get masks for

Load model state in-place for unsharded chechpoint saved in `dir`, works for sharded and unsharded models