Get the rank within the process group.

Get the appropriate video path, preferring extracted clips when available.

Selects a multiple-choice option based on the model output The output is should exactly match one of the option, but contains some heuri

Build a batch by interleaving the text tokens from tokenizing `message_list` and the multi-modal tokens from `multi_model_tokens`

Get cache file path for a video directory.

Turns an image into a string that can be used as a src in html images

Check if a local or remote directory is empty. This also returns true if the directory does not exist. :param dir: Path/URL to the direc

Download HF annotations and create videos for each source. Args: sources: list of source names to download, or None for all.

Gets the default evaluation config for task (or task:split string) `name`

Get a human-readable prompt by compressing the image tokens

Save to a YAML file.

Flatten *dims* into a single dimension called *name*. :param device_mesh: The world-mesh object. Only views of *device_mesh* are actually mu

Helper to normalize and round points.

Select a normalized label and find its human-readable equivalent.

Clear out the contents of a local or remote directory. .. warning:: This function is potentially very destructive! By defau

Computes the hash of a string.

Get the data parallel process group associated with a ``DeviceMesh`` created from :func:`build_world_mesh()`. Like :func:`get_dp_mesh()`

Build a HTML element containing `image_src` and the boxes in `boxes` on top of it. Provides a way to draw annotated images without have to load/m

Reshape the mask to match the pooled patches index. Args: mask (torch.Tensor): Input tensor of shape (B*T, N). pooled_pa

Upload source file to a target location on GCS or S3. :param source: Path to the file to upload. :param target: Target URL to upload to.

Wait for the condition function to return True.

Convert HF row to message_list format matching TrackingDataset.

Get scale factor from example.

Get scale factor from example.

Parse timestamp string like '00:00:01.089' to seconds.

Encode GT mask PNGs to COCO RLE and save as individual JSON files per query. Saves to: VIDEO_HOME/{data_split}/MasksRLE/{video_id}/{query_id}

Building pointing output using the pointing string

Building pointing string using the coordinate string

Build a preprocessor that converts 'raw' image/text data from various tasks into tensors inputs/targets that can be passed to the mod

Tokenizer this model uses

Tokenizer this model uses

Groups a pixmo datasets so each row contains all annotation for one image, and add images path using `url_to_path`, removing rows that do not

TODO [QUESTION]: Merge with `format_timestamps` in DataFormatter? Format time value for model input/output. Args: time_value: Ti

Format video points for counting objects across frames.

Generate a unique ID.

Get a range of bytes from a local or remote file. :param source: Path/URL to the file. :param bytes_start: Byte offset to start at.

Run a query through an OpenAI model

Get a human-readable string representation of a ``DeviceMesh``. :param device_mesh: The device mesh to get info for.

Load a video frames by decoding all frames with pyav More robust than `load_video_decord` but can by much slower for long videos

Post-process the output of the model to decode the text. Args: generated_outputs (`torch.Tensor` or `np.ndarray`):

Returns an actual path for local or remote file, potentially downloading it if a copy doesn't exist locally yet.