Aggregate accuracy for a specific data source.

Build the textual prompt: modality instruction + question + choices + post_prompt.

Build shape prompt matching original codebase format. Options list mixes shape candidates + scene candidates (simple + complex). task_typ

Helper function to compute per-domain statistics Args: results: List of dicts with 'score' and 'domain' fields metric_key: N

Create a zero-score result dict

Decode an AudioDecoder object or audio dict to a standard dict format.

Auto-detect environment activation command. Builds: cd <repo_root> && source .venv/bin/activate Returns empty string if no venv found.

Convert a PIL Image to RGB, handling None and dict (lazy HF Image) gracefully.

Robust extraction: 1) Try <answer>...<\answer> (case-insensitive, flexible whitespace) 2) If not found, try to extract last n

Extract a count number from text. Handles thinking model outputs by stripping <think>...</think> blocks, then tries \\boxed{X} format first,

Shared logic: populate score buckets and per-question-type scores from results.

Format seconds into human-readable time string.

Match GT to predicted elements using Hungarian algorithm. Returns list of (gt_idx, pred_idx, normalized_edit_distance) for valid matches.

Normalize LaTeX formula for edit distance comparison.

Normalize all images to have consistent dimensions. This ensures that the image processor can create homogeneous arrays.

Normalize a 1..max_score score to a 0..100 scale (matching summarize.py).

Normalize text for edit distance: lowercase, strip non-alnum (keep CJK), collapse whitespace.

Try to parse a single answer.

Process the text prompt and the input medias when the media files contain both image and video. In this case, sample frames from the video an

Remove model special tokens from the prediction.

Format refs and preds for sacrebleu corpus calculation. It is very particular

Stage 1: Generate auxiliary visualization image. Uses original image + prompt + question to generate an annotated / auxiliar

Extract sub-category scores from metric_stats.

Create a filtering pipeline.

Calls the OpenAI API's chat completion endpoint with exponential backoff.

Normalize gen_kwargs for consistent hashing. Only includes keys that affect generation output. Coerces ``0.0`` to ``0`` so float/int variants

Args: results: a list of values returned by process_results Returns: A score

Remove common noise characters from text.

Classification-gated F1 = 100 * pmF1 * IL_MCC.

The current version only supports non-overlapping segments. You need to modify the timestamp computation to support overlapping segments.

Convert Dictionary into Tree Example: input(dict) { "menu": [

Create a graph representation from a cognitive map. Args: cogmap: The cognitive map dictionary Returns: A directed graph (Di

Method for creating a (potentially) sliced and limited iterator from a raw document iterator. Used for splitting data among ranks in mult

Decrypt text fields in a single sample using the provided canary password. Args: sample: Dataset sample with encrypted fields

Return the device.