Human-readable explanation of this confidence level.

Download the dataset files into self.data_dir.

Encodes an image file into a base64 string.

Evaluate how 'correct' the candidate answer is relative to the gold_answer.

Evaluate the exact match between the ground truth and candidate strings. Args: ground_truth (str): The ground truth string.

Extract the application name(s) as a comma-separated string. .. deprecated:: Use :func:`extract_apps` instead.

Compute F1 score between one or more gold answers and a prediction string. If `gold_truths` is a single string, it is automatically converted

Return actions from an event stream.

Return an ordered mapping of every error code → display name. Walks every top-level category in ``error_taxonomy_analysis.md`` and extracts e

Get the full instruction including follow-up tasks from user messages.

Return the content of the most recent UserMessage, or None if absent.

Returns a list of examples that belong to the specified set (e.g. 'dev', 'test', or 'gaia').

Handle captcha detection and solving.

Decorator to handle TargetClosedError and tunnel connection errors by attempting to recover the page. Args: max_retries: Maximum num

Enhanced decorator that can also handle browser context recreation. Use this for critical operations where you have access to the browser con

Initialize worker process with global system and benchmark

Initialize worker process with global system

Loads the data from a JSONL file and the references from a JSON file. Matches entries by: - item["web_name"] (e.g. 'Allreci

Get the current page.

Get the playwright controller.

Replace a string containing a URL with just the netloc.

Unwrap an optional value, raising if None.

Run the rubric verification pipeline (Steps 0–8). Reads the :class:`DataPoint` from ``run_context.data_point``. Returns a list with