Apply standard migration logic for a field. Priority order (highest to lowest): 1. Task-specific migration (experiment + task) 2. Ex

Apply migrations to normalize a row's structure based on experiment and task config.

Parse command line arguments.

Parse command line arguments.

Parse the file diff log content into structured data.

Parse the agent log file and extract structured information. Args: log_path: Path to the log file to parse Retu

Parse the Claude Code agent log file and extract structured information.

Read SQL from file or stdin.

Render a comparison result as a self-contained HTML page.

Resolve failing test names to (actual, expected) relation pairs.

Save the diff log to file.

Scan task.yaml files and return matching tasks. Args: task_ids: Only include tasks whose task_id is in this list. db_

Send a terminal notification when a run completes. Uses OSC 9 (iTerm2/Windows Terminal) and OSC 777 (Gnome Terminal/VTE) for desktop notifica

Set the variant configuration for the current task.

Setup base files - copy setup files and run scripts.

Setup dbt project by copying project files. Returns: Tuple of (success, error_message). error_message is empty string on success.

Setup DuckDB by copying the database file. Returns: Tuple of (success, error_message). error_message is empty string on success.

Setup migration by copying migration files.

Setup Snowflake by cloning database and creating user/role. Args: terminal: Terminal instance session: Tmux session task_

Setup a task for the given variant.

Check if file should be excluded from processing.

Check if file should be included in processing.

Stop docker compose services but keep containers alive for debugging.

Remove ANSI color codes from text.

Update dbt_project.yml and profiles.yml files based on variant configuration.

Update a specific task's row from log data dictionary.

Wipe the contents of the sandbox directory.

Write benchmark results to a TSV file. Args: results: BenchmarkResults object containing all trial results output_path: Path

Initialize the DuckDB extractor. Args: source_dir: Path to directory containing DuckDB files. If None, uses default.

Runs the Terminal-Bench harness. Args: dataset_path: The path to the dataset. output_path: The path to the o

Initialize the scanner. Args: tasks_dir: Path to the directory containing task subdirectories.

Initialize the dataset. Args: dataset_path: Path to the dataset directorygs task_ids: Optional list of specific task

Initialize DbtParser with explicit parser type. Args: parser_type: Either "dbt" or "dbt-fusion" to specify which format

Initialize the AsciinemaHandler. Args: markers: List of tuples containing (timestamp, label) for each marker

Iterate over the tasks in the dataset.

Get the number of tasks in the dataset.

Human-readable representation for logging.

Custom error formatter that adds a blank line before the error output.

Environment variables to use when running the agent (e.g. ANTHROPIC_API_KEY).

Execute a single task and return its results. Args: task_path: Path to the task directory task_key: Optional key for

Format time in seconds to a readable string.

Script to install the agent in the container.

Use a local or default Macro install script depending on environment.

Normalize mixed str/dict solution_seeds from YAML into dicts.

Parse Macro agent output using the Macro parser.

Parse Claude agent output to extract metrics.

Parse Gemini agent output to extract metrics.

Parse Codex agent output to extract metrics.