Open a specific run in your browser.

Run the benchmark with all configured plugin sets.

Helper: create a parquet file from a SQL query.

Execute a command in the tmux session. Args: keys (str): The keys to send to the tmux session. block (bool):

Load the plugin sets configuration from YAML.

Clean text for spreadsheet compatibility.

List recent benchmark runs.

Configure MCP servers after agent installation.

Create a mock TmuxSession with configurable exec_run results.

Resolve which plugin sets to use for a run. Args: plugin_set_names: Explicit plugin set names from --plugin-set, or None for defa

Capture a snapshot of files in the container directory using tar archive.

Generic function to copy files or directories.

Check if a file path should be excluded from diffing.

Install skills from the plugin set into the container. Args: plugin_set: The plugin set configuration terminal: The D

Check if this plugin set is compatible with the given agent.

Kill the tmux session and all processes running in it.

Remove line breaks and normalize whitespace for display.

Helper to copy a configuration file to the container.

Load a single task, applying all filters and key selection logic.

Run a docker-compose command with the appropriate environment variables.

Extract a comma-separated unique set of a variant field.

Generate all HTML pages for the experiment.

Get a plugin set by name.

Resolve all timeouts for a task, applying overrides as needed. Task-specific overrides take precedence over global config.

Execute multiple SQL queries separated by semicolons.

Get the git repository root directory. This handles both regular repos and worktrees by finding the actual repository root, not the w

Find a database file by name and type.

Create diff from dictionary.

Generate Snowflake credentials for a specific task (the user created during setup).

Get multiple plugin sets by name. Raises if any not found.

Handle file diffing operations for a specific phase.

Return True if all skills should be installed from this origin.

Generate a JSON summary of benchmark results.

Convert diff to dictionary for serialization.

Build a single TSV export row from a TaskInfo and prompt.

Create a new table with current task data.

Fetch rows from a table as a list of dicts.

Get the most recent run directory that has results.

Get Snowflake connection using environment variables for setup operations.

Render a list of row dicts as an HTML table.

Save the database catalog to disk.

Save a run to the saved directory. Returns True if successful.

Truncate text in the middle with ellipsis to fit available space.

Wrap content in a full HTML page with VS Code dark theme styling.

Write a detail page using the template.

Write a single marker event to the output file.

Create a diff between two snapshots.

Create a new tmux session with the given name.

Flip IS_SUPERHOST for given host IDs. UPDATED_AT stays at 1980-01-01.

Flip IS_SUPERHOST for given host IDs and set UPDATED_AT.

Format tool result output, limiting length.

Format a single trial result with calculated statistics. This is a shared utility used by both the TSV writer and summarize_results. Ar

Generate HTML content from parsed diff phases.

Get file content by hash.

Get all plugin sets marked as default.

Determine the failure type based on the result. Returns: - Empty string if test passed - "eval_error" if dbt compile passed

Get file content from optimized JSON data structure.

Get file content by path.

Find the most recent experiment directory that has results.json.

Get the path to a specific migration directory.

Get the number of snapshots captured.

Get Snowflake connection using environment variables.

Render a file diff log as HTML. Args: diff_log_path: Path to the file_diff_log.txt file task_id: Optional task ID to include in t

Setup agent-specific configuration files and resources.