Stop and remove the docker compose services.

Read file from container, apply update function, write back to container.

Calculate total experiment duration.

Clone a Snowflake database. Returns: Tuple of (success, error_message). error_message is None on success.

Inner comparison logic operating on an open DuckDB connection.

Compute the differences between the two snapshots.

Copy task data as TSV to the system clipboard.

Copy TSV files as plaintext.

Create a fresh agent for a specific task. Args: task_id: The ID of the task to create an agent for. Returns:

Create the final status message based on parsed results. The PASS/FAIL logic here mirrors harness._is_resolved() exactly: -

Create user and role for the database. Returns: Tuple of (success, error_message). error_message is None on success.

Pre-scan: try excluding each column from EXCEPT ALL to find systematic diffs. If excluding a single column recovers >threshold of rows as exact m

Detect columns that differ in >= threshold fraction of paired rows. Returns a dict of column_name -> {diff_count, total_paired, sample_values}.

Disable console handlers to prevent double printing.

Execute multiple SQL queries separated by semicolons.

Execute all tasks in parallel and collect their results.

Execute trials for the current skill set. Returns: BenchmarkResults: The results of the trials.

Extract comparison artifacts from the container after test execution.

Extract CSV files and schema file from the database after agent execution.

Extract CSV files from DuckDB database.

Extract table name from Parquet filename.

Extract schema information from a database table.

Format duration in seconds to H:MM:SS format.

Fuzzy-match unmatched rows by similarity score.

Generate data comparisons detail page from diff.html artifacts.

Generate diffs detail page.

Generate panes detail page. Displays terminal output in chronological order: 1. pre-agent.txt (setup before agent runs) 2. HT

Generate results.json detail page.

Generate solution tests for tables specified in solution_seeds. Args: trial_handler: The trial handler containing task configurat

Generate the main summary page.

Generate detail pages for a specific task.

Extract base task ID from variant task ID. e.g., 'foo.hard.1-of-1' -> 'foo' 'foo.base.2-of-3' -> 'foo' 'foo' -> '

Get dynamic environment variables for dbt MCP server.

Get the expected test count from the '[ade-bench] expected_test_count=N' line.

Get the current git commit hash.

Get the user who ran the experiment. First tries to get the git user.name, falls back to system username.

Check if the content contains actual test results (not just compilation errors).

Initialize or reinitialize file logging for the current run path.

Initialize console logging. File logging is initialized per plugin set.

Load and resolve plugin sets from configuration.

Install skills from a single skill origin. Args: skill_origin: The skill origin configuration terminal: The Docker co

Load all tasks from directory, filtering to only status=ready.

Load the database catalog from disk.

Load experiment data from results.json and run_metadata.json.

Load tasks from an experiment set file.

Load specific tasks from the task_ids.

Load task.yaml contents for each task and return as a dictionary.

Load all tasks that start with the given prefix.

Parse the agent output to extract metrics. Override in subclasses if needed.

Parse the JSON response data to extract metrics.

Parse the JSON response data to extract metrics.

Calculates 1 - comb(n - c, k) / comb(n, k).

Prepare keys for sending to the terminal. Args: keys (str | list[str]): The keys to send to the terminal. bl

Process a single line from the recording, inserting markers as needed.

Rebuild the table with current task data.

Remove tasks that are in the excluded list.

Commands to run the agent with the given task prompt.

Return the command(s) that will run the Macro agent inside the container.

Run comprehensive setup including files, migration, and scripts.

Run tests with proper timeout management.

Save a human-readable summary of file changes.

Save detailed diff information to the trial log file.

Save the most recent run with results.

Check if two values match, with numeric tolerance for floats.

Write a new recording file with markers merged in at the correct timestamps.

Write the panes page with pre-built HTML content. Unlike _write_detail_page which escapes content, this method accepts raw HTML that

Write any remaining markers that come after all recorded events.

Add files to exclusion list.

Add file content and return its hash.

Analyze schema and tables in a DuckDB file.

Analyze schema and tables in a DuckDB file.

Check ANTHROPIC_API_KEY.

Check GEMINI_API_KEY.

Check MACRO_API_KEY.

Check OPENAI_API_KEY.

Check if task exists in tasks directory.

Compare two parquet files and return a structured diff.

Convert all discovered DuckDB files to Snowflake databases.