MCPcopy Create free account

hub / github.com/dbt-labs/ade-bench / functions

Functions626 in github.com/dbt-labs/ade-bench

Functionsave_callback
Save ADE-bench results.
ade_bench/cli/ab/save.py:57
Functionsave_run
Save a run to the experiments_saved directory. If no run_id is provided, saves the most recent run with results.
ade_bench/cli/ab/save.py:92
Methodseeds_dir
(self)
ade_bench/handlers/trial_handler.py:342
Methodsessions_path
(self)
ade_bench/handlers/trial_handler.py:370
Methodset_variant_config
(self, config: dict)
ade_bench/agents/installed_agents/abstract_installed_agent.py:60
Methodsetup_method
(self)
tests/scripts/test_dump_tables.py:21
Methodsetup_timeout_sec
Timeout for setup operations (task setup scripts and agent installation) in seconds.
ade_bench/config.py:55
Methodshared_config_path
Path to the shared config directory.
ade_bench/handlers/trial_handler.py:456
Methodshared_databases_path
Path to the shared databases directory.
ade_bench/handlers/trial_handler.py:431
Methodshared_databases_root_path
Path to the shared databases directory in the main repository root. This always points to the main repo's databases, even when running
ade_bench/handlers/trial_handler.py:421
Methodshared_duckdb_path
Path to the shared DuckDB databases directory in the main repository.
ade_bench/handlers/trial_handler.py:436
Methodshared_migrations_path
Path to the shared migrations directory.
ade_bench/handlers/trial_handler.py:451
Methodshared_projects_path
Path to the shared projects directory.
ade_bench/handlers/trial_handler.py:446
Methodshared_snowflake_path
Path to the shared Snowflake databases directory in the main repository.
ade_bench/handlers/trial_handler.py:441
Methodshould_exclude_file
Check if file should be excluded from conversion.
scripts_python/migrate_duckdb_to_snowflake.py:81
Methodshould_include_file
Check if file should be included in conversion.
scripts_python/migrate_duckdb_to_snowflake.py:85
Methodskill_locations
Get list of skill locations as strings (for result tracking).
ade_bench/harness_models.py:277
Methodsolution_path
(self)
ade_bench/handlers/trial_handler.py:323
Methodsolutions_dir
(self)
ade_bench/handlers/trial_handler.py:346
Functionspin_up_container
( client_container_name: str, client_image_name: str, docker_compose_path: Path, docker_name_p
ade_bench/terminal/docker_compose_manager.py:202
Methodstop
Stop the live display and re-enable console handlers.
ade_bench/utils/logger.py:124
Methodstop
(self)
ade_bench/terminal/tmux_session.py:127
Methodstop_services_only
Keep containers alive for debugging without stopping services.
ade_bench/terminal/docker_compose_manager.py:145
Methodtask_id
(self)
ade_bench/handlers/trial_handler.py:204
Methodtask_ids
Get the list of task IDs in the dataset.
ade_bench/utils/dataset.py:166
Methodtask_prompt
(self)
ade_bench/handlers/trial_handler.py:214
Methodtask_prompt_dict
(self)
ade_bench/handlers/trial_handler.py:121
Methodtask_setup_dir_path
Path to the task's setup directory.
ade_bench/handlers/trial_handler.py:416
Methodtask_setup_script_path
Path to the task's setup.sh script.
ade_bench/handlers/trial_handler.py:411
Methodtasks
Get the list of tasks in the dataset.
ade_bench/utils/dataset.py:161
Methodtasks_dir
Return the tasks directory path.
ade_bench/utils/task_scanner.py:62
Functiontemp_output_dir
Create temporary output directory.
tests/handlers/test_file_diff_handler.py:40
Methodtest_all_rows_differ_in_one_column
When every row differs in one column, pre-scan detects it.
tests/scripts/test_compare_tables.py:152
Methodtest_all_rows_match
(self)
tests/scripts/test_compare_tables.py:73
Methodtest_both_empty
(self)
tests/scripts/test_compare_tables.py:245
Functiontest_capture_snapshot_basic
Test basic snapshot capture without exclusions.
tests/handlers/test_file_diff_handler.py:46
Functiontest_capture_snapshot_excludes_dotfiles_by_default
Test that hidden files (.*) are excluded by find command.
tests/handlers/test_file_diff_handler.py:84
Functiontest_capture_snapshot_with_exclusions
Test snapshot capture with path exclusions.
tests/handlers/test_file_diff_handler.py:64
Methodtest_column_comparison_case_insensitive
(self)
tests/scripts/test_compare_tables.py:62
Methodtest_column_diff_section
(self)
tests/scripts/test_compare_tables_html.py:113
Methodtest_compact_table_with_mixed_diffs
Rows with non-systematic diffs show in a compact table.
tests/scripts/test_compare_tables_html.py:76
Functiontest_container
Create a test container with sample files.
tests/handlers/test_file_diff_handler.py:18
Functiontest_create_diff
Test diff creation between snapshots.
tests/handlers/test_file_diff_handler.py:127
Methodtest_dbt_server_by_args
dbt MCP server detected by 'dbt-mcp' in args.
tests/agents/installed_agents/test_abstract_installed_agent.py:138
Methodtest_dbt_server_by_name
dbt MCP server detected by server name 'dbt'.
tests/agents/installed_agents/test_abstract_installed_agent.py:102
Methodtest_dir
(self)
ade_bench/handlers/trial_handler.py:338
Methodtest_dump_empty_table
(self)
tests/scripts/test_dump_tables.py:86
Methodtest_dump_multiple_tables
(self)
tests/scripts/test_dump_tables.py:45
Methodtest_dump_nonexistent_table
(self)
tests/scripts/test_dump_tables.py:58
Methodtest_dump_single_table
(self)
tests/scripts/test_dump_tables.py:31
Methodtest_dump_with_row_limit
(self)
tests/scripts/test_dump_tables.py:69
Methodtest_env_file_write_failure_logs_warning
Non-zero exit on env file write logs warning but continues.
tests/agents/installed_agents/test_abstract_installed_agent.py:196
Methodtest_env_var_emits_config_flag
When set, the value is passed via -c model_reasoning_effort=<value>.
tests/agents/installed_agents/test_openai_codex_agent.py:16
Methodtest_env_var_value_is_shell_quoted
Value goes through shlex.quote so injection attempts are inert.
tests/agents/installed_agents/test_openai_codex_agent.py:24
Methodtest_expected_empty_actual_has_rows
(self)
tests/scripts/test_compare_tables.py:263
Methodtest_extra_column
(self)
tests/scripts/test_compare_tables.py:52
Methodtest_extra_rows
(self)
tests/scripts/test_compare_tables.py:100
Methodtest_fuzzy_cap_exceeded
When unmatched rows > fuzzy_row_limit, skip fuzzy matching.
tests/scripts/test_compare_tables.py:135
Methodtest_generates_valid_html
(self)
tests/scripts/test_compare_tables_html.py:28
Functiontest_get_repo_root_from_main_directory
Test finding repo root from main directory. This test verifies that _get_repo_root finds the ade-bench repository root, not the task's tempor
tests/test_trial_handler.py:5
Functiontest_get_repo_root_from_worktree
Test finding repo root from worktree. This verifies that when code runs from a worktree, _get_repo_root correctly finds the main repository r
tests/test_trial_handler.py:40
Methodtest_handles_error_status
Tests with status 'error' should also be detected.
tests/scripts/test_detect_failing_equality_tests.py:58
Methodtest_identical_columns
(self)
tests/scripts/test_compare_tables.py:28
Methodtest_ignores_model_nodes
(self)
tests/scripts/test_detect_failing_equality_tests.py:49
Methodtest_ignores_non_equality_failures
(self)
tests/scripts/test_detect_failing_equality_tests.py:40
Functiontest_loader_file_not_found
()
tests/plugins/test_loader.py:23
Functiontest_loader_loads_yaml
(tmp_path)
tests/plugins/test_loader.py:7
Functiontest_loader_resolve_plugin_sets_defaults
When no --plugin-set, use defaults.
tests/plugins/test_loader.py:51
Functiontest_loader_resolve_plugin_sets_error_on_incompatible_explicit
Error when explicitly requested plugin set is incompatible.
tests/plugins/test_loader.py:106
Functiontest_loader_resolve_plugin_sets_error_when_none_compatible
Error when no plugin sets are compatible with agent.
tests/plugins/test_loader.py:125
Functiontest_loader_resolve_plugin_sets_explicit
Explicit --plugin-set names are resolved.
tests/plugins/test_loader.py:29
Functiontest_loader_resolve_plugin_sets_filters_incompatible
Plugin sets incompatible with agent are filtered out.
tests/plugins/test_loader.py:77
Methodtest_mcp_add_failure_logs_warning
Non-zero exit on mcp add logs warning but doesn't raise.
tests/agents/installed_agents/test_abstract_installed_agent.py:217
Functiontest_mcp_server_config_minimal
()
tests/test_plugin_set.py:5
Functiontest_mcp_server_config_with_env
()
tests/test_plugin_set.py:12
Methodtest_missing_column
(self)
tests/scripts/test_compare_tables.py:42
Methodtest_missing_relation_html
(self)
tests/scripts/test_compare_tables_html.py:43
Methodtest_missing_rows
(self)
tests/scripts/test_compare_tables.py:87
Methodtest_mixed_columns_prescan_and_postscan
Pre-scan catches the column-wide diff; post-scan catches sporadic diffs.
tests/scripts/test_compare_tables.py:178
Methodtest_no_env_var_omits_flag
When OPENAI_CODEX_REASONING_EFFORT is unset, no -c flag is emitted.
tests/agents/installed_agents/test_openai_codex_agent.py:9
Methodtest_no_failures
(self)
tests/scripts/test_detect_failing_equality_tests.py:31
Methodtest_no_match_beyond_tolerance
(self)
tests/scripts/test_compare_tables.py:125
Methodtest_no_mcp_servers
No calls when mcp_servers is empty.
tests/agents/installed_agents/test_abstract_installed_agent.py:37
Methodtest_no_systematic_when_below_threshold
Columns differing in <90% of rows are not systematic.
tests/scripts/test_compare_tables.py:200
Methodtest_not_in_manifest
(self)
tests/scripts/test_detect_failing_equality_tests.py:91
Methodtest_null_vs_value
(self)
tests/scripts/test_compare_tables.py:233
Methodtest_nulls_match
(self)
tests/scripts/test_compare_tables.py:225
Methodtest_numeric_tolerance
(self)
tests/scripts/test_compare_tables.py:114
Methodtest_one_failing_equality
(self)
tests/scripts/test_detect_failing_equality_tests.py:19
Functiontest_plugin_set_full
()
tests/test_plugin_set.py:42
Functiontest_plugin_set_is_compatible_with_agent_all
When agents is None, compatible with all agents.
tests/test_plugin_set.py:59
Functiontest_plugin_set_is_compatible_with_agent_restricted
When agents is set, only compatible with listed agents.
tests/test_plugin_set.py:66
Functiontest_plugin_set_minimal
()
tests/test_plugin_set.py:31
Functiontest_plugin_sets_config_from_yaml
()
tests/test_plugin_set.py:73
Functiontest_plugin_sets_config_get_by_name
()
tests/test_plugin_set.py:111
Functiontest_plugin_sets_config_get_by_names
()
tests/test_plugin_set.py:123
Functiontest_plugin_sets_config_get_by_names_unknown_raises
()
tests/test_plugin_set.py:137
Functiontest_plugin_sets_config_get_defaults
()
tests/test_plugin_set.py:97
Methodtest_resolve_from_manifest
(self)
tests/scripts/test_detect_failing_equality_tests.py:70
Methodtest_script_paths
(self)
ade_bench/handlers/trial_handler.py:166
← previousnext →501–600 of 626, ranked by callers