Print JSON data in a readable format.

Process a user query and return tool calls made.

Record an error occurrence. Args: operation_name: Name of the operation that failed exception: The exception

Reload configuration from file and environment.

Request variant effect prediction.

Search for locations using WeatherAPI

Search for clinical trials using NCI CTS API. Returns: Dictionary with: - trials: List of trial records - total: Tot

Search Wikipedia using the MCP server.

Set tool call result cache Args: server_name: Server name tool_name: Tool name params: T

Store a response in the cache. Args: endpoint: The API endpoint URL params: Query parameters response: Response data

Set the current search context.

Starts the HTTP MCP server process with automatic port conflict resolution.

* Subtract one number from another * @param minuend - The number to subtract from * @param subtrahend - The number to subtract * @retur

Convert paper to dictionary format for serialization

Convert to standard ResultItem format.

Validate gene symbol with caching.

Validate a recall record. Args: recall: Recall record Returns: True if valid Raises: OpenFDAValidationErro

Wait if rate limited before allowing request.

Initialize the LLM judge. Args: llm_provider: LLM provider for evaluation enable_judge_stability: Whether to

Initialize retry configuration. Args: max_attempts: Maximum number of retry attempts initial_delay: Initial delay bet

Initialize circuit breaker. Args: name: Circuit breaker name for logging config: Configuration (uses defaults if not

Initialize rate limiter. Args: requests_per_second: Sustained request rate burst_size: Maximum burst capacity

Add CIViC link if id is present.

Add ClinVar link if variant_id is present.

Add COSMIC link if cosmic_id is present.

Add dbSNP and Ensembl links if rsid is present.

Add helpful links to disease result.

Add external database links for the drug.

Add advanced eligibility criteria parameters.

Add HGNC link if gene name is present.

Add location parameters if present.

Add UCSC Genome Browser link if chromosome and position are present.

Aggregate mutations by study.

Aggregate mutation data into summary statistics. Args: matching_mutations: List of mutations matching the query criteria

Recursively aggregate nested metrics

Apply environment variable configuration overrides. Scans environment variables starting with 'BENCHMARK_' and applies them t

Basic fallback generation

Timer that flushes the batch after timeout.

Convert binary representation to traditional hexagram number. Args: binary_string: 6-character binary string rep

Build query parameters for biomarker search.

Build the search query for device events.

Build query parameters for disease search.

Build the field registry with all searchable fields.

Build authorization headers if token is available.

Build the complete 64-hexagram database. Returns: Dictionary mapping hexagram numbers to their attributes

Build query parameters for intervention search.

Build the traditional King Wen sequence mapping binary to hexagram numbers

Build query string from structured request. Note: Preprint servers use plain text search, not PubMed syntax.

Build the search query for adverse events.

Build standardized task result

Calculate average scores from multiple evaluations

Calculate metrics about server utilization and coordination

Calculate tool selection and execution accuracy metrics

Calculate the end position for UCSC Genome Browser link.

Call tool using HTTP transport.

Call tool using HTTP transport.

Call tool using STDIO transport.

Check which success criteria were met.

Check ast.ImportFrom node for violations.

Check ast.Import node for violations.

Check if parameters comply with tool schema

Helper method to cleanup individual HTTP server with error handling.

Collect patient outcome information.

Connects to an HTTP MCP server.

Connects to an HTTP MCP server.

Connects to a single server and maintains the session.

Connects to a single server and discovers its tools.

Connects to a STDIO MCP server.

Creates and maintains a persistent STDIO connection using AsyncExitStack.

Convert environment variable values to appropriate types. Args: value: String value from environment variable

Create a comprehensive summary of task execution including accumulated information

Create benchmark runner and get available models

Create a deep copy of a dictionary, handling nested dicts and lists.

Delete a nested path from a dictionary using dot notation.

Determine which models to test based on arguments

Discover available NixOS channels by testing API patterns.

Retrieves detailed information for a disease from MyDisease.info. This tool provides real-time disease annotations including: - Official

Get drug/chemical information from MyChem.info. This tool retrieves comprehensive drug information including: - Drug identifiers (DrugBank, C

Evaluate which expected behaviors were observed.

Evaluate task execution result and format output

Executes a list of planned tool calls, handling sequential-only tools separately.

Expand disease terms with synonyms if requested.

Extract ChEBI fields from response.

Extract ChEMBL fields from response.

Extract DrugBank fields from response.

Extract mutation pattern from keywords asynchronously.

Extract PubChem fields from response.

Extracts plain text from a CallToolResult object.

Extract UNII fields from response.

Extract URL from disclaimer text

Extract year from publication info