Handle an incoming message. Args: message: Message to handle session: Server session resource_ow

Create a new task. Args: context_key: Authorization context key scoping this task. ttl: Requested TTL in ms. None mea

Add tool definitions directly from MCP-style dictionaries. Args: tools: List of tool definitions. Each must have:

Add a new evaluation case to the suite. Args: name: The name of the evaluation case. user_message: The user'

Run the evaluation suite. Args: client: The LLM client instance (AsyncOpenAI or AsyncAnthropic). model: The

Handle CLI error reporting with optional debug traceback and exit.

Analyze a function to create corresponding Pydantic models for its input and output.

Escape HTML special characters.

Add a function to the catalog as a tool. Args: tool_func: The function to add to the catalog. toolkit_or_nam

Create a mock CaptureResult for testing.

Create mock evaluation results structure.

Set up a session with 2025-11-25 and full tasks capability.

Add a tool to the registry. Args: tool_descriptor: MCP-style tool definition. materialized_tool: Optional Materialize

Atomically update task status. - If current is terminal and new==current: idempotent no-op. - If current is terminal and different: r

Escape HTML special characters.

Add track-specific configuration. Args: track_name: The track name (must be registered via add_*_tools). expected_too

Add a critic result to the list of critic results. Args: field: The field name for the critic result. result

Override invoke to track command execution.

Get a tool from the catalog by fully-qualified name and version. If the version is not specified, the any version is returned.

Given a tool function, create a ToolDefinition

Helper for canonical_url tests: minimal valid kwargs.

Format evaluation results as JSON.

Handle tool call request.

Capture event in a detached subprocess that is non-blocking. Spawns a completely independent subprocess that continues running even a

Find a tool by its function.

Create a comparative case that runs against multiple tool tracks. Use .for_track() on the returned builder to configure track-specific

Retrieve the secret for the tool invocation. Raises a ValueError if the secret is not found.

Send initialize + notifications/initialized, return init result.

Create a fresh session with mock streams.

Convert OpenAI-format messages to Anthropic format. Anthropic only supports "user" and "assistant" roles (system is a separate parameter).

Create a mock EvaluationResult with the specified properties.

Serialize ``data`` as JSON (indent=2) and atomically write to ``path``.

Register a resource at build time (before server start). If the URI contains ``{`` it is treated as a URI template and a :class:`Reso

Create settings from environment variables. Automatically discovers and loads .env file by traversing upward from the current directo

Create a copy with updated fields.

Create an MCP-compatible tool definition from a MaterializedTool. Computes MCP annotations from tool metadata behavior fields and builds

Load or create usage.json file with atomic writes and file locking. Returns: dict: The usage data containing anon_id and optional

Create mock comparative evaluation results structure. Args: model: The model name. suite_name: Base suite name (will have track s

Parse output format string into a list of formats. Supports: - Single format: "md" -> ["md"] - Comma-separated: "md,html" -> ["md",

Convert a MaterializedTool to Anthropic tool schema format. Args: tool: The MaterializedTool to convert Returns: The Anthrop

Create a minimal Starlette Request with given headers.

Validate JWT and return authenticated resource owner. Always validates (cannot be disabled): - Token signature using JWKS public key

Get a task by ID, scoped to context. Raises NotFoundError for missing task OR context mismatch (no info leak). Performs an O(1) TTL

Set the session for this context.

Start the task manager. Spawns a periodic TTL-cleanup task so that expired tasks are removed from memory even without explicit access

Create an error response.

Create a mock APIError following Microsoft Graph error structure: { "error": { "code": "string", "m

Handle non-API Slack SDK errors.

Map Slack SlackApiError to appropriate ToolRuntimeError.

Store a template together with its handler and compiled regex. Warns at registration time if the new template overlaps with any exist

Display evaluation results in a format inspired by pytest's output. Args: results: List of dictionaries containing evaluation result

List tasks owned by a context key with deterministic pagination. Contract: - Ordering: ``createdAt`` descending (newest first), with

Send data to stdout.

Stop the task manager and cancel all background tasks.

Create a mock errors module with all necessary error classes.

Cancel a task. Raises NotFoundError or InvalidTaskStateError. Performs the same O(1) TTL check as ``get_task`` so an expired task can

Compare the tool names by replacing all separators with the TOOL_NAME_SEPARATOR and comparing the normalized names. Converts names like

Configure an MCP client to connect to an Arcade Cloud gateway. If *auth_token* is ``None`` the config contains only the URL and the MCP clien

Convert a Python value to MCP-compatible content.

Evaluate the actual tool calls against the expected tool calls and critics. Args: actual_tool_calls: A list of tuples co

Returns True if the evaluation failed (excluding warnings).