The result of converting a document to text.

Main class for checking benchmark progress

Generic benchmark evaluation result structure

Generic benchmark task data structure

Generic benchmark evaluator for JSONL format

Class to maintain a persistent Playwright MCP session.

A custom version of markdownify's MarkdownConverter. Changes include: - Altering the default heading style to use '#', '##', etc. - Remo

Generator for final answers with context management support. Handles the generation of final answers, failure summaries for retry, and v

Custom formatter that adds colors for better developer visualization

Main class to run a benchmark

Summary statistics across all runs

Statistics for a single task

Grader for GAIA-Text-103 tasks using LLM judgement

Copy GAIA-Text-103 tasks from GAIA validation logs

Result of grading a single task

Main orchestrator for coordinating agent task execution. Manages the execution loop for main and sub-agents, coordinating LLM calls, too

Formatter for processing and formatting agent outputs.

Record detailed information of task execution steps

Handler for streaming events in SSE protocol format. Manages the sending of various event types including workflow lifecycle, agent life

Summary statistics across all runs

Statistics for a single task

Thread-safe async queue wrapper

Unified token usage tracking across different LLM providers. We unify OpenAI and Anthropic formats. There are four usage types: - input/

Executor for tool calls with support for duplicate detection and result processing. Handles the execution of tool calls, including parameter

Class for analyzing trace JSON files, convenient for reading and accessing important information Supports two tool call formats: 1. Old

Abstract base class for LLM provider clients. This class provides the common interface and utilities for interacting with different LLM

Abstract base class for benchmark evaluators

A wrapper class for error messages. Use this to wrap error messages that should be distinguishable from normal responses. Example:

Main class for checking GAIA benchmark progress

Record technical details of LLM calls

A wrapper class for responses with optional extra information. Use this to wrap responses that may include additional metadata. Example

Tests for basic message formatting without tools.

Integration tests for complete conversation flows.

Tests for edge cases and error handling.

Tests for <think> tag handling.

Tests for tool call formatting in assistant messages.

Tests for tool definition formatting in system prompt.

Tests for tool response handling.

Record detailed information of tool calls

this enables other kinds of tool manager.