Generate and return the statistics summary string.

Get statistics about loaded documents and chunks.

Get a summary of folder contents.

Get the trace ID for this error.

Handle max turns reached summary prompt

Parse tool_calls or <use_mcp_tool> tags from LLM response text. Returns a list containing tool call information.

Parse RAG retrieval information directly from the execution log. This extracts RAG tool calls and their results from the main execution

Process tool call results

Preview folder contents without running a task. Args: folder_path: Path to the folder to preview recursive: Whether to s

Process all files in a folder and prepare task description for LLM. This function: 1. Scans the folder and categorizes all files

Resolve query from a jsonl file if the query is a file path. If query ends with .jsonl, it reads the file and finds the matching query

Run batch evaluation across multiple cases.

Run a task on a folder containing multiple files. Args: cfg: Hydra configuration object folder_path: Path to the folder

Simplified interface to run a folder task with default config. Args: folder_path: Path to the folder to process query: T

Run sub agent Note: APIConnectionSkipError, APIRateLimitError, and APITimeoutError are propagated to the caller for proper ha

Safely parse a JSON string with multiple fallbacks: 1. Try standard json.loads() 2. If it fails, try json_repair 3. If it still fails

Save both original and final reports to a comparison file. Args: results_dir: Directory to save results folder_name: Nam

Save the final report to a markdown file with timestamp. Args: results_dir: Directory to save results folder_name: Name

Scan a folder and categorize all files. Args: folder_path: Path to the folder to scan recursive: Whether to scan subdire

Set the task logger for structured logging.

Use GAIA-style judge to verify if the predicted answer is correct.

Automatically select provider and create LLM client based on configuration

Converts PPTX files to Markdown. Supports headings, tables, images with alt text, and recursively extracts text from GROUP shapes. Args:

Converts Excel files to plain Markdown tables using openpyxl. Does NOT preserve color/style formatting to keep output concise. Args:

Extracts ZIP files to a permanent local directory and returns a listing of extracted files.

Initialize the benchmark run Args: cfg: Hydra configuration object

Apply cache control to the last user message and system message (if applicable)

Build the batch summary prompt.

Build the rerank prompt.

Call Anthropic Claude Vision API.

Call custom video API (OpenAI-compatible)

Analyze video by extracting key frames and using VQA model. This approach: 1. Extracts N evenly-spaced frames from t

Call Google Gemini API for video understanding

Call Gemini Audio API via OpenAI-compatible endpoint (supports audio_url format)

Call GPT-4o-audio API (best for complex audio understanding)

Call GPT-4o-video API

Call OpenAI-compatible endpoint (Qwen, custom, etc.).

Call OpenAI-compatible VLLM (GPT-4V, GPT-4o, etc.).

Call OpenAI Whisper API

Call Qwen-Audio API (supports both transcription and audio QA)

Call Qwen-VL-Video API

Call Open-Source Whisper API

Check if cache is valid for the current file.

Check which required titles are cited in the report.

Collect streaming response chunks into a complete response object (async version)

Collect streaming response chunks into a complete response object (sync version)

Compress a single item. Args: idx: Index of item to compress compress_func: Async function to compre

Display summary statistics

Display summary statistics

Encode image to base64 or return URL.

Extract all explicit citations from the report text. Supported formats: - [long_context: "..."] - [filename.md, page

Extract text content from a file using existing converters. Args: file_info: FileInfo object for the file max_content_le

Extract run number from directory path for sorting

Extract evenly-spaced frames from video and convert to base64. Args: video_path: Path to video file

Fallback compression using simple truncation

Filter message history, keep specified number of tool results

Find claims that failed verification in existing evaluation data.

Generate follow-up questions for multi-turn verification.

Get API key with priority: 1. Environment variable based on provider (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY) 2. Config file

Determine the appropriate audio file extension from URL or content type. Args: url: The URL of the audio file content_type:

Get audio information string.

Get base URL with priority: 1. Environment variable based on provider (e.g., OPENAI_BASE_URL, ANTHROPIC_BASE_URL) 2. Config f

Get SQLite database path for caching embeddings.