Container describing a single model generation outcome.

Request-scoped view over a shared dictionary with deferred commits.

A framework for managing and executing a network of nodes using a language model. This class enables the creation of a graph structure for p

Stopping criteria to capture time-to-first-token during generation.

Central coordinator for anchor-related KV cache interactions.

Tracks per-request agent outputs, reuse rates, and mode-specific TTFT.

Container tracking deferred mutations for a single request.

Research/analysis agent that can search wiki and compose context.

Programming agent that validates peers with internal tests.

Final referencing/answer agent assembling the final response.

Final code synthesis agent that integrates peers and executes tests.

Thin wrapper around OpenAI-style chat completions.

Prompt set for GSM8K-style math problems with multiple agent roles.

Prompt set for HumanEval-style coding problems with multiple roles.

Configuration for KV communication and scheduling. Args: threshold (float): The threshold for the KV communication. thread_p

Abstract base class for text-generating language models. Subclasses implement synchronous and asynchronous generation. Defaults are provided

Local HF model chat with KV reuse and anchor-based dense prefill. Provides utilities to construct chat prompts, manage shared KV caches acros

Factory and registry for constructing concrete `LLM` implementations.

Prompt set for 4-option multiple-choice QA (MMLU-style).

Math agent that aggregates peer signals and computes final answers.

Represents a processing unit within a graph-based framework. This class encapsulates the functionality for a node in a graph, managing c

Abstract base class for a set of prompts.