Retrieve a concrete `LLM` instance by name and provider hints. Args: model_name: Provider/model identifier; defaults to a general

Escape Loguru markup tokens in free-form text.

Synchronous execution wrapper that normalizes outputs to a list.

Return the shared memory slot for a given agent id.

Return number of transformer layers tracked in the cache.

In-place concatenation of the token inputs from another TokenInput along the time dimension.

Execute nodes topologically for `num_rounds` and return final answers.

Return key/value tensors (or None) for a given layer index.

Return True if the cache uses the newer `layers` structure.

Create a config from environment variables with safe defaults.

在子进程执行 code，超过 timeout 秒直接终止，并返回 answer 变量（或错误信息）。

Synchronously generate a completion. Args: messages: Conversation history to condition the model. max_tokens: Maximum

Public alias to access or create the request-scoped state.

Bind the chat instance to a graph node id and role.

Return stacked key/value tensors when all layers are dense tensors.

Asynchronously generate a completion. Args: messages: Conversation history to condition the model. max_tokens: Maximu

Execute a node asynchronously with optional KV cache handling modes.

Generate a response using the requested strategy with sensible fallbacks.

TODO

Check if prefix KV has been initialized for an agent.

Materialize and store prefix KV segments and placeholder indices. The rendered prompt is tokenized and executed once to obtain the KV

Ensure the user input placeholder cache is ready and choose a strategy.

Best-effort sequence length from cache (APIs vary across transformers).

Configure the logging settings for the application. Args: print_level (str): The logging level for console output. logfile_l

Materialise condition KV cache for another agent and update anchors.

Overwrite cache layers with stacked tensors maintaining per-layer metadata.

Assign key/value tensors to a specific layer, updating metadata if present.

Return a copy with provided non-None fields overridden.

Rotate placeholder cache keys by absolute offset (with optional drop).

Asynchronous execution entry supporting standard and KV-reuse workflows.

Connect every node to the decision node with a spatial edge.

Rebuild spatial edges from masks while preventing cycles.

Rebuild temporal edges for non-zero rounds from masks while preventing cycles.

TODO

TODO

TODO

Return a dict that maps id to info.

To be overriden by the descendant class

Returns a new TokenInput with input_ids and attention_mask sliced along the time dimension. Negative indexing is allowed.

Tokenize a single chat segment and move tensors to the model device.

Tokenize chat messages and return model inputs, text, and prompt length.

Prepare prompts, optionally populate anchors, and return mode hints.

To be overriden by the descendant class

To be overriden by the descendant class

Prepare prompts, run quick internal checks, and return mode hints.

Prepare prompts, optionally populate anchors, and return mode hints.

Prepare prompts, possibly set anchors, and return mode hints.

Return per-request state used by the KV engine.

Log the cumulative average reuse rate across all processed requests.

Propagate memory update across nodes; supports multi-request keys.

Validate value ranges and return self.

Persist a latency record to JSON, tolerating malformed or missing files.

To be overriden by the descendant class

In-place concatenate multiple caches along sequence dimension.

Apply an elementwise binary op to two caches layer-by-layer.

Return the content inside a \\boxed expression, handling nested braces.

Convert None/negative indices into bounded absolute [start, end).

Convert negative indices and None into concrete slice bounds.

Render base messages from the current template with provided text.

In-place replace [start, end) with the content from `real`.