In-place slice of KV cache along sequence dimension.

Core KV-aware generation entry. Builds merged prefix KV and token ids from stored segments and per-request placeholder caches, then r

Detect if adding edges would create a cycle starting at `new_node`.

Returns a new TokenInput instance with token inputs concatenated along the time dimension.

Retrieve shared KV cache and ids for a placeholder given message context.

Return role-specific analysis constraints.

TODO

TODO

Normalize answer into a single string and strip prefix markers.

Log an agent's generation output and update reuse statistics.

Initialise bookkeeping for a new request.

Commit all staged request states into the global dictionaries.

Compute attention-like weights between the real placeholder segment and stored anchors.

Return a new cache that is the concatenation of base and others.

Ensure the global input buckets exist and return the shared store.

Initialise or resize the shared thread pool used for CPU work.

To be overriden by the descendant class

Extract trailing assistant prompt from a legacy text prefix.

Parse tokenizer chat template to identify structural markers.

Extract pure python code from a fenced block.

Look up cached anchor weights for a specific placeholder/message/signature.

Return or create a request-scoped state container under a lock.

Lazy-load shared tokenizer/model and shared KV memory storage.

Monkey-patch DynamicCache with convenience methods used by KVCOMM.

Fallback prompt renderer when chat_template is unavailable.

Convert mixed message representations into chat dicts.

Normalise various prefix formats into a base messages template.

To be overriden by the descendant class

Return a copy of cache with [start, end) replaced by `real`.

Pick anchors whose placeholder span covers the current placeholder length.

Select positions by index tensor, preserving layout and metadata.

Store computed anchor weights for reuse within the same request.

Return a sliced copy of the KV cache.

Split a cache into multiple segments by the given lengths (sum sizes).

Split a cache into placeholder and prefix segments per provided spans.

Move all tensors in the cache to the specified device.

Call an OpenAI-compatible chat endpoint asynchronously.

Core KV-aware generation entry. Builds merged prefix KV and token ids from stored segments and per-request placeholder caches, then r

Create a lightweight fingerprint for the active anchors.

Clear all the spatial connection of the nodes in the graph.

Clear all the temporal connection of the nodes in the graph.

Evaluate a graph on the provided dataset.

Render a single chat block for the given role and content.

Generate with dense prefix prefill and optional anchor update.

Generate by reusing existing prefix KV (fast path).