Submit an event (hot path — non-blocking). The event's ``timestamp`` is set to ``time.time()`` at call time. When the queue is full t

Return a simple health-check string. Returns: The literal string ``"OK"``.

Get the blender by instance_id.

Start no background thread.

Encode an asymmetric (K, V) pair into one EncodedKV blob. If `precomputed_v_quant` is provided it is used as-is: the caller has alrea

Get the data pointer of the MemoryObj. This is used to access the raw data in the memory.

Stop no background thread.

Get the tensor from the MemoryObj.

Decode an EncodedKV into (K, V_quant_or_dequant, scales). If `out_v_dtype` is None, V is returned at its native FP8 dtype (storage_on

Return whether one bit is set.

Record a top-level metric (not inside any named section). Args: key: Machine-readable key (used in JSON output). labe

Load the encoder output tensor for ``mm_hash`` if present. Args: mm_hash: multimodal-input identifier produced by vLLM.

Set one bit by index.

Close the backend.

Return ``(span, context)`` for an open span, or ``None``. Args: session_id: The request session identifier. span_name

Get the result of the future. Args: timeout (Optional[float]): Maximum time to wait in seconds. If None,

Decrease ref count for the given MemoryObj by one.

Register an ``EventSubscriber`` and wire up its callbacks.

Update the request tracker when a running request is scheduled again vllm_cached_tokens: the number of tokens that are cached in vLLM

Return a context manager that sets the current device. Args: device: The device to select. Ignored in the stub. Returns:

Return the current L2 storage utilization as an ``AdapterUsage``. The default implementation returns the totals maintained by the

Helper to create ObjectKey instances.

Reserve write access for the given keys. Args: keys: The list of object keys to reserve write access for. is_temporar

Instead of Tuple[Tuple[Tensor, Tensor]], return List[Tensor] where KV are in the same tensor

Close the L1Manager and free all resources.

Match request tokens against the directory. Rolls a chunk-window hash over the request, then probes the scope's direct-address table

Return the instance with the given id, or ``None`` if unknown. Args: instance_id: Identifier to look up. Returns:

Wait for I/O completion and return results. Returns: List of MockIorResult, each with .result containing bytes transferred

Allocate a block of memory from the virtual address space of a given size. The actual allocated size could be larger than the request

Return values selected by set bit indices.

Check if the address manager is consistent. Returns: True if consistent, False otherwise.

Finish read access for the given keys. Will delete the object if it is temporary and read count reaches zero. Args:

Finish write access for the given keys. Args: keys: The list of object keys to finish write access for. Returns:

Return set bit indices in ascending order.

Get the size of the MemoryObj in bytes. Note that this number could be smaller than the physical size. The physical size is a

Create a platform-appropriate EventNotifier. On Linux (Python 3.10+), returns an ``EventfdNotifier``. On other platforms, returns a ``PipeNot

DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.

Remove and return ``(span, context)``, or ``None`` if absent. Args: session_id: The request session identifier. span_

Record no native callback registration.

Mark this event as recorded on the given stream. Args: stream: The stream to record on. If None, uses the current

Create a test MemoryObj using AdHocMemoryAllocator for testing.

Get the total free size in the address space. Returns: The total free size in bytes.

Set the result of the future and mark it as done. This function is NOT SUPPOSED TO BE CALLED by users directly. It should be only cal

Return ``True`` when the store task succeeded.

Return the number of set bits.

Get a value.

Returns a dummy MemoryObj for testing purposes.

check whether two paged kv caches are the same at slot_mapping

Free a previously allocated block of memory. Args: address: The starting address of the block to free. size:

Handle batched KV operations with single lock acquisition. Logs warning and calls callback if sequence out of order is detected.

Reserve read access for the given keys. Args: keys: The list of object keys to reserve read access for.

Create or update the quota for ``cache_salt``. Raises ``ValueError`` for a negative limit. A zero limit is accepted and behaves ident

Close the storage manager and release all resources.

Return the length of the leading contiguous set-bit prefix.

Deterministic chunk_size tokens derived from seed; unique per seed.

Create an ObjectKey for testing.

--- helpers ---

Builds a new LMCacheEngine instance if it doesn't already exist for the given ID. raises: ValueError if the instance already