A single observable event in the MP system. Attributes: event_type: The type of event. timestamp: Wall-clock time (``time.time()`

Small Python Bitmap fallback for source-only distributed tests.

The unique identifier for an object in the distributed storage manager

Describes the layout of a memory object

Even if local_cpu is False (the hot_cache is not used), contains(), insert_key(), remove(), get_blocking(), get_keys(), and clear() are s

LMCacheMetadata should be extracted from the northbound serving engine configuration and wrap the extraction of attributes (e.g. model na

Object lifecycle state machine for L1 cache +--------+ | None | <---------------------------------------+ +-----

Manages a virtual address space starting from 0 for memory allocation. Key interfaces: - allocate(size): Allocate a block of memory of t

Lightweight KV operation event for queue storage (without common fields)

Batched KV operation message with common fields and lightweight operations Design: Common fields (instance_id, worker_id, location) are stored on

Configuration for the EventBus. Attributes: enabled: Whether the event bus is active. When disabled, ``publish()`` is a no-o

Manages event ingestion, queueing, and dispatch to subscribers. Events are appended to a deque on the hot path (``publish()``) and drained by

Default prefetch policy: for each key, pick the first adapter (lowest index) that has it.

AdHocMemoryAllocator is a simple allocator that does not actually allocate memory. It is used for testing purposes only.

HTTP-mappable runtime L2 reconfiguration error.

Allocates memory in the pre-allocated pinned memory.

One LMCache KV group: layers of one engine group that share a copy kernel. Carries the layer indices and which engine group they belong to. Sever

The configuration for L1 memory manager.

Defines the structure and behavior of a protocol request. Attributes: payload_classes: List of expected payload types in order

Specification for a single message queue handler. Args: request_type: The ZMQ request type this handler serves. handler: The call

LRU (Least Recently Used) eviction policy. This policy tracks the order of key accesses and evicts the least recently used keys first wh

Asynchronously prefetches data from L2 adapters into L1 memory. The controller: 1. Accepts prefetch requests via submit_prefetch_request

Wraps a raw flat tensor with some metadata

ObjectKey-shape stand-in for L1 event metadata.

Legacy raw-block storage plugin wrapper. The durable raw-device/index/checkpoint logic now lives in RawBlockCore. This wrapper preserves

Declarative description of a single configurable parameter. Attributes: key: State dict key (matches argparse attr name, e.g., ``"engine_

Storage plugin backend for /dev/dax mmap-backed KV cache.

Mimics BlockAllocationRecord for testing without importing vLLM types.

A transfer-channel-specific address, with starting position and size (in bytes). A single address corresponds to a single memory object.

Select one path from a comma-separated list using a sharding policy. Args: raw_csv: Comma-separated directory paths (e.g. ``"

Minimal TTLLock fallback for tests that only import the symbol.

Special config for the L1 Object/Key manager

The configuration for eviction policies (L1 and optionally L2).

Thread-safe fleet-wide chunk fingerprint directory. Hashes published token ranges into per-scope direct-address tables, mutated in place, and

L1MemoryManager manages the allocation and deallocation of L1 memory. Observability metrics to emit: 1. Memory usage 2. Active alloc

Thread-safe registry of byte quotas keyed by ``cache_salt``. Quotas are dynamic — CRUD operations (``set_quota``, ``delete_quota``) are cheap

Message for Registration (REQ-REP mode)

Default store policy: store all keys to all adapters, never delete from L1.

Config for a mock L2 adapter (for testing). Fields: - max_size_gb: maximum size in GB. - mock_bandwidth_gb: simulated bandwidth in G

Thread-safe stats aggregation for benchmark results. Receives ``RequestResult`` objects from the request sender, maintains running totals, an

Thread-safe in-memory ledger of L2 byte usage per ``cache_salt``, plus a per-key size map so re-stores don't double-count.

Asynchronously stores L1 data to L2 adapters after write completion. The controller: 1. Registers a StoreListener with L1Manager to rece

Records calls and serves canned responses for adapter methods.

Bloom Filter for memory-efficient.

Describes the L1 memory buffer registered with an external backend (e.g. Nixl).

The configuration for the distributed storage manager.

Test subscriber that records events.

Control Error Message

LMCacheManager manages the lifecycle of LMCache internal components. For an integration to utilize the Manager, define a ServiceFactory

Minimal vLLM Request stub.

Immutable result of a completed L2 store task. Encodes both the success flag and bytes transferred in the int value: ``>= 0`` means success (

MP L2 adapter that persists KV objects into raw-block slots.

Central registry managing the tree structure of instances and workers. Structure: instance_id -> InstanceNode -> WorkerNode Lock hierarc

Async streaming request sender for inference engines. Each ``send_request`` call is a self-contained coroutine. Concurrency is controlled ext

Registry of active OTel spans, keyed by ``(session_id, span_name)``. ``span_name`` is a logical identifier (e.g. ``"request"``, ``"retrieve"``,

The caller asked the codec to emit something it cannot. Examples: a runtime_layout this codec doesn't implement, or an FP8 variant (e5m2) res

Minimal adapter that satisfies the abstract surface so we can exercise base-class behavior in isolation.

Audit wrapper for RemoteConnector that dynamically wraps all methods. Features: - Automatically wraps all RemoteConnector methods - Metho

CLI command for sustained performance benchmarking.

Allocates (1) memory in the pre-allocated pinned memory. (2) byte_array buffer memory.

V3 matcher: token-level probe (any offset) + full-hash collision rejection. Self-contained (does not inherit a base matcher).

The encoded blob's header or payload failed an integrity check. Examples: codec_magic does not match, codec_version is unknown, payload_c

Thread-safe in-memory registry of mp servers. All public methods acquire an internal lock, so the registry stays consistent under concurrent

Helper class to facilitate testing MessageQueueServer and MessageQueueClient. Supports testing with single or multiple concurrent clients, w

A simple read-write lock with timeout support. Multiple readers can hold the lock simultaneously, but only one writer. Note: This lock i

Manager for creating connectors based on URL. This class maintains a registry of connector adapters and creates the appropriate connecto

Per-``cache_salt`` LRU eviction policy. Maintains one ``OrderedDict`` per ``cache_salt`` (keyed by ``ObjectKey``). New and touched keys move

A key for the layer cache engine

Wraps a pybind-wrapped C++ IStorageConnector to implement L2AdapterInterface. The native_client must expose: - event_fd() -> int

Bundle a config, one token database, and a fake SM. A shared :class:`ChunkedTokenDatabase` instance is essential here: the database lazily re

Implements a paged memory allocator.

Real-time terminal display for benchmark progress. Runs a daemon thread that redraws stats every second using ANSI cursor control for in-plac

Implements a "explicit list" memory allocator. Uses AddressManager for address space management.

L1 memory manager for the GDS slab-file tier. A peer of :class:`~lmcache.v1.distributed.memory_manager.l1_memory_manager.L1MemoryManager`

Main config for L2 adapters. Holds an ordered list of adapter configs. Each element corresponds to one L2 adapter instance (e.g. two dis

Configuration for the ZMQ-based multiprocess cache server.

Mock MemoryObj for testing.

Workload-specific config for the prefix-suffix-tuner workload. Attributes: context_length: Total tokens per request (prefix + breaker + s

Configuration for ZMQ benchmark parameters

Manages the temporary GPU buffer for GPUCacheContext The logical layout of the temp GPU buffer is (batch size, object group, kernel grou

Wraps sync Serializer/Deserializer into async SerdeProcessor. Runs each submitted task in a thread pool. On completion, stores the result and

Creates OTel spans from CB (Cache Blending) START/END event pairs. Each session gets one root ``"cb.request"`` span that nests all child span

Registry for dynamically loaded check modes

Mock configuration object.