Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 7 callers
Method
close
Close all modules and release shared resources.
lmcache/v1/multiprocess/server.py:108
↓ 7 callers
Method
close
Stop the daemon, drain the match pool, and close the HTTP client.
lmcache/v1/mp_coordinator/blend_client.py:199
↓ 7 callers
Method
close
(self)
lmcache/v1/platform/event_notifier.py:213
↓ 7 callers
Method
close
(self)
lmcache/v1/storage_backend/nixl_storage_backend.py:286
↓ 7 callers
Method
close
Stop checkpointing, write a final checkpoint, and close the device.
lmcache/v1/storage_backend/raw_block/core.py:938
↓ 7 callers
Function
compute_new_token_ids
Reproduces the fixed token slicing logic from build_connector_meta. Uses min(num_computed_tokens, tracker_len) as slice base.
tests/v1/test_decode_save_and_preemption.py:48
↓ 7 callers
Method
contains
(self, path: str)
tests/v1/distributed/test_hfbucket_l2_adapter.py:131
↓ 7 callers
Method
contains_key
Return whether one encoded key is present in the raw-block index. Args: encoded_key: Encoded raw-block key string. lo
lmcache/v1/storage_backend/raw_block/core.py:481
↓ 7 callers
Function
create_interleaved_lookup_keys
Create interleaved lookup keys for scheduler-style TP lookup. The order matches what the scheduler expects: [chunk0_worker0, chunk0_work
tests/v1/distributed/test_object_key_parallel.py:115
↓ 7 callers
Function
create_object_key
(chunk_id: int, model_name: str = "test_model")
tests/v1/distributed/test_resp_l2_adapter_integration.py:68
↓ 7 callers
Function
create_test_key
Create a test CacheEngineKey. Args: key_id: Key ID for testing Returns: CacheEngineKey with test configuration
tests/v1/storage_backend/test_hf3fs_connector.py:136
↓ 7 callers
Function
create_test_key
Create a test CacheEngineKey.
tests/v1/storage_backend/test_fs_connector.py:75
↓ 7 callers
Function
create_test_metadata
Create a test metadata for LMCacheMetadata.
tests/v1/storage_backend/test_gds_backend.py:77
↓ 7 callers
Function
create_test_metadata
Create a test metadata for LMCacheMetadata.
tests/v1/storage_backend/test_fs_connector.py:62
↓ 7 callers
Method
delete_many
Delete keys from the in-memory DAX index. When ``force`` is ``False``, externally locked and in-flight keys are skipped. Slots borrow
lmcache/v1/storage_backend/dax/core.py:496
↓ 7 callers
Function
encode_tokens
Encode token ids into a compact base64 wire string. Token ids fit in ``uint32``, so a little-endian ``uint32`` buffer is far smaller than a J
lmcache/v1/mp_coordinator/schemas.py:23
↓ 7 callers
Function
ensure_adapter_loaded
Import pending modules until *name* appears in the factory registry (or all pending modules have been tried). Raises: ImportError
lmcache/v1/distributed/l2_adapters/factory.py:101
↓ 7 callers
Function
envValue
envValue returns the value of the named env var on the container, or "".
operator/internal/webhook/pod_injector_test.go:180
↓ 7 callers
Method
estimate_serialized_size
( self, layout_descs: LayoutDescGroup, )
lmcache/v1/distributed/serde/asym_k16_v8.py:133
↓ 7 callers
Method
execute
(self, args: argparse.Namespace)
lmcache/cli/commands/ping.py:99
↓ 7 callers
Method
exists
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/connector/redis_connector.py:68
↓ 7 callers
Method
exists_sync
Check if key exists synchronously.
tests/v1/storage_backend/test_remote_storage_plugin.py:48
↓ 7 callers
Function
extract_plugin_type
Extract the type portion from a plugin name. Plugin name format: ``{type}`` or ``{type}.{instance}``. Returns the *type* part so that adapter
lmcache/v1/storage_backend/connector/__init__.py:97
↓ 7 callers
Function
f
(key)
tests/v1/storage_backend/test_pd_backend_async.py:432
↓ 7 callers
Method
finalize_reads
Release read reservations. Args: reservations: Reservations previously returned by ``reserve_reads``.
lmcache/v1/storage_backend/dax/core.py:638
↓ 7 callers
Function
fold_unfold_ranked
Compose :func:`fold` -> :func:`highest_set_bit` -> :func:`unfold`. Convenience for the full pipeline over the ``group x chunk x kv_rank`` loo
lmcache/v1/distributed/bitmap_ops/fold.py:166
↓ 7 callers
Function
formatFloat
formatFloat renders a float flag value with the minimal decimal representation (e.g. 30.0 -> "30", 0.2 -> "0.2").
operator/internal/resources/coordinator_deployment.go:117
↓ 7 callers
Method
from_gpu
Expect a kwarg 'kvcaches' which is a nested tuple of K and V tensors. The kvcaches should correspond to the "WHOLE token sequence". W
lmcache/v1/gpu_connector/gpu_connectors.py:1558
↓ 7 callers
Function
get_concrete_engine_kv_shape_from_shape_desc
Return the concrete shape for a single kernel group's ``shape_desc``. Like :func:`get_concrete_engine_kv_shape`, but the numeric values are r
lmcache/v1/gpu_connector/utils.py:150
↓ 7 callers
Function
get_extra_config_bool
Extract a boolean value from the config's extra_config dict. Args: key: The key to look up in extra_config. config: The LMCacheEn
lmcache/v1/storage_backend/gds_backend.py:155
↓ 7 callers
Function
get_handler_type
Get the handler type for a request type. Args: req_type: The request type to look up Returns: The handler type (SYNC, B
lmcache/v1/multiprocess/protocol.py:70
↓ 7 callers
Function
get_item
Look up a ConfigItem by key. Raises KeyError if not found.
lmcache/cli/commands/bench/engine_bench/interactive/schema.py:409
↓ 7 callers
Method
get_kv_buffer_shape
Returns the shape of the KV buffer for the given number of tokens
lmcache/v1/platform/cuda/cache_context.py:582
↓ 7 callers
Method
get_memory_usage
Get the current memory usage of L1 cache. Returns: A tuple of (used_memory_bytes, total_memory_bytes). Note:
lmcache/v1/distributed/l1_manager.py:802
↓ 7 callers
Function
get_registered_l2_adapter_types
Return all known adapter type names (eager and lazy).
lmcache/v1/distributed/l2_adapters/config.py:116
↓ 7 callers
Method
get_seq_discontinuity_count
Get the count of sequence discontinuities (thread-safe).
lmcache/v1/cache_controller/utils.py:369
↓ 7 callers
Method
get_seq_num
Get sequence number for a location.
lmcache/v1/cache_controller/utils.py:192
↓ 7 callers
Method
get_serialize_event_fd
Return the fd signaled on serialize completion.
lmcache/v1/distributed/serde/async_processor.py:74
↓ 7 callers
Method
get_tensor
Get the tensor from the MemoryObj at the given index(group).
lmcache/v1/memory_management.py:404
↓ 7 callers
Function
get_zmq_rpc_path_lmcache
Get the ZMQ RPC path for LMCache lookup and offload communication. Args: engine_id: The engine ID for the RPC path. service_name:
lmcache/v1/rpc_utils.py:107
↓ 7 callers
Method
heartbeat
Send periodic heartbeats to the controller (DEALER-ROUTER mode). Process any commands received in the heartbeat response. Us
lmcache/v1/cache_controller/worker.py:443
↓ 7 callers
Method
is_backend_bypassed
Check if a backend is currently bypassed. Args: backend_name: The name of the backend to check Returns:
lmcache/v1/storage_backend/storage_manager.py:903
↓ 7 callers
Method
is_finished
Whether the transfer reached a terminal state (done or errored).
lmcache/v1/distributed/transfer_channel/api.py:45
↓ 7 callers
Function
isinstance_mock_adapter
Tests MockTestConnectorAdapter as isinstance() not identifying the child class
tests/v1/storage_backend/test_remote_storage_plugin.py:134
↓ 7 callers
Method
list_quotas
Return a snapshot of all registered quotas. The returned list is a detached copy; mutating it does not affect the registry.
lmcache/v1/distributed/quota_manager.py:84
↓ 7 callers
Function
load_lookup_events
Load and return lookup events from one or more JSONL files or directories. Parameters ---------- paths: Each element may be
lmcache/tools/cache_simulator/simulator.py:98
↓ 7 callers
Method
lock_refcount
Return the L2 lock refcount for a legacy encoded key.
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:213
↓ 7 callers
Method
lookup
Checks the existence of KV cache of the tokens from the cache engine. :param Optional[Union[torch.Tensor, List[int]]] tokens: the in
lmcache/v1/cache_engine.py:1125
↓ 7 callers
Method
lookup_unpin
(self, lookup_id: str)
lmcache/v1/cache_engine.py:1539
↓ 7 callers
Function
m
(cur_st: int, cur_ed: int)
tests/v1/multiprocess/test_blend_v3_load_store_opts.py:432
↓ 7 callers
Function
make_hotplug_adapter
(tmp_path, *, slot_bytes: int = 2048)
tests/v1/distributed/test_dax_l2_adapter.py:146
↓ 7 callers
Function
make_layout
Create a small MemoryLayoutDesc for testing (bf16, ~200KB/chunk).
tests/v1/distributed/serde/test_serde_e2e.py:64
↓ 7 callers
Function
make_memory_obj
Wrap payload bytes in a binary tensor memory object. Args: payload: Byte-compatible data to expose through TensorMemoryObj. Returns:
tests/v1/storage_backend/raw_block_test_utils.py:105
↓ 7 callers
Function
mla_enabled
(model_config: "ModelConfig")
lmcache/integration/vllm/utils.py:190
↓ 7 callers
Function
msgspec_decode
(b_obj: bytes, cls: Any)
lmcache/v1/multiprocess/mq.py:95
↓ 7 callers
Function
msgspec_encode
(obj: Any, cls: Any)
lmcache/v1/multiprocess/mq.py:82
↓ 7 callers
Method
name
Return the subcommand name (e.g. ``"mock"``).
lmcache/cli/commands/base.py:50
↓ 7 callers
Function
normalize_url
Ensure *url* has an ``http://`` or ``https://`` scheme.
lmcache/cli/commands/describe.py:29
↓ 7 callers
Method
on_lookup_request
This function is called when a lookup request is sent to the cache. It will record the number of tokens requested.
lmcache/observability.py:333
↓ 7 callers
Function
pad_shape_to_4d
Pad a shape with fewer than 4 dimensions to 4D using trailing zeros. Shapes that are already 4D are returned as-is. For shapes with fewe
lmcache/v1/protocol.py:99
↓ 7 callers
Method
pop_pending_keys
Pop all pending keys from the queue. This is non-blocking and should be called by the StoreController's main loop after sele
lmcache/v1/distributed/storage_controllers/store_controller.py:99
↓ 7 callers
Method
pread_into
(self, offset, out, payload_len, total_len=None)
tests/v1/storage_backend/test_rust_raw_block_backend.py:135
↓ 7 callers
Method
prefetch_all_done_callback
Callback function when all prefetch tasks (i.e., prefetching from all backends for the entire request) are done. :param int
lmcache/v1/storage_backend/storage_manager.py:554
↓ 7 callers
Method
put
(self, key, value)
tests/v1/test_remote_mla_worker_id_as0.py:32
↓ 7 callers
Method
query_prefetch_result
Query the result of a prefetch request. Thread-safe. Returns the retained-key bitmap if the request has completed, None if s
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:395
↓ 7 callers
Function
register_codec
Register a codec for type ``t``. Raises: ValueError: If ``t`` or ``codec.tag`` is already registered.
lmcache/v1/mp_observability/trace/codecs.py:56
↓ 7 callers
Method
register_kv_caches
Register TRT-LLM's 4-D KV pool tensor with the connector. Reshapes the 4-D tensor to canonical 6-D form, runs format discovery, and c
lmcache/v1/gpu_connector/gpu_connectors.py:2018
↓ 7 callers
Method
release
Release the lock.
lmcache/v1/cache_controller/locks.py:133
↓ 7 callers
Method
reset
Clear all accumulated results and restart the timer. Used between warmup and benchmark phases so warmup stats don't pollute benchmark
lmcache/cli/commands/bench/engine_bench/stats.py:111
↓ 7 callers
Method
reshape_partial_chunk
( self, memory_obj: MemoryObj, bytes_read: int, )
lmcache/v1/storage_backend/connector/base_connector.py:86
↓ 7 callers
Function
safe_asdict
Convert a dataclass instance to a JSON-safe ``dict``. Args: obj: A dataclass instance. Returns: Dictionary with all fields r
lmcache/v1/utils/json_utils.py:41
↓ 7 callers
Method
serialize
(self, src: MemoryObjGroup, dst: MemoryObj)
lmcache/v1/distributed/serde/asym_k16_v8.py:106
↓ 7 callers
Method
start
(self)
examples/blend_kv/online_kv.py:52
↓ 7 callers
Method
stop
(self)
lmcache/v1/multiprocess/native_completion.py:91
↓ 7 callers
Method
subscriber_exception_counts
Snapshot of per-subscriber callback exception counts. Maps ``subscriber_name`` (the owning class name for bound methods, ``__qualname
lmcache/v1/mp_observability/event_bus.py:291
↓ 7 callers
Method
summary
Return a per-qualname :class:`OpStats` snapshot. Returns: A dict keyed by qualname. A qualname with only errors stil
lmcache/cli/commands/trace/_stats.py:171
↓ 7 callers
Method
to_object_key
Recover the corresponding :class:`ObjectKey`. Raises: ValueError: ``chunk_hash_hex`` is not valid hex, or one of
lmcache/v1/distributed/api.py:191
↓ 7 callers
Method
update_interval_remote_write_metrics
(self, write_bytes: int)
lmcache/observability.py:536
↓ 7 callers
Method
update_remote_ping_error_code
Update ping error code
lmcache/observability.py:557
↓ 7 callers
Function
wait_for_condition
Poll until a predicate returns True or timeout.
tests/v1/distributed/serde/test_serde_e2e.py:72
↓ 7 callers
Method
wait_for_save
Blocking until the KV cache is saved to the connector buffer.
lmcache/integration/vllm/vllm_v1_adapter.py:1098
↓ 7 callers
Function
write_and_wait_for_l2
Write keys to L1 via StorageManager and wait for L2 store. Fills each chunk with deterministic data so round-trip can be verified.
tests/v1/distributed/serde/test_serde_e2e.py:141
↓ 6 callers
Method
Reconcile
+kubebuilder:rbac:groups=lmcache.lmcache.ai,resources=lmcacheengines,verbs=get;list;watch;create;update;patch;delete +kubebuilder:rbac:groups=lmcache.
operator/internal/controller/lmcacheengine_controller.go:53
↓ 6 callers
Function
SelectorLabels
SelectorLabels returns the immutable label subset used for pod selectors.
operator/internal/resources/labels.go:22
↓ 6 callers
Method
_adapt_shapes_and_dtypes
( self, shapes: Union[torch.Size, list[torch.Size]], dtypes: Union[torch.dtype, list[t
lmcache/v1/memory_management.py:1236
↓ 6 callers
Function
_allocate_cpu_memory
( size: int, numa_mapping: Optional[NUMAMapping] = None, shm_name: Optional[str] = None, use_h
lmcache/v1/memory_management.py:521
↓ 6 callers
Function
_check
Convert a non-zero ``CUfileError_t`` into a Python exception.
lmcache/v1/gpu_connector/_cufile_async.py:119
↓ 6 callers
Function
_config_dict
(**overrides)
tests/v1/distributed/test_raw_block_l2_adapter.py:159
↓ 6 callers
Method
_drain_finished_queue
Drain all completed requests and call ``on_request_finished``.
lmcache/cli/commands/bench/engine_bench/workloads/base.py:125
↓ 6 callers
Method
_empty_table
(self, size: int = 8)
tests/v1/multiprocess/test_token_hasher.py:253
↓ 6 callers
Function
_enable_polling
Point a controller's poll loop at a mock coordinator client.
tests/v1/multiprocess/test_p2p_controller.py:384
↓ 6 callers
Function
_extract_content
Return text content from a streaming chunk, or ``""`` if none. Ported from Tensormesh-Benchmark ``streaming_utils.py``.
lmcache/cli/commands/bench/engine_bench/request_sender.py:33
↓ 6 callers
Function
_fail
(reason: str)
lmcache/v1/gpu_connector/kv_format/contiguity.py:113
↓ 6 callers
Method
_fmt
(key: CacheEngineKey)
tests/v1/test_nixl_doca_memos.py:58
↓ 6 callers
Function
_free_cpu_memory
( buffer: torch.Tensor, size: int | None = None, numa_mapping: Optional[NUMAMapping] = None, s
lmcache/v1/memory_management.py:572
↓ 6 callers
Function
_generate_contexts
Generate ``num_contexts`` unique context blocks of ~``length`` tokens. Each context draws from ``vocab_pool`` with a per-context seed so the
lmcache/cli/commands/bench/engine_bench/workloads/long_doc_permutator.py:166
↓ 6 callers
Method
_get_client
Lazy initialization of independent Bigtable async data client.
lmcache/v1/storage_backend/connector/bigtable_connector.py:97
↓ 6 callers
Method
_get_connector_metadata
Get the connector metadata. This function should only be called inside the connector. Returns: ConnectorMetadata: the co
lmcache/integration/vllm/lmcache_mp_connector.py:665
↓ 6 callers
Function
_l1_desc
()
tests/v1/distributed/transfer_channel/test_context_lifecycle.py:75
↓ 6 callers
Function
_layout
(nbytes: int)
tests/v1/distributed/memory_manager/test_gds_l1_memory_manager.py:26
← previous
next →
901–1,000 of 13,371, ranked by callers