Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 1 callers
Method
stop
(self)
examples/online_session/openai_chat_completion_client.py:81
↓ 1 callers
Method
stop
(self)
lmcache/v1/internal_api_server/api_server.py:127
↓ 1 callers
Method
stop
Stop the health monitor thread
lmcache/v1/health_monitor/base.py:525
↓ 1 callers
Method
stop_monitoring
Stop the background monitoring thread.
lmcache/v1/pin_monitor.py:235
↓ 1 callers
Method
stop_plugins
Terminate all plugin processes. Note: This method may be called via atexit during interpreter shutdown, when the logging system may a
lmcache/v1/plugin/runtime_plugin_launcher.py:198
↓ 1 callers
Function
storage_plugin_launcher
Loads custom storage backends based on configuration. Looks for backend configurations in config.extra_config and instantiates them usin
lmcache/v1/storage_backend/__init__.py:40
↓ 1 callers
Method
store_layer
(self, token_ids, **kwargs)
tests/v1/test_vllm_layerwise_wait_for_save.py:37
↓ 1 callers
Method
store_speed
(self)
lmcache/observability.py:203
↓ 1 callers
Function
stream_service_response
Asynchronously stream the response from a service.
examples/disagg_prefill_mp/disagg_proxy_server.py:195
↓ 1 callers
Function
strerror
Convert error code to human-readable string. Args: error (ErrorType): The error code. Returns: str: The human-readable strin
lmcache/v1/distributed/error.py:39
↓ 1 callers
Method
submit_batch_delete
(self, keys: list[str])
tests/v1/distributed/test_native_connector_l2_adapter.py:113
↓ 1 callers
Method
submit_deserialize
Submit a batch deserialization task. Args: src_objs: Source byte-buffer MemoryObjs (filled by L2 load). dst_objs: Des
lmcache/v1/distributed/serde/base.py:174
↓ 1 callers
Method
submit_deserialize
Submit a batch deserialize task to the thread pool.
lmcache/v1/distributed/serde/async_processor.py:112
↓ 1 callers
Method
submit_load_task
( self, keys: list[ObjectKey], objects: list[MemoryObj], )
examples/lmc_external_l2_adapter/src/lmc_external_l2_adapter/adapter.py:210
↓ 1 callers
Method
submit_load_task
( self, keys: list[ObjectKey], objects: list[MemoryObj], )
lmcache/v1/distributed/l2_adapters/p2p_l2_adapter.py:299
↓ 1 callers
Method
submit_load_task
( self, keys: list[ObjectKey], objects: list[MemoryObj], )
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:449
↓ 1 callers
Method
submit_lookup_and_lock_task
( self, keys: list[ObjectKey], layout_desc: MemoryLayoutDesc, )
lmcache/v1/distributed/l2_adapters/p2p_l2_adapter.py:213
↓ 1 callers
Method
submit_lookup_and_lock_task
Submit an asynchronous lookup-and-lock task. Found keys have their DAX external lock refcount incremented until ``submit_unlock`` is
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:436
↓ 1 callers
Method
submit_lookup_and_lock_task
Submit a non-blocking lookup-and-lock task. Args: keys: Object keys to look up in raw-block L2. Returns: Tas
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:451
↓ 1 callers
Method
submit_put_task
Submit a single put task to store KV cache to disk asynchronously. :param key: The cache key for this KV chunk. :param memor
lmcache/v1/storage_backend/local_disk_backend.py:304
↓ 1 callers
Method
submit_retrieve
Submit a retrieve request and return a completion future. Args: request_id: External request identifier. key: LMCache
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:185
↓ 1 callers
Method
submit_retrieve
( self, _request_id: str, key: Any, instance_id: int, kv_caches: dict[
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:455
↓ 1 callers
Method
submit_serialize
Submit a batch serialization task. Args: src_objs: Source KV-shaped MemoryObjs (read-locked). dst_objs: Destination b
lmcache/v1/distributed/serde/base.py:146
↓ 1 callers
Method
submit_store
Submit a store request and return a completion future. Args: request_id: External request identifier. key: LMCache ke
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:156
↓ 1 callers
Method
submit_store
( self, _request_id: str, key: Any, instance_id: int, kv_caches: dict[
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:413
↓ 1 callers
Method
submit_store_task
Submit a store task to store a batch of memory objects associated with a batch of keys. Args: keys (list[ObjectK
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:488
↓ 1 callers
Method
submit_store_task
( self, keys: list[ObjectKey], objects: list[MemoryObj], )
lmcache/v1/distributed/l2_adapters/native_connector_l2_adapter.py:189
↓ 1 callers
Method
submit_store_task
Submit a wrapped store (serialize → inner.store). All-or-nothing: if temp alloc fails for any key or serialize submission raises, the
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:164
↓ 1 callers
Method
submit_store_task
Submit a non-blocking raw-block store task. Args: keys: Object keys to persist. objects: Memory objects containing pa
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:407
↓ 1 callers
Method
submit_store_task
Submit a store task to store a batch of memory objects associated with a batch of keys. For the mock adapter, the store oper
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:155
↓ 1 callers
Method
submit_unlock
(self, keys: list[ObjectKey])
lmcache/v1/distributed/l2_adapters/p2p_l2_adapter.py:284
↓ 1 callers
Method
support_batched_contains
(self)
lmcache/v1/storage_backend/connector/redis_connector.py:164
↓ 1 callers
Method
support_batched_put
(self)
lmcache/v1/storage_backend/connector/eic_connector.py:621
↓ 1 callers
Method
support_ping
(self)
lmcache/v1/storage_backend/connector/s3_connector.py:655
↓ 1 callers
Method
supports_producer_reuse
(self)
tests/v1/lookup_client/test_chunk_statistics_lookup_client.py:29
↓ 1 callers
Function
test_exists
(eic_client)
tests/v1/storage_backend/test_eic.py:112
↓ 1 callers
Function
test_set
(eic_client)
tests/v1/storage_backend/test_eic.py:67
↓ 1 callers
Method
time_to_lookup
(self)
lmcache/observability.py:127
↓ 1 callers
Method
to_bytes
Serialize a pytorch tensor to bytes. The serialized bytes should contain both the data and the metadata (shape, dtype, etc.) of the t
lmcache/storage_backend/serde/serde.py:18
↓ 1 callers
Method
to_core_config
Convert this adapter config to the shared RawBlockCore config.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:276
↓ 1 callers
Method
to_gpu
Store KV data from a memory object into MUSA paged KV caches. Args: memory_obj: The memory object containing KV data.
lmcache/v1/gpu_connector/musa_connectors.py:108
↓ 1 callers
Method
to_gpu
Expect a kwarg 'kvcaches' which is a nested tuple of K and V tensors. The kvcaches should correspond to the "WHOLE token sequence". N
lmcache/v1/gpu_connector/hpu_connector.py:83
↓ 1 callers
Method
to_tensor
Note: This function may break if the accelerator is not initialized. We should call `torch_dev.init()` before using t
lmcache/v1/multiprocess/custom_types.py:138
↓ 1 callers
Method
tokens_per_layer
Return the token capacity per layer (``num_blocks * block_size``). Non-MP only (see the class docstring).
lmcache/v1/gpu_connector/kv_format/specs/base.py:146
↓ 1 callers
Method
touch_cache
(self)
lmcache/v1/storage_backend/storage_manager.py:1037
↓ 1 callers
Method
touch_instance
Refresh the worker's last-seen time and mark it ping-proven. A no-op if the instance is not tracked. Args: instance_id:
lmcache/v1/multiprocess/modules/engine_driven_transfer.py:172
↓ 1 callers
Method
touch_keys
Touch the given keys, marking the keys as accessed(retrieved or stored). Args: keys: The list of object keys to touch.
lmcache/v1/distributed/l1_manager.py:707
↓ 1 callers
Method
touch_l1_keys
Touch the keys in L1 storage, marking the keys as accessed(retrieved or stored). Args: keys (list[ObjectKey]): L
lmcache/v1/distributed/storage_manager.py:684
↓ 1 callers
Method
track_engine
(self)
lmcache/usage_context.py:188
↓ 1 callers
Method
track_env
(self)
lmcache/usage_context.py:165
↓ 1 callers
Method
track_metadata
(self)
lmcache/usage_context.py:192
↓ 1 callers
Method
tracked_instance_count
Return the number of currently registered non-GPU instances.
lmcache/v1/multiprocess/modules/engine_driven_transfer.py:187
↓ 1 callers
Method
transfer_speed
(self)
lmcache/observability.py:244
↓ 1 callers
Function
translate_text
Send a pre-built user message to the endpoint and return the cleaned translation. Raises: RuntimeError: If the response has no message co
tools/translate_docs_zh.py:291
↓ 1 callers
Function
truncate_to_tokens
( text: str, max_tokens: int, tok: PreTrainedTokenizerBase, )
examples/online_session/openai_chat_completion_client.py:45
↓ 1 callers
Function
try_get_vllm_kv_cache_layout
Try to query the KV cache layout from vLLM at runtime. Returns ``"NHD"`` or ``"HND"`` if vLLM is available and the layout has been configured
lmcache/integration/vllm/utils.py:48
↓ 1 callers
Function
try_native_multi_layer_block_kv_transfer
Try native MUSA acceleration for block-based paged-KV transfer. This is the optional native fast path used by ``lmcache.v1.platform.musa.ops`
lmcache/v1/platform/musa/native_kv_transfer.py:84
↓ 1 callers
Function
uniqueNamespaceName
uniqueNamespaceName produces a DNS-1123-safe namespace name that's unique within the test process. The time component disambiguates across runs / Kind
operator/test/e2e/smoke_helpers_test.go:86
↓ 1 callers
Function
unknown_documents
Raise an error for a missing documents placeholder. Args: key: The name of the missing placeholder. Raises: ValueError: Alwa
lmcache/cli/commands/query/_prompt.py:102
↓ 1 callers
Method
unpin
(self)
tests/v1/test_memory_management.py:430
↓ 1 callers
Method
unpin
(self)
lmcache/v1/storage_backend/nixl_storage_backend.py:427
↓ 1 callers
Method
unpin
( self, key: CacheEngineKey, )
lmcache/v1/storage_backend/p2p_backend.py:780
↓ 1 callers
Method
unpin
( self, key: CacheEngineKey, )
lmcache/v1/storage_backend/local_disk_backend.py:224
↓ 1 callers
Method
unregister_all
Unregister all periodic threads.
lmcache/v1/periodic_thread.py:468
↓ 1 callers
Method
unsafe_read
Read already read-locked objects without acquiring new read locks.
lmcache/v1/distributed/storage_manager.py:694
↓ 1 callers
Function
updateCurrentTime
()
lmcache/v1/cache_controller/frontend/static/js/controller_app.js:65
↓ 1 callers
Method
updateStatus
updateStatus queries the DaemonSet and pods to compute status fields. It re-fetches the engine to get the latest resourceVersion, avoiding conflicts f
operator/internal/controller/reconcile_helpers.go:297
↓ 1 callers
Method
updateStatus
updateStatus queries the Deployment to compute status fields. It re-fetches the coordinator to get the latest resourceVersion, avoiding conflicts from
operator/internal/controller/coordinator_reconcile_helpers.go:226
↓ 1 callers
Method
updateStatus
updateStatus queries the DaemonSet and pods to compute status fields. It re-fetches the engine to get the latest resourceVersion, avoiding conflicts f
operator/internal/controller/cacheblend_reconcile_helpers.go:247
↓ 1 callers
Method
update_event_status
Update the status of the event with the given type and id.
lmcache/v1/event_manager.py:68
↓ 1 callers
Function
update_file
Translate up to ``remaining_budget`` empty/fuzzy messages in one ``.po`` file. For each entry needing translation, sends the focal msgid along wi
tools/translate_docs_zh.py:347
↓ 1 callers
Method
update_forced_unpin_count
(self, delta: int)
lmcache/observability.py:575
↓ 1 callers
Method
update_from_top_indices
(self, top_indices: torch.Tensor)
lmcache/v1/compute/attention/metadata.py:19
↓ 1 callers
Method
update_interval_prompt_tokens
(self, delta: int)
lmcache/observability.py:591
↓ 1 callers
Method
update_interval_vllm_hit_tokens
(self, delta: int)
lmcache/observability.py:587
↓ 1 callers
Method
update_lookup_socket
(self, new_lookup_socket: zmq.asyncio.Socket)
lmcache/v1/storage_backend/p2p_backend.py:149
↓ 1 callers
Method
update_peer_lookup_url
(self, new_peer_lookup_url: str)
lmcache/v1/storage_backend/p2p_backend.py:139
↓ 1 callers
Method
update_seq_num
Update sequence number for a location.
lmcache/v1/cache_controller/utils.py:187
↓ 1 callers
Method
validateAndSetCondition
validateAndSetCondition runs validation and updates the ConfigValid condition. Returns an error if validation fails (to stop reconciliation).
operator/internal/controller/coordinator_reconcile_helpers.go:40
↓ 1 callers
Method
validateAndSetCondition
validateAndSetCondition runs validation and updates the ConfigValid condition. Returns an error if validation fails (to stop reconciliation).
operator/internal/controller/cacheblend_reconcile_helpers.go:40
↓ 1 callers
Function
validate_args
(args)
benchmarks/long_doc_qa/long_doc_qa.py:758
↓ 1 callers
Function
validate_environment
Exit with an error if any required API env var is unset. Raises: SystemExit: If ``TRANSLATION_API_BASE_URL``, ``TRANSLATION_API_KEY``,
tools/translate_docs_zh.py:326
↓ 1 callers
Function
validate_mamba_step_alignment
Reject scheduler configs that can skip Mamba state snapshots. In ``mamba_cache_mode="align"`` vLLM snapshots the recurrent state only at the
lmcache/integration/vllm/lmcache_mp_connector.py:104
↓ 1 callers
Function
validate_nodes
Validate a list of node dicts; see :func:`validate_node`.
lmcache/lmcache_frontend/app.py:328
↓ 1 callers
Function
validate_reverse_correctness
(rope, reverse_rope, fused_rope, head_size)
lmcache/v1/compute/positional_encoding.py:110
↓ 1 callers
Function
validate_rope_params
( head_size: int, rotary_dim: int, max_position: int, base: float, is_neox_style: bool = T
lmcache/v1/compute/positional_encoding.py:83
↓ 1 callers
Function
validate_storage_manager_config
Validate storage manager configuration. This rejects L2 adapters that require a single contiguous L1 memory descriptor when hybrid L1 Device-
lmcache/v1/distributed/config.py:276
↓ 1 callers
Function
verify_kv_cache_pattern
Verify that the KV cache contains the expected pattern at the specified slot mappings
tests/disagg/test_nixl_cache_engine.py:68
↓ 1 callers
Function
verify_round_trip
Verify that loaded data matches what was stored. Compares the underlying ``raw_data`` tensors directly (more efficient than converting via ``
lmcache/cli/commands/bench/l2_adapter_bench/data.py:176
↓ 1 callers
Function
visualize_results
(warmup_df, benchmark_df)
benchmarks/long_doc_qa/long_doc_qa.py:400
↓ 1 callers
Function
vllmContainerArgs
vllmContainerArgs returns the args of the vLLM container in the pod (the one named "vllm", falling back to the first container). The mutating webhook
operator/test/e2e/cacheblend_integration_smoke_test.go:274
↓ 1 callers
Method
wait
Wait for the future to be done, with the CUDA stream. Args: timeout (Optional[float]): Maximum time to wait for the UNDE
lmcache/v1/multiprocess/futures.py:116
↓ 1 callers
Method
wait
Make the given stream wait until this event completes. Args: stream: The stream that should wait. If None, uses the
lmcache/v1/platform/cpu/stub_cpu_device.py:54
↓ 1 callers
Function
waitDeploymentAvailable
waitDeploymentAvailable polls a Deployment's status until the Available condition is True, or until ctx is cancelled / timeout elapses. Uses kubectl r
operator/test/e2e/e2e_suite_test.go:168
↓ 1 callers
Function
waitForLocalPort
waitForLocalPort polls 127.0.0.1:port until a TCP connection succeeds or timeout elapses. kubectl port-forward briefly accepts connections and immedia
operator/test/utils/portforward.go:161
↓ 1 callers
Function
wait_for_condition
(predicate, timeout: float = 10.0, poll_interval: float = 0.05)
tests/v1/distributed/test_multimodel.py:49
↓ 1 callers
Function
wait_for_keys_in_cache
Wait for keys to appear in the backend's cache.
tests/v1/storage_backend/test_p2p_backend_with_controller.py:38
↓ 1 callers
Function
wait_for_prefetch_status
Poll query_prefetch_status until non-None or timeout.
tests/v1/distributed/serde/test_serde_fs_e2e.py:67
↓ 1 callers
Function
wait_put_tasks_complete
Wait for remote backend put tasks to complete
lmcache/v1/check/utils.py:164
↓ 1 callers
Method
warmup
Run warmup requests. Blocks until all warmup is done.
lmcache/cli/commands/bench/engine_bench/workloads/base.py:43
← previous
next →
4,701–4,800 of 13,371, ranked by callers