Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 2 callers
Function
run_gen_dataset
Execute the gen-dataset workflow from a parsed argument namespace. Loads events via :func:`~lmcache.tools.cache_simulator.simulator.load_look
lmcache/tools/cache_simulator/gen_bench_dataset.py:295
↓ 2 callers
Function
run_http_server
Run the LMCache HTTP server with integrated MP (ZMQ) server. Args: http_config: Configuration for the HTTP frontend mp_confi
lmcache/v1/multiprocess/http_server.py:198
↓ 2 callers
Function
run_simulate
Execute the simulate workflow from a parsed argument namespace. Loads events, resolves ``kv_bytes_per_chunk``, runs the simulator, prints a t
lmcache/tools/cache_simulator/simulator.py:722
↓ 2 callers
Function
run_sweep
Execute the sweep workflow from a parsed argument namespace. Loads events, resolves ``kv_bytes_per_chunk``, sweeps across a log-spaced range
lmcache/tools/cache_simulator/plot_hit_rate.py:127
↓ 2 callers
Function
safe_storage_config_dict
Best-effort JSON-serializable dict view of a StorageManagerConfig. Falls back to ``str(config)`` for fields that are not directly serializabl
lmcache/v1/mp_observability/trace/recorder.py:254
↓ 2 callers
Function
sections_to_dict
Convert a title and sections to a JSON-serialisable dictionary. Named sections become nested dicts keyed by machine key. The unnamed default
lmcache/cli/metrics/section.py:42
↓ 2 callers
Method
select_l1_deletions
Never delete from L1. Args: keys: Keys that were successfully stored to L2. Returns: Empty list (ke
lmcache/v1/distributed/storage_controllers/store_policy.py:167
↓ 2 callers
Method
select_load_plan
Decide which adapter loads which keys. Args: keys: Full list of keys being prefetched from L2. lookup_result
lmcache/v1/distributed/storage_controllers/prefetch_policy.py:31
↓ 2 callers
Method
send_request
Send a message via ZMQ DEALER socket and wait for reply Args: message: Message to send Returns: Tuple of (ti
lmcache/tools/controller_benchmark/benchmark.py:221
↓ 2 callers
Method
send_response_to_scheduler
(self, lookup_id: str, num_hit_tokens: int)
lmcache/v1/lookup_client/lmcache_async_lookup_client.py:387
↓ 2 callers
Method
serialize
Cast src tensor to fp8 and copy bytes into dst buffer.
lmcache/v1/distributed/serde/fp8.py:34
↓ 2 callers
Function
serialize_header
Pack the EncodedKV header into bytes (excludes payload). Computes payload CRC32 from `enc.payload` and writes it as the last field. Caller i
lmcache/v1/kv_codec/encoded_kv.py:236
↓ 2 callers
Method
serialize_into
(self, buffer)
lmcache/v1/protocol.py:174
↓ 2 callers
Function
setImageSpec
setImageSpec rewrites an ImageSpec's repository+tag from a "repo:tag" image reference (the env-knob form), allocating the ImageSpec if nil. A referenc
operator/test/e2e/cacheblend_integration_smoke_test.go:240
↓ 2 callers
Method
set_error
(self, msg: str | None)
tests/v1/distributed/test_s3_l2_adapter.py:79
↓ 2 callers
Method
set_hot_cache
Dynamically enable or disable the hot cache on LocalCPUBackend. When disabled, the existing hot cache entries will be cleared
lmcache/v1/storage_backend/storage_manager.py:828
↓ 2 callers
Function
setup
()
tests/benchmarks/test_benchmark.py:206
↓ 2 callers
Function
setup_mooncake_store
Initialize a MooncakeDistributedStore instance. Calls ``store.setup()`` using the dict-based API introduced in Mooncake PR #1445 when availab
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:67
↓ 2 callers
Method
should_skip
(self)
tests/v1/test_health_monitor.py:42
↓ 2 callers
Method
should_skip
Check if this health check should be skipped. Override this method to conditionally skip checks (e.g., if the component does
lmcache/v1/health_monitor/base.py:103
↓ 2 callers
Function
should_translate
Return True if the entry has English content and is empty or fuzzy.
tools/translate_docs_zh.py:197
↓ 2 callers
Function
should_use_lazy_alloc
Determine if lazy allocation should be used based on CUDA availability.
tests/v1/distributed/test_l1_memory_manager.py:48
↓ 2 callers
Function
should_use_lazy_alloc
Determine if lazy allocation should be used based on CUDA availability.
tests/v1/distributed/test_distributed_storage_manager.py:43
↓ 2 callers
Function
should_use_lazy_alloc
()
tests/v1/distributed/test_runtime_adapters.py:66
↓ 2 callers
Function
should_use_lazy_alloc
Determine if lazy allocation should be used based on CUDA availability.
tests/v1/distributed/test_l1_manager.py:74
↓ 2 callers
Method
simulate_successful_ping
Mimic one successful heartbeat cycle: on the unhealthy->healthy edge the recover callback runs first, and the event is set only when t
tests/v1/test_vllm_mp_adapter.py:83
↓ 2 callers
Function
single_to_multi_deserializer
Wrap a single-tensor :class:`Deserializer` as a length-1 :class:`MultiDeserializer`. The wrapper is layout-equivalent: identical bytes are wr
lmcache/v1/distributed/serde/multi.py:261
↓ 2 callers
Method
size_bytes
(self)
tests/v1/storage_backend/test_rust_raw_block_backend.py:132
↓ 2 callers
Method
slice_kv_cache_on_tokens
Slices the KV cache tensor on the token dimension
lmcache/v1/platform/cuda/cache_context.py:594
↓ 2 callers
Method
stage_block_ids
Stage per-group block IDs into the shared staging buffer. Returns one non-overlapping view per LMCache group.
lmcache/v1/platform/base_cache_context.py:226
↓ 2 callers
Method
stale
Return the ids of instances whose heartbeat has expired. Args: timeout: Maximum allowed seconds since the last heartbeat.
lmcache/v1/mp_coordinator/registry.py:159
↓ 2 callers
Method
start
Start the LMCache engine
lmcache/v1/standalone/__main__.py:289
↓ 2 callers
Method
start
(self)
lmcache/v1/multiprocess/native_completion.py:80
↓ 2 callers
Method
start
Start the health monitor thread. Returns: Optional[threading.Thread]: The started thread, or None if no
lmcache/v1/health_monitor/base.py:490
↓ 2 callers
Method
start
Start heartbeat thread
lmcache/lmcache_frontend/heartbeat.py:192
↓ 2 callers
Method
start_all
(self)
lmcache/v1/cache_controller/worker.py:633
↓ 2 callers
Method
status
Return a snapshot of the heartbeat thread state. Returns: Dict containing ``running`` (bool), ``local_ip``, ``startup
lmcache/lmcache_frontend/heartbeat.py:220
↓ 2 callers
Method
stop
Stop the LMCache engine
lmcache/v1/standalone/__main__.py:331
↓ 2 callers
Method
stop_plugins
Terminate all plugin processes.
lmcache/v1/multiprocess/mp_runtime_plugin_launcher.py:107
↓ 2 callers
Function
store_keys_in_l2
Push test data into L2 directly so subsequent prefetches hit.
tests/v1/distributed/test_l1_l2_state_metrics.py:104
↓ 2 callers
Method
store_layer
Store the KV cache in a layerwise manner. :param torch.Tensor tokens: The tokens of the corresponding KV caches. :param Opt
lmcache/v1/cache_engine.py:590
↓ 2 callers
Method
stream
(self, _stream: object)
tests/v1/test_musa_connector.py:118
↓ 2 callers
Method
stream
Returns the GPU stream for KV cache operations
lmcache/v1/platform/cuda/cache_context.py:447
↓ 2 callers
Function
stream_service_response
Asynchronously stream the response from a service using a persistent client.
examples/disagg_prefill/disagg_proxy_server.py:412
↓ 2 callers
Method
submit_get_into
Submit a GET-into-buffer operation.
lmcache/v1/storage_backend/connector/valkey_connector.py:258
↓ 2 callers
Method
submit_load_task
Submit a non-blocking raw-block load task. Args: keys: Object keys to load. objects: Caller-provided destination buff
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:491
↓ 2 callers
Method
submit_task
( self, task_type: str, task: Callable, *args, **kwargs, )
lmcache/v1/storage_backend/local_disk_backend.py:52
↓ 2 callers
Function
substituteVLLMPlaceholders
substituteVLLMPlaceholders renders the vllm Deployment fixture for a concrete run. Kept as a tiny function so the template stays readable and so we do
operator/test/e2e/vllm_integration_smoke_test.go:186
↓ 2 callers
Method
summary
(self)
benchmarks/multi_round_qa/multi-round-qa.py:339
↓ 2 callers
Method
summary
(self, start_time: float, end_time: float)
benchmarks/multi_round_qa/multi-round-qa.py:548
↓ 2 callers
Method
summary
(self, start_time: float, end_time: float)
benchmarks/rag/rag.py:303
↓ 2 callers
Method
support_batched_async_contains
(self)
lmcache/v1/storage_backend/remote_backend.py:506
↓ 2 callers
Method
support_batched_get
(self)
lmcache/v1/storage_backend/connector/s3_connector.py:662
↓ 2 callers
Function
test_get
(eic_client)
tests/v1/storage_backend/test_eic.py:90
↓ 2 callers
Function
test_long_document_qa
Test long document QA with the given prompts and sampling parameters. Process prompts concurrently with a limit on inflight requests. Ar
benchmarks/multi_doc_qa/multi_doc_qa.py:158
↓ 2 callers
Method
title
Set the report title. Args: title: New report title shown in the header.
lmcache/cli/metrics/metrics.py:50
↓ 2 callers
Method
to_dict
Serialize the slot descriptor into the MQ context schema. Returns: Dict payload shared between the server and worker for one SHM
lmcache/v1/multiprocess/transfer_context/shm.py:39
↓ 2 callers
Method
to_gpu
Expect a kwarg 'kvcaches' which is a nested tuple of K and V tensors. The kvcaches should correspond to the "WHOLE token sequence". N
lmcache/v1/gpu_connector/gpu_connectors.py:267
↓ 2 callers
Method
to_gpu
(self, memory_obj: MemoryObj, start: int, end: int, **kwargs)
lmcache/v1/gpu_connector/gpu_connectors.py:531
↓ 2 callers
Method
to_gpu
Expect a kwarg 'kvcaches' which is a nested tuple of K and V tensors. The kvcaches should correspond to the "WHOLE token sequence". N
lmcache/v1/gpu_connector/xpu_connectors.py:142
↓ 2 callers
Method
to_gpu
(self, memory_obj: MemoryObj, start: int, end: int, **kwargs)
lmcache/v1/gpu_connector/xpu_connectors.py:369
↓ 2 callers
Method
to_json
Serialize to a JSON-compatible dict for config export. Excludes environment-specific keys (``engine_url``, ``lmcache_url``) and inter
lmcache/cli/commands/bench/engine_bench/interactive/state.py:258
↓ 2 callers
Method
to_json
(self)
lmcache/v1/multiprocess/mp_runtime_plugin_launcher.py:45
↓ 2 callers
Method
to_tensor
Reconstruct the tensor by mapping the same SHM segment. The returned tensor owns the mmap: a ``weakref.finalize`` hook runs ``munmap`
lmcache/v1/platform/cpu/shm.py:100
↓ 2 callers
Function
torch_quant
Quantize a float tensor to fixed number of bins Input: bins: number of bins qA: the input tensor Returns: xq: t
lmcache/storage_backend/serde/cachegen_encoder.py:27
↓ 2 callers
Function
torch_quant_vectorized
Quantize each group of a tensor to fixed number of bins Input: bins: number of bins for different layers, with shape [nlayer]
lmcache/storage_backend/serde/cachegen_encoder.py:48
↓ 2 callers
Method
touch_instance
(self, instance_id: int)
tests/v1/multiprocess/test_worker_liveness.py:189
↓ 2 callers
Function
transfer_kv_per_object_group
Helper function to transfer memory objects of a single object group to/from GPU, with batching support. Args: cache_context: The GPU
lmcache/v1/multiprocess/modules/lmcache_driven_transfer.py:233
↓ 2 callers
Function
trim_load_plan_with_mask
Trim the load plan to the key indices set in ``mask`` (gap-tolerant). Args: load_plan: Mapping from adapter index to Bitmap of key indice
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:92
↓ 2 callers
Function
trimmed_mean
Exclude the smallest trim_fraction and largest trim_fraction and take mean of the rest. If trim_fraction <= 0, returns normal mean.
benchmarks/long_doc_qa/long_doc_qa.py:459
↓ 2 callers
Function
try_native_from_gpu
Try native paged-KV-to-contiguous-buffer gather. Args: use_mla: Whether the active layout is MLA. memory_tensor: LMCache contiguo
lmcache/v1/platform/musa/native_kv_transfer.py:244
↓ 2 callers
Function
try_native_to_gpu
Try native contiguous-buffer-to-paged-KV scatter. Args: use_mla: Whether the active layout is MLA. memory_tensor: LMCache contigu
lmcache/v1/platform/musa/native_kv_transfer.py:173
↓ 2 callers
Function
ttft_stream
( client: OpenAI, model: str, messages: list[dict], printer: Printer | None = None, )
examples/online_session/openai_chat_completion_client.py:98
↓ 2 callers
Function
unescape_salt
Translate the URL sentinel back to the empty-string salt.
lmcache/cli/commands/quota/_helpers.py:33
↓ 2 callers
Method
unpin
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/local_cpu_backend.py:263
↓ 2 callers
Method
unpin
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/pd_backend.py:1084
↓ 2 callers
Method
unpin
Decrement the external lock count for ``key``. Args: key: The cache key to unpin. Returns: ``True`` if ``key
lmcache/v1/storage_backend/plugins/dax_backend.py:289
↓ 2 callers
Method
unregister_kv_cache
Unregister the KV cache tensors for a given GPU instance ID. Args: instance_id: The GPU instance ID (such as PID).
lmcache/v1/multiprocess/modules/lmcache_driven_transfer.py:691
↓ 2 callers
Function
unwrap_request_payloads
( b_payloads: list[bytes], payload_clss: list[Any] )
lmcache/v1/multiprocess/mq.py:53
↓ 2 callers
Method
update_chunk_hash_dict
(self, key: KeyType)
lmcache/v1/storage_backend/cache_policy/lru.py:30
↓ 2 callers
Method
update_heartbeat
Update worker heartbeat timestamp. Returns True if successful.
lmcache/v1/cache_controller/utils.py:541
↓ 2 callers
Method
update_local_cpu_evict_failed_count
(self, evict_failed_count: int)
lmcache/observability.py:571
↓ 2 callers
Method
update_local_cpu_evict_metrics
(self, evict_keys_count: int)
lmcache/observability.py:566
↓ 2 callers
Method
update_pinned_memory_objs_count
(self, delta: int)
lmcache/observability.py:583
↓ 2 callers
Method
update_remote_cache_usage
(self, usage: int)
lmcache/observability.py:523
↓ 2 callers
Method
upload_files
Upload files in a single batch operation.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:151
↓ 2 callers
Method
usage
Return current and post-eviction slot usage fractions. Returns: ``(current_usage, usage_after_ongoing_eviction)`` where both valu
lmcache/v1/storage_backend/dax/core.py:546
↓ 2 callers
Function
validateCoordinatorConnectionSpec
validateCoordinatorConnectionSpec validates the shared coordinator connection configuration used by both LMCacheEngine and CacheBlendEngine. A nil spe
operator/api/v1alpha1/lmcacheengine_validation.go:53
↓ 2 callers
Function
validateEvictionSpec
validateEvictionSpec validates the shared eviction configuration used by both LMCacheEngine and CacheBlendEngine. A nil eviction is valid (defaults ap
operator/api/v1alpha1/lmcacheengine_validation.go:82
↓ 2 callers
Function
validateL2BackendSpec
validateL2BackendSpec validates the shared L2 backend configuration used by both LMCacheEngine and CacheBlendEngine. A nil backend is valid (L2 is opt
operator/api/v1alpha1/lmcacheengine_validation.go:113
↓ 2 callers
Function
validate_kv_cache_groups
Reject KV cache group specs the transfer path cannot serve correctly. Rejected, with one aggregated error listing every offending group: - `
lmcache/integration/vllm/kv_cache_group_edits.py:97
↓ 2 callers
Function
wait_decode_kv_ready
(req_id: str, num_tp_rank: int)
examples/disagg_prefill/disagg_proxy_server.py:442
↓ 2 callers
Method
wait_for_async_processing
(self, timeout: float = 5.0)
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:118
↓ 2 callers
Function
wait_for_backend_ready
Wait for P2P backend to be ready for operations.
tests/v1/storage_backend/test_p2p_backend_with_controller.py:58
↓ 2 callers
Function
wait_for_prefetch_result_bitmap
Poll query_prefetch_result, returning the raw retained Bitmap. Unlike :func:`wait_for_prefetch_result`, this keeps the full bitmap so a calle
tests/v1/distributed/test_prefetch_controller.py:124
↓ 2 callers
Function
wait_for_sparse_found
Poll query_prefetch_status; return the found-key index set. For SPARSE prefetches the result bitmap is gap-tolerant, so callers read the full
tests/v1/distributed/test_distributed_storage_manager.py:178
↓ 2 callers
Method
workload_items_all_default
True if no workload-specific items have been explicitly set.
lmcache/cli/commands/bench/engine_bench/interactive/state.py:146
↓ 1 callers
Function
BuildCBArgs
BuildCBArgs returns the target vLLM container's args with the CacheBlend required flag set applied (M5). Each flag is appended-or-replaced (design §9.
operator/internal/webhook/cacheblend_inject_builders.go:209
↓ 1 callers
Function
BuildCBEngineLookupService
BuildCBEngineLookupService creates the node-local lookup Service (internalTrafficPolicy=Local) for the CacheBlendEngine, so opted-in vLLM pods reach t
operator/internal/resources/cacheblend_engine.go:129
↓ 1 callers
Function
BuildCBEngineMetricsService
BuildCBEngineMetricsService creates the headless metrics Service for the CacheBlendEngine.
operator/internal/resources/cacheblend_engine.go:135
← previous
next →
2,901–3,000 of 13,371, ranked by callers