Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 1 callers
Method
_sparse_classify
Classify each prefetched chunk as found or stale, and finalize state. A chunk is found only if every TP rank's key loaded; stale chunks take
lmcache/v1/multiprocess/modules/blend_v3.py:674
↓ 1 callers
Method
_sparse_prefetch_submit
Coalesce all matches into one sparse L2->L1 prefetch and submit it. Non-blocking. Dedups object keys before submit (sparse keeps one read
lmcache/v1/multiprocess/modules/blend_v3.py:621
↓ 1 callers
Function
_split_kv
Split a blob KV tensor to K and V tensors with the merged heads Input: tensor: the KV tensor with shape [num_layers, 2,
lmcache/storage_backend/serde/cachegen_encoder.py:86
↓ 1 callers
Function
_sse_piece
(obj: dict[str, Any], chat: bool)
lmcache/cli/commands/query/_request.py:108
↓ 1 callers
Function
_stable_u64_from_str
Hash an arbitrary string into a stable 64-bit unsigned int. Used to project arbitrary multimodal-hash strings (which may be opaque, not hex)
lmcache/v1/ec_engine.py:43
↓ 1 callers
Method
_start_background_thread
Start background thread for periodic flushing.
lmcache/v1/storage_backend/batched_message_sender.py:94
↓ 1 callers
Method
_start_heartbeat
(self)
lmcache/integration/sglang/multi_process_adapter.py:170
↓ 1 callers
Method
_start_lookup_phase
Submit lookup_and_lock to all live (non-draining) adapters for a new request.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:760
↓ 1 callers
Function
_start_loop
Start an async event loop in a background thread.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:77
↓ 1 callers
Method
_start_orchestration
Initialize the transfer channel and start the peer-poll thread. Args: coordinator_config: Coordinator connection used for peer di
lmcache/v1/multiprocess/modules/p2p_controller.py:132
↓ 1 callers
Method
_start_pending_requests
Start pending requests up to the max in-flight limit.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:745
↓ 1 callers
Function
_startup_resync
One-shot backfill of usage + eviction trackers from a live MP server's actual L2 contents.
lmcache/v1/mp_coordinator/app.py:107
↓ 1 callers
Function
_stop_loop
Stop the async event loop.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:85
↓ 1 callers
Method
_store_one_locked
(self, key: ObjectKey, obj: MemoryObj)
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:1156
↓ 1 callers
Function
_stream
POST with ``stream: true``; parse SSE; return TTFT/TPOT and token metrics.
lmcache/cli/commands/query/_request.py:125
↓ 1 callers
Function
_string_to_object_key
Reverse of :func:`_object_key_to_string`. Expects ``<model_name>@<kv_rank_hex>@<object_group_id_hex>@<chunk_hash_hex>[@<cache_salt>]``.
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:67
↓ 1 callers
Function
_stub_dynamic_agent_runtime
Stub the NixlDynamicStorageAgent runtime methods invoked by storage_to_mem. The constructor was already stubbed by _stub_dynamic_agent; this fill
tests/v1/storage_backend/test_nixl_shared_pool.py:271
↓ 1 callers
Function
_stub_gpu_allocator
(monkeypatch, allocator)
tests/v1/storage_backend/test_nixl_shared_pool.py:228
↓ 1 callers
Method
_submit_async_mem_to_storage
Post the transfer and hand cleanup off to a background task. On any failure before the task is scheduled, ownership has not transferr
lmcache/v1/storage_backend/nixl_storage_backend.py:1796
↓ 1 callers
Method
_submit_coordinator_match
Issue a fleet directory match query for this request (best-effort). Args: key: The lookup request key. Returns:
lmcache/v1/multiprocess/modules/blend_v3.py:1242
↓ 1 callers
Method
_submit_prefix_leg
Submit the CB prefix prefetch (non-blocking). Opens the ``cb.prefix_lookup`` span (CB namespace — CB requests no longer feed the MP r
lmcache/v1/multiprocess/modules/blend_v3.py:747
↓ 1 callers
Method
_submit_put_many
Persist multiple legacy raw-block keys in one background batch. Args: pending: Ordered ``(key, spec, memory_obj)`` tuples to pers
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:451
↓ 1 callers
Method
_submit_put_one
( self, key: CacheEngineKey, spec: RawBlockKeySpec, memory_obj: MemoryObj,
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:426
↓ 1 callers
Method
_submit_put_tasks
Submit put tasks using callback pattern.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:310
↓ 1 callers
Method
_submit_retrieve
( self, request_id: str, token_ids: list[int], offset: int, matched_en
lmcache/integration/sglang/multi_process_adapter.py:380
↓ 1 callers
Method
_submit_store_for_single_shape
Submit ``keys`` (all same shape) to their target adapters.
lmcache/v1/distributed/storage_controllers/store_controller.py:573
↓ 1 callers
Method
_submit_tasks
Submit tasks (put or get) based on the benchmark mode. Args: keys: List of cache keys. objs: List of memory objects f
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:293
↓ 1 callers
Method
_submit_write
( self, key: CacheEngineKey, memory_obj: MemoryObj, on_complete_callback: Opti
lmcache/v1/storage_backend/plugins/dax_backend.py:901
↓ 1 callers
Function
_tensor_from_cpu_ptr
Zero-copy CPU tensor from a raw host pointer via ctypes + torch.frombuffer.
lmcache/python_ops_fallback.py:138
↓ 1 callers
Function
_tensor_from_cuda_ptr
Zero-copy CUDA tensor from a raw device pointer.
lmcache/python_ops_fallback.py:158
↓ 1 callers
Function
_tensor_list
Return ``value`` as ``list[torch.Tensor]`` when it is tensor-backed.
lmcache/v1/platform/musa/ops.py:19
↓ 1 callers
Function
_test_single_adapter
Run all test phases against one adapter.
lmcache/v1/check/check_mode_test_l2_adapter.py:192
↓ 1 callers
Function
_time_call
Time a synchronized MUSA callable.
benchmarks/musa/bench_inprocess_transfer.py:249
↓ 1 callers
Function
_to_block_id_list
Convert block IDs from tensor/list form into Python integers.
lmcache/v1/platform/musa/native_kv_transfer.py:353
↓ 1 callers
Function
_to_bool
(value: Any)
lmcache/v1/storage_backend/plugins/dax_backend.py:33
↓ 1 callers
Function
_transfer_cross_layer
Handle cross-layer formats: single tensor [NB, NL, 2, ...].
lmcache/python_ops_fallback.py:1280
↓ 1 callers
Function
_transfer_per_layer_hnd
Handle per-layer HND formats: heads before block tokens.
lmcache/python_ops_fallback.py:1516
↓ 1 callers
Function
_transfer_per_layer_mla
Handle MLA per-layer formats: [NB, BS, HS].
lmcache/python_ops_fallback.py:1442
↓ 1 callers
Function
_transfer_per_layer_nhd
Handle per-layer NHD formats: block tokens before heads.
lmcache/python_ops_fallback.py:1640
↓ 1 callers
Function
_transfer_sglang_mha
Handle SGLang MHA formats: 2*NL tensors (list[list[Tensor]]).
lmcache/python_ops_fallback.py:1362
↓ 1 callers
Method
_transition_to_load_phase
Compute load plan, reserve L1 buffers, and submit load tasks.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:814
↓ 1 callers
Method
_trigger_stop
(self, reason: str)
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:190
↓ 1 callers
Method
_try_get_hash
Try to get hash function, handling sha256_cbor_64bit rename.
lmcache/v1/token_database.py:178
↓ 1 callers
Method
_try_get_hash
Try to get hash function, handling sha256_cbor_64bit rename. Adapted from TokenDatabase._try_get_hash (token_database.py:152-168).
lmcache/v1/multiprocess/token_hasher.py:129
↓ 1 callers
Function
_try_load_tokenizer
Best-effort load of the model's tokenizer. Used to generate body content as random valid token-IDs, then decoded back to text — guaranteeing
lmcache/cli/commands/bench/engine_bench/workloads/prefix_suffix_tuner.py:312
↓ 1 callers
Function
_try_native_block_transfer_from_gpu
Try native paged-KV-to-object gather for each LMCache object.
lmcache/v1/platform/musa/native_kv_transfer.py:402
↓ 1 callers
Function
_try_native_block_transfer_to_gpu
Try native object-to-paged-KV scatter for each LMCache object.
lmcache/v1/platform/musa/native_kv_transfer.py:444
↓ 1 callers
Method
_try_reinitialize_connection
Try to reinitialize the connection if connector is None. Returns: bool: True if connection was successfully initialized,
lmcache/v1/health_monitor/checks/remote_backend_check.py:141
↓ 1 callers
Function
_tuple_kv_to_blob
( kv_tensors, )
tests/v1/test_mem_kernels.py:59
↓ 1 callers
Function
_unique_token_coverage
Return the number of unique query tokens covered by a set of CBMatchResults. match_sub_sequence is a sliding-window probe, so two results from di
lmcache/v1/multiprocess/modules/blend.py:286
↓ 1 callers
Function
_unique_type_name
()
tests/v1/distributed/transfer_channel/test_context_lifecycle.py:34
↓ 1 callers
Function
_unlink_stale_shm
Remove a stale LMCache shm segment if it exists.
lmcache/v1/distributed/memory_manager/l1_memory_manager.py:25
↓ 1 callers
Method
_unlock_unneeded_keys
Phase 1 unlock: keys locked in lookup but not in the load plan.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:1160
↓ 1 callers
Method
_unpin_if_needed
(self, encoded_key: str)
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:678
↓ 1 callers
Method
_unregister_cpu_buffer
Unregister CPU buffer.
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:464
↓ 1 callers
Method
_unregister_cuda_host_memory
(self)
lmcache/v1/memory_management.py:2723
↓ 1 callers
Method
_unwrap_reconfigurable_l2_adapter
( self, adapter: L2AdapterInterface, )
lmcache/v1/distributed/storage_manager.py:1050
↓ 1 callers
Method
_update_and_get_finished_store
Converge the internal states about finished stores and returns the 'safe finished store request ids' back
lmcache/integration/vllm/vllm_multi_process_adapter.py:1737
↓ 1 callers
Method
_update_result
(self, response: Response)
benchmarks/multi_round_qa/multi-round-qa.py:214
↓ 1 callers
Method
_uses_futures_pattern
Check if this backend uses futures pattern. Override in subclass to indicate futures pattern usage.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:490
↓ 1 callers
Function
_v2_retrieve_vllm_contract
Mimic vLLM non-layerwise retrieve contract for V2 connector.
tests/benchmarks/test_xpu_connector_benchmark.py:123
↓ 1 callers
Function
_v3_filtered_to_block_aligned
Set of V3 matches that survive retrieve's ``cur_st % block_size == 0`` drop. Strided probe's correctness target: it must find at least every
tests/v1/multiprocess/test_optimized_lookup_v3.py:593
↓ 1 callers
Method
_validate_config
Validate the configuration. Returns: True if valid, False otherwise
lmcache/v1/storage_backend/connector/hf3fs_adapter.py:152
↓ 1 callers
Function
_validate_dim0_padded_layout
Validate that *tensor* matches the dim-0-padding-only strided layout. Mainly used for DeepSeek V4 integration, where compressor / indexer KV
lmcache/v1/gpu_connector/kv_format/contiguity.py:76
↓ 1 callers
Method
_validate_format_transition
( self, mem: MemoryObj, target_fmt: MemoryFormat )
lmcache/v1/gpu_connector/musa_connectors.py:480
↓ 1 callers
Method
_validate_full_chunk_for_upload
Reject partial or metadata-bearing uploads for the conservative MVP.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:531
↓ 1 callers
Function
_validate_huggingface_hub_support
Raise a clear error when the installed ``huggingface_hub`` is too old. Assumes ``huggingface_hub`` is importable (guaranteed by the declared
lmcache/v1/storage_backend/connector/hfbucket_connector.py:863
↓ 1 callers
Method
_validate_loaded_entries
Drop recovered entries whose slot headers do not match metadata.
lmcache/v1/storage_backend/raw_block/core.py:1877
↓ 1 callers
Method
_validate_num_workers
Validate and return a positive integer worker count. Raises: ValueError: If ``raw`` is not a positive integer.
lmcache/v1/distributed/l2_adapters/config.py:282
↓ 1 callers
Method
_validate_paths
parse the base_paths_str which separated by comma, and validate the paths are subdirectory of mount point path
lmcache/v1/storage_backend/connector/hf3fs_adapter.py:189
↓ 1 callers
Method
_validate_per_op_workers
Validate per-operation worker counts (``None`` is a no-op). Raises: ValueError: If any worker count is not a positive integer.
lmcache/v1/distributed/l2_adapters/config.py:293
↓ 1 callers
Function
_validate_per_tp_device_paths
Validate that each TP rank uses a distinct raw-block device path. Args: per_tp_devices: Mapping from TP rank to raw-block device path.
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:43
↓ 1 callers
Method
_validate_remove_mode
(mode: str)
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:1560
↓ 1 callers
Method
_validate_resize_mode
(mode: str)
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:1565
↓ 1 callers
Function
_validate_vllm_device_features
Reject vLLM configurations that can't run on the active device. Three classes of misconfiguration are rejected here, all *before* any device-
lmcache/v1/gpu_connector/__init__.py:23
↓ 1 callers
Function
_verify
Check each destination object holds the expected per-object byte pattern. Raises: RuntimeError: If any destination object does not match.
lmcache/tools/transfer_channel_benchmark/benchmark.py:320
↓ 1 callers
Method
_verify_integrity
Verify data integrity between original and read data. Returns: Tuple of (error_count, is_passed)
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:593
↓ 1 callers
Function
_wait_fd
(fd: int, timeout: float = 5.0)
tests/v1/distributed/test_cache_salt_l2_eviction.py:75
↓ 1 callers
Method
_wait_for_completion
Wait using Event pattern.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:345
↓ 1 callers
Method
_wait_for_futures
Wait for all futures to complete.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:497
↓ 1 callers
Method
_wait_for_lookup
Wait for the LOOKUP's prefetch to finish and return the matched bytes. Sends a single blocking WAIT_PREFETCH_STATUS request so the daemon
lmcache/integration/sglang/multi_process_adapter.py:235
↓ 1 callers
Method
_wait_for_specific_worker
Wait for specific worker to be available
tests/v1/internal_api_server/test_worker_info_api.py:164
↓ 1 callers
Method
_wait_for_transfer
Asynchronously wait for transfer to complete without blocking. On any non-DONE outcome (``ERR`` or earlier exception), unlink any FIL
lmcache/v1/storage_backend/nixl_storage_backend.py:1730
↓ 1 callers
Function
_wait_load_finished
Wait for all load tasks to finish. Returns ``{task_id: bitmap}``. ``query_load_result`` consumes the per-task result, so we cache the bitmaps
lmcache/cli/commands/bench/l2_adapter_bench/runner.py:76
↓ 1 callers
Function
_wait_lookup_finished
Wait for all lookup-and-lock tasks to finish. Returns ``{task_id: bitmap}``. ``query_lookup_and_lock_result`` consumes the per-task result, s
lmcache/cli/commands/bench/l2_adapter_bench/runner.py:102
↓ 1 callers
Function
_wait_store_finished
Wait for all store tasks to finish. Returns the accumulated ``{task_id: L2StoreResult}`` dict. ``pop_completed_store_tasks`` consumes the ada
lmcache/cli/commands/bench/l2_adapter_bench/runner.py:52
↓ 1 callers
Function
_walk
( current_pkg: ModuleType, current_path: Sequence[str], depth: int, )
lmcache/v1/utils/subclass_discovery.py:137
↓ 1 callers
Method
_warn_if_loaded_metadata_looks_cross_rank
(self)
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:319
↓ 1 callers
Function
_weak_completions_error
(msg: str)
lmcache/cli/commands/query/_request.py:241
↓ 1 callers
Method
_worker
(self)
lmcache/v1/storage_backend/job_executor/pq_executor.py:55
↓ 1 callers
Method
_worker
(self)
lmcache/v1/storage_backend/job_executor/pq_executor.py:159
↓ 1 callers
Function
_wrap_sglang_kv_caches
Flatten SGLang's depth-2 ``[K_layers, V_layers]`` KV layout into a single flat ``KVCache`` so it fits upstream's wire ``KVCache`` payload type
lmcache/integration/sglang/multi_process_adapter.py:47
↓ 1 callers
Method
_write
(self, fd: int, buffer: memoryview, length: int, file_offset: int)
lmcache/v1/storage_backend/connector/hf3fs_connector.py:501
↓ 1 callers
Method
_write_checkpoint
Write one checkpoint copy and advance persisted metadata counters.
lmcache/v1/storage_backend/raw_block/core.py:1627
↓ 1 callers
Method
_write_mem_obj
(self, mem_obj: MemoryObj, data: bytes, offset: int)
lmcache/v1/storage_backend/connector/s3_connector.py:294
↓ 1 callers
Method
_write_one
Write one object header and payload into a raw-block slot. Args: key: Raw-block key spec with the slot-header identity.
lmcache/v1/storage_backend/raw_block/core.py:1350
↓ 1 callers
Method
_write_uring_cmd_buffers
Write buffers as bounded NVMe raw-command chunks. Args: offsets: Device offsets for each logical write. buffers: Sour
lmcache/v1/storage_backend/raw_block/core.py:1127
↓ 1 callers
Function
_zero_fill_client_blocks
Zero out a paged block range across all client tensors. Used right before a warm-pass ``RETRIEVE`` so that any non-zero bytes observed afterw
lmcache/cli/commands/bench/server_bench/helpers.py:658
↓ 1 callers
Method
access
Mark an existing entry as most-recently used. O(1).
lmcache/tools/cache_simulator/lru_cache.py:45
↓ 1 callers
Function
active_adapter_ids
Set of active adapter ids exposed by the public ``l2_adapters()`` API.
tests/v1/distributed/test_runtime_adapters.py:100
← previous
next →
3,701–3,800 of 13,371, ranked by callers