Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 1 callers
Function
_group_keys_by_shape
Group ``keys`` by the fields that determine their KV cache shape. Each bucket shares a single ``(shape, dtype)``, so each bucket can be submi
lmcache/v1/distributed/storage_controllers/store_controller.py:49
↓ 1 callers
Method
_handle_batched_lookup_and_get
( self, msg: BatchedLookupAndGetMsg )
lmcache/v1/storage_backend/p2p_backend.py:384
↓ 1 callers
Method
_handle_batched_lookup_and_put
( self, msg: BatchedLookupAndPutMsg )
lmcache/v1/storage_backend/p2p_backend.py:445
↓ 1 callers
Function
_handle_export
Prompt for filename, resolve values, save JSON, and exit.
lmcache/cli/commands/bench/engine_bench/interactive/__init__.py:177
↓ 1 callers
Method
_handle_heartbeat_commands
Handle commands received in heartbeat response. Uses polymorphic dispatch - each command class implements its own execute()
lmcache/v1/cache_controller/worker.py:488
↓ 1 callers
Method
_handle_metadata_write_completion
Handle completion of metadata write task, checking for exceptions.
lmcache/v1/storage_backend/gds_backend.py:689
↓ 1 callers
Method
_handle_msg
(self, req: HandshakeMsg)
lmcache/v1/distributed/transfer_channel/impl/nixl_impl.py:267
↓ 1 callers
Method
_handle_peer_requests
Handle `BatchedLookupAndGetMsg` issued by peers in `batched_get_non_blocking`.
lmcache/v1/storage_backend/p2p_backend.py:345
↓ 1 callers
Method
_handle_publish
Send a best-effort register/evict; failures are logged and dropped.
lmcache/v1/mp_coordinator/blend_client.py:282
↓ 1 callers
Function
_has_ext
()
tests/v1/distributed/test_raw_block_l2_adapter.py:33
↓ 1 callers
Function
_has_ext
Check if the Rust raw block I/O extension is available.
tests/v1/storage_backend/test_raw_block_uring_cmd.py:47
↓ 1 callers
Function
_has_real_names
Check if parameter names are real (not arg0, arg1, ...).
tests/v1/test_c_ops_fallback_parity.py:207
↓ 1 callers
Function
_has_working_new_shared_cuda
()
tests/v1/multiprocess/test_cache_server.py:44
↓ 1 callers
Method
_head_request
(self, key_str: str)
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:849
↓ 1 callers
Function
_health_loop
Evict stale instances on a timer until cancelled.
lmcache/v1/mp_coordinator/app.py:94
↓ 1 callers
Function
_health_str
(is_healthy: bool)
lmcache/tools/mp_status_viewer/__main__.py:22
↓ 1 callers
Method
_heartbeat_stop_requested
Whether a created heartbeat thread has a stop requested. Returns: ``True`` only if a heartbeat exists and its stop was requested.
lmcache/integration/vllm/vllm_multi_process_adapter.py:1333
↓ 1 callers
Function
_hipify_wrapper
Run torch hipify on csrc/ and return hipified source paths.
setup_extensions/build_profiles/rocm.py:25
↓ 1 callers
Function
_http_request
Send an HTTP request and return the parsed JSON response. Args: method: HTTP method (GET, POST, PUT, DELETE). url: Full URL to re
lmcache/cli/commands/kvcache.py:23
↓ 1 callers
Function
_infer_kv_dtype
Infer the KV element dtype from whichever inputs carry it. Inference order (first match wins): 1. ``shape_desc.dtype`` — authoritative when s
lmcache/python_ops_fallback.py:856
↓ 1 callers
Function
_infer_l1_devdax_overflow_from_dax_adapter
( memory_config: "L1MemoryManagerConfig", l2_adapter_config: L2AdaptersConfig, )
lmcache/v1/distributed/config.py:47
↓ 1 callers
Method
_init_connector_state
Initialize connector-specific state variables.
lmcache/integration/vllm/vllm_v1_adapter.py:518
↓ 1 callers
Method
_init_dynamic_metrics
Initialize dynamic metrics that will be updated by lambda functions.
lmcache/v1/cache_controller/observability.py:39
↓ 1 callers
Method
_init_kv_caches_from_forward_context
(self, forward_context: "ForwardContext")
lmcache/integration/vllm/vllm_v1_adapter.py:738
↓ 1 callers
Method
_init_mem_handlers
(self, device, buffer_ptr, buffer_size, page_size, device_id)
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:127
↓ 1 callers
Method
_init_pipe
Spawn a local worker subprocess.
tests/v1/shm_allocator/shmfile_connector.py:155
↓ 1 callers
Method
_init_receiver
(self)
lmcache/v1/storage_backend/pd_backend.py:872
↓ 1 callers
Method
_init_receiver
Launch the async memory allocation server coroutine on the already-running receiver event loop (self._recv_loop, created before the t
lmcache/v1/storage_backend/pd_backend_async.py:1162
↓ 1 callers
Method
_init_sender
Initialize sender-side sockets and locks on the sender event loop.
lmcache/v1/storage_backend/pd_backend_async.py:665
↓ 1 callers
Method
_init_shared_memory
Initialize shared memory connection to ai-toolkit daemon.
lmcache/v1/storage_backend/connector/sagemaker_hyperpod_connector.py:163
↓ 1 callers
Method
_init_side_channels
(self)
lmcache/v1/transfer_channel/nixl_channel.py:245
↓ 1 callers
Method
_init_side_channels
Initialize side channel for handling incoming connections
lmcache/v1/transfer_channel/py_socket_channel.py:154
↓ 1 callers
Method
_init_tcp
Connect to a remote worker via TCP.
tests/v1/shm_allocator/shmfile_connector.py:144
↓ 1 callers
Function
_initialize_remote_backend
Initialize RemoteBackend with FSConnector.
lmcache/v1/internal_api_server/vllm/load_fs_chunks_api.py:202
↓ 1 callers
Method
_initialize_sync
Initialize the sync process. Handles startup delay, entering freeze mode, getting keys, and sending the start message with r
lmcache/v1/cache_controller/full_sync_sender.py:195
↓ 1 callers
Method
_inspect_handler_signature
Inspect the handler signature to ensure it matches the expected payload classes. Args: handler (callable): The handler fu
lmcache/v1/multiprocess/mq.py:642
↓ 1 callers
Function
_install_nixl_mock_if_absent
Install a fake ``nixl`` package so this module can import ``nixl_storage_backend`` and exercise its construction paths without a real NIXL bui
tests/v1/storage_backend/test_nixl_shared_pool.py:26
↓ 1 callers
Method
_is_buffer_aligned
Check if a buffer is aligned to the block alignment boundary. Args: buf: Object implementing the Python buffer protocol.
lmcache/v1/storage_backend/raw_block/core.py:999
↓ 1 callers
Function
_is_connection_error
(error_msg: str)
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:297
↓ 1 callers
Function
_is_hnd_format
Return True when a per-layer KV format stores heads before block tokens (HND).
lmcache/python_ops_fallback.py:777
↓ 1 callers
Function
_is_mla_block_transfer_format
Return whether ``engine_kv_format`` is a native-supported MLA layout.
lmcache/v1/platform/musa/native_kv_transfer.py:331
↓ 1 callers
Function
_is_musa_block_transfer_candidate
Return whether normalized paged layers can use native MUSA kernels.
lmcache/v1/platform/musa/native_kv_transfer.py:345
↓ 1 callers
Function
_is_mutable_config
Check if a config key is mutable at runtime. NOTE: This is currently experimental. All configs default to mutable=True unless explicitly set
lmcache/v1/internal_api_server/vllm/conf_api.py:22
↓ 1 callers
Method
_is_self_registered
Return whether this instance appears in ``instances`` with our URL. Args: instances: The peer instances parsed from the coordinat
lmcache/v1/multiprocess/modules/p2p_controller.py:417
↓ 1 callers
Function
_is_sliding_window_spec
Return whether the KV cache spec is a vLLM sliding-window spec. Checked by class name so this module stays importable without vLLM. Subclasse
lmcache/integration/vllm/kv_cache_groups.py:22
↓ 1 callers
Function
_is_supported_musa_block_transfer_format
Return whether the native MUSA block path supports ``engine_kv_format``.
lmcache/v1/platform/musa/native_kv_transfer.py:326
↓ 1 callers
Function
_is_tty
()
lmcache/cli/commands/bench/engine_bench/interactive/terminal.py:29
↓ 1 callers
Method
_is_valid_checkpoint_entry
Return whether a checkpoint entry references a valid data slot.
lmcache/v1/storage_backend/raw_block/core.py:1688
↓ 1 callers
Function
_key
(seed: int = 0)
tests/v1/distributed/test_devdax_l1_allocator.py:53
↓ 1 callers
Function
_key
(i: int)
tests/v1/distributed/l2_adapters/test_p2p_l2_adapter_integration.py:99
↓ 1 callers
Method
_key_exists_on_disk
Check whether the file for *key* exists on disk. Uses ``aiofiles.os.path.exists`` so the check is non-blocking and always reflects th
lmcache/v1/distributed/l2_adapters/fs_l2_adapter.py:480
↓ 1 callers
Method
_key_to_file_and_tmp_path
Return ``(final_path, tmp_path)``. When ``relative_tmp_dir`` is configured, the temp file is placed under that sub-directory (same be
lmcache/v1/distributed/l2_adapters/fs_l2_adapter.py:493
↓ 1 callers
Method
_kv_present
True if KV exists in any active backend, or unknown when SM unbound.
lmcache/v1/hidden_state_store.py:327
↓ 1 callers
Function
_l1_usage_ratio_or_zero
Return ``target.get_memory_usage()`` as a 0.0-1.0 ratio. Returns 0.0 when ``target`` is None or ``total_bytes`` is zero so the observable-gau
lmcache/v1/distributed/l1_manager.py:119
↓ 1 callers
Method
_launch_plugin
Launch a plugin
lmcache/v1/plugin/runtime_plugin_launcher.py:96
↓ 1 callers
Method
_launch_plugins
Launch plugins from specified location
lmcache/v1/plugin/runtime_plugin_launcher.py:40
↓ 1 callers
Function
_layerwise_retrieve_vllm_contract
Mimic vLLM layerwise retrieve contract. For layerwise mode, benchmark should call retrieve_layer with the same slot mapping + KV cache contex
tests/benchmarks/test_xpu_layerwise_connector_benchmark.py:167
↓ 1 callers
Function
_leaf_specs
Return a group spec's leaf specs, unwrapping ``UniformTypeKVCacheSpecs``.
lmcache/integration/vllm/kv_cache_group_edits.py:89
↓ 1 callers
Method
_list_request
Build a ListObjectsV2 request. Returns ``(s3_req, body_chunks, captured)``. The caller awaits ``s3_req.finished_future`` and assemble
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:957
↓ 1 callers
Method
_load_bytes_from_disk_with_memory
Load byte array from disk into a pre-allocated memory object. Args: key: Cache key for error handling path:
lmcache/v1/storage_backend/gds_backend.py:814
↓ 1 callers
Method
_load_checkpoint_from_device
Load the newest valid checkpoint from the raw device if present.
lmcache/v1/storage_backend/raw_block/core.py:1922
↓ 1 callers
Function
_load_chunks_from_fs_connector
Load chunks from FSConnector into LocalCPUBackend.
lmcache/v1/internal_api_server/vllm/load_fs_chunks_api.py:223
↓ 1 callers
Function
_load_completed
( request_id: int, t: float, adapter_index: int = 0, l2_name: str = "fs", task_id: int = 0
tests/v1/mp_observability/subscribers/metrics/test_l2_throughput.py:103
↓ 1 callers
Function
_load_config_from_file
Load configuration from yaml file.
lmcache/v1/internal_api_server/vllm/load_fs_chunks_api.py:191
↓ 1 callers
Method
_load_gds
Read data from disk into a GPU buffer
lmcache/v1/storage_backend/gds_backend.py:1002
↓ 1 callers
Method
_load_meta_payload
Load and CRC-validate a checkpoint payload for a metadata header.
lmcache/v1/storage_backend/raw_block/core.py:1533
↓ 1 callers
Function
_load_nixl
Import the nixl Python bindings, tolerating the cuXX-suffixed packages.
lmcache/v1/distributed/transfer_channel/impl/nixl_impl.py:46
↓ 1 callers
Method
_load_sharegpt_data
(self)
benchmarks/multi_round_qa/multi-round-qa.py:390
↓ 1 callers
Method
_local_available_count
( self, shapes: Union[torch.Size, list[torch.Size]], dtypes: Union[torch.dtype, list[t
lmcache/v1/memory_management.py:2755
↓ 1 callers
Method
_local_memory_usage
(self)
lmcache/v1/memory_management.py:2771
↓ 1 callers
Method
_log_expansion_progress
Log the cumulative expansion progress since the last log.
lmcache/v1/lazy_memory_allocator.py:254
↓ 1 callers
Method
_log_stats
(self)
examples/disagg_prefill/disagg_proxy_server.py:220
↓ 1 callers
Function
_lookup
(salt: str, **kw)
tests/v1/mp_coordinator/test_l2_api.py:30
↓ 1 callers
Function
_lookup_attrs
Build ``{"model_name": ..., "cache_salt": ...}`` from the event. Missing fields are dropped from the returned dict so future emission sites t
lmcache/v1/mp_observability/subscribers/metrics/lookup.py:35
↓ 1 callers
Function
_lookup_side_effect
submit_request dispatcher keyed on request type.
tests/v1/distributed/l2_adapters/test_p2p_l2_adapter.py:78
↓ 1 callers
Function
_make_allocator
Create the LMCache benchmark buffer allocator.
benchmarks/musa/bench_inprocess_transfer.py:162
↓ 1 callers
Function
_make_blake3_hash_func
Create a blake3-based hash function compatible with the (prefix_hash, tuple(tokens), None) calling convention.
lmcache/v1/multiprocess/token_hasher.py:28
↓ 1 callers
Function
_make_config
()
tests/cli/commands/bench/engine_bench/test_stats.py:51
↓ 1 callers
Function
_make_connector
( requests: list[SimpleNamespace], )
tests/v1/test_v1_adapter_state_desync.py:79
↓ 1 callers
Function
_make_credentials_provider
Build an awscrt credentials provider for the S3 L2 adapter. Resolution: 1. Static keys from ``config.aws_access_key_id`` / ``config.a
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:199
↓ 1 callers
Function
_make_desync_request
Build a request whose ``token_ids`` and ``slot_mapping`` lengths disagree, simulating a state desync.
tests/v1/test_v1_adapter_state_desync.py:63
↓ 1 callers
Function
_make_dir
(path: str)
lmcache/v1/storage_backend/connector/eic_connector.py:98
↓ 1 callers
Function
_make_handler
(code: int, body: bytes = b"")
tests/cli/test_ping.py:40
↓ 1 callers
Function
_make_hnd_kv_caches
Build per-layer HND KV tensors for non-CUDA data transfer tests.
tests/v1/multiprocess/test_engine_driven_transfer.py:118
↓ 1 callers
Function
_make_keys
Create cache keys for benchmark.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:180
↓ 1 callers
Function
_make_kv_tensors
Create deterministic CPU KV tensors for testing.
tests/v1/multiprocess/test_http_server.py:17
↓ 1 callers
Function
_make_kvcaches
Allocate synthetic non-MLA vLLM MUSA paged KV caches.
benchmarks/musa/bench_inprocess_transfer.py:193
↓ 1 callers
Function
_make_memory_objs
Create memory objects for benchmark.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:114
↓ 1 callers
Function
_make_metadata
Create metadata matching the synthetic benchmark KV cache.
benchmarks/musa/bench_inprocess_transfer.py:174
↓ 1 callers
Function
_make_mock_handler
Create a mock MaruHandler with mmap-backed regions.
tests/v1/storage_backend/test_maru_backend.py:52
↓ 1 callers
Function
_make_mock_sender
()
tests/cli/commands/bench/engine_bench/workloads/test_random_prefill.py:98
↓ 1 callers
Function
_make_mock_sender
()
tests/cli/commands/bench/engine_bench/workloads/test_long_doc_qa.py:157
↓ 1 callers
Function
_make_mock_sender
()
tests/cli/commands/bench/engine_bench/workloads/test_multi_round_chat.py:255
↓ 1 callers
Function
_make_mock_sender
()
tests/cli/commands/bench/engine_bench/workloads/test_long_doc_permutator.py:253
↓ 1 callers
Function
_make_paged_allocator
(metadata: LMCacheMetadata)
tests/v1/storage_backend/test_nixl_shared_pool.py:150
↓ 1 callers
Function
_make_storage_manager
(size_bytes: int)
tests/v1/distributed/l2_adapters/test_p2p_l2_adapter_integration.py:73
↓ 1 callers
Method
_map_devdax
(self)
lmcache/v1/memory_management.py:2667
↓ 1 callers
Method
_mark_slot_committed_locked
(self, slot_id: int, generation: int)
lmcache/v1/storage_backend/dax/core.py:956
↓ 1 callers
Method
_match_fingerprints
Drain pending registrations and fingerprint-match sub-sequences. Returns the raw matches (any order, possibly overlapping); the caller
lmcache/v1/multiprocess/modules/blend_v3.py:559
↓ 1 callers
Function
_maybe_refresh_from_supplier
Refresh the node registry from supplier, at most once per interval. The first caller within the interval performs the refresh; other concurre
lmcache/lmcache_frontend/app.py:795
← previous
next →
3,401–3,500 of 13,371, ranked by callers