MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 2 callersMethod_get_size_for_kernel_group
Returns the size in bytes of the temp GPU buffer for the given kernel group index **Assumes the size is lmcache_tokens_per_c
lmcache/v1/platform/cuda/cache_context.py:300
↓ 2 callersMethod_get_size_for_single_batch
Returns the size in bytes of the temp GPU buffer for a single batch (i.e., a single chunk) **Assumes the size is lmcache_tok
lmcache/v1/platform/cuda/cache_context.py:330
↓ 2 callersMethod_get_slices
Get slices for parallel task submission.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:443
↓ 2 callersFunction_get_storage_manager
Resolve the shared ``StorageManager`` or raise ``HTTPException``. Returns the live ``StorageManager`` instance. Raises ``HTTPException(503)``
lmcache/v1/multiprocess/http_apis/l2_api.py:54
↓ 2 callersMethod_get_storage_manager
Get the storage manager from the cache engine.
lmcache/v1/health_monitor/base.py:285
↓ 2 callersFunction_get_sysfs_path
Derive sysfs path from device path.
tests/v1/storage_backend/test_raw_block_uring_cmd.py:35
↓ 2 callersMethod_get_unfull_chunk_shapes
(self, num_tokens: int)
lmcache/v1/storage_backend/p2p_backend.py:653
↓ 2 callersMethod_get_writable_view
Get a writable memoryview from byte array.
lmcache/v1/storage_backend/connector/sagemaker_hyperpod_connector.py:633
↓ 2 callersFunction_handle_disagg_request
Common handler for disaggregated prefill/decode requests. Works for both /v1/completions and /v1/chat/completions — the only difference
examples/disagg_prefill_mp/disagg_proxy_server.py:248
↓ 2 callersMethod_handle_init_msg
( self, req: Union[NixlMsg, InitSideMsgBase] )
lmcache/v1/transfer_channel/nixl_channel.py:258
↓ 2 callersMethod_handle_init_msg
Handle initialization messages from peers
lmcache/v1/transfer_channel/py_socket_channel.py:174
↓ 2 callersMethod_handle_post_init_failure
Handle initialization failure during post_init. Args: e: The exception that caused the failure
lmcache/v1/manager.py:224
↓ 2 callersMethod_has_l2_adapters
Return whether any L2 adapter is currently active.
lmcache/v1/distributed/storage_manager.py:991
↓ 2 callersFunction_header_size
(group_size: int)
tests/v1/distributed/serde/test_multi.py:93
↓ 2 callersMethod_headers
(self, **extras: str)
lmcache/connections.py:50
↓ 2 callersMethod_http_request_impl
Execute HTTP request with connection pooling and error handling.
lmcache/v1/storage_backend/connector/sagemaker_hyperpod_connector.py:238
↓ 2 callersFunction_hybrid_storage_config
(path: str, adapter_config: object)
tests/v1/distributed/test_devdax_l1_allocator.py:102
↓ 2 callersMethod_init_none_hash
Initialize NONE_HASH. Adapted from TokenDatabase.__init__ (token_database.py:64-82).
lmcache/v1/multiprocess/token_hasher.py:152
↓ 2 callersMethod_init_sender
(self)
lmcache/v1/storage_backend/pd_backend.py:400
↓ 2 callersMethod_initialize_attributes
Initialize attributes from KV caches using utils functions. Args: kv_caches: The KV cache tensors from which to discover layout.
lmcache/v1/gpu_connector/musa_connectors.py:346
↓ 2 callersMethod_initialize_attributes
(self, kv_caches: List[torch.Tensor])
lmcache/v1/gpu_connector/hpu_connector.py:268
↓ 2 callersMethod_initialize_kv_cache_pointers
Discover KV-cache layout, build the layer-groups manager, and capture per-group GPU pointer tensors. All layout-adjacent work lives h
lmcache/v1/gpu_connector/gpu_connectors.py:455
↓ 2 callersMethod_initialize_kv_cache_pointers
(self)
lmcache/v1/gpu_connector/xpu_connectors.py:325
↓ 2 callersMethod_initialize_pointers
(self, kv_caches: List[torch.Tensor])
lmcache/v1/gpu_connector/gpu_connectors.py:243
↓ 2 callersMethod_initialize_pointers
(self, kv_caches: DiscoverableKVCache)
lmcache/v1/gpu_connector/gpu_connectors.py:1476
↓ 2 callersMethod_initialize_pointers
(self, kv_caches: List[torch.Tensor])
lmcache/v1/gpu_connector/xpu_connectors.py:120
↓ 2 callersMethod_initialize_pointers
(self, kv_caches: List[torch.Tensor])
lmcache/v1/gpu_connector/xpu_connectors.py:1282
↓ 2 callersMethod_inject_key
Insert a key into backend.dict without writing anything to disk.
tests/v1/storage_backend/test_local_disk_backend.py:393
↓ 2 callersFunction_install_fake_raw_block_device
(monkeypatch, *, size_bytes: int = 64 * 1024)
tests/v1/storage_backend/test_rust_raw_block_backend.py:76
↓ 2 callersMethod_invoke_on_complete_callback
( self, key: CacheEngineKey, on_complete_callback: Optional[Callable[[CacheEngineKey],
lmcache/v1/storage_backend/plugins/dax_backend.py:654
↓ 2 callersFunction_is_cross_layer_format
Return True when a KV format uses a single cross-layer tensor.
lmcache/python_ops_fallback.py:761
↓ 2 callersMethod_is_devdax_obj
(self, memory_obj: MemoryObj)
lmcache/v1/memory_management.py:2752
↓ 2 callersMethod_is_local_obj
(self, memory_obj: MemoryObj)
lmcache/v1/memory_management.py:2746
↓ 2 callersFunction_is_mla_format
Return True when a KV format uses MLA paged layout.
lmcache/python_ops_fallback.py:786
↓ 2 callersMethod_is_power_of_two
(value: int)
lmcache/v1/storage_backend/local_cpu_backend.py:552
↓ 2 callersFunction_is_rocm
Check if we're running on ROCm (AMD GPU).
lmcache/v1/compute/attention/utils.py:21
↓ 2 callersFunction_is_sglang_mha_format
Return True when a KV format uses SGLang MHA layout (2*NL tensors).
lmcache/python_ops_fallback.py:769
↓ 2 callersFunction_is_skip_safe_io_error
(exc: BaseException)
tests/v1/storage_backend/test_raw_block_device.py:25
↓ 2 callersMethod_is_sync_pd_backend
Check if the PD backend is the sync variant. :return: True when PD is enabled and ``pd_backend_mode`` is ``"sync"``. :rtype: bool
lmcache/v1/cache_engine.py:1919
↓ 2 callersMethod_key_to_path
(self, key: ObjectKey)
lmcache/v1/distributed/l2_adapters/fs_l2_adapter.py:477
↓ 2 callersMethod_key_to_path
( self, key: CacheEngineKey, )
lmcache/v1/storage_backend/local_disk_backend.py:187
↓ 2 callersFunction_l1_desc
()
tests/v1/distributed/transfer_channel/test_factory.py:61
↓ 2 callersFunction_l2_name_attrs
Build ``{"l2_name": ...}`` if the event carries an ``l2_name`` metadata key, else ``{}``. Keeps the counter dimensionless when the emission s
lmcache/v1/mp_observability/subscribers/metrics/l2.py:22
↓ 2 callersMethod_launch_new_request
(self, timestamp: float, request_executor: RequestExecutor)
benchmarks/multi_round_qa/multi-round-qa.py:242
↓ 2 callersFunction_layout
(num_bytes: int = 4096)
tests/v1/distributed/test_devdax_l1_allocator.py:61
↓ 2 callersMethod_lazy_initialize_buffer
(self, kv_caches: List[torch.Tensor])
lmcache/v1/gpu_connector/musa_connectors.py:489
↓ 2 callersMethod_lazy_initialize_buffer
Lazily initialize the GPU buffer allocator if it is not initialized yet. Currently, we use the `kv_caches` (kv cache pointer) to dete
lmcache/v1/gpu_connector/gpu_connectors.py:722
↓ 2 callersMethod_lazy_initialize_buffer
Lazily initialize the GPU buffer allocator if it is not initialized yet. Currently, we use the `kv_caches` (kv cache pointer) to dete
lmcache/v1/gpu_connector/gpu_connectors.py:1137
↓ 2 callersMethod_lazy_initialize_buffer
Lazily initialize the GPU buffer allocator if it is not initialized yet. Currently, we use the `kv_caches` (kv cache pointer) to dete
lmcache/v1/gpu_connector/gpu_connectors.py:1678
↓ 2 callersMethod_lazy_initialize_buffer
Lazily initialize the GPU buffer allocator if it is not initialized yet. Currently, we use the `kv_caches` (kv cache pointer) to dete
lmcache/v1/gpu_connector/xpu_connectors.py:547
↓ 2 callersMethod_lazy_initialize_buffer
Lazily initialize the GPU buffer allocator if it is not initialized yet. Currently, we use the `kv_caches` (kv cache pointer) to dete
lmcache/v1/gpu_connector/xpu_connectors.py:948
↓ 2 callersMethod_lazy_initialize_buffer
Lazily initialize the GPU buffer allocator if it is not initialized yet. Currently, we use the `kv_caches` (kv cache pointer) to dete
lmcache/v1/gpu_connector/xpu_connectors.py:1486
↓ 2 callersMethod_list_reconfigurable_l2_adapters
( self, )
lmcache/v1/distributed/storage_manager.py:1063
↓ 2 callersMethod_load_bytes_from_disk_with_allocation
Load byte array from disk by first allocating memory, then loading. Args: key: Cache key for error handling
lmcache/v1/storage_backend/gds_backend.py:780
↓ 2 callersMethod_load_key
Build the ``(request_id, adapter_index)`` correlation key. Returns ``None`` if either field is missing.
lmcache/v1/mp_observability/subscribers/metrics/l2_throughput.py:139
↓ 2 callersFunction_load_submitted
( request_id: int, t: float, adapter_index: int = 0, l2_name: str = "fs", total_bytes: int
tests/v1/mp_observability/subscribers/metrics/test_l2_throughput.py:81
↓ 2 callersFunction_make_adapter
(tmp_path: Path)
tests/v1/multiprocess/test_raw_block_l2_adapter.py:41
↓ 2 callersMethod_make_backend_stub
Create a minimal PDBackend-like object for testing.
tests/v1/storage_backend/test_pd_backend_bidirectional.py:187
↓ 2 callersMethod_make_client
(self, endpoint: str)
tests/cli/commands/bench/test_server_bench.py:470
↓ 2 callersMethod_make_client
(self, endpoint: str)
tests/cli/commands/bench/test_server_bench.py:566
↓ 2 callersFunction_make_config
(**overrides)
tests/cli/commands/bench/test_bench_command.py:71
↓ 2 callersFunction_make_config
(port_start: int)
tests/v1/internal_api_server/test_per_instance_app.py:28
↓ 2 callersFunction_make_context_pair
Create two contexts backed by two distinct CPU buffers. ``buf_b`` holds a known byte pattern so transfers can be content-verified; ``buf_a``
tests/v1/distributed/transfer_channel/test_nixl_impl.py:51
↓ 2 callersMethod_make_dynamic_backend
(backend_name: str)
tests/v1/test_nixl_doca_memos.py:188
↓ 2 callersFunction_make_event_handle
Create a CUDA event IPC handle for GPU mode. CPU mode does not need a cross-process event (SHM mappings are coherent without device-side sync
lmcache/cli/commands/bench/server_bench/helpers.py:490
↓ 2 callersFunction_make_final_stats
(**overrides)
tests/cli/commands/bench/test_bench_command.py:88
↓ 2 callersFunction_make_key
Create an ObjectKey with the given raw hash bytes.
tests/v1/distributed/serde/test_serde_fs_e2e.py:44
↓ 2 callersFunction_make_keys
(prefix: str, count: int)
tests/v1/mp_observability/subscribers/metrics/test_l1_lifecycle.py:36
↓ 2 callersFunction_make_local_cpu_flat
LocalCPUBackend memory_allocator = MixedMemoryAllocator(use_paging=False).
tests/v1/storage_backend/test_nixl_shared_pool.py:202
↓ 2 callersFunction_make_mock_result
(request_id: str = "req_0")
tests/cli/commands/bench/engine_bench/workloads/test_random_prefill.py:81
↓ 2 callersFunction_make_mock_result
(request_id: str = "req_0")
tests/cli/commands/bench/engine_bench/workloads/test_long_doc_qa.py:140
↓ 2 callersFunction_make_mock_result
(request_id: str = "req_0")
tests/cli/commands/bench/engine_bench/workloads/test_multi_round_chat.py:238
↓ 2 callersFunction_make_mock_result
(request_id: str = "req_0")
tests/cli/commands/bench/engine_bench/workloads/test_long_doc_permutator.py:236
↓ 2 callersFunction_make_mock_sender
()
tests/cli/commands/bench/engine_bench/workloads/test_prefix_suffix_tuner.py:228
↓ 2 callersFunction_make_object_key
(**overrides)
tests/v1/distributed/test_encoded_object_key.py:19
↓ 2 callersFunction_make_prompt_payload
Build an OpenAI-compatible /v1/completions payload.
examples/kv_cache_reuse/remote_backends/valkey/benchmark_l2.py:66
↓ 2 callersFunction_make_raw_block_core
(*, use_odirect: bool = False)
tests/v1/storage_backend/test_rust_raw_block_backend.py:87
↓ 2 callersFunction_make_scheduler_manager
Mock LMCacheManager representing a scheduler (no engine).
tests/v1/internal_api_server/test_per_instance_app.py:58
↓ 2 callersFunction_make_serde
(chunk_size)
tests/benchmarks/test_cachegen.py:43
↓ 2 callersFunction_make_tensor_obj
(size: int, fill: float)
tests/v1/distributed/test_aerospike_l2_integration.py:85
↓ 2 callersMethod_make_tensor_view
Create a tensor view over a SHM slot via ``torch.frombuffer``.
lmcache/v1/multiprocess/transfer_context/shm.py:109
↓ 2 callersFunction_make_worker_manager
Mock LMCacheManager representing a worker (engine present).
tests/v1/internal_api_server/test_per_instance_app.py:36
↓ 2 callersFunction_memory_obj
Build a filled TensorMemoryObj for use as a store/load buffer.
tests/v1/distributed/test_fault_inject_l2_adapter.py:48
↓ 2 callersMethod_meta_payload_capacity
Return usable bytes in one metadata checkpoint payload area.
lmcache/v1/storage_backend/raw_block/core.py:1495
↓ 2 callersFunction_missing_chat_template
(exc: BaseException)
lmcache/cli/commands/query/_request.py:226
↓ 2 callersFunction_native_module_if_ready
Return a usable native module when opt-in and ABI-compatible.
lmcache/v1/platform/musa/native_kv_transfer.py:308
↓ 2 callersFunction_native_tensors_ready
Return whether native MUSA dispatch can consume these tensors directly.
lmcache/v1/platform/musa/native_kv_transfer.py:71
↓ 2 callersFunction_next_url
()
tests/v1/distributed/transfer_channel/test_nixl_impl.py:47
↓ 2 callersMethod_nixl_cpu_defaults
(**overrides: Any)
tests/v1/test_config.py:981
↓ 2 callersFunction_normalize_adapter_init_args
Normalize adapter constructor args from old and new vLLM connectors. Args: vllm_block_size: The vLLM block size for the current connector
lmcache/integration/vllm/vllm_multi_process_adapter.py:354
↓ 2 callersFunction_normalize_backend
(backend: str)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:150
↓ 2 callersFunction_normalize_default
Normalize a default value string for comparison. Handles differences between pybind11 docstring representation and Python inspect representat
tests/v1/test_c_ops_fallback_parity.py:93
↓ 2 callersFunction_normalize_hash_to_int
Normalize hash outputs to LMCache's int chunk-hash representation. This function is triggered when vLLM's ``sha256_cbor`` hash function is us
lmcache/v1/token_database.py:34
↓ 2 callersFunction_obj_to_memoryview
Extract a byte-oriented memoryview from a MemoryObj. Uses the MemoryObj's byte_array property which returns a ctypes-backed memoryview w
lmcache/v1/distributed/l2_adapters/native_connector_l2_adapter.py:71
↓ 2 callersFunction_object_block_ids
Return block IDs belonging to one LMCache object.
lmcache/v1/platform/musa/native_kv_transfer.py:391
↓ 2 callersFunction_object_layout
Return the layout for a single ``object_size``-byte uint8 object.
lmcache/tools/transfer_channel_benchmark/benchmark.py:84
↓ 2 callersFunction_object_slot_identity
Return the stable 64-bit slot identity for an encoded ObjectKey. Args: encoded: Raw-block ObjectKey string produced by ``object_key_to_st
lmcache/v1/storage_backend/raw_block/key_codec.py:162
↓ 2 callersMethod_on_load_submitted
(self, event: Event)
lmcache/v1/mp_observability/subscribers/metrics/l2.py:168
↓ 2 callersMethod_on_raw_future_complete
Update the CUDA event and result when the raw future is complete.
lmcache/v1/multiprocess/futures.py:98
↓ 2 callersMethod_on_retrieve_start
(self, event: Event)
lmcache/v1/mp_observability/subscribers/metrics/cb_server.py:183
← previousnext →2,301–2,400 of 13,371, ranked by callers