Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 1 callers
Method
_alloc_chunk
(self, n_tokens: int, hidden_dim: int)
lmcache/v1/hidden_state_store.py:335
↓ 1 callers
Method
_allocate_and_put
(self, alloc_request: AllocRequest)
lmcache/v1/storage_backend/pd_backend.py:908
↓ 1 callers
Function
_allocate_cpu_shm_kv_cache
Allocate paged CPU KV cache tensors backed by POSIX SHM. For each (group, layer) we ``shm_open`` a fresh segment and ``mmap`` it into the cli
lmcache/cli/commands/bench/server_bench/helpers.py:265
↓ 1 callers
Method
_allocate_for_read
Allocate one MemoryObj per key. Returns ``(None, [], [])`` if any allocation fails, after freeing what was already taken; otherwise r
lmcache/v1/storage_backend/nixl_storage_backend.py:1619
↓ 1 callers
Method
_allocate_from_devdax
( self, shapes: Union[torch.Size, list[torch.Size]], dtypes: Union[torch.dtype, list[t
lmcache/v1/memory_management.py:2780
↓ 1 callers
Function
_allocate_gpu_kv_cache
Allocate paged GPU KV cache tensors. Each layer is a tensor of shape ``(kv_size, num_blocks, block_size, num_heads, head_size)`` matching
lmcache/cli/commands/bench/server_bench/helpers.py:200
↓ 1 callers
Function
_allocate_gpu_memory
( size: int, device: str, )
lmcache/v1/memory_management.py:591
↓ 1 callers
Method
_allocate_restore_outputs
(self, reserved: Sequence[_RestoreItem])
lmcache/v1/storage_backend/plugins/dax_backend.py:699
↓ 1 callers
Method
_allocate_slot_locked
Allocate a slot offset while ``self._lock`` is held.
lmcache/v1/storage_backend/raw_block/core.py:1467
↓ 1 callers
Function
_apply_ec_storage_defaults
Apply EC-only storage defaults after EC override ingestion.
lmcache/v1/config.py:1185
↓ 1 callers
Method
_apply_extra_config
Apply extra config from vLLM to LMCache config.
lmcache/integration/vllm/vllm_v1_adapter.py:501
↓ 1 callers
Method
_apply_loaded_state
Apply decoded checkpoint state after validating layout fields.
lmcache/v1/storage_backend/raw_block/core.py:1700
↓ 1 callers
Method
_apply_local_cpu_fallback
Apply LOCAL_CPU fallback policy for a failed health check. This will: 1. Enable bypass for the specified backend in StorageM
lmcache/v1/health_monitor/base.py:312
↓ 1 callers
Method
_apply_pending_adapter_ops
Apply queued add/remove ops on the store loop thread.
lmcache/v1/distributed/storage_controllers/store_controller.py:504
↓ 1 callers
Method
_apply_pending_adapter_ops
Apply queued add/remove ops on the prefetch loop thread.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:675
↓ 1 callers
Function
_assert_copied_slots_match
Verify destination slots match source slots after connector round-trip.
benchmarks/musa/bench_inprocess_transfer.py:269
↓ 1 callers
Method
_assert_single_worker_response
(self, response, instance_id, worker_id)
tests/v1/internal_api_server/test_worker_info_api.py:197
↓ 1 callers
Method
_async_batch_store
Register multiple KV metadata entries via single batch_store RPC.
lmcache/v1/storage_backend/maru_backend.py:381
↓ 1 callers
Method
_async_create_alloc_socket
( self, receiver_id: str, receiver_mem_alloc_url: str )
lmcache/v1/storage_backend/pd_backend_async.py:727
↓ 1 callers
Method
_async_init_loop
(self)
lmcache/v1/transfer_channel/nixl_channel.py:334
↓ 1 callers
Method
_async_init_loop
Asynchronous initialization loop for handling incoming connections
lmcache/v1/transfer_channel/py_socket_channel.py:224
↓ 1 callers
Method
_async_init_proxy_socket
Create the async ZMQ PUSH socket for ProxyNotif messages. Must run on the sender event loop so the socket is loop-bound. :param prox
lmcache/v1/storage_backend/pd_backend_async.py:674
↓ 1 callers
Method
_async_launch_request
(self, messages, max_tokens, extra_headers=None)
benchmarks/multi_round_qa/multi-round-qa.py:130
↓ 1 callers
Method
_async_launch_request
(self, request_id, prompt, max_tokens)
benchmarks/rag/rag.py:162
↓ 1 callers
Method
_async_mem_alloc_server
Async ZMQ ROUTER server for memory allocation requests. Replaces the blocking _mem_alloc_loop / _mem_alloc_thread. Uses a ROU
lmcache/v1/storage_backend/pd_backend_async.py:1171
↓ 1 callers
Method
_async_process_tokens_internal
This function is used to get the memory objects from the event manager. Args: tokens: Input tokens to process
lmcache/v1/cache_engine.py:1638
↓ 1 callers
Method
_async_store
Register KV metadata with MaruServer (runs in event loop). Uses CxlMemoryAdapter.create_store_handle() to extract (region_id, page_in
lmcache/v1/storage_backend/maru_backend.py:333
↓ 1 callers
Method
_audit_get
Retrieve data with optional integrity check
lmcache/v1/storage_backend/connector/audit_connector.py:270
↓ 1 callers
Method
_audit_put
Store data with optional checksum tracking
lmcache/v1/storage_backend/connector/audit_connector.py:239
↓ 1 callers
Function
_available_backend_names
(status: dict)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:187
↓ 1 callers
Function
_backend_status_response
(status: dict, backend: str)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:202
↓ 1 callers
Function
_backends_response
(status: dict)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:218
↓ 1 callers
Function
_banner_disabled
Return whether ``LMCACHE_DISABLE_BANNER`` is set to a truthy value.
lmcache/banner.py:57
↓ 1 callers
Method
_batch_get_buffer
Batch get using batch_get_buffer when metadata is stored remotely. This is used when save_chunk_meta=True (metadata stored with data)
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:603
↓ 1 callers
Method
_batch_get_into
Zero-copy batch get using batch_get_into when metadata is available locally. This is used when save_chunk_meta=False (metadata not st
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:523
↓ 1 callers
Method
_batched_allocate_from_devdax
( self, shapes: Union[torch.Size, list[torch.Size]], dtypes: Union[torch.dtype, list[t
lmcache/v1/memory_management.py:2793
↓ 1 callers
Method
_batched_free_devdax_objs
( self, memory_objs: List[MemoryObj], update_stats: bool, )
lmcache/v1/memory_management.py:2818
↓ 1 callers
Method
_batched_get
Internal: retrieve multiple memory objects from Valkey in parallel. Note: Once an allocation failure occurs, all subsequent slots are
lmcache/v1/storage_backend/connector/valkey_connector.py:468
↓ 1 callers
Method
_batched_get_blocking_by_thread_pool_impl
( self, keys: List[CacheEngineKey], )
lmcache/v1/storage_backend/gds_backend.py:885
↓ 1 callers
Method
_batched_put_with_metadata
( self, keys: List[CacheEngineKey], memory_objs: List[MemoryObj], )
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:746
↓ 1 callers
Method
_batched_put_zero_copy
( self, keys: List[CacheEngineKey], memory_objs: List[MemoryObj], )
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:717
↓ 1 callers
Method
_batched_to_gpu_gen
Generator: CPU token2d -> (optional staging) -> MUSA paged KV.
lmcache/v1/gpu_connector/musa_connectors.py:515
↓ 1 callers
Method
_bind_dynamic_metric_children
(self)
lmcache/observability.py:1490
↓ 1 callers
Function
_bootstrap_backends
Import every direct sub-package under ``lmcache.v1.platform``. Each backend sub-package self-registers with :mod:`lmcache.v1.platform._regist
lmcache/v1/platform/__init__.py:47
↓ 1 callers
Method
_broadcast_or_receive_memory_objs
Handles broadcasting or receiving memory objects in a distributed environment. This function implements the communication logic wher
lmcache/v1/cache_engine.py:1811
↓ 1 callers
Method
_build_addresses
Build the per-key transfer addresses for a completed lookup. The first ``hit_count`` keys form the locked L1 prefix; their addresses
lmcache/v1/multiprocess/modules/p2p_controller.py:321
↓ 1 callers
Function
_build_backend_params
Build pytest parameter list for the backend fixture. Returns one entry per available backend configuration: - cuda_c_ops: uses lmcache.c_ops
tests/v1/test_python_ops_fallback.py:53
↓ 1 callers
Method
_build_core_config
(self, extra: Mapping[str, Any])
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:237
↓ 1 callers
Method
_build_descs
Build NixlDescs for ``keys``. For FILE backends this opens one fd per key; the caller owns FD lifetime once this method returns succe
lmcache/v1/storage_backend/nixl_storage_backend.py:1523
↓ 1 callers
Method
_build_group_report_map
Map each kernel-group index to its owning object-group index.
lmcache/v1/platform/base_cache_context.py:280
↓ 1 callers
Function
_build_list_objects_v2_response
Render a minimal ListObjectsV2 XML response from the fake backend. The adapter's parser only consumes ``<Contents>/<Key>`` + ``<Contents>/<Si
tests/v1/distributed/test_s3_l2_adapter.py:105
↓ 1 callers
Function
_build_lmcache_driven_context
Build a :class:`LMCacheDrivenTransferContext` after capability check.
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:80
↓ 1 callers
Function
_build_metadata
Build test metadata for benchmark.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:92
↓ 1 callers
Function
_build_modules
Assemble the list of engine modules based on configuration. Args: ctx: The shared engine context. mp_config: Server configuration
lmcache/v1/multiprocess/server.py:162
↓ 1 callers
Method
_build_new_question
(self)
benchmarks/multi_round_qa/multi-round-qa.py:235
↓ 1 callers
Method
_build_put_stream
Build streaming payload: [RemoteMetadata (28 bytes)] + [KV cache data] Args: memory_obj: The memory object to stream
lmcache/v1/storage_backend/connector/sagemaker_hyperpod_connector.py:813
↓ 1 callers
Method
_build_request_list
Build the full request list from the enumerated permutations. Each entry is ``(messages, permutation_index)`` where messages concaten
lmcache/cli/commands/bench/engine_bench/workloads/long_doc_permutator.py:309
↓ 1 callers
Method
_build_restore_waves
( self, reserved: Sequence[_RestoreItem], )
lmcache/v1/storage_backend/plugins/dax_backend.py:741
↓ 1 callers
Method
_build_schedule
Build the request schedule as ``(doc_index, query_index)`` pairs. Tile policy uses query-major order: all documents for query 0, then
lmcache/cli/commands/bench/engine_bench/workloads/long_doc_qa.py:176
↓ 1 callers
Method
_build_single_group_report
Build a status dict for a single kernel group. Override this in subclasses to inject extra per-group fields without duplicating the w
lmcache/v1/platform/base_cache_context.py:288
↓ 1 callers
Method
_build_system_prompt
(self)
benchmarks/multi_round_qa/multi-round-qa.py:222
↓ 1 callers
Function
_build_worker_binary
Build shm_file_worker from tests/v1/shm_allocator/csrc/ and return the path to the compiled binary.
tests/v1/shm_allocator/test_shm_allocator.py:53
↓ 1 callers
Method
_cache_contains
(self, chunk_hash: int)
lmcache/v1/storage_backend/nixl_storage_backend.py:1447
↓ 1 callers
Method
_calculate_effective_cpu_size
Calculate the effective CPU memory size based on system available memory and reserve memory configuration. Args:
lmcache/v1/storage_backend/local_cpu_backend.py:296
↓ 1 callers
Method
_call_blocking_handler
Call the blocking handler in a separate thread and send the response back to the client. Args: handler_entry (Bl
lmcache/v1/multiprocess/mq.py:541
↓ 1 callers
Method
_call_handler
( self, handler_entry: RequestHandlerBase[Any], payloads: list[bytes], prefix_
lmcache/v1/multiprocess/mq.py:579
↓ 1 callers
Method
_call_sync_handler
Call the sync handler and send the response back to the client. Args: handler_entry (SyncRequestHandler[Any]): The handl
lmcache/v1/multiprocess/mq.py:519
↓ 1 callers
Method
_can_join_user
(self, timestamp: float)
benchmarks/multi_round_qa/multi-round-qa.py:435
↓ 1 callers
Method
_can_merge_with_prev
Hook: Check if curr_block can merge with prev_block.
lmcache/v1/memory_management.py:1308
↓ 1 callers
Method
_can_merge_with_succ
Hook: Check if curr_block can merge with succ_block.
lmcache/v1/memory_management.py:1314
↓ 1 callers
Method
_canonicalize_hash_inputs
Canonicalize hash inputs so that semantically identical requests produce structurally identical hash inputs across instances.
lmcache/v1/token_database.py:249
↓ 1 callers
Method
_check_and_evict_by_cache_salt
Per-``cache_salt`` eviction driven by :class:`QuotaManager`. For every salt with non-zero bytes, compare its usage against ``watermar
lmcache/v1/distributed/storage_controllers/eviction_controller.py:330
↓ 1 callers
Method
_check_and_evict_global
Aggregate-usage eviction (``LRU`` / ``noop``).
lmcache/v1/distributed/storage_controllers/eviction_controller.py:302
↓ 1 callers
Method
_check_and_send_proxy_notif
Send ProxyNotif once all RDMA batches for req_id are complete. Fires only when both conditions hold: 1. All reserved chunks have comp
lmcache/v1/storage_backend/pd_backend_async.py:997
↓ 1 callers
Method
_check_exit_conditions
(self)
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:171
↓ 1 callers
Method
_check_hash_match
(got: CodecHashes, expected: CodecHashes)
lmcache/v1/kv_codec/asym_k16_v8.py:403
↓ 1 callers
Method
_check_legacy_register_kv_caches
Check for legacy connector without register_kv_caches implementation.
lmcache/integration/vllm/vllm_v1_adapter.py:598
↓ 1 callers
Method
_check_shm_capacity
Warn if /dev/shm free space is smaller than the KV cache.
lmcache/v1/platform/cpu/cache_context.py:198
↓ 1 callers
Method
_checkpoint_dtype_name
Return a durable checkpoint string for a torch dtype. Args: dtype: Torch dtype from recovered or live memory metadata. R
lmcache/v1/storage_backend/raw_block/core.py:1613
↓ 1 callers
Method
_checkpoint_submissions_locked
Close ``st``'s current batch behind a CUDA event on ``stream`` and drop earlier batches whose event has completed. Hold ``self._submis
lmcache/v1/gpu_connector/gds_context.py:373
↓ 1 callers
Method
_chunk_tokens
Chunk the tokens into chunks of size self.chunk_size. :param tokens: the input tokens, with shape [seq_len] device: the
lmcache/v1/token_database.py:333
↓ 1 callers
Method
_cleanup_after_init_failure
(self)
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:773
↓ 1 callers
Method
_cleanup_after_init_failure
Close resources that may have been opened before init failed.
lmcache/v1/storage_backend/raw_block/core.py:965
↓ 1 callers
Method
_cleanup_device
(self)
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:362
↓ 1 callers
Method
_cleanup_finished_aborted_lookups
Check for finished aborted lookups and send cleanup messages to workers.
lmcache/v1/lookup_client/lmcache_async_lookup_client.py:260
↓ 1 callers
Method
_cleanup_in_flight_requests
Release resources for any in-flight requests during shutdown.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:1205
↓ 1 callers
Method
_cleanup_in_flight_tasks
Release all held read locks for any in-flight tasks that haven't completed. Called during stop().
lmcache/v1/distributed/storage_controllers/store_controller.py:783
↓ 1 callers
Method
_cleanup_read_results
Cleanup read results by releasing memory objects.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:616
↓ 1 callers
Method
_cleanup_request_tracker
Clean up request tracker and associated lookup future for a request. This should be called when a request is finished to prevent memo
lmcache/integration/vllm/lmcache_mp_connector.py:1344
↓ 1 callers
Method
_cleanup_request_tracker
Clean up request tracker and associated lookup future for a request. This should be called when a request is finished to prevent memo
lmcache/integration/vllm/lmcache_mp_connector_0201.py:1191
↓ 1 callers
Method
_cleanup_request_tracker
Clean up request tracker and associated lookup future for a request. This should be called when a request is finished to prevent memo
lmcache/integration/vllm/lmcache_mp_connector_0180.py:1070
↓ 1 callers
Method
_cleanup_restore_outputs
(self, reserved: Sequence[_RestoreItem])
lmcache/v1/storage_backend/plugins/dax_backend.py:843
↓ 1 callers
Method
_clear
Clear all the distribution stats
lmcache/observability.py:594
↓ 1 callers
Method
_clear_queue
Clear all items from async queue.
lmcache/v1/lookup_client/record_strategies/base.py:307
↓ 1 callers
Function
_clone_lmcache_engine_config
Create a detached LMCacheEngineConfig copy from an existing config.
lmcache/v1/config.py:1171
↓ 1 callers
Function
_clone_tensor_memory_obj
Deep-clone a TensorMemoryObj.
examples/lmc_external_l2_adapter/src/lmc_external_l2_adapter/adapter.py:42
↓ 1 callers
Method
_close
(self)
lmcache/v1/storage_backend/connector/hf3fs_connector.py:119
↓ 1 callers
Method
_close_backend
Close the backend and perform cleanup.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:289
↓ 1 callers
Method
_close_devdax_mapping_locked
(self)
lmcache/v1/memory_management.py:2677
↓ 1 callers
Function
_coerce_bool
Coerce config values that may arrive as bools or strings.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:950
← previous
next →
3,101–3,200 of 13,371, ranked by callers