Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 1 callers
Method
_execute_load_in_loop
Coroutine that performs a batched load from Nixl storage into L1 memory. For each key that exists in ``_memory_objects``, the corres
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:842
↓ 1 callers
Method
_execute_load_in_loop
Execute the load operation in the event loop thread. This is a helper function to avoid blocking the main thread.
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:468
↓ 1 callers
Method
_execute_lookup
( self, keys: list[ObjectKey], task_id: L2TaskId, )
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:1115
↓ 1 callers
Method
_execute_lookup
( self, keys: list[ObjectKey], task_id: L2TaskId, )
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:560
↓ 1 callers
Method
_execute_lookup
( self, keys: list[ObjectKey], task_id: L2TaskId, )
lmcache/v1/distributed/l2_adapters/fs_l2_adapter.py:648
↓ 1 callers
Method
_execute_read_phase
Execute the read phase and return results. Override in subclass to customize read behavior.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:572
↓ 1 callers
Method
_execute_store
( self, keys: list[ObjectKey], objects: list[MemoryObj], task_id: L2TaskId,
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:1042
↓ 1 callers
Method
_execute_store
( self, keys: list[ObjectKey], objects: list[MemoryObj], task_id: L2TaskId,
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:524
↓ 1 callers
Method
_execute_store
( self, keys: list[ObjectKey], objects: list[MemoryObj], task_id: L2TaskId,
lmcache/v1/distributed/l2_adapters/fs_l2_adapter.py:575
↓ 1 callers
Method
_execute_store_in_the_loop
Store each key-object pair to its own file via dynamic DMA write.
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:589
↓ 1 callers
Method
_execute_store_in_the_loop
Coroutine that performs a batched store to Nixl storage. For each key-object pair, memory page indices are mapped to storage
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:705
↓ 1 callers
Method
_execute_store_in_the_loop
Execute the store operation in the event loop thread. This is a helper function to avoid blocking the main thread.
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:376
↓ 1 callers
Method
_execute_two_phase_benchmark
Execute two-phase benchmark: write then read. This method handles the write phase, release of write objects, read phase, and optional
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:509
↓ 1 callers
Function
_expected_retained_indices
The ranked retain-mask indices the pipeline should produce for ``hit``.
tests/v1/distributed/test_bitmap_ops.py:436
↓ 1 callers
Function
_export_config
Export resolved config to JSON and exit. Builds a standalone config dict from the resolved ``EngineBenchConfig`` and workload-specific CLI ar
lmcache/cli/commands/bench/engine_bench/command.py:399
↓ 1 callers
Method
_extra_cfg
Read extra_config['<key>'] (no prefix).
tests/v1/shm_allocator/shmfile_connector.py:132
↓ 1 callers
Function
_extract_exact_object_size
Extract a file size from an exact path info object.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:906
↓ 1 callers
Function
_extract_kv_at_slots
Extract KV data at specified slot positions from kv_tensor. Handles different kv_tensor formats: - MHA (5D): [2, num_blocks, block_size, num_
lmcache/v1/internal_api_server/vllm/cache_api.py:160
↓ 1 callers
Method
_fast_split_by_subtensor
Match the `sep_tokens` with sliding windows
lmcache/v1/token_database.py:469
↓ 1 callers
Function
_fetch_catalog
Fetch the source object catalog from the server's control socket.
lmcache/tools/transfer_channel_benchmark/benchmark.py:300
↓ 1 callers
Function
_fetch_child_nodes_from_api_nodes
Try to fetch child nodes via /api/nodes from a multiProcess server. Returns a non-empty list of child node dicts when the target is a multiPr
lmcache/lmcache_frontend/app.py:175
↓ 1 callers
Method
_fetch_instances
Fetch and parse the coordinator's active-instance list. Returns: The parsed peer instances. Raises: httpx.HT
lmcache/v1/multiprocess/modules/p2p_controller.py:386
↓ 1 callers
Function
_fetch_lmcache_status
Fetch ``/status`` from the LMCache HTTP server. Returns: Parsed JSON response. Raises: RuntimeError: If the server is unreac
lmcache/cli/commands/bench/engine_bench/config.py:96
↓ 1 callers
Function
_fetch_node_metrics
Fetch metrics from a single node
lmcache/lmcache_frontend/app.py:837
↓ 1 callers
Method
_finalize_drained_adapters
Detach draining adapters that no longer have in-flight tasks.
lmcache/v1/distributed/storage_controllers/store_controller.py:539
↓ 1 callers
Method
_finalize_drained_adapters
Detach draining adapters no longer referenced by any request.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:716
↓ 1 callers
Method
_finalize_inflight_locked
(self, key: KeyT, *, write_failed: bool)
lmcache/v1/storage_backend/dax/core.py:980
↓ 1 callers
Method
_finalize_load
Finalize a completed load: build result bitmap, transition L1 state, release read locks outside the retained set, and report the
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:1075
↓ 1 callers
Method
_finalize_store
Release read locks, publish completion, apply policy L1 deletions on success, and remove the tracking entry.
lmcache/v1/distributed/storage_controllers/store_controller.py:722
↓ 1 callers
Method
_find_device_for_key_locked
( self, key: ObjectKey, *, lock: bool, )
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:1173
↓ 1 callers
Method
_first_model_id
Return the first model ID from ``GET /v1/models``.
lmcache/cli/commands/query/_request.py:295
↓ 1 callers
Function
_flush
(bus: EventBus)
tests/cli/commands/trace/test_driver.py:101
↓ 1 callers
Method
_flush
Send buffered events to the coordinator.
lmcache/v1/mp_coordinator/l2/event_listener.py:99
↓ 1 callers
Function
_fmt_bytes
(n: int)
lmcache/tools/mp_status_viewer/__main__.py:26
↓ 1 callers
Method
_force_unpin_timeout_object
Force unpin a timeout object and log the event.
lmcache/v1/pin_monitor.py:169
↓ 1 callers
Function
_format_safe_path
URL-encode the object name to form a safe HTTP path.
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:194
↓ 1 callers
Function
_format_throughput
Format throughput as a human-readable string.
lmcache/v1/check/utils.py:271
↓ 1 callers
Function
_format_value
Format a metric value for terminal display.
lmcache/cli/metrics/formatter.py:86
↓ 1 callers
Method
_free_devdax_obj
(self, memory_obj: MemoryObj)
lmcache/v1/memory_management.py:2811
↓ 1 callers
Method
_free_inconsistent_lookup_locks
Release over-hit tail locks on servers that reported more than min. When servers disagree on hit chunk counts, servers reporting more
lmcache/integration/vllm/vllm_multi_process_adapter.py:776
↓ 1 callers
Function
_gather_paged_to_flat_chunks
Gather paged client tensors into flat per-chunk CPU tensors. Output layout matches the server's expected ``commit_store`` payload (set up at
lmcache/cli/commands/bench/server_bench/helpers.py:533
↓ 1 callers
Function
_generate_breaker
Generate a per-request breaker of fresh random tokens. Sampled fresh per request: each request's breaker tokens differ both in IDs (defeats c
lmcache/cli/commands/bench/engine_bench/workloads/prefix_suffix_tuner.py:410
↓ 1 callers
Method
_generate_documents
Generate synthetic documents of approximately ``document_length`` tokens.
lmcache/cli/commands/bench/engine_bench/workloads/long_doc_qa.py:162
↓ 1 callers
Method
_generate_fixed_kvcaches
Generate fixed pattern kvcaches for testing and MD5 verification. Supports both single group and multiple layer groups. Args:
lmcache/v1/standalone/__main__.py:236
↓ 1 callers
Method
_generate_history
Generate pre-filled history of approximately ``chat_history_length`` tokens.
lmcache/cli/commands/bench/engine_bench/workloads/multi_round_chat.py:226
↓ 1 callers
Method
_generate_multi_group_kvcaches
Generate kvcaches for multiple layer groups configuration Args: device: Device to create tensors on (default: "cpu")
lmcache/v1/standalone/__main__.py:246
↓ 1 callers
Function
_generate_prefix
Generate a prefix with unique-ID header + per-prefix random body. The unique-ID header makes the prefix's chained block hash unique per ``ind
lmcache/cli/commands/bench/engine_bench/workloads/prefix_suffix_tuner.py:360
↓ 1 callers
Method
_generate_prompts
Generate synthetic prompts of approximately ``request_length`` tokens.
lmcache/cli/commands/bench/engine_bench/workloads/random_prefill.py:97
↓ 1 callers
Method
_generate_random_endpoint
Generate random IP and port for test messages
lmcache/tools/controller_benchmark/handlers/deregister.py:45
↓ 1 callers
Function
_generate_suffix
Generate the single shared suffix used by every request. The suffix is bit-identical across all requests by design — its content-hash fingerp
lmcache/cli/commands/bench/engine_bench/workloads/prefix_suffix_tuner.py:390
↓ 1 callers
Method
_generate_system_prompt
Generate a system prompt of approximately ``shared_prompt_length`` tokens.
lmcache/cli/commands/bench/engine_bench/workloads/multi_round_chat.py:220
↓ 1 callers
Function
_generic_operation_payload
( payload: dict[str, object], )
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:296
↓ 1 callers
Method
_get
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/connector/redis_connector.py:278
↓ 1 callers
Method
_get
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/connector/redis_connector.py:656
↓ 1 callers
Method
_get_all_worker_infos_fresh
Internal method to get fresh WorkerInfo without cache.
lmcache/v1/cache_controller/utils.py:504
↓ 1 callers
Method
_get_all_worker_nodes_fresh
Internal method to get fresh (instance_id, WorkerNode) tuples without cache.
lmcache/v1/cache_controller/utils.py:531
↓ 1 callers
Method
_get_allocator_backend
( self, config: LMCacheEngineConfig )
lmcache/v1/storage_backend/storage_manager.py:311
↓ 1 callers
Function
_get_allowed_imports
Extract script_allowed_imports from either inProcess or mp mode.
lmcache/v1/internal_api_server/common/run_script_api.py:19
↓ 1 callers
Function
_get_backend
Try backends in order, first successful import wins.
lmcache/__init__.py:68
↓ 1 callers
Method
_get_cached_object_size_locked
(self, object_path: str)
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:864
↓ 1 callers
Function
_get_chunk_size
Query the server's chunk size.
lmcache/cli/commands/bench/server_bench/helpers.py:1161
↓ 1 callers
Function
_get_color_for_nvtx
(name)
lmcache/utils.py:666
↓ 1 callers
Function
_get_config_dict
Get the config dict filtered by keys
lmcache/v1/internal_api_server/vllm/conf_api.py:32
↓ 1 callers
Method
_get_cpu_info
(self)
lmcache/usage_context.py:232
↓ 1 callers
Method
_get_extra_result_fields
Get extra fields to add to the benchmark result. Override in subclass to add backend-specific fields.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:502
↓ 1 callers
Method
_get_file_name
(self, key: CacheEngineKey)
tests/v1/shm_allocator/shmfile_connector.py:199
↓ 1 callers
Method
_get_full_sync_sender
Lazy initialization of FullSyncSender
lmcache/v1/cache_controller/worker.py:340
↓ 1 callers
Method
_get_gpu_info
(self)
lmcache/usage_context.py:245
↓ 1 callers
Method
_get_hash_func
Load hash function with vLLM version compatibility. Adapted from TokenDatabase._get_vllm_hash_func (token_database.py:97-168).
lmcache/v1/multiprocess/token_hasher.py:73
↓ 1 callers
Function
_get_histogram_attrs
Return a list of attribute dicts from all data points of a histogram.
tests/v1/mp_observability/subscribers/metrics/test_l0_lifecycle.py:86
↓ 1 callers
Method
_get_init_hash
(self)
lmcache/v1/token_database.py:330
↓ 1 callers
Function
_get_metadata
(use_mla: bool)
tests/v1/test_connector.py:400
↓ 1 callers
Function
_get_metadata
Helper to create test metadata.
tests/v1/storage_backend/test_resp_connector.py:47
↓ 1 callers
Method
_get_next_sequence_number
Get next sequence number for message tracking. Thread-safe: Uses dedicated lock for sequence number generation.
lmcache/v1/storage_backend/batched_message_sender.py:119
↓ 1 callers
Method
_get_numa_node_count
(self)
lmcache/v1/storage_backend/connector/hf3fs_adapter.py:238
↓ 1 callers
Method
_get_object_size
(self, key_str: str)
lmcache/v1/storage_backend/connector/s3_connector.py:179
↓ 1 callers
Method
_get_object_size_bytes
Return the cached or fetched size for a specific LMCache key string.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:607
↓ 1 callers
Function
_get_or_create_engine
Return the LMCache engine singleton, creating it on first call. Called by the worker's ``register_kv_caches``. On subsequent calls (e.g. afte
lmcache/integration/tensorrt_llm/tensorrt_adapter.py:47
↓ 1 callers
Method
_get_or_create_instance
Get or create an instance node. Internal use only.
lmcache/v1/cache_controller/utils.py:379
↓ 1 callers
Method
_get_or_create_request_tracker
( self, request: "Request" )
lmcache/integration/vllm/lmcache_mp_connector.py:1320
↓ 1 callers
Method
_get_or_create_request_tracker
( self, request: "Request" )
lmcache/integration/vllm/lmcache_mp_connector_0201.py:1167
↓ 1 callers
Method
_get_or_create_request_tracker
( self, request: "Request" )
lmcache/integration/vllm/lmcache_mp_connector_0180.py:1046
↓ 1 callers
Method
_get_or_create_scope
Return the table for ``model_scope``, creating it if absent. Args: model_scope: The reuse scope to fetch a table for. Re
lmcache/v1/mp_coordinator/blend_directory.py:238
↓ 1 callers
Function
_get_path_info_path
Read a Hugging Face path-info object's path field defensively.
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:113
↓ 1 callers
Function
_get_path_info_size
Read a Hugging Face path-info object's size field defensively.
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:125
↓ 1 callers
Function
_get_path_info_type
Read a Hugging Face path-info object's type field defensively.
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:119
↓ 1 callers
Function
_get_per_tp_device_path
Return the device path configured for a TP rank. Args: per_tp_devices: Mapping with string or integer rank keys. tp_rank: Tensor-
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:59
↓ 1 callers
Method
_get_positive_float
Extract a positive float from config with validation. Args: config_dict: Configuration dictionary key: Confi
lmcache/v1/storage_backend/connector/sagemaker_hyperpod_adapter.py:211
↓ 1 callers
Method
_get_provider
(self)
lmcache/usage_context.py:198
↓ 1 callers
Method
_get_reconfigurable_l2_adapter
( self, adapter_index: int, )
lmcache/v1/distributed/storage_manager.py:1075
↓ 1 callers
Method
_get_remote_alloc_request
Get the allocation request given the keys and memory objects. Let's say there are N memory objects in total. We have the fol
lmcache/v1/storage_backend/pd_backend.py:465
↓ 1 callers
Method
_get_remote_alloc_request
Get the allocation request given the keys and memory objects. Let's say there are N memory objects in total. We have the fol
lmcache/v1/storage_backend/pd_backend_async.py:758
↓ 1 callers
Method
_get_request
(self, key_str: str, mem_obj: MemoryObj)
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:873
↓ 1 callers
Method
_get_scratch
Per-thread reusable staging buffer for buffer-GET (issue #6215). GLIDE writes into this thread-private bytearray (not slab memory), t
lmcache/v1/storage_backend/connector/valkey_connector.py:208
↓ 1 callers
Method
_get_size_for_object_group
Returns the size in bytes of the temp GPU buffer for the given object group **Assumes the size is lmcache_tokens_per_chunk
lmcache/v1/platform/cuda/cache_context.py:316
↓ 1 callers
Method
_get_slot_mapping_list
Convert slot_mapping to list if it's a tensor, otherwise return as is. :param slot_mapping: The slot_mapping to convert,
lmcache/v1/cache_engine.py:1927
↓ 1 callers
Method
_get_source
(self)
lmcache/usage_context.py:257
↓ 1 callers
Method
_get_version_from_nodes
Get version from target nodes by querying each node directly.
lmcache/lmcache_frontend/heartbeat.py:129
↓ 1 callers
Method
_get_vllm_hash_func
Get hash function from vLLM with version compatibility. Tries multiple import paths to support different vLLM versions: - vllm.utils.
lmcache/v1/token_database.py:123
↓ 1 callers
Method
_global_min_tokens
(self, local_tokens: int)
lmcache/integration/sglang/multi_process_adapter.py:189
← previous
next →
3,301–3,400 of 13,371, ranked by callers