Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 8 callers
Function
get_spec_class
Return the spec class for *fmt*, for static facts (``is_mla``, ...). Raises: ValueError: If *fmt* has no spec.
lmcache/v1/gpu_connector/kv_format/specs/registry.py:41
↓ 8 callers
Method
get_temp_object_group_buffer
( self, batch_idx: int, object_group_idx: int )
tests/v1/platform/test_cache_context_dispatch.py:103
↓ 8 callers
Method
get_transfer_channel_client
(self, peer_advertise_url: str)
tests/v1/distributed/transfer_channel/test_factory.py:45
↓ 8 callers
Method
get_worker_ids
Get sorted list of worker IDs.
lmcache/v1/cache_controller/utils.py:277
↓ 8 callers
Method
has_chunk
Return True if a chunk is cached for ``(key, layer_idx)``.
lmcache/v1/hidden_state_store.py:288
↓ 8 callers
Method
is_first_rank
Check if the current worker is the first rank
lmcache/v1/metadata.py:72
↓ 8 callers
Function
lmcache_get_or_create_config
Get the LMCache configuration from the environment variable `LMCACHE_CONFIG_FILE`. If the environment variable is not set, this function will
lmcache/integration/vllm/utils.py:75
↓ 8 callers
Function
lookup_all
Lookup all keys individually and return total found count. Uses the two-phase lookup protocol: LOOKUP registers the job server-side, then QUE
tests/v1/multiprocess/test_cache_server.py:152
↓ 8 callers
Function
make_adapter
( tmp_path, *, slot_bytes: int = 4096, max_slots: int = 2, num_store_workers: int = 1,
tests/v1/distributed/test_dax_l2_adapter.py:117
↓ 8 callers
Function
make_mock_adapter_config
Create a MockL2AdapterConfig, optionally with serde.
tests/v1/distributed/serde/test_serde_e2e.py:102
↓ 8 callers
Function
make_object_key
Create a deterministic object key for raw block tests. Args: chunk_id: Integer chunk identifier encoded into the object key hash.
tests/v1/storage_backend/raw_block_test_utils.py:88
↓ 8 callers
Function
make_page_buffer_shape_desc
Build a :class:`PageBufferShapeDesc` from a representative layer. Args: kv_caches: Full kv_caches structure. engine_kv_format: Fo
lmcache/v1/gpu_connector/utils.py:554
↓ 8 callers
Function
make_storage_manager_config
Build a StorageManagerConfig with the given L2 adapter configs.
tests/v1/distributed/serde/test_serde_e2e.py:116
↓ 8 callers
Method
name
(self)
tests/v1/test_health_monitor.py:35
↓ 8 callers
Function
newReconciler
newReconciler wires a LMCacheEngineReconciler around the suite's envtest client. Kept as a one-liner because every spec needs one.
operator/internal/controller/reconcile_helpers_test.go:89
↓ 8 callers
Method
no_worker_id_version
Create a copy with worker_id=None for lookup requests.
lmcache/v1/multiprocess/custom_types.py:329
↓ 8 callers
Method
num_blocks_per_chunk
Returns: The number of vllm blocks in a LMCache data chunk
lmcache/integration/vllm/vllm_multi_process_adapter.py:1682
↓ 8 callers
Function
optimal_lookup
Chain walk + filtered matcher probe + three-bucket partition. Args: matcher: V3 matcher with registered chunks. query: Query toke
tests/v1/multiprocess/test_optimized_lookup_v3.py:94
↓ 8 callers
Method
parent
Get the allocator that allocates this memory object
lmcache/v1/memory_management.py:411
↓ 8 callers
Method
put
(self, key: str, val: bool)
lmcache/v1/storage_backend/connector/bigtable_connector.py:47
↓ 8 callers
Method
query_prefetch_status
Query the status of the prefetch task. Args: handle (PrefetchHandle): The handle of the prefetch task. Returns:
lmcache/v1/distributed/storage_manager.py:637
↓ 8 callers
Method
query_serialize_result
Pop and return the serialize task result, or None if pending.
lmcache/v1/distributed/serde/async_processor.py:105
↓ 8 callers
Function
reasonFromReady
(ready bool, trueReason, falseReason string)
operator/internal/controller/reconcile_helpers.go:410
↓ 8 callers
Function
recordOnFailure
recordOnFailure runs the standard dump bundle iff the current spec failed. Use as the body of an AfterEach so passing specs stay quiet.
operator/test/e2e/smoke_helpers_test.go:195
↓ 8 callers
Method
records
Yield every record in the file in order. Yields each :class:`Record` as it is read. When the file ends cleanly (boundary aligned to
lmcache/v1/mp_observability/trace/reader.py:78
↓ 8 callers
Method
release
(self, n: int = 1)
lmcache/v1/storage_backend/storage_manager.py:156
↓ 8 callers
Method
release_write
Release a write lock.
lmcache/v1/cache_controller/locks.py:78
↓ 8 callers
Function
resolve_tokens_per_gb
Query the LMCache server and compute tokens per GB of KV cache. Fetches ``/status``, finds the model entry matching *model_name*, and compute
lmcache/cli/commands/bench/engine_bench/config.py:149
↓ 8 callers
Function
run
()
tests/v1/mp_coordinator/test_registrar.py:35
↓ 8 callers
Method
run
Run the benchmark and return results.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:369
↓ 8 callers
Function
run_engine_bench
Centralized orchestrator: create all modules and run the engine bench. Args: command: The outer ``BenchCommand`` instance, used for
lmcache/cli/commands/bench/engine_bench/command.py:522
↓ 8 callers
Method
sbrk
Expand the virtual address space by a given size. Args: size: The size to expand the address space. Will be aligned inte
lmcache/v1/memory_management.py:1522
↓ 8 callers
Function
server_module_factory
Create a patched server module/context with configurable mocks.
tests/v1/multiprocess/test_engine_driven_transfer.py:915
↓ 8 callers
Function
slice_block_ids_per_group
Slice each engine group's block IDs for a token range. The range is given in tokens — the only unit shared by every engine group. A group who
lmcache/v1/multiprocess/group_view.py:144
↓ 8 callers
Function
snapshot
()
tests/v1/mp_observability/subscribers/metrics/test_l1_failures.py:47
↓ 8 callers
Method
stop
(self, timeout: float = 5.0)
tests/v1/test_vllm_mp_adapter.py:79
↓ 8 callers
Method
stop
Stop the display thread and print final state.
lmcache/cli/commands/bench/engine_bench/progress.py:64
↓ 8 callers
Method
store
Paged store, then register the stored chunks as match fingerprints. Delegates the KV write to ``LMCacheDrivenTransfer.store``, then (worker 0
lmcache/v1/multiprocess/modules/blend_v3.py:1135
↓ 8 callers
Method
to_cuda_future
( self, device: Any | None = None, )
lmcache/v1/multiprocess/futures.py:70
↓ 8 callers
Method
update_active_memory_objs_count
(self, active_memory_objs_count: int)
lmcache/observability.py:579
↓ 8 callers
Method
update_interval_remote_read_metrics
(self, read_bytes: int)
lmcache/observability.py:531
↓ 8 callers
Function
wait_for_condition
(predicate, timeout: float = 5.0)
tests/v1/distributed/test_l1_l2_state_metrics.py:78
↓ 8 callers
Function
wait_for_prefetch_status
Poll query_prefetch_status until it returns a non-None value.
tests/v1/distributed/serde/test_serde_e2e.py:86
↓ 8 callers
Method
write_lock
Context manager for write lock with timeout. Args: timeout: Timeout in seconds. None means wait forever.
lmcache/v1/cache_controller/locks.py:99
↓ 7 callers
Function
MergeLabels
MergeLabels merges multiple label maps. Later maps take precedence.
operator/internal/resources/labels.go:38
↓ 7 callers
Function
StandardLabels
StandardLabels returns the full label set for all owned resources.
operator/internal/resources/labels.go:31
↓ 7 callers
Method
__init__
(self, size: int)
lmcache/v1/storage_backend/nixl_storage_backend.py:265
↓ 7 callers
Function
_apply_env_converter_safely
Apply env_converter to a value safely.
lmcache/v1/config_base.py:28
↓ 7 callers
Function
_bf16_tensor
(*shape: int, seed: int)
tests/v1/distributed/serde/test_asym_k16_v8_multi.py:41
↓ 7 callers
Function
_build_dynamic
(monkeypatch, metadata, config, local_cpu_backend=None)
tests/v1/storage_backend/test_nixl_shared_pool.py:236
↓ 7 callers
Function
_byte_buffer
(num_bytes: int)
tests/v1/distributed/serde/test_asym_k16_v8_v_only.py:61
↓ 7 callers
Function
_clip
(text: str, limit: int = _MAX_ERR)
lmcache/cli/commands/query/_request.py:26
↓ 7 callers
Function
_create_local_cpu_backend
(memory_allocator, use_mla, config=None)
tests/v1/test_connector.py:416
↓ 7 callers
Method
_format_object_key
Format the NIXL object name for ``key`` per the configured backend.
lmcache/v1/storage_backend/nixl_storage_backend.py:1494
↓ 7 callers
Function
_generate_system_prompt
Generate a deterministic shared system prompt of ~``length`` tokens. Args: length: Approximate token length of the system prompt.
lmcache/cli/commands/bench/engine_bench/workloads/long_doc_permutator.py:135
↓ 7 callers
Method
_get_instance
Get an instance node. Uses optimistic locking (lock-free read).
lmcache/v1/cache_controller/utils.py:395
↓ 7 callers
Method
_get_setup_key
(self, key: str)
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:197
↓ 7 callers
Function
_load_bitmap
Run a load for ``keys`` and return its result bitmap.
tests/v1/distributed/test_fault_inject_l2_adapter.py:117
↓ 7 callers
Function
_make_5d_kv
Create deterministic 5D CPU KV tensors: ``[kv, NB, BS, NH, HS]``. Matches the ``NL_X_TWO_NB_BS_NH_HS`` layout (block_axis=1). ``kv_size=1`` m
tests/v1/multiprocess/test_compute_mp_checksums.py:13
↓ 7 callers
Function
_make_completions_chunk
Build a minimal ``Completion`` chunk.
tests/cli/commands/bench/engine_bench/test_request_sender.py:50
↓ 7 callers
Function
_make_config
Default config plus the requested overrides.
tests/v1/test_musa_support.py:53
↓ 7 callers
Function
_make_end_session_event
(request_id: str)
tests/v1/mp_observability/subscribers/metrics/test_l0_lifecycle.py:60
↓ 7 callers
Function
_make_fake_module
Create a fake Python module containing *adapter_cls*.
tests/v1/distributed/test_l2_adapter_factory.py:61
↓ 7 callers
Function
_make_key
Build an IPCCacheServerKey.
lmcache/cli/commands/bench/server_bench/helpers.py:171
↓ 7 callers
Function
_make_local_cpu_paged
LocalCPUBackend with memory_allocator = MixedMemoryAllocator(use_paging=True). Sized to exactly 4 chunks so tests that need to drive the pool to
tests/v1/storage_backend/test_nixl_shared_pool.py:165
↓ 7 callers
Function
_make_native_module
Create a fake native module with the Stage2 LMCache ABI.
tests/v1/test_musa_native.py:17
↓ 7 callers
Function
_make_notifier_fd
Create a test fd suitable for the periodic notifier to write to. Returns (poll_fd, write_fd, close_func) where poll_fd is for select.poll() a
tests/v1/native_storage_ops/test_periodic_event_notifier.py:27
↓ 7 callers
Function
_make_storage_manager_config
Build a StorageManagerConfig for multiprocess engine-context tests.
tests/v1/multiprocess/test_engine_driven_transfer.py:150
↓ 7 callers
Function
_make_transfer_spec
( receiver_host="127.0.0.1", init_port=9100, alloc_port=9101, req_id="req-0", is_last_pref
tests/v1/storage_backend/test_pd_backend_async.py:87
↓ 7 callers
Function
_make_unique_slot_mapping
( *, total_slots: int, num_tokens: int, device: torch.device )
tests/v1/test_xpu_sglang_connector.py:31
↓ 7 callers
Method
_nixl_cpu_defaults
(**overrides: Any)
tests/v1/test_config.py:892
↓ 7 callers
Method
_normalize_heartbeat_url
Normalize heartbeat URL based on controller connection. If benchmark connects to controller via localhost (127.0.0.1), but heartbeat_
lmcache/tools/controller_benchmark/benchmark.py:327
↓ 7 callers
Function
_object_key_to_string
Serialize an MP ``ObjectKey`` to the shared L2 object-name format. Unsalted keys use ``<model_name>@<kv_rank_hex>@<object_group_id_hex>@<chun
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:80
↓ 7 callers
Function
_op
Build a minimal four-token ``LoadStoreOp`` over *block_ids*.
tests/v1/test_vllm_mp_adapter.py:120
↓ 7 callers
Function
_parse
(argv: list[str])
tests/v1/multiprocess/test_config.py:33
↓ 7 callers
Method
_process_tokens_internal
Process tokens and populate the reordered lists. This function is used to process tokens and populate the reordered lists. Args:
lmcache/v1/cache_engine.py:1703
↓ 7 callers
Function
_put_object
Place a key in the fake S3 backend with deterministic content. Mirrors the production PUT pipeline: ``_object_key_to_string`` → ``_format_saf
tests/v1/distributed/test_s3_l2_adapter.py:786
↓ 7 callers
Method
_rawdev
Return the lazily opened Rust raw-block device binding.
lmcache/v1/storage_backend/raw_block/core.py:406
↓ 7 callers
Method
_release_write_temps
Release write-locked temps and delete them. No-op on empty.
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:621
↓ 7 callers
Function
_resolve_args
Resolve args via config file, interactive mode, or pass through.
lmcache/cli/commands/bench/engine_bench/command.py:353
↓ 7 callers
Method
_run_benchmark
Run a benchmark with the given worker function.
benchmarks/microbenchmark/ttl_lock_benchmark.py:95
↓ 7 callers
Method
_set_cached_object_size_locked
(self, object_path: str, size: int)
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:873
↓ 7 callers
Function
_skip_if_no_xpu
()
tests/v1/test_xpu_sglang_connector.py:26
↓ 7 callers
Function
_store_and_wait
Helper: store one key and wait for the store event fd to fire.
tests/v1/distributed/test_nixl_store_l2_adapter.py:842
↓ 7 callers
Function
_tensor_from_ptr
Create a tensor view over a raw pointer (zero-copy where possible). Supports both CPU (pinned or regular) and CUDA device pointers. Arg
lmcache/python_ops_fallback.py:61
↓ 7 callers
Method
_validate
( self, slots: int, tokens: int, chunk: int = 256, sw: int = -1 )
tests/v1/test_kv_layer_groups_manager.py:304
↓ 7 callers
Function
_wait_event_fd
(event_fd: int, timeout: float = 5.0)
tests/v1/distributed/test_raw_block_l2_adapter.py:121
↓ 7 callers
Function
_wait_for_fd
Wait until ``fd`` is readable or timeout. Drains the pending signal.
tests/v1/distributed/serde/test_async_processor.py:55
↓ 7 callers
Method
add_event
Add an event with the given type and id.
lmcache/v1/event_manager.py:33
↓ 7 callers
Method
add_normal_thread_pool
Assign a ThreadPoolExecutor to specific request types. Use this for non-GPU blocking handlers (e.g. LOOKUP, END_SESSION). Must be ca
lmcache/v1/multiprocess/mq.py:780
↓ 7 callers
Method
allocate
( self, shapes: Union[torch.Size, list[torch.Size]], dtypes: Union[torch.dtype, list[t
lmcache/v1/memory_management.py:1622
↓ 7 callers
Method
batched_remove
( self, keys: list[CacheEngineKey], free_obj: bool = True, )
lmcache/v1/storage_backend/audit_backend.py:218
↓ 7 callers
Method
calculate_num_blocks
Calculate the number of blocks for *num_tokens* in a kernel group.
lmcache/v1/platform/base_cache_context.py:192
↓ 7 callers
Function
cbSpecToEngineSpec
cbSpecToEngineSpec projects a CacheBlendEngineSpec onto an LMCacheEngineSpec so the shared, spec-keyed resource builders (ComputeResources, BuildConta
operator/internal/resources/cacheblend_engine.go:64
↓ 7 callers
Method
clear
Clear all stored KV cache data from the storage manager.
lmcache/v1/multiprocess/modules/management.py:171
↓ 7 callers
Function
clear_and_wait_drained
Clear L1 and poll until every object is evicted. After an L2 store the StoreController holds read locks on the stored objects for a short win
tests/v1/distributed/serde/test_serde_e2e.py:198
↓ 7 callers
Method
close
Release EC engine resources.
lmcache/integration/vllm/vllm_ec_adapter.py:191
↓ 7 callers
Method
close
(self)
lmcache/v1/cache_controller/worker.py:653
↓ 7 callers
Method
close
Release L1 memory handlers.
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:321
↓ 7 callers
Method
close
(self)
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:527
← previous
next →
801–900 of 13,371, ranked by callers