Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 5 callers
Method
get_num_layers
(self, parallel_config)
tests/v1/test_ec_connector.py:45
↓ 5 callers
Function
get_page_buffer_size
Return the page buffer size (num_blocks * block_size) from ``kv_caches``.
lmcache/v1/gpu_connector/utils.py:266
↓ 5 callers
Method
get_paged_buffers
Get the paged buffers for fixed buffer registration. Returns: Tuple of paged buffer tensors if using paged allocator, No
lmcache/v1/memory_management.py:2554
↓ 5 callers
Method
get_shape
(self, num_tokens: int)
lmcache/v1/gpu_connector/gpu_connectors.py:1422
↓ 5 callers
Method
get_shape
(self, num_tokens: int)
lmcache/v1/gpu_connector/xpu_connectors.py:1228
↓ 5 callers
Method
get_shape
(self, num_tokens: int)
lmcache/v1/gpu_connector/xpu_connectors.py:1791
↓ 5 callers
Method
get_store_event_fd
Get the event fd for store operation, which will be signaled on the completion of the store tasks. Returns: int:
lmcache/v1/distributed/l2_adapters/base.py:152
↓ 5 callers
Function
get_transfer_channel_context
Get the global transfer channel context. Raises: RuntimeError: If the context has not been initialized yet.
lmcache/v1/distributed/transfer_channel/__init__.py:87
↓ 5 callers
Method
handleFinalizer
handleFinalizer migrates any LMCacheEngine carrying the legacy "lmcache.ai/cleanup" finalizer off of it. Owner references on every child resource (Dae
operator/internal/controller/reconcile_helpers.go:57
↓ 5 callers
Method
handle_full_sync_end
Handle full sync end message from a worker. This marks the sync as end-received and records actual total keys.
lmcache/v1/cache_controller/controllers/kv_controller.py:306
↓ 5 callers
Method
hotplug_remove_device
Remove or drain a DAX device. Args: device_path: Device path to remove. mode: ``migrate`` preserves KV on another dev
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:833
↓ 5 callers
Function
init_event_bus
Replace the global singleton with a new EventBus built from *config*. Returns the newly created bus.
lmcache/v1/mp_observability/event_bus.py:379
↓ 5 callers
Function
init_logger
(name: str, log_level=logging.DEBUG)
benchmarks/rag/utils.py:48
↓ 5 callers
Method
insert
Insert one fingerprint, growing the table when needed. Args: poly: The chunk's full 64-bit poly hash. loc: The chunk'
lmcache/v1/mp_coordinator/blend_directory.py:145
↓ 5 callers
Method
is_frozen
Get the current freeze mode status. Returns: bool: True if freeze mode is enabled, False otherwise
lmcache/v1/cache_engine.py:348
↓ 5 callers
Method
is_healthy
Check if the LMCache system is healthy. This method returns False if: - Initialization failed (irrecoverable error)
lmcache/v1/cache_engine.py:257
↓ 5 callers
Method
is_ready
True when all required items (whose conditions are met) have values.
lmcache/cli/commands/bench/engine_bench/interactive/state.py:85
↓ 5 callers
Method
l2_adapters
Return all active L2 adapters paired with descriptors, in ascending adapter-id order (== configuration order for the initial set, then
lmcache/v1/distributed/storage_manager.py:888
↓ 5 callers
Function
loadNodeListForManagement
()
lmcache/lmcache_frontend/static/js/app.js:247
↓ 5 callers
Function
make_adapter
()
tests/v1/distributed/test_runtime_adapters.py:83
↓ 5 callers
Function
make_adapter
(bandwidth_gb: float = 10.0)
tests/v1/distributed/test_l1_l2_state_metrics.py:68
↓ 5 callers
Function
make_aligned_tensor
Create a 1-D uint8 tensor whose data pointer is aligned. Args: num_bytes: Number of bytes in the returned tensor. align_bytes: Re
lmcache/cli/commands/bench/l2_adapter_bench/data.py:27
↓ 5 callers
Function
make_descriptor
(index: int)
tests/v1/distributed/test_runtime_adapters.py:87
↓ 5 callers
Function
make_descriptor
(index: int)
tests/v1/distributed/test_l1_l2_state_metrics.py:73
↓ 5 callers
Function
make_descriptor
Create an AdapterDescriptor for testing.
tests/v1/distributed/test_store_policy.py:32
↓ 5 callers
Function
make_layout
()
tests/v1/distributed/test_runtime_adapters.py:59
↓ 5 callers
Function
make_layout
()
tests/v1/distributed/test_l1_l2_state_metrics.py:61
↓ 5 callers
Function
make_object_key
(chunk_id: int)
tests/v1/distributed/test_runtime_adapters.py:51
↓ 5 callers
Function
make_object_key
(chunk_id: int)
tests/v1/distributed/test_l1_l2_state_metrics.py:53
↓ 5 callers
Function
merge_bitmaps
Merge bitmaps with a bitwise OR into a ``num_keys``-sized bitmap. Always returns a ``num_keys``-sized bitmap (empty input -> all zeros), so d
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:54
↓ 5 callers
Function
minimalCoordinator
()
operator/internal/resources/coordinator_test.go:29
↓ 5 callers
Method
notify
(self)
lmcache/v1/platform/event_notifier.py:193
↓ 5 callers
Method
on_l1_keys_accessed
(self, keys: list[ObjectKey])
lmcache/v1/distributed/eviction.py:166
↓ 5 callers
Method
on_l1_keys_write_finished
Enqueue keys and signal the notifier. Called inside L1Manager's lock. Must be fast and must not call any L1Manager methods (
lmcache/v1/distributed/storage_controllers/store_controller.py:121
↓ 5 callers
Method
on_lookup_finished
This function is called when a lookup request is finished. It will record the number of tokens hit and track by node type. A
lmcache/observability.py:353
↓ 5 callers
Method
on_request_finished
Called when a request completes. Decrements in-flight.
lmcache/cli/commands/bench/engine_bench/progress.py:78
↓ 5 callers
Method
on_retrieve_finished
( self, retrieve_stats: RetrieveRequestStats, num_retrieved_tokens: int, )
lmcache/observability.py:399
↓ 5 callers
Method
on_store_finished
( self, store_stats: StoreRequestStats, num_stored_tokens: int = -1, )
lmcache/observability.py:478
↓ 5 callers
Method
on_store_request
Returns the internal "request id" that will be used in on_store_finished
lmcache/observability.py:460
↓ 5 callers
Function
p2p_backend_context
Context manager for P2P backend with automatic cleanup
tests/v1/storage_backend/test_p2p_backend_with_controller.py:330
↓ 5 callers
Function
parse_args_to_l2_adapters_config
Build L2AdaptersConfig from parsed command-line arguments. Expects args to have the attribute added by add_l2_adapters_args (a list of J
lmcache/v1/distributed/l2_adapters/config.py:438
↓ 5 callers
Method
poll_match
Return the match state for a request. Args: rid: Request id used in :meth:`submit_match`. Returns: :data:`PE
lmcache/v1/mp_coordinator/blend_client.py:177
↓ 5 callers
Method
put
(self, key: CacheEngineKey, mock_obj: MockMemoryObj)
lmcache/v1/storage_backend/connector/mock_connector.py:76
↓ 5 callers
Method
put_many
Persist a batch of memory objects into raw-block slots. Args: keys: Ordered raw-block key specs corresponding to ``objs``.
lmcache/v1/storage_backend/raw_block/core.py:614
↓ 5 callers
Method
pwrite_from_buffer
(self, offset, data, payload_len=None, total_len=None)
tests/v1/storage_backend/test_rust_raw_block_backend.py:139
↓ 5 callers
Method
query_lookup_and_lock_result
(self, task_id: L2TaskId)
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:433
↓ 5 callers
Method
query_lookup_result
Query the keys that are found during the lookup for a specific request. Thread-safe. Returns the prefix-hit count if the lookup phas
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:371
↓ 5 callers
Method
query_worker_info
Query worker info.
lmcache/v1/cache_controller/controllers/registration_controller.py:245
↓ 5 callers
Method
random_instance
Return a uniformly random registered instance, or ``None`` when the registry is empty.
lmcache/v1/mp_coordinator/registry.py:131
↓ 5 callers
Method
read
(self, buffer: memoryview, length: int, file_offset: int = 0)
lmcache/v1/storage_backend/connector/hf3fs_connector.py:618
↓ 5 callers
Method
receive_batch
Record receipt of a sync batch. Returns: True if batch was recorded, False if invalid
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:183
↓ 5 callers
Method
register
Register KV caches with the server and wait for ACK. Args: instance_id: Worker process instance identifier. kv_caches
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:122
↓ 5 callers
Method
register_client
Register a client for the given key (peer advertise url). A client already registered for ``key`` is kept; ``client`` is then a
lmcache/v1/distributed/transfer_channel/impl/nixl_impl.py:436
↓ 5 callers
Method
register_kv_cache
Register the KV cache tensors for a given GPU instance ID. Args: instance_id: The GPU instance ID (such as PID). kv_c
lmcache/v1/multiprocess/modules/lmcache_driven_transfer.py:623
↓ 5 callers
Function
register_transfer_channel_factory
Register a factory that creates a context for ``transfer_channel_type``. Each implementation module should call this at import time. Args:
lmcache/v1/distributed/transfer_channel/factory.py:37
↓ 5 callers
Method
release_storage_handler
Release storage handler resources. :param reg_descs: Memory descriptors to deregister. :param xfer_handler: Transfer dlist h
lmcache/v1/storage_backend/nixl_storage_backend.py:669
↓ 5 callers
Method
reserve_reads
Reserve committed entries for a read pipeline. Each reservation increments the slot borrow count so eviction can remove the key from
lmcache/v1/storage_backend/dax/core.py:568
↓ 5 callers
Method
resync_from
Page through ``instance``'s L2 keys and record each one. Returns the number of keys recorded; stops early on HTTP failure and returns
lmcache/v1/mp_coordinator/l2/resync_manager.py:53
↓ 5 callers
Method
save_kv_layer
Start saving the a layer of KV cache from vLLM's paged buffer to the connector. Args: layer_name (str): the name of the l
lmcache/integration/vllm/vllm_v1_adapter.py:996
↓ 5 callers
Method
select_l1_retentions
Determine which keys to retain in L1 after prefetched objects are consumed. Called by PrefetchController just before ``l1_mgr
lmcache/v1/distributed/storage_controllers/prefetch_policy.py:53
↓ 5 callers
Method
select_store_targets
Store all keys to all adapters. Args: keys: Keys that were just written to L1. adapters: Descriptors of avai
lmcache/v1/distributed/storage_controllers/store_policy.py:150
↓ 5 callers
Method
send_response
Send a response back to the client. Args: identity: The client identity from recv_request. response: The raw response
lmcache/v1/rpc/transport.py:89
↓ 5 callers
Method
set_healthy
(self, healthy: bool)
tests/v1/test_health_monitor.py:45
↓ 5 callers
Method
set_need_full_sync_all
Set whether all workers need full sync (e.g., after controller restart)
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:71
↓ 5 callers
Method
setup_stats_client
(self, **kwargs)
tests/v1/lookup_client/test_chunk_statistics_lookup_client.py:68
↓ 5 callers
Function
snapshot
Capture counters before the test; yield a callable that returns deltas.
tests/v1/mp_observability/subscribers/metrics/test_cb_server.py:38
↓ 5 callers
Function
snapshot
()
tests/v1/mp_observability/subscribers/metrics/test_l2_failures.py:46
↓ 5 callers
Function
snapshot
Capture counters before the test; yield a callable that returns deltas.
tests/v1/mp_observability/subscribers/metrics/test_lookup.py:96
↓ 5 callers
Method
stop
Stop the event loop and thread
lmcache/v1/check/utils.py:557
↓ 5 callers
Method
stop_services
Stop all managed components gracefully.
lmcache/v1/manager.py:241
↓ 5 callers
Method
submit_batch_exists
(self, keys)
tests/v1/distributed/test_l2_adapter_factory.py:711
↓ 5 callers
Method
submit_batch_get
(self, keys, memoryviews)
tests/v1/distributed/test_l2_adapter_factory.py:705
↓ 5 callers
Method
submit_batch_set
(self, keys, memoryviews)
tests/v1/distributed/test_l2_adapter_factory.py:708
↓ 5 callers
Method
submit_lookup_and_lock_task
( self, keys: list[ObjectKey], layout_desc: MemoryLayoutDesc )
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:424
↓ 5 callers
Method
submit_read
Read data from the remote addresses into the local addresses. Args: local_addresses: addresses in the local L1 buffer.
lmcache/v1/distributed/transfer_channel/abstract.py:61
↓ 5 callers
Method
submit_retrieve_request
Submit a KV cache retrieve request to LMCache When the server is unhealthy the request is not submitted: blocks are flagged
lmcache/integration/vllm/vllm_multi_process_adapter.py:1429
↓ 5 callers
Method
submit_unlock
Submit an unlock task to unlock a batch of objects by the given keys. Args: keys (list[ObjectKey]): the list of keys to
lmcache/v1/distributed/l2_adapters/base.py:280
↓ 5 callers
Method
summary_lines
Return ``(label, value_str)`` pairs for the config summary.
lmcache/cli/commands/bench/engine_bench/interactive/state.py:298
↓ 5 callers
Method
time_to_retrieve
(self)
lmcache/observability.py:151
↓ 5 callers
Method
to_dict
(self)
lmcache/utils.py:510
↓ 5 callers
Method
to_json
(self)
tests/v1/plugin/test_runtime_plugin_launcher.py:24
↓ 5 callers
Method
to_namespace
Convert to ``argparse.Namespace`` compatible with ``run_engine_bench``. Fills defaults for any unset items, then builds the namespace
lmcache/cli/commands/bench/engine_bench/interactive/state.py:212
↓ 5 callers
Method
to_string
(self)
lmcache/utils.py:588
↓ 5 callers
Function
unfold
Expand a model-wide hit length into the per-group retain mask. Each group retains the chunks it needs to serve ``hit_length``: ``[0, hit_leng
lmcache/v1/distributed/bitmap_ops/fold.py:127
↓ 5 callers
Method
unlock_many
Release external locks for keys. Args: keys: Keys whose external lock refcount should be decremented.
lmcache/v1/storage_backend/dax/core.py:482
↓ 5 callers
Method
unpin
Unpin the memory obj so that it can be evicted.
lmcache/v1/memory_management.py:308
↓ 5 callers
Method
unregister_all_metrics
()
lmcache/observability.py:842
↓ 5 callers
Function
wait_for_condition
(predicate, timeout: float = 5.0, poll: float = 0.02)
tests/v1/distributed/test_runtime_adapters.py:70
↓ 5 callers
Function
wait_for_condition
Poll until a predicate returns True or timeout.
tests/v1/distributed/test_prefetch_controller.py:78
↓ 5 callers
Function
wait_for_event_fd
Wait for an eventfd notification and consume it when present. Args: event_fd: Event file descriptor to poll. timeout: Maximum wai
tests/v1/storage_backend/raw_block_test_utils.py:160
↓ 5 callers
Method
wait_for_in_flight_dispatches
Await every outstanding fire-and-forget dispatch.
lmcache/v1/mp_coordinator/l2/eviction_manager.py:167
↓ 5 callers
Function
wait_for_lookup_result
Poll query_lookup_result until it returns a non-None value.
tests/v1/distributed/test_prefetch_controller.py:108
↓ 5 callers
Method
write
(self, data: memoryview, length: int, file_offset: int = 0)
lmcache/v1/storage_backend/connector/hf3fs_connector.py:615
↓ 5 callers
Function
write_resp
Write text to the specified output file (if any), otherwise to stdout.
benchmarks/multi_doc_qa/multi_doc_qa.py:90
↓ 4 callers
Function
ComputeResources
ComputeResources returns resource requirements, either from overrides or auto-computed from L1 size.
operator/internal/resources/compute.go:31
↓ 4 callers
Function
CoordinatorPort
CoordinatorPort returns the coordinator's HTTP port, applying the default.
operator/internal/resources/coordinator_deployment.go:78
↓ 4 callers
Function
_add_latency_section
Add a latency summary section to the metrics report. Computes count, mean, min, max, p50, and p99 from the raw latency list. Skipped if the l
lmcache/cli/commands/bench/server_bench/command.py:668
↓ 4 callers
Function
_app_with_all_apis
()
tests/v1/multiprocess/http_apis/test_common_api.py:31
↓ 4 callers
Method
_apply_cb_rope_batched
Re-RoPE the given tmp-pool slots in place (K-only, per kernel group). Args: gpu_context (GPUCacheContext): The instance's GPU cac
lmcache/v1/multiprocess/modules/blend_v3.py:1368
↓ 4 callers
Function
_build_headers
Build common HTTP headers, including auth if OPENAI_API_KEY is set.
examples/disagg_prefill_mp/disagg_proxy_server.py:156
← previous
next →
1,301–1,400 of 13,371, ranked by callers