Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 3 callers
Method
is_valid
Whether the address is valid (non-negative offset and size).
lmcache/v1/distributed/transfer_channel/api.py:29
↓ 3 callers
Method
list
(self)
lmcache/v1/storage_backend/connector/mock_connector.py:93
↓ 3 callers
Function
lmcache_memcpy_async_h2d
Helper function to copy memory object allocated by different allocators to GPU buffer. This function is non-blocking and won't do stream sync
lmcache/v1/gpu_connector/gpu_ops.py:13
↓ 3 callers
Function
loadNodes
()
lmcache/lmcache_frontend/static/js/app.js:238
↓ 3 callers
Function
loadTargetNodes
(proxyName, filteredNodes = null)
lmcache/lmcache_frontend/static/js/app.js:196
↓ 3 callers
Function
load_ec_engine_config
Build EC config from base config plus EC-prefixed overrides. Precedence is: 1) base LMCache config, 2) YAML keys prefixed with ``ec_`` (o
lmcache/v1/config.py:1204
↓ 3 callers
Method
load_many_into
Load DAX entries directly into caller-provided buffers. Args: keys: Keys to load. objs: Destination memory objects. E
lmcache/v1/storage_backend/dax/core.py:316
↓ 3 callers
Method
log_config
Log key workload config before the benchmark starts.
lmcache/cli/commands/bench/engine_bench/workloads/base.py:59
↓ 3 callers
Method
lookup
( self, token_ids: Union[torch.Tensor, list[int]], lookup_id: str, request_con
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:121
↓ 3 callers
Function
make_chain_walk_storage
Build a chain-walk callable backed by a dict. Models the real chain-walk path: walks contiguous chunks from position 0, probes a fast in-memo
tests/v1/multiprocess/test_optimized_lookup_v3.py:149
↓ 3 callers
Function
make_ipc_key
Create an IPCCacheServerKey for testing.
tests/v1/multiprocess/test_unified_touch.py:37
↓ 3 callers
Function
make_memory_objects
Create MemoryObj views backed by a shared L1 benchmark buffer. Each returned object is a ``data_size``-byte slice of ``buffer``, pre-filled w
lmcache/cli/commands/bench/l2_adapter_bench/data.py:87
↓ 3 callers
Function
make_object_keys
Generate *num_keys* unique ``ObjectKey`` instances for benchmarking. ``ObjectKey`` is a frozen dataclass with field order: (chunk_hash, model
lmcache/cli/commands/bench/l2_adapter_bench/data.py:59
↓ 3 callers
Method
mark_end
Record the wall-clock time when replay finished. Args: wall_time_s: ``time.time()`` at replay end.
lmcache/cli/commands/trace/_stats.py:125
↓ 3 callers
Method
mark_failed
Mark a worker's sync as failed
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:291
↓ 3 callers
Method
mark_start
Record the wall-clock time when replay began. Called once before the first ``record()``; replay-duration metrics are derived from sta
lmcache/cli/commands/trace/_stats.py:113
↓ 3 callers
Method
match_sub_sequence
Find every registered chunk reused anywhere in a query sequence. Vectorized direct-address probe over all token positions, then a small
lmcache/v1/multiprocess/modules/blend_v3.py:212
↓ 3 callers
Function
measure_list_depth_until_tensor
Return ``(list_depth, tensor_ndim, first_tensor)`` for *kv_caches*. Descends the first element of each list down to the inner tensor, counting
lmcache/v1/gpu_connector/kv_format/detectors/base.py:20
↓ 3 callers
Method
memcheck
(self)
lmcache/v1/memory_management.py:2535
↓ 3 callers
Method
memcheck
For debug purposes; logs allocator state and checks consistency. Mirrors ``TensorMemoryAllocator.memcheck`` for the GDS slab address
lmcache/v1/distributed/memory_manager/gds_l1_memory_manager.py:127
↓ 3 callers
Function
memory_obj_to_bytes
Convert a test memory object to raw bytes.
tests/v1/storage_backend/test_hfbucket_connector.py:227
↓ 3 callers
Function
merge_bitmaps
Merge bitmaps with a bitwise OR into a ``num_keys``-sized bitmap. Always returns a ``num_keys``-sized bitmap (empty input -> all zeros), so d
lmcache/v1/distributed/bitmap_ops/fold.py:299
↓ 3 callers
Function
mock_controller_context
Context manager for mock controller process
tests/v1/storage_backend/test_p2p_backend_with_controller.py:302
↓ 3 callers
Function
need_gpu_interm_buffer
Check if the GPU Connector needs to create an intermediate buffer on the GPU
lmcache/v1/gpu_connector/utils.py:57
↓ 3 callers
Method
num_allocated_blocks
(self)
lmcache/integration/vllm/lmcache_mp_connector.py:265
↓ 3 callers
Method
num_cached_chunks
Return the number of distinct chunk keys currently cached.
lmcache/v1/hidden_state_store.py:284
↓ 3 callers
Method
oldest_event_lag_seconds
Wall-clock age of the oldest queued event, or 0.0 when empty. Used as a drain-lag gauge: a rising value means the drain thread is not
lmcache/v1/mp_observability/event_bus.py:274
↓ 3 callers
Method
on_keys_removed
Notify the eviction policy that keys have been deleted. Args: keys (list[ObjectKey]): The keys that have been deleted.
lmcache/v1/distributed/eviction.py:76
↓ 3 callers
Method
on_l1_keys_read_finished
(self, keys: list[ObjectKey])
lmcache/v1/distributed/eviction.py:146
↓ 3 callers
Method
on_lookup
Touch ``key`` in the LRU (move to MRU end).
lmcache/v1/mp_coordinator/l2/eviction_manager.py:66
↓ 3 callers
Method
on_new_token_hashes
Register a new token sequence and index its non-overlapping chunks. Args: token_ids: Raw token IDs for the full sequence (num_tok
lmcache/v1/multiprocess/modules/blend.py:96
↓ 3 callers
Method
on_pin
Register a pinned memory object for timeout monitoring. Note: The same memory_obj can be pinned multiple times, so this function may
lmcache/v1/pin_monitor.py:91
↓ 3 callers
Method
on_store
Register a stored key in the LRU. Per-salt bytes are the caller's responsibility (see :meth:`L2UsageManager.record_stored`).
lmcache/v1/mp_coordinator/l2/eviction_manager.py:61
↓ 3 callers
Method
on_unpin
Unregister a memory object from timeout monitoring.
lmcache/v1/pin_monitor.py:109
↓ 3 callers
Function
override_controller_config_from_dict
Override configuration with dictionary
lmcache/v1/cache_controller/config.py:119
↓ 3 callers
Function
parseCBConnectionConfig
=========================== BuildCBConnectionConfigMap =========================== parseCBConnectionConfig unmarshals the CBKVConnector kv-transfer-co
operator/internal/resources/cacheblend_engine_test.go:258
↓ 3 callers
Function
parse_args_to_config
Convert parsed CLI arguments into a fully-resolved EngineBenchConfig. Handles model auto-detection and tokens-per-GB resolution from the LMCa
lmcache/cli/commands/bench/engine_bench/config.py:215
↓ 3 callers
Function
parse_args_to_coordinator_config
Convert parsed command line arguments to a CoordinatorConfig. A flag value takes precedence over its environment variable. The heartbeat inte
lmcache/v1/multiprocess/config.py:586
↓ 3 callers
Function
parse_kv_dtype
Parse a kv_dtype string to a torch.dtype. Returns None if the string is not recognized.
lmcache/v1/check/utils.py:51
↓ 3 callers
Method
pin
(self)
lmcache/v1/memory_management.py:758
↓ 3 callers
Method
pin
Increment the external lock count for ``key`` if it exists. Args: key: The cache key to pin. Returns: ``True
lmcache/v1/storage_backend/plugins/dax_backend.py:278
↓ 3 callers
Method
pop_completed_store_tasks
Pop all completed store tasks. Returns: dict[L2TaskId, L2StoreResult]: a dictionary mapping the task id to an ``L2Sto
lmcache/v1/distributed/l2_adapters/base.py:222
↓ 3 callers
Function
postCompletion
postCompletion sends a single /v1/completions request and discards the result. Failures (non-2xx, network errors, JSON parse errors) propagate to the
operator/test/e2e/vllm_integration_smoke_test.go:210
↓ 3 callers
Method
preprocess
Preprocess token IDs before recording. This method is called to transform raw token IDs into a format suitable for the specific recor
lmcache/v1/lookup_client/record_strategies/base.py:70
↓ 3 callers
Method
process_tokens
Process the tokens/hashes and return the corresponding cache engine keys. :param Optional[Union[torch.Tensor, List[int]]] tokens: The tokens
lmcache/v1/token_database.py:367
↓ 3 callers
Function
prompt_choice
Prompt user to select from a list using arrow keys. Each choice is a ``(value, description)`` tuple. The description is shown next to the va
lmcache/cli/commands/bench/engine_bench/interactive/terminal.py:199
↓ 3 callers
Method
query_deserialize_result
Pop and return the deserialize task result, or None if pending.
lmcache/v1/distributed/serde/async_processor.py:133
↓ 3 callers
Method
query_load_result
Non-blockingly query the result of a load task by its task id. The result is a bitmap indicating the success or failure of each key-o
lmcache/v1/distributed/l2_adapters/base.py:330
↓ 3 callers
Method
query_load_result
Return and remove a completed load bitmap if available.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:528
↓ 3 callers
Method
query_lookup_and_lock_result
Non-blockingly query the result of a lookup and lock task by its task id. The result is a bitmap indicating the success or failure of
lmcache/v1/distributed/l2_adapters/base.py:260
↓ 3 callers
Method
reconcileServiceMonitor
reconcileServiceMonitor creates, updates, or deletes the ServiceMonitor.
operator/internal/controller/reconcile_helpers.go:244
↓ 3 callers
Function
recv_cfg
(max_len)
tests/v1/storage_backend/test_pd_backend_async.py:812
↓ 3 callers
Function
reformat_block_ids
(block_ids: tuple[list[int], ...] | None)
lmcache/integration/vllm/lmcache_mp_connector_0201.py:71
↓ 3 callers
Function
refreshActiveTab
()
lmcache/lmcache_frontend/static/js/app.js:422
↓ 3 callers
Method
register
Register a telemetry reporter with lazy-loading module and class name. Args: name: The name to register the telemetry reporter un
lmcache/integration/request_telemetry/factory.py:23
↓ 3 callers
Method
register
Register the lmcache worker with the controller via DEALER-ROUTER. This method sends a RegisterMsg and waits for RegisterRetMsg
lmcache/v1/cache_controller/worker.py:191
↓ 3 callers
Method
register
Register KV caches with the non-GPU context server. ``engine_group_infos`` is accepted to satisfy the base interface but is currently
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:327
↓ 3 callers
Method
register_all_apis
Discover and register all ``*_api`` modules under the ``http_apis`` directory.
lmcache/v1/multiprocess/http_api_registry.py:29
↓ 3 callers
Method
register_listener
Register a listener to receive L2 adapter events.
lmcache/v1/distributed/l2_adapters/base.py:353
↓ 3 callers
Method
register_listener
Register a listener and seed it with currently indexed keys.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:551
↓ 3 callers
Method
register_recover_callback
Register a callback fired on the unhealthy->healthy transition. The callback runs **before** the health event is set. It must return
lmcache/integration/vllm/vllm_multi_process_adapter.py:443
↓ 3 callers
Function
register_serde_factory
Register a serde factory under a type name. Args: name: Serde type name (used in the JSON config ``"type"`` field). factory: Call
lmcache/v1/distributed/serde/factory.py:29
↓ 3 callers
Method
remove
Evict all fingerprints for the given storage keys. Args: object_keys: Storage keys of chunks to evict. Returns:
lmcache/v1/mp_coordinator/blend_directory.py:313
↓ 3 callers
Method
remove
(fname: Path)
lmcache/v1/storage_backend/connector/hf3fs_connector.py:598
↓ 3 callers
Method
remove_sync
Remove the file associated with the given key. Args: key: The key to remove. Returns: bool: True if
lmcache/v1/storage_backend/connector/fs_connector.py:375
↓ 3 callers
Method
remove_transfer_channel_client
(self, peer_advertise_url: str)
tests/v1/distributed/transfer_channel/test_factory.py:48
↓ 3 callers
Method
repeat
(self, n)
tests/v1/multiprocess/test_blend_v3_load_store_opts.py:286
↓ 3 callers
Method
report_status
Return a status dict for the store controller.
lmcache/v1/distributed/storage_controllers/store_controller.py:323
↓ 3 callers
Method
reset
Reset the singleton instance. Mainly for testing.
lmcache/v1/periodic_thread.py:429
↓ 3 callers
Method
reset_statistics
(self)
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:80
↓ 3 callers
Function
resolve_hfbucket_connector_config
Resolve plugin-scoped configuration for the HFBucket connector. Args: config: LMCache engine config used to resolve ``extra_config`` valu
lmcache/v1/storage_backend/connector/hfbucket_connector.py:764
↓ 3 callers
Method
resolve_obj_keys
Resolve per-object-group object keys from an IPC cache key. Uses the session manager to track token state and the token hasher to com
lmcache/v1/multiprocess/engine_context.py:202
↓ 3 callers
Function
round_robin_pick_client
(clients, idx)
examples/disagg_prefill/disagg_proxy_server.py:427
↓ 3 callers
Method
run
Drain the buffer on a timer until cancelled. Flush failures drop the batch to bound buffer growth when the coordinator is down.
lmcache/v1/mp_coordinator/l2/event_listener.py:91
↓ 3 callers
Method
run
( self, coro_fn: Coroutine[Any, Any, Any], num_chunks: int, )
lmcache/v1/storage_backend/storage_manager.py:182
↓ 3 callers
Function
run_cache_server
Run the LMCache cache server with ZMQ message queue. Args: mp_config: Configuration for the ZMQ multiprocess server. storage_mana
lmcache/v1/multiprocess/server.py:282
↓ 3 callers
Method
save_json
Export the current state to a JSON file.
lmcache/cli/commands/bench/engine_bench/interactive/state.py:276
↓ 3 callers
Function
select_retained
Select the retained subset of ``found`` for the non-windowed selections. ``PREFIX`` (LONGEST) keeps the leading contiguous run and drops everythi
lmcache/v1/distributed/bitmap_ops/fold.py:318
↓ 3 callers
Method
send_message_local
(self, msg, message_type)
lmcache/usage_context.py:139
↓ 3 callers
Method
send_message_server
(self, msg, message_type)
lmcache/usage_context.py:123
↓ 3 callers
Method
send_request
Send request and return stats.
lmcache/cli/commands/query/_request.py:286
↓ 3 callers
Method
serialize
(self, src: MemoryObjGroup, dst)
tests/v1/distributed/serde/test_multi.py:116
↓ 3 callers
Method
set_size
(self, size)
lmcache/v1/storage_backend/connector/eic_connector.py:43
↓ 3 callers
Function
shm_open_pool_as_mmap
Open an existing segment as an independent ``mmap.mmap`` object. Convenience helper for non-GPU SHM transports that consume the segment via `
lmcache/v1/multiprocess/posix_shm.py:250
↓ 3 callers
Method
shutdown_async
Async shutdown — safe to call from a coroutine on the event loop. Args: wait: If True, drain the queue before shutting down.
lmcache/v1/storage_backend/job_executor/pq_executor.py:89
↓ 3 callers
Function
single_to_multi_serializer
Wrap a single-tensor :class:`Serializer` as a length-1 :class:`MultiSerializer`. The wrapper is layout-equivalent: identical bytes are writte
lmcache/v1/distributed/serde/multi.py:251
↓ 3 callers
Method
snapshot_keys
Return committed keys currently indexed by this DAX arena. Returns: A snapshot list of keys whose slots are committed and readabl
lmcache/v1/storage_backend/dax/core.py:369
↓ 3 callers
Method
stop
(self)
examples/blend_kv/online_kv.py:58
↓ 3 callers
Method
stop
Stop heartbeat thread
lmcache/lmcache_frontend/heartbeat.py:207
↓ 3 callers
Method
stop_statistics
(self)
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:76
↓ 3 callers
Method
storage_to_mem
Combined method: collect metadata with lock, then do NIXL transfer.
lmcache/v1/storage_backend/nixl_storage_backend.py:1181
↓ 3 callers
Method
storage_to_mem
( self, keys: list[CacheEngineKey], pin: bool = False )
lmcache/v1/storage_backend/nixl_storage_backend.py:1668
↓ 3 callers
Method
store
Store the tokens/hashes and mask into the cache engine. :param Optional[torch.Tensor] tokens: The tokens of the corresponding KV caches.
lmcache/v1/cache_engine.py:385
↓ 3 callers
Method
stream
(self)
tests/v1/platform/test_cache_context_dispatch.py:79
↓ 3 callers
Method
submit_load_task
Submit a load task to load a batch of objects by the given keys. The load operation is not guaranteed to succeed, and the caller shou
lmcache/v1/distributed/l2_adapters/base.py:304
↓ 3 callers
Method
submit_lookup_and_lock_task
Submit a lookup and lock task to look up and lock a batch of objects by the given keys. Args: keys (list[ObjectK
lmcache/v1/distributed/l2_adapters/base.py:239
↓ 3 callers
Method
submit_put_task
Synchronously put the MemoryObj into the local cpu backend. :param on_complete_callback: Optional callback invoked after the
lmcache/v1/storage_backend/local_cpu_backend.py:147
↓ 3 callers
Method
submit_store_task
( self, keys: list[ObjectKey], objects: list[MemoryObj], )
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:401
↓ 3 callers
Method
submit_store_task
Submit a store task to store a batch of memory objects associated with a batch of keys. Args: keys (list[ObjectK
lmcache/v1/distributed/l2_adapters/base.py:201
↓ 3 callers
Method
take_match
Drop a request's stored match state once consumed. Args: rid: Request id to clear.
lmcache/v1/mp_coordinator/blend_client.py:190
← previous
next →
2,001–2,100 of 13,371, ranked by callers