Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/LMCache/LMCache
/ functions
Functions
13,371 in github.com/LMCache/LMCache
⨍
Functions
13,371
◇
Types & classes
2,044
↳
Endpoints
181
↓ 2 callers
Function
loadOverview
()
lmcache/v1/cache_controller/frontend/static/js/controller_app.js:168
↓ 2 callers
Function
loadTabData
(tabId)
lmcache/v1/cache_controller/frontend/static/js/controller_app.js:142
↓ 2 callers
Function
loadWorkers
()
lmcache/v1/cache_controller/frontend/static/js/controller_app.js:424
↓ 2 callers
Function
load_and_wait
( adapter: DaxL2Adapter, keys: list[ObjectKey], objs: list[MemoryObj], )
tests/v1/distributed/test_dax_l2_adapter.py:190
↓ 2 callers
Function
load_config_with_overrides
Load configuration with support for file, environment variables, and overrides. This is a generic utility function that can be reused across
lmcache/v1/config_base.py:544
↓ 2 callers
Function
load_controller_config_with_overrides
Load controller configuration with support for file, env vars, and overrides. This function uses the generic load_config_with_overrides util
lmcache/v1/cache_controller/config.py:154
↓ 2 callers
Function
load_dataset
(dataset_path)
benchmarks/rag/utils.py:141
↓ 2 callers
Method
load_json
Load state from a JSON config file.
lmcache/cli/commands/bench/engine_bench/interactive/state.py:283
↓ 2 callers
Function
log_jsonl
(path: Path, rec: dict)
examples/online_session/openai_chat_completion_client.py:56
↓ 2 callers
Method
lookup
(self, msg: LookupMsg)
lmcache/v1/cache_controller/controllers/kv_controller.py:388
↓ 2 callers
Method
lookup
( self, token_ids: Union[torch.Tensor, list[int]], lookup_id: str, request_con
lmcache/v1/lookup_client/lmcache_lookup_client.py:86
↓ 2 callers
Method
make_meta
Create metadata for a single multimodal hash.
lmcache/integration/vllm/vllm_ec_adapter.py:39
↓ 2 callers
Function
make_object_key
(chunk_id: int, model_name: str)
tests/v1/distributed/test_multimodel.py:41
↓ 2 callers
Method
mem_to_storage
( self, keys: Sequence[CacheEngineKey], mem_objs: List[MemoryObj] )
lmcache/v1/storage_backend/nixl_storage_backend.py:1773
↓ 2 callers
Method
memcheck
(self)
lmcache/v1/memory_management.py:3105
↓ 2 callers
Method
memcheck
Check the integrity of the underlying storage backend's memory allocators Returns: True if everything is good ot
lmcache/v1/storage_backend/storage_manager.py:1146
↓ 2 callers
Method
merge_cli_args
Merge CLI args on top of existing state (CLI args win).
lmcache/cli/commands/bench/engine_bench/interactive/state.py:289
↓ 2 callers
Method
move
Execute a move cache operation with error handling.
lmcache/v1/cache_controller/executor.py:281
↓ 2 callers
Method
name
(self)
lmcache/v1/health_monitor/checks/remote_backend_check.py:118
↓ 2 callers
Method
needs_retrieve
Check whether the current request needs retrieve, will be used update_stage_after_alloc
lmcache/integration/vllm/lmcache_mp_connector.py:224
↓ 2 callers
Method
needs_retrieve
Check whether the current request needs retrieve, will be used update_stage_after_alloc
lmcache/integration/vllm/lmcache_mp_connector_0201.py:237
↓ 2 callers
Method
needs_retrieve
Check whether the current request needs retrieve, will be used update_stage_after_alloc
lmcache/integration/vllm/lmcache_mp_connector_0180.py:219
↓ 2 callers
Function
normalize_answer
(s)
benchmarks/rag/utils.py:163
↓ 2 callers
Method
num_blocks
Returns the number of blocks in the KV cache.
lmcache/v1/platform/base_cache_context.py:170
↓ 2 callers
Function
num_engine_group_infos
Return the number of LMCache KV groups visible to transfer requests. Args: groups: The LMCache KV groups, in protocol order. Returns
lmcache/v1/multiprocess/group_view.py:74
↓ 2 callers
Method
on_exists
(self)
lmcache/v1/storage_backend/connector/mock_connector.py:124
↓ 2 callers
Method
on_keys_removed
Notify the eviction policy that keys have been deleted. Deleted keys are removed from tracking. Args: keys (list
lmcache/v1/distributed/eviction_policy/lru.py:112
↓ 2 callers
Method
on_l2_keys_stored
(self, keys: list[ObjectKey], sizes: list[int])
lmcache/v1/distributed/eviction.py:185
↓ 2 callers
Method
on_new_token_hashes
Index a stored sequence's non-overlapping chunks into the matcher. Records each new chunk's poly hash + start position so a later mat
lmcache/v1/multiprocess/modules/blend_v3.py:136
↓ 2 callers
Method
on_remove
Drop ``key`` from the LRU. Per-salt bytes are the caller's responsibility (see :meth:`L2UsageManager.record_evicted`).
lmcache/v1/mp_coordinator/l2/eviction_manager.py:70
↓ 2 callers
Method
on_server_message
(self, message, display=True)
examples/frontend/chat_session.py:45
↓ 2 callers
Method
on_system_response
(self, response: str)
benchmarks/multi_round_qa/multi-round-qa.py:101
↓ 2 callers
Method
on_user_message
(self, message, display=True)
examples/frontend/chat_session.py:40
↓ 2 callers
Method
on_user_query
(self, query: str)
benchmarks/multi_round_qa/multi-round-qa.py:94
↓ 2 callers
Function
one_read
()
lmcache/tools/transfer_channel_benchmark/benchmark.py:268
↓ 2 callers
Function
ownedByCoordinator
ownedByCoordinator reports whether owners contains a controller reference to the LMCacheCoordinator test fixture.
operator/internal/controller/lmcachecoordinator_controller_test.go:41
↓ 2 callers
Method
p2p_unlock_objects
Unlock the specified object keys. Args: keys: the list of object keys to unlock.
lmcache/v1/multiprocess/modules/p2p_controller.py:304
↓ 2 callers
Function
pair_hosts_and_ports
Flexible host-port pairing with expansion strategies.
examples/disagg_prefill_mp/disagg_proxy_server.py:45
↓ 2 callers
Function
pair_hosts_and_ports
Flexible host-port pairing with expansion strategies. Multiple pairing strategies: 1. Single host + single port + count: Gen
examples/disagg_prefill/disagg_proxy_server.py:92
↓ 2 callers
Function
parse_args_to_http_frontend_config
Convert parsed command line arguments to an HTTPFrontendConfig. Args: args: Parsed arguments from the argument parser. Returns:
lmcache/v1/multiprocess/config.py:510
↓ 2 callers
Function
parse_command_line_extra_params
Parse extra command-line parameters in key=value format. Args: extra_args: List of strings in format "key=value" Returns:
lmcache/v1/config_base.py:615
↓ 2 callers
Function
parse_entries
Parse a ``.po`` file into one :class:`PoEntry` per message.
tools/translate_docs_zh.py:134
↓ 2 callers
Method
pending_count
Return the number of pending keys waiting to be processed.
lmcache/v1/distributed/storage_controllers/store_controller.py:114
↓ 2 callers
Function
pick_up_clients
(request: Request)
examples/disagg_prefill/disagg_proxy_server.py:466
↓ 2 callers
Method
pin
(self)
lmcache/v1/storage_backend/nixl_storage_backend.py:423
↓ 2 callers
Method
pin
Pin a key to prevent eviction on MaruServer. Increments the server-side pin_count. Args: key: The cache key. Re
lmcache/v1/storage_backend/maru_backend.py:657
↓ 2 callers
Function
plot_bars
(df, title, filename)
benchmarks/long_doc_qa/long_doc_qa.py:401
↓ 2 callers
Method
post_non_blocking
Post a Nixl transfer handle and await until the transfer is done.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:370
↓ 2 callers
Method
prepare
Prepare I/O operation - stores parameters for later use.
tests/conftest.py:396
↓ 2 callers
Method
prepare_retrieve
(self, *_args: Any, **_kwargs: Any)
tests/v1/multiprocess/test_engine_driven_transfer.py:522
↓ 2 callers
Function
print_output
Generate text using the LLM and print the output with timing information. Args: llm: The vLLM LLM instance to use for generation.
examples/kv_cache_reuse/local_backends/rust_backend_offload.py:93
↓ 2 callers
Function
print_output
( llm: LLM, prompt: list[str], sampling_params: SamplingParams, req_str: str, )
examples/kv_cache_reuse/local_backends/offload.py:77
↓ 2 callers
Function
print_performance_results
Print performance results in a formatted table. Args: stats_data: list of (op, stats, results, pass_count). obj_bytes: size of on
lmcache/v1/check/utils.py:283
↓ 2 callers
Method
profile_process_tokens
(self)
lmcache/observability.py:209
↓ 2 callers
Function
prompt_number
Prompt for a numeric value with validation. Args: label: The config item name. description: One-line explanation. default
lmcache/cli/commands/bench/engine_bench/interactive/terminal.py:70
↓ 2 callers
Function
prompt_text
Prompt for free-form text input. Args: label: The config item name shown as a heading. description: One-line explanation shown be
lmcache/cli/commands/bench/engine_bench/interactive/terminal.py:38
↓ 2 callers
Method
put
Write memory_obj data to a file via the worker.
tests/v1/shm_allocator/shmfile_connector.py:214
↓ 2 callers
Method
put
Store one encoder output tensor under ``mm_hash``. Args: mm_hash: multimodal-input identifier produced by vLLM. tenso
lmcache/v1/ec_engine.py:130
↓ 2 callers
Function
quantize_v_fp8
Quantize V to FP8 using the given scales. Saturates at +/- finfo(fp8).max (via clamp before cast). Returns a tensor with dtype=fp8_dtype, sam
lmcache/v1/kv_codec/asym_k16_v8.py:151
↓ 2 callers
Function
query_and_measure_ttft
( client: OpenAI, model: str, messages: list[dict], max_tokens: int, )
benchmarks/multi_doc_qa/shuffle_doc_qa.py:169
↓ 2 callers
Function
query_and_measure_ttft
(prompt)
benchmarks/ttft-estimator/ttft-estimator.py:40
↓ 2 callers
Method
query_lookup_and_lock_result
Return and remove a completed lookup bitmap if available.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:481
↓ 2 callers
Method
query_prefetch_lookup_hits
Query the number of prefix-hit chunks for a prefetch task before the L2 prefetching is done. Args: handle (Prefe
lmcache/v1/distributed/storage_manager.py:572
↓ 2 callers
Method
query_remote_cache
Query the decoder for which keys are cached in its GPU memory. Returns a CacheQueryResponse with cached_keys and cached_indexes.
lmcache/v1/storage_backend/pd_backend.py:728
↓ 2 callers
Method
queue_depth
Number of events currently queued waiting for dispatch. Reads ``len`` on the underlying deque, which is atomic in CPython, so callers
lmcache/v1/mp_observability/event_bus.py:265
↓ 2 callers
Function
rand_ascii
(n: int)
examples/online_session/openai_chat_completion_client.py:41
↓ 2 callers
Method
raw_device
Return the lazily opened Rust raw-block device. Returns: The underlying Rust ``RawBlockDevice`` object. Raises:
lmcache/v1/storage_backend/raw_block/core.py:428
↓ 2 callers
Method
read_bytes
(self, offset: int, length: int)
tests/v1/shm_allocator/verify_shmfile_tcp.py:97
↓ 2 callers
Method
read_file
(self, key, buffer, path)
lmcache/v1/storage_backend/local_disk_backend.py:639
↓ 2 callers
Method
reap_stale_instances
( self, reap_timeout_s: float, registration_grace_s: float )
tests/v1/multiprocess/test_worker_liveness.py:192
↓ 2 callers
Method
receive_all
(self, client_socket, n)
lmcache/v1/server/__main__.py:34
↓ 2 callers
Method
reconfigure
(self, operation: str, payload: dict[str, object])
tests/v1/distributed/test_dax_l2_adapter.py:428
↓ 2 callers
Method
reconfigure_status
Return generic runtime reconfiguration status for this adapter. Returns: Standard reconfiguration status with DAX hotplug details
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:694
↓ 2 callers
Method
record
Record the preprocessed chunk data. This method performs the actual recording operation using the preprocessed data. It should update
lmcache/v1/lookup_client/record_strategies/base.py:87
↓ 2 callers
Function
reformat_block_ids
(block_ids: tuple[list[int], ...] | None)
lmcache/integration/vllm/lmcache_mp_connector_0180.py:64
↓ 2 callers
Method
reg_worker_async
Register worker via registration controller.
tests/v1/cache_controller/conftest.py:167
↓ 2 callers
Method
register_all_apis
Discover and register API modules from specified categories Args: categories: List of categories to register.
lmcache/v1/internal_api_server/api_registry.py:29
↓ 2 callers
Function
register_and_write
Register a buffer on ``stream`` and write ``pattern`` to a chunk.
tests/v1/gpu_connector/test_gds_context.py:182
↓ 2 callers
Method
register_kv_caches
Register the KV cache tensor and create the LMCache engine. Called once by the runtime after KV cache allocation. Caches the engine a
lmcache/integration/tensorrt_llm/tensorrt_adapter.py:287
↓ 2 callers
Function
register_kv_wrapper
Register a KV-cache IPC wrapper factory for ``device_type``. Args: device_type: The device type string (e.g., ``"cuda"``). factor
lmcache/v1/platform/_registry.py:49
↓ 2 callers
Function
register_prefetch_policy
Register a prefetch policy class under a name. Each policy module should call this at import time. Args: name: Policy name (e.g
lmcache/v1/distributed/storage_controllers/prefetch_policy.py:87
↓ 2 callers
Function
register_store_policy
Register a store policy class under a name. Each policy module should call this at import time. Args: name: Policy name (e.g. "
lmcache/v1/distributed/storage_controllers/store_policy.py:102
↓ 2 callers
Function
relative_time
Relative time to the start of the benchmark.
benchmarks/long_doc_qa/long_doc_qa.py:391
↓ 2 callers
Method
release_handle
(self, handle)
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:408
↓ 2 callers
Method
remap
Remap this arena to a new byte size while preserving metadata. Args: new_max_dax_size_bytes: New number of bytes to map.
lmcache/v1/storage_backend/dax/core.py:707
↓ 2 callers
Method
remove_worker
Remove and return a worker from this instance.
lmcache/v1/cache_controller/utils.py:267
↓ 2 callers
Method
report_block_allocations
Report vLLM GPU block allocation deltas to LMCache server. Fire-and-forget: does not wait for a response. If the server is unhealthy
lmcache/integration/vllm/vllm_multi_process_adapter.py:982
↓ 2 callers
Method
report_status
Return a status dict for the Nixl L2 adapter.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:653
↓ 2 callers
Method
report_status
Return adapter health, task counters, and core status.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:592
↓ 2 callers
Method
reserve_reads_for_keys
Reserve readable slots for explicit keys. Args: keys: Keys to reserve for a lock-safe read or migration copy. Returns:
lmcache/v1/storage_backend/dax/core.py:387
↓ 2 callers
Method
reset_counters
Reset all Prometheus Counter metrics by calling clear(). After clearing, re-initialize with labels so metrics remain visible.
lmcache/observability.py:1879
↓ 2 callers
Method
retrieve
Retrieve the CPU KV cache and put into GPU blocks. Args: key: The IPC key for the KV cache blocks. Must have work
lmcache/v1/multiprocess/modules/lmcache_driven_transfer.py:919
↓ 2 callers
Method
retrieve_speed
(self)
lmcache/observability.py:156
↓ 2 callers
Function
round_robin_pick_client
(clients, idx)
examples/disagg_prefill_mp/disagg_proxy_server.py:142
↓ 2 callers
Function
round_robin_pick_clients
()
examples/disagg_prefill/disagg_proxy_server.py:434
↓ 2 callers
Method
run
Replay every record in the trace. Dispatch is always paced to the recorded ``t_mono`` offsets via ``time.sleep``: the replay never ru
lmcache/cli/commands/trace/_driver.py:245
↓ 2 callers
Function
run_benchmark
Dispatch to the server or client role. Args: args: Parsed CLI arguments produced by ``add_benchmark_arguments``. Returns: ``
lmcache/tools/transfer_channel_benchmark/benchmark.py:372
↓ 2 callers
Function
run_common_test_framework
Common test framework for both storage manager and remote backend tests. Args: test_context: A dictionary containing test-specific f
lmcache/v1/check/utils.py:389
↓ 2 callers
Function
run_func
(tokens, slot_mappings)
tests/benchmarks/test_xpu_layerwise_connector_benchmark.py:373
↓ 2 callers
Function
run_func
(tokens, slot_mappings)
tests/benchmarks/test_xpu_connector_benchmark.py:276
← previous
next →
2,801–2,900 of 13,371, ranked by callers