MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 1 callersFunctionbuild_llm_with_lmcache
(lmcache_connector: str, model: str)
examples/blend_kv_v1/blend.py:62
↓ 1 callersFunctionbuild_messages
Same structure as two-request-demo.py: system, then for each slot (user label, user body), then final summarize user message. `perm[k]` i
benchmarks/multi_doc_qa/shuffle_doc_qa.py:106
↓ 1 callersFunctionbuild_parallel_strategy_from_vllm_config
Build a ParallelStrategy from a vLLM config. Centralises the (vllm_config -> KV parallel geometry) mapping. Args: vllm_config: The v
lmcache/integration/vllm/lmcache_mp_connector.py:141
↓ 1 callersFunctionbuild_rag_prompt
( system_prompt, example, query_prompt, separator: str, prompt_build_method: PromptBuildMe
benchmarks/rag/utils.py:217
↓ 1 callersMethodbuild_request
Build request payload and metadata for the provided prompt.
lmcache/cli/commands/query/_request.py:273
↓ 1 callersFunctionbuild_safe_vocab
Return token IDs that round-trip stably through the tokenizer. A *safe* token is one where:: tokenizer.encode(tokenizer.decode([id]), ad
lmcache/tools/cache_simulator/gen_bench_dataset.py:57
↓ 1 callersFunctionbuild_user_message
Build the chat-completions user message for one translation call. Wraps the focal entry in labeled blocks and prepends the neighboring entrie
tools/translate_docs_zh.py:204
↓ 1 callersFunctionbuild_warmup_messages
()
benchmarks/multi_doc_qa/shuffle_doc_qa.py:197
↓ 1 callersMethodcache_size_per_token
(self)
tests/v1/platform/test_cache_context_dispatch.py:115
↓ 1 callersMethodcache_size_per_token
Returns cache size per logical token in bytes (all groups).
lmcache/v1/platform/base_cache_context.py:141
↓ 1 callersMethodcalculate_chunk_budget
Calculate the maximum number of chunks that can be allocated concurrently without causing memory deadlocks in the async loading syste
lmcache/v1/storage_backend/local_cpu_backend.py:885
↓ 1 callersFunctioncalculate_local_rank_and_world_size
Calculate the local worker id and local world size. Current assumption (TODO: add custom logic in the future): - Tensor Parallel is intr
lmcache/integration/vllm/utils.py:342
↓ 1 callersMethodcancel_request
Cancel an in-flight or pending request. Delegates to all storage backends. Backends that track per-request state will cancel
lmcache/v1/storage_backend/storage_manager.py:1372
↓ 1 callersFunctioncapacity_range_bytes
Return *num_points* byte capacities log-spaced between *min_gib* and *max_gib* GiB.
lmcache/tools/cache_simulator/plot_hit_rate.py:35
↓ 1 callersMethodchat
(self, question)
examples/frontend/chat_session.py:50
↓ 1 callersMethodchat
(self)
examples/blend_kv/online_kv.py:98
↓ 1 callersMethodcheck
Perform the health check. Returns: bool: True if healthy, False otherwise
lmcache/v1/health_monitor/base.py:81
↓ 1 callersFunctioncheck_native_abi
Return whether ``module`` exposes the Stage2 LMCache KV-transfer ABI.
lmcache/v1/platform/musa/native_kv_transfer.py:54
↓ 1 callersFunctioncheck_other
()
tests/v1/storage_backend/test_pd_backend_async.py:627
↓ 1 callersFunctioncheck_spdx_header
(file_path)
tools/check_spdx_header.py:13
↓ 1 callersMethodcheckpoint_now
Synchronously write a metadata checkpoint.
lmcache/v1/storage_backend/raw_block/core.py:890
↓ 1 callersMethodclean
(self)
lmcache/v1/compute/blend/metadata.py:31
↓ 1 callersFunctionclean_fuzzy_flag
Remove the ``"fuzzy"`` marker from a message, modifying ``lines`` in place.
tools/translate_docs_zh.py:183
↓ 1 callersFunctionclean_translation_response
Strip incidental formatting the model occasionally adds to the output. Trims whitespace, removes a leading ``msgstr:`` / ``Translation:`` label
tools/translate_docs_zh.py:255
↓ 1 callersMethodcleanup_temp_files
Remove leftover ``*.tmp.*`` files in the storage directory. These can be left behind if a store crashed between opening the temp file
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:297
↓ 1 callersFunctionclearAllTabs
()
lmcache/lmcache_frontend/static/js/app.js:994
↓ 1 callersMethodclear_current_retrieve_stats
(self)
lmcache/observability.py:329
↓ 1 callersMethodclear_kv_store
Clear all KV data for this worker.
lmcache/v1/cache_controller/utils.py:176
↓ 1 callersMethodclear_lookup_status
(self, lookup_id: str)
tests/v1/lookup_client/test_chunk_statistics_lookup_client.py:26
↓ 1 callersMethodclear_lookup_status
Clear temporary lookup status for a given lookup ID. Args: lookup_id: The lookup ID whose status needs to be cleared.
lmcache/v1/lookup_client/abstract_client.py:70
↓ 1 callersMethodclear_lookup_status
(self, lookup_id: str)
lmcache/v1/lookup_client/lmcache_async_lookup_client.py:251
↓ 1 callersFunctionclient_main
Run the benchmark client: read a subset of source objects and report. Args: cfg: The benchmark configuration (``role == "client"``).
lmcache/tools/transfer_channel_benchmark/benchmark.py:198
↓ 1 callersFunctionclone_tensor_memory_obj
(obj: MemoryObj)
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:40
↓ 1 callersMethodclose
(self)
tests/v1/distributed/test_native_connector_l2_adapter.py:141
↓ 1 callersMethodclose
Close the connector.
tests/v1/storage_backend/test_remote_storage_plugin.py:68
↓ 1 callersMethodclose
(self)
tests/v1/shm_allocator/verify_shmfile_tcp.py:131
↓ 1 callersMethodclose
(self)
lmcache/integration/request_telemetry/base.py:47
↓ 1 callersMethodclose
Free every cached chunk and the underlying pinned pool.
lmcache/v1/hidden_state_store.py:300
↓ 1 callersMethodclose
Close the cache engine and free all the resources
lmcache/v1/cache_engine.py:1609
↓ 1 callersMethodclose
Clean up all resources owned by the controller manager.
lmcache/v1/cache_controller/controller_manager.py:528
↓ 1 callersMethodclose
Close all sockets and cleanup resources
lmcache/v1/transfer_channel/py_socket_channel.py:251
↓ 1 callersMethodclose
Close the L2 adapter and release all the resources. After calling this function, the L2 adapter should not be used anymore.
lmcache/v1/distributed/l2_adapters/base.py:531
↓ 1 callersMethodclose
Clear the per-task key map, then close the inner adapter.
lmcache/v1/distributed/l2_adapters/fault_inject_l2_adapter.py:415
↓ 1 callersMethodclose
Free the resources created by the server.
lmcache/v1/distributed/transfer_channel/abstract.py:47
↓ 1 callersMethodclose
(self)
lmcache/v1/distributed/transfer_channel/impl/nixl_impl.py:499
↓ 1 callersMethodclose
(self)
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:165
↓ 1 callersMethodclose
(self)
lmcache/v1/lookup_client/lmcache_lookup_client.py:282
↓ 1 callersMethodclose
Release GPU resources owned by this module.
lmcache/v1/multiprocess/modules/lmcache_driven_transfer.py:609
↓ 1 callersMethodclose
Release resources held by this context.
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:217
↓ 1 callersMethodclose
Release underlying OS resources. Idempotent.
lmcache/v1/platform/event_notifier.py:81
↓ 1 callersMethodclose
Close the storage backend.
lmcache/v1/storage_backend/pd_backend.py:1069
↓ 1 callersMethodclose
Close the client and cleanup resources.
lmcache/v1/storage_backend/resp_client.py:222
↓ 1 callersMethodclose
Shut down the PQ executor and the thread pool.
lmcache/v1/storage_backend/connector/valkey_connector.py:652
↓ 1 callersMethodclose
(self)
lmcache/v1/storage_backend/connector/mock_connector.py:97
↓ 1 callersMethodclose
Flush, fsync, and close the trace file. Idempotent. Flips the trace gate off so any straggler publishes after this point are no-ops.
lmcache/v1/mp_observability/trace/recorder.py:134
↓ 1 callersMethodclose_backend
Close and remove a specific storage backend by name. The backend will be closed and removed from the internal dict so that n
lmcache/v1/storage_backend/storage_manager.py:1221
↓ 1 callersFunctioncmd_generate
Generate test prompt and disjoint flood prompts.
examples/kv_cache_reuse/remote_backends/valkey/benchmark_l2.py:80
↓ 1 callersFunctioncmd_run
Run the full L2 benchmark: cold → flood → L2 retrieval.
examples/kv_cache_reuse/remote_backends/valkey/benchmark_l2.py:189
↓ 1 callersFunctioncollect_bytes
Collect a byte tensor from the output_buffer + output_lengths
lmcache/storage_backend/serde/cachegen_encoder.py:237
↓ 1 callersMethodcollect_extensions
Build all extensions and return requirements file name. Args: profile: Resolved build profile, or ``None``. Returns:
setup_extensions/policy.py:195
↓ 1 callersMethodcollect_storage_backends
Discover and build optional L2 storage backend extensions. Each storage backend can be enabled via its ``BUILD_WITH_*`` env var or au
setup_extensions/policy.py:232
↓ 1 callersMethodcommit_retrieve
Commit retrieve. Pickle: no-op. Shm: release read locks.
lmcache/v1/multiprocess/transfer_context/base.py:188
↓ 1 callersMethodcommit_retrieve
(self, key: IPCCacheServerKey, instance_id: int)
lmcache/v1/multiprocess/transfer_context/shm.py:200
↓ 1 callersMethodcommit_store
Finalize a store request. Args: key: Cache key identifying the requested token range. instance_id: Worker instance id
lmcache/v1/multiprocess/modules/server_transfer.py:109
↓ 1 callersMethodcommit_store
Commit store. Pickle: serialize and send. Shm: notify server.
lmcache/v1/multiprocess/transfer_context/base.py:174
↓ 1 callersMethodcommit_store
( self, key: IPCCacheServerKey, instance_id: int, _chunks: list[torch.Tensor] )
lmcache/v1/multiprocess/transfer_context/shm.py:170
↓ 1 callersFunctioncompute_f1
(a_pred, a_gold, tokenizer)
benchmarks/rag/utils.py:194
↓ 1 callersFunctioncompute_kv_bytes_per_token
Return the number of KV cache bytes per token for *model_name*. Reads num_hidden_layers, num_key_value_heads, head_dim, and torch_dtype from
examples/disagg_prefill/disagg_proxy_server.py:249
↓ 1 callersFunctioncompute_kvcache_checksums
Compute MD5 checksums for kvcaches at specified slot positions. This method is used by the kvcache check API to verify that stored and retrie
lmcache/v1/internal_api_server/vllm/cache_api.py:218
↓ 1 callersMethodcompute_layer
( self, input_ids: torch.Tensor, )
lmcache/v1/compute/models/base.py:68
↓ 1 callersFunctioncompute_rl
(pred, gold)
benchmarks/rag/utils.py:211
↓ 1 callersFunctionconnectToController
()
lmcache/v1/cache_controller/frontend/static/js/controller_app.js:77
↓ 1 callersMethodcontains
( self, key: CacheEngineKey, )
lmcache/v1/server/storage_backend/local_backend.py:32
↓ 1 callersMethodcontains
(self, key: int)
lmcache/v1/storage_backend/nixl_storage_backend.py:376
↓ 1 callersMethodcontains
(self, key: CacheEngineKey, pin: bool = False)
lmcache/v1/storage_backend/pd_backend.py:385
↓ 1 callersMethodcontains
Check whether key is in the storage backend. :param CacheEngineKey key: The key of the MemoryObj. :param bool pin: Whether
lmcache/v1/storage_backend/abstract_backend.py:48
↓ 1 callersMethodcontains
(self, key: CacheEngineKey, pin: bool = False)
lmcache/v1/storage_backend/p2p_backend.py:748
↓ 1 callersMethodcontains
Check whether the key exists in the storage backend. :param CacheEngineKey key: The key to check. :param Optional[List[str]
lmcache/v1/storage_backend/storage_manager.py:935
↓ 1 callersMethodcontains
Check whether the given key exists in the local data store. :param key: The cache engine key to look up. :param pin: If True and the
lmcache/v1/storage_backend/pd_backend_async.py:632
↓ 1 callersFunctioncontrollerPodNameOnce
controllerPodNameOnce returns the first running controller-manager pod name. We resolve fresh each call rather than caching because the pod can be evi
operator/test/e2e/smoke_helpers_test.go:177
↓ 1 callersFunctioncoordinatorPrometheusEnabled
coordinatorPrometheusEnabled reports whether prometheus metrics are enabled for the coordinator. Unlike the engines, the coordinator defaults to disab
operator/internal/resources/coordinator_service.go:55
↓ 1 callersFunctioncountFlag
countFlag returns how many times flag appears (two-token or single-token).
operator/internal/webhook/pod_injector_test.go:565
↓ 1 callersMethodcreate
Create a request telemetry reporter based on the specified type. Args: telemetry_type: The type of telemetry reporter to create.
lmcache/integration/request_telemetry/factory.py:44
↓ 1 callersFunctioncreate_app
Build the coordinator FastAPI app. Args: config: The coordinator configuration. Returns: A configured FastAPI application. `
lmcache/v1/mp_coordinator/app.py:64
↓ 1 callersFunctioncreate_app
Create a FastAPI application with endpoints for LMCache operations.
lmcache/v1/api_server/__main__.py:85
↓ 1 callersFunctioncreate_argument_parser
()
benchmarks/long_doc_qa/long_doc_qa.py:604
↓ 1 callersFunctioncreate_argument_parser
()
benchmarks/multi_doc_qa/multi_doc_qa.py:372
↓ 1 callersMethodcreate_batched_storage_handler
(self, descs: list[NixlDesc], page_size: int)
lmcache/v1/storage_backend/nixl_storage_backend.py:649
↓ 1 callersFunctioncreate_config
Create a configuration for the LMCacheEngine with Nixl backend.
tests/disagg/test_nixl_cache_engine.py:99
↓ 1 callersMethodcreate_connector
Create a connector using the given context.
lmcache/v1/storage_backend/connector/__init__.py:167
↓ 1 callersMethodcreate_connector
Create a configured ``HFBucketConnector`` for the given context.
lmcache/v1/storage_backend/connector/hfbucket_adapter.py:32
↓ 1 callersFunctioncreate_lmcache_ec_config
Create EC config from LMCache config plus EC-specific overrides.
lmcache/integration/vllm/utils.py:138
↓ 1 callersFunctioncreate_lmcache_metadata
Create LMCacheMetadata from vLLM configuration. This function extracts common metadata creation logic that was duplicated across multipl
lmcache/integration/vllm/utils.py:198
↓ 1 callersMethodcreate_lookup_client
Create a lookup client based on the configuration. Args: config: The LMCache engine configuration metadata:
lmcache/v1/lookup_client/factory.py:40
↓ 1 callersMethodcreate_lookup_server
Create a lookup server based on the configuration. Args: lmcache_engine: The LMCache engine instance metadat
lmcache/v1/lookup_client/factory.py:102
↓ 1 callersFunctioncreate_memory_allocator
Create a memory allocator based on the provided configuration. Args: config (L1MemoryManagerConfig): Configuration for the memory ma
lmcache/v1/distributed/memory_manager/l1_memory_manager.py:45
↓ 1 callersFunctioncreate_memory_objects_batch
Create a batch of memory objects for reuse
lmcache/v1/check/utils.py:189
↓ 1 callersMethodcreate_message
( self, benchmark: "ZMQControllerBenchmark", test_data: "TestData" )
lmcache/tools/controller_benchmark/handlers/heartbeat.py:31
↓ 1 callersMethodcreate_message
( self, benchmark: "ZMQControllerBenchmark", test_data: "TestData" )
lmcache/tools/controller_benchmark/handlers/deregister.py:27
↓ 1 callersMethodcreate_message
( self, benchmark: "ZMQControllerBenchmark", test_data: "TestData" )
lmcache/tools/controller_benchmark/handlers/admit.py:31
← previousnext →3,901–4,000 of 13,371, ranked by callers