MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 1 callersFunctionfill_kv_cache_with_pattern
Fill the KV cache at the specified slot mappings with a recognizable pattern
tests/disagg/test_nixl_cache_engine.py:53
↓ 1 callersFunctionfindDocsNavbar
* Locate the top nav bar that holds the GitHub / profile / theme-toggle * icons. Prefers the structural `header nav` selector; falls back to * the G
docs/source/_static/custom.js:148
↓ 1 callersMethodfind_kv_with_worker_info
Find a KV chunk and return worker info in one lookup. Optimized for batched_p2p_lookup to avoid multiple lookups. Returns: (
lmcache/v1/cache_controller/utils.py:608
↓ 1 callersFunctionfirstFoundEnvtestBinaryDir
firstFoundEnvtestBinaryDir locates the envtest binary directory under bin/k8s so the suite runs from an IDE without KUBEBUILDER_ASSETS set, mirroring
operator/internal/webhook/webhook_suite_test.go:161
↓ 1 callersMethodfirst_encoded_key
Return one indexed encoded key for diagnostics. Returns: The first indexed key according to dictionary iteration order, or
lmcache/v1/storage_backend/raw_block/core.py:556
↓ 1 callersFunctionflow_control_check
Check flow control and wait if necessary
lmcache/v1/check/utils.py:204
↓ 1 callersFunctionflush_kv_cache
(client: OpenAI, model: str)
examples/online_session/openai_chat_completion_client.py:133
↓ 1 callersMethodformat
(self, record)
benchmarks/multi_round_qa/utils.py:34
↓ 1 callersMethodforward_contiguous
Perform forward pass of the attention mechanism.
lmcache/v1/compute/attention/abstract.py:16
↓ 1 callersFunctionfree_later
()
tests/v1/storage_backend/test_pd_backend_async.py:551
↓ 1 callersFunctionfree_pinned_ptr
Non-CUDA equivalent of freeing a previously allocated pinned pointer.
lmcache/python_ops_fallback.py:430
↓ 1 callersMethodfrom_bytes
Deserialize a pytorch tensor from bytes. Input: bytes: a stream of bytes Output: torch.Tensor: the
lmcache/storage_backend/serde/serde.py:51
↓ 1 callersMethodfrom_cache_engine_config
( config: LMCacheEngineConfig, metadata: LMCacheMetadata )
lmcache/v1/storage_backend/nixl_storage_backend.py:132
↓ 1 callersMethodfrom_cache_engine_config
Convert the LMCacheEngineConfig to PDConfig
lmcache/v1/storage_backend/pd_backend_async.py:230
↓ 1 callersMethodfrom_dict
Build a config instance from a dict (e.g. from parsed JSON). The dict will contain the "type" key used for dispatch; the concrete
lmcache/v1/distributed/l2_adapters/config.py:337
↓ 1 callersMethodfrom_dict
(cls, d: dict)
lmcache/v1/distributed/l2_adapters/plugin_l2_adapter.py:68
↓ 1 callersMethodfrom_dict
Parse a slot descriptor from the MQ context schema. Args: d: Mapping containing ``offset``, ``length``, ``shape``, and
lmcache/v1/multiprocess/transfer_context/shm.py:53
↓ 1 callersMethodfrom_fd
Wrap an already-opened fd and registered cuFile handle. For callers that open + register the file themselves (e.g. a slab that must b
lmcache/v1/gpu_connector/_cufile_async.py:281
↓ 1 callersMethodfrom_gpu
Expect a kwarg 'kvcaches' which is a nested tuple of K and V tensors. The kvcaches should correspond to the "WHOLE token sequence". W
lmcache/v1/gpu_connector/hpu_connector.py:142
↓ 1 callersMethodfrom_new_request
Create the request tracker from a new request. Args: lmcache_config (LMCacheEngineConfig): the LMCache engine config.
lmcache/integration/vllm/vllm_v1_adapter.py:149
↓ 1 callersMethodfrom_pool
Mock implementation of Redis.from_pool
tests/conftest.py:206
↓ 1 callersMethodfrom_tensor_memory_obj
(tensor_memory_obj: MemoryObj)
lmcache/v1/storage_backend/connector/mock_connector.py:35
↓ 1 callersMethodfrom_url
Mock implementation of Redis.from_url
tests/conftest.py:201
↓ 1 callersMethodfused_encode
(self, old_positions, new_positions, k)
lmcache/v1/compute/positional_encoding.py:65
↓ 1 callersFunctiongen_bench_dataset
Convert lookup-hash events into a vllm bench serve custom dataset. Each event becomes one JSONL line with fields ``"prompt"`` (text) and ``"o
lmcache/tools/cache_simulator/gen_bench_dataset.py:141
↓ 1 callersFunctiongenerate_prompt_ids
( doc_prompts: list[str], sys_prompts: list[str], query_prompts: list[str], num_requests: int,
benchmarks/multi_doc_qa/multi_doc_qa.py:219
↓ 1 callersFunctiongenerate_random_slot_mapping
(num_blocks, block_size, num_tokens, device)
tests/v1/test_cache_engine.py:1406
↓ 1 callersFunctiongenerate_request_id
Generate a unique request ID.
examples/disagg_prefill_mp/disagg_proxy_server.py:210
↓ 1 callersFunctiongenerate_stream
()
examples/disagg_prefill_mp/disagg_proxy_server.py:311
↓ 1 callersFunctiongenerate_test_data
( num_objs: int, shape: torch.Size, dtype: torch.dtype = torch.bfloat16 )
tests/disagg/test_nixl_channel.py:23
↓ 1 callersFunctiongenerate_test_tokens
Generate test tokens for testing. The sequence is [0, 1, 2, ..., num_chunks * chunk_size - 1]
tests/disagg/test_nixl_cache_engine.py:23
↓ 1 callersFunctiongenerate_warmup_prompt_ids
( doc_prompts, sys_prompts, query_prompts, blend_special_str, tokenizer, offset=1 )
benchmarks/multi_doc_qa/multi_doc_qa.py:198
↓ 1 callersMethodget
( self, key: CacheEngineKey, )
lmcache/v1/server/storage_backend/local_backend.py:62
↓ 1 callersMethodget
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/connector/instrumented_connector.py:46
↓ 1 callersMethodget
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/connector/infinistore_connector.py:90
↓ 1 callersFunctiongetChunkSize
(spec *lmcachev1alpha1.LMCacheEngineSpec)
operator/internal/resources/compute.go:203
↓ 1 callersFunctiongetHashAlgorithm
(spec *lmcachev1alpha1.LMCacheEngineSpec)
operator/internal/resources/compute.go:217
↓ 1 callersFunctiongetMaxWorkers
(spec *lmcachev1alpha1.LMCacheEngineSpec)
operator/internal/resources/compute.go:210
↓ 1 callersMethodget_adapter_state_observations
``(count, {"state": ...})`` tuples for the ``lmcache_mp.l2_adapters`` gauge. ``len()`` reads are GIL-atomic, safe from the OTel thread.
lmcache/v1/distributed/storage_controllers/store_controller.py:389
↓ 1 callersMethodget_adapter_state_observations
``(count, {"state": ...})`` tuples for the ``lmcache_mp.l2_adapters`` gauge. ``len()`` reads are GIL-atomic, safe from the OTel thread.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:467
↓ 1 callersMethodget_all
Get all registered threads.
lmcache/v1/periodic_thread.py:479
↓ 1 callersFunctionget_all_registered_names
Return all known adapter type names. Forces import of all pending modules so that the returned list is complete.
lmcache/v1/distributed/l2_adapters/factory.py:155
↓ 1 callersMethodget_all_worker_nodes_cached
Get all (instance_id, WorkerNode) tuples across all instances, with caching. Args: timeout_seconds: Cache timeout in seconds.
lmcache/v1/cache_controller/utils.py:512
↓ 1 callersMethodget_allocator_backend
(self)
lmcache/v1/storage_backend/pd_backend.py:346
↓ 1 callersFunctionget_arg_parser
Get a standalone argument parser for storage manager configuration. This creates a new parser with only storage manager arguments. For i
lmcache/v1/distributed/config.py:520
↓ 1 callersMethodget_async_client
(self)
lmcache/connections.py:36
↓ 1 callersMethodget_available_memory_gb
Get system available memory in GB using psutil. This method is cross-platform and doesn't require subprocess calls. Returns:
lmcache/v1/system_detection.py:32
↓ 1 callersMethodget_block_mapping
Get block mapping for the given chunk infos, works by prefix match. :param List[Tuple[CacheEngineKey, int, int]] chunk_infos:
lmcache/v1/storage_backend/storage_manager.py:1007
↓ 1 callersMethodget_blocking
A blocking function to get the kv cache from the storage backend. :param CacheEngineKey key: The key of the MemoryObj. :ret
lmcache/v1/storage_backend/abstract_backend.py:119
↓ 1 callersMethodget_cache_age
Get the age of the cache in seconds. Returns: Age in seconds. Returns float('inf') if cache is uninitialized.
lmcache/v1/utils/cache_utils.py:100
↓ 1 callersMethodget_chunk_size
Return the chunk size used for KV cache operations. Returns: The chunk size.
lmcache/v1/multiprocess/modules/management.py:163
↓ 1 callersFunctionget_concrete_engine_kv_shape
Return the shape with actual numeric values substituted. For example, instead of ``NL x [2, NB, BS, NH, HS]`` this returns ``80 x [2, 2048, 1
lmcache/v1/gpu_connector/utils.py:136
↓ 1 callersFunctionget_configs_for_app
Determine which configs to return based on app_id and current state. This is where you would implement your custom logic, such as: - Loo
examples/remote_config_server/config_server.py:59
↓ 1 callersFunctionget_cpu_offload_GiB_per_gpu
( per_gpu_kv_cache_GiB: float, GiB_1K_tokens_per_gpu: float, tp: int )
benchmarks/long_doc_qa/long_doc_qa_recommender.py:124
↓ 1 callersMethodget_data
(self, key_str: str, meta: RemoteMetadata)
lmcache/v1/storage_backend/connector/eic_connector.py:350
↓ 1 callersMethodget_deserialize_event_fd
(self)
tests/v1/distributed/serde/test_factory.py:84
↓ 1 callersMethodget_deserialize_event_fd
Event fd signaled when deserialize tasks complete. Must be distinct from the serialize event fd.
lmcache/v1/distributed/serde/base.py:136
↓ 1 callersFunctionget_detector
Return a detector instance for *engine_type*, or ``None`` if none exists.
lmcache/v1/gpu_connector/kv_format/detectors/registry.py:40
↓ 1 callersMethodget_engine_instance_id
(self)
lmcache/v1/standalone/standalone_service_factory.py:42
↓ 1 callersMethodget_event_future
Pop and return the event with the given type and id.
lmcache/v1/event_manager.py:128
↓ 1 callersMethodget_events_count_by_status
Get the count of events for the given event type and status. This is a lightweight O(1) operation using dict length.
lmcache/v1/event_manager.py:113
↓ 1 callersMethodget_eviction_actions
( self, expected_ratio: float, key_eligible_filter: Callable[[ObjectKey], bool] | None
lmcache/v1/distributed/eviction_policy/noop.py:44
↓ 1 callersMethodget_first_layer
Return the key for the first layer
lmcache/utils.py:475
↓ 1 callersFunctionget_fstype
(path)
lmcache/v1/storage_backend/gds_backend.py:77
↓ 1 callersMethodget_full_chunk_size_bytes
(self)
lmcache/v1/storage_backend/local_cpu_backend.py:851
↓ 1 callersMethodget_full_chunk_size_bytes
return the number of bytes in a full chunk useful for S3Connector where we need to preallocate filesystem buffers in ramfs fo
lmcache/v1/storage_backend/connector/__init__.py:148
↓ 1 callersFunctionget_gpu_pci_bus_id
Get the PCI bus ID via CUDA/ROCm runtime. Other backends return None. Args: device_id (int): CUDA/ROCm device index. Return
lmcache/python_ops_fallback.py:2521
↓ 1 callersMethodget_handler_type
(self)
lmcache/v1/multiprocess/mq.py:402
↓ 1 callersMethodget_handlers
Return handler specs for all request types this module serves. Returns: List of handler specs for lookup-related request types.
lmcache/v1/multiprocess/modules/p2p_controller.py:160
↓ 1 callersMethodget_indices_list
(self)
tests/v1/storage_backend/raw_block_test_utils.py:214
↓ 1 callersMethodget_inference_info
Get inference information by delegating to the connector. Returns: dict: Dictionary containing inference information,
lmcache/v1/manager.py:528
↓ 1 callersMethodget_inference_version
Get vLLM version information. Returns: str: vLLM version string
lmcache/integration/vllm/vllm_v1_adapter.py:726
↓ 1 callersMethodget_inflight_load_bytes_observations
Per-adapter ``(reserved_bytes, attributes)`` for the ``lmcache_mp.inflight_load_memory_usage_bytes`` gauge.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:507
↓ 1 callersMethodget_inflight_loads_observations
Per-adapter ``(count, attributes)`` for the ``lmcache_mp.num_inflight_l2_loads`` gauge.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:492
↓ 1 callersMethodget_inflight_stores_observations
Per-adapter ``(count, attributes)`` snapshot for the ``lmcache_mp.num_inflight_l2_stores`` gauge. ``dict.copy()`` is GIL-atomic in CP
lmcache/v1/distributed/storage_controllers/store_controller.py:400
↓ 1 callersMethodget_instance
Get or create the singleton instance.
lmcache/v1/periodic_thread.py:420
↓ 1 callersMethodget_instance
Get an instance by instance_id.
lmcache/v1/cache_controller/utils.py:457
↓ 1 callersMethodget_json
(self, url: str, *, timeout: Optional[float] = None)
lmcache/connections.py:121
↓ 1 callersMethodget_kernel_group_kv_pointers
( self, kernel_group_idx: int )
tests/v1/platform/test_cache_context_dispatch.py:93
↓ 1 callersMethodget_kernel_group_shape_dtype
Returns the shape and dtype for the given kernel group index and number of tokens. Mirrors :meth:`GPUCacheContext.get_kernel_group_sh
lmcache/v1/platform/cpu/cache_context.py:295
↓ 1 callersMethodget_kernel_group_shape_dtype
Returns the shape and dtype for the given kernel group index and number of tokens. Will be exported by GPUCacheContext and u
lmcache/v1/platform/cuda/cache_context.py:229
↓ 1 callersMethodget_keys
array ordering of keys from LRU to MRU
lmcache/v1/storage_backend/local_cpu_backend.py:906
↓ 1 callersMethodget_kv
Get the KV cache for the given layer ID. This function is used to get the KV cache from the GPU buffer.
lmcache/v1/gpu_connector/gpu_connectors.py:755
↓ 1 callersMethodget_kv_keys
Get all keys for a location.
lmcache/v1/cache_controller/utils.py:167
↓ 1 callersMethodget_kv_size
(self, token_cnt: int)
benchmarks/rag/precompute.py:56
↓ 1 callersFunctionget_l2_adapter_config_class
Resolve a registered L2-adapter config class by type name. Lazily imports the defining module if needed (via :func:`_ensure_config_loaded`),
lmcache/v1/distributed/l2_adapters/config.py:94
↓ 1 callersMethodget_l2_adapter_reconfigure_status
(self)
tests/v1/multiprocess/http_apis/test_reconfigure_api.py:40
↓ 1 callersFunctionget_l2_stored_object_count
Return the total stored object count across all L2 adapters.
tests/v1/distributed/serde/test_serde_e2e.py:134
↓ 1 callersMethodget_load_event_fd
(self)
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:394
↓ 1 callersMethodget_load_event_fd
(self)
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:152
↓ 1 callersFunctionget_log_level
Try to read LMCACHE_LOG_LEVEL from environment variables. Could be: - DEBUG - INFO - WARNING - ERROR - CRITICAL If n
lmcache/logging.py:39
↓ 1 callersMethodget_lookup_and_lock_event_fd
Return the pollable fd signaled when lookup tasks complete. Returns: File descriptor owned by this adapter's lookup notifier.
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:370
↓ 1 callersMethodget_lookup_and_lock_event_fd
Return the eventfd signaled when lookup-and-lock tasks complete.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:395
↓ 1 callersMethodget_lookup_info
Get information about the current lookup client and server. Returns: dict: Information about lookup client/server types
lmcache/v1/manager.py:494
↓ 1 callersMethodget_loop
Get the event loop
lmcache/v1/check/utils.py:564
↓ 1 callersMethodget_mem_to_storage_handle
Get a Nixl transfer handle for transferring data from memory to storage.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:335
↓ 1 callersMethodget_memory_allocator
(self)
lmcache/v1/storage_backend/nixl_storage_backend.py:865
↓ 1 callersMethodget_message_count
(self, benchmark: "ZMQControllerBenchmark")
lmcache/tools/controller_benchmark/handlers/p2p_lookup.py:46
↓ 1 callersMethodget_message_count
(self, benchmark: "ZMQControllerBenchmark")
lmcache/tools/controller_benchmark/handlers/register.py:48
↓ 1 callersMethodget_message_count
(self, benchmark: "ZMQControllerBenchmark")
lmcache/tools/controller_benchmark/handlers/heartbeat.py:48
← previousnext →4,101–4,200 of 13,371, ranked by callers