MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 1 callersMethodget_message_count
(self, benchmark: "ZMQControllerBenchmark")
lmcache/tools/controller_benchmark/handlers/deregister.py:41
↓ 1 callersMethodget_message_count
(self, benchmark: "ZMQControllerBenchmark")
lmcache/tools/controller_benchmark/handlers/admit.py:51
↓ 1 callersMethodget_messages_for_openai
(self)
benchmarks/multi_round_qa/multi-round-qa.py:106
↓ 1 callersMethodget_meta
(self, key_str: str)
lmcache/v1/storage_backend/connector/eic_connector.py:295
↓ 1 callersMethodget_metadata_many
Return metadata for encoded keys without loading payload bytes. Args: encoded_keys: Ordered encoded raw-block keys to inspect.
lmcache/v1/storage_backend/raw_block/core.py:505
↓ 1 callersMethodget_metadata_prefix
Return leading-hit metadata and optionally lock those entries. Args: encoded_keys: Ordered encoded raw-block keys to inspect.
lmcache/v1/storage_backend/raw_block/core.py:523
↓ 1 callersMethodget_model
Get the vllm model by instance_id.
lmcache/v1/compute/models/utils.py:59
↓ 1 callersMethodget_non_blocking
( self, key: CacheEngineKey, location: Optional[str] = None, )
lmcache/v1/storage_backend/gds_backend.py:865
↓ 1 callersMethodget_num_available_descs
(self)
lmcache/v1/storage_backend/nixl_storage_backend.py:271
↓ 1 callersMethodget_num_tokens
Get token number for the given MemoryObj.
lmcache/v1/memory_management.py:329
↓ 1 callersFunctionget_numa_id
Get the NUMA ID for the current GPU Args: numa_mapping (NUMAMapping): The NUMA mapping object. Returns: int: The NUMA I
lmcache/v1/lazy_memory_allocator.py:27
↓ 1 callersMethodget_or_create_lmcache_engine
(self)
lmcache/integration/vllm/vllm_service_factory.py:162
↓ 1 callersMethodget_or_create_lmcache_engine
(self)
lmcache/v1/standalone/standalone_service_factory.py:48
↓ 1 callersMethodget_or_create_metadata
(self)
lmcache/integration/vllm/vllm_service_factory.py:67
↓ 1 callersMethodget_or_create_metadata
(self)
lmcache/v1/standalone/standalone_service_factory.py:45
↓ 1 callersMethodget_output_buffer
(self, nlayers: int, nchannels: int, ntokens: int)
lmcache/v1/storage_backend/naive_serde/cachegen_decoder.py:55
↓ 1 callersMethodget_output_buffer
(self, nlayers: int, nchannels: int, ntokens: int)
lmcache/storage_backend/serde/cachegen_decoder.py:143
↓ 1 callersFunctionget_prefix_cache_token_size
(model_name: str, tp: int)
benchmarks/long_doc_qa/long_doc_qa_recommender.py:51
↓ 1 callersMethodget_ref_count
(self)
lmcache/v1/memory_management.py:749
↓ 1 callersFunctionget_registered_prefetch_policies
Return the list of registered prefetch policy names.
lmcache/v1/distributed/storage_controllers/prefetch_policy.py:105
↓ 1 callersFunctionget_registered_serde_types
Return the list of registered serde type names.
lmcache/v1/distributed/serde/factory.py:47
↓ 1 callersFunctionget_registered_store_policies
Return the list of registered store policy names.
lmcache/v1/distributed/storage_controllers/store_policy.py:120
↓ 1 callersMethodget_results
Return benchmark results for aggregation
lmcache/tools/controller_benchmark/benchmark.py:658
↓ 1 callersMethodget_serialize_event_fd
(self)
tests/v1/distributed/serde/test_factory.py:81
↓ 1 callersMethodget_serialize_event_fd
Event fd signaled when serialize tasks complete. Must be distinct from the deserialize event fd.
lmcache/v1/distributed/serde/base.py:128
↓ 1 callersMethodget_shape
(self)
lmcache/v1/memory_management.py:924
↓ 1 callersMethodget_shape
(self, num_tokens: int)
lmcache/v1/gpu_connector/gpu_connectors.py:2079
↓ 1 callersMethodget_shape_desc
Returns the PageBufferShapeDesc for *group_idx*.
lmcache/v1/platform/base_cache_context.py:199
↓ 1 callersFunctionget_span_registry
Return the process-level singleton :class:`SpanRegistry`. The registry is created on first call. Pass a fresh :class:`SpanRegistry` directly
lmcache/v1/mp_observability/subscribers/tracing/span_registry.py:136
↓ 1 callersMethodget_statistics
(self)
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:86
↓ 1 callersMethodget_storage_indices
(self, num_objs: int)
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:387
↓ 1 callersMethodget_storage_to_mem_handle
Get a Nixl transfer handle for transferring data from storage to memory.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:346
↓ 1 callersMethodget_store_event_fd
(self)
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:475
↓ 1 callersMethodget_store_event_fd
(self)
lmcache/v1/distributed/l2_adapters/native_connector_l2_adapter.py:176
↓ 1 callersMethodget_store_event_fd
(self)
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:149
↓ 1 callersMethodget_store_event_fd
Return the eventfd signaled when store tasks complete.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:389
↓ 1 callersMethodget_store_event_fd
(self)
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:142
↓ 1 callersMethodget_subscriptions
Return event_type -> callback mapping. Called once during ``register()``. The EventBus stores these callbacks directly.
lmcache/v1/mp_observability/event_bus.py:77
↓ 1 callersMethodget_subscriptions
(self)
lmcache/v1/mp_observability/subscribers/metrics/event_bus.py:60
↓ 1 callersMethodget_summary
Get a summary of all registered threads. Returns: Dict containing: - total_count: Total number of regist
lmcache/v1/periodic_thread.py:514
↓ 1 callersMethodget_sync_id
Get sync ID for a worker. Returns sync ID if worker is syncing, None otherwise.
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:118
↓ 1 callersMethodget_sync_status
Get sync status for a specific worker. Returns: Tuple of (is_complete, global_progress, can_exit_freeze, missing_batches
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:453
↓ 1 callersMethodget_temp_kernel_group_buffer
Returns the temp GPU buffer for the given batch index and kernel group index. The returned buffer is with the correct shape and dtype
lmcache/v1/platform/cuda/cache_context.py:179
↓ 1 callersMethodget_temp_object_group_buffer
Returns the temp GPU buffer for the given batch index and object group index The returned buffer is a flat uint8 raw tensor.
lmcache/v1/platform/cuda/cache_context.py:206
↓ 1 callersFunctionget_tensor_parallel_recommendation
(model_name: str)
benchmarks/long_doc_qa/long_doc_qa_recommender.py:22
↓ 1 callersFunctionget_tokenizer
()
examples/frontend/frontend.py:14
↓ 1 callersMethodget_transfer_channel_server
(self)
tests/v1/distributed/transfer_channel/test_factory.py:42
↓ 1 callersMethodget_underlying_buffer
Get the underlying buffer tensor. Will be used by RDMA registrations.
lmcache/v1/lazy_memory_allocator.py:214
↓ 1 callersFunctionget_url_from_args
Get the base URL from command line arguments. Args: args: Command line arguments. Returns: str: The base URL.
benchmarks/long_doc_qa/long_doc_qa.py:94
↓ 1 callersMethodglobal_min_tokens
( self, local_tokens: int, tp_group: dist.ProcessGroup, device: torch.device )
lmcache/integration/sglang/sglang_adapter.py:229
↓ 1 callersMethodhandle_batched_kv_operations
Handle batched KV operations by forwarding to registry.
lmcache/v1/cache_controller/controllers/kv_controller.py:149
↓ 1 callersMethodhandle_batched_push_request
(self, socket)
lmcache/v1/cache_controller/controller_manager.py:326
↓ 1 callersMethodhandle_batched_req_request
Handle requests on ROUTER socket. ROUTER socket receives multi-part messages: [identity, empty_frame, payload] and must reply
lmcache/v1/cache_controller/controller_manager.py:353
↓ 1 callersMethodhandle_heartbeat_request
Handle heartbeat requests on dedicated ROUTER socket. This runs on a separate socket to ensure heartbeats are processed without being
lmcache/v1/cache_controller/controller_manager.py:412
↓ 1 callersMethodhandle_request
Handle incoming requests (control msgs) from the controller.
lmcache/v1/cache_controller/worker.py:529
↓ 1 callersMethodhandle_worker_message
(self, msg: WorkerMsg)
lmcache/v1/cache_controller/controller_manager.py:162
↓ 1 callersMethodhandle_worker_req_message
( self, msg: WorkerReqMsg )
lmcache/v1/cache_controller/controller_manager.py:176
↓ 1 callersFunctionhandler
(request: httpx.Request)
tests/v1/mp_coordinator/test_registrar.py:27
↓ 1 callersMethodhandler_exception_counts
(self)
lmcache/v1/multiprocess/native_completion.py:101
↓ 1 callersFunctionhasReadOnlyMount
(c *corev1.Container, name string)
operator/internal/webhook/pod_injector_envtest_test.go:139
↓ 1 callersMethodhas_commands
Check if there are commands to execute
lmcache/v1/cache_controller/message.py:310
↓ 1 callersFunctionhas_content
Check if the chunk has content in the choices. Args: chunk: The response chunk from OpenAI Chat Completions API. Returns:
benchmarks/long_doc_qa/long_doc_qa.py:155
↓ 1 callersFunctionhas_content
Check if the chunk has content in the choices. Args: chunk: The response chunk from OpenAI API. Returns: bool: True if c
benchmarks/multi_doc_qa/multi_doc_qa.py:64
↓ 1 callersFunctionhas_content_completions
Completions streaming emits text at choices[0].text.
benchmarks/long_doc_qa/long_doc_qa.py:148
↓ 1 callersMethodhas_inflight
Check whether a key currently has an uncommitted write. Args: key: Key to check. Returns: ``True`` if a writ
lmcache/v1/storage_backend/dax/core.py:196
↓ 1 callersMethodhas_unconfigured_general
True if there are general items the user hasn't explicitly set.
lmcache/cli/commands/bench/engine_bench/interactive/state.py:138
↓ 1 callersMethodhas_workload_items
True if there are workload-specific items to configure.
lmcache/cli/commands/bench/engine_bench/interactive/state.py:142
↓ 1 callersMethodhead_size
Return the per-head dimension for ``layer_idx``.
lmcache/v1/gpu_connector/kv_format/specs/base.py:142
↓ 1 callersMethodheader
Return the file header. Always present; populated by ``__init__``.
lmcache/v1/mp_observability/trace/reader.py:68
↓ 1 callersMethodhealth
Check the health of the cache engine. return: 0 if healthy, otherwise the error code
lmcache/v1/cache_engine.py:1599
↓ 1 callersMethodhealth_check
(self)
lmcache/v1/cache_controller/controller_manager.py:477
↓ 1 callersMethodhelp
(self)
lmcache/cli/commands/bench/engine_bench/__init__.py:21
↓ 1 callersMethodhelp
(self)
lmcache/cli/commands/tool/transfer_channel_benchmark.py:27
↓ 1 callersMethodhelp
(self)
lmcache/cli/commands/tool/cache_simulator/sweep_command.py:19
↓ 1 callersMethodhelp
(self)
lmcache/cli/commands/tool/cache_simulator/simulate_command.py:20
↓ 1 callersMethodhelp
(self)
lmcache/cli/commands/tool/cache_simulator/gen_dataset_command.py:20
↓ 1 callersMethodhelp
Return a help string describing the config fields for this adapter type. This is used in command-line help to explain the expected J
lmcache/v1/distributed/l2_adapters/config.py:357
↓ 1 callersMethodhidden_dim
Return the hidden dimension (``num_heads * head_size``) for a layer.
lmcache/v1/gpu_connector/kv_format/specs/base.py:138
↓ 1 callersMethodhotplug_resize_device
Resize a mapped DAX device. Args: device_path: Device path to resize. size_bytes: New mapped byte size. m
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:925
↓ 1 callersFunctionimagePullBackOff
imagePullBackOff scans pods matching sel and returns a one-line description (pod, container, image, kubelet message) of the first container or init co
operator/test/utils/wait.go:156
↓ 1 callersMethodincr_or_send_stats
(self, stats: "LMCacheStats")
lmcache/usage_context.py:382
↓ 1 callersMethodindexed_key_count
Return the number of keys currently indexed by raw-block.
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:221
↓ 1 callersFunctioninfer_attn_backend_from_vllm
( vllm_attn: "torch.nn.Module", enable_sparse: bool = False, )
lmcache/v1/compute/attention/utils.py:37
↓ 1 callersFunctioninfer_attn_func_from_vllm
(device_type)
lmcache/v1/compute/attention/utils.py:84
↓ 1 callersFunctioninfer_model_from_vllm
(vllm_model, blender, enable_sparse: bool = False)
lmcache/v1/compute/models/utils.py:14
↓ 1 callersMethodinit_async_admit_condition
Create asyncio.Condition bound to the current event loop. Must be called from within the target event loop.
lmcache/v1/storage_backend/pd_backend_async.py:131
↓ 1 callersMethodinit_attn_metadata
Initialize attention metadata.
lmcache/v1/compute/attention/abstract.py:31
↓ 1 callersFunctioninit_eic_client
()
tests/v1/storage_backend/test_eic.py:35
↓ 1 callersFunctioninit_lmcache_engine
Initialize LMCache engine for SGLang integration. Args: model_config: SGLang model configuration tp_size: Tensor parallel si
lmcache/integration/sglang/sglang_adapter.py:48
↓ 1 callersMethodinit_mem_handlers
Initialize memory handlers for the given device and buffer.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:212
↓ 1 callersMethodinit_mem_handlers
(self, device, buffer_ptr, buffer_size, page_size, device_id)
lmcache/v1/storage_backend/nixl_storage_backend.py:477
↓ 1 callersFunctioninit_otel_tracing
Set up the OpenTelemetry TracerProvider with an OTLP exporter. Tracing requires an OTLP endpoint — there is no local fallback. When *otlp_end
lmcache/v1/mp_observability/otel_init.py:122
↓ 1 callersMethodinit_storage_handlers_file
Initialize storage handlers for file-based backends. Each file holds ``file_size // page_size`` pages at successive offsets. ``file_s
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:236
↓ 1 callersMethodinit_storage_handlers_file
(self, page_size, fds)
lmcache/v1/storage_backend/nixl_storage_backend.py:581
↓ 1 callersMethodinit_storage_handlers_object
Initialize storage handlers for object-based backends.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:305
↓ 1 callersMethodinit_storage_handlers_object
(self, page_size, keys)
lmcache/v1/storage_backend/nixl_storage_backend.py:597
↓ 1 callersMethodinitialize_allocator
( self, config: LMCacheEngineConfig, metadata: LMCacheMetadata, )
lmcache/v1/storage_backend/nixl_storage_backend.py:824
↓ 1 callersMethodinitialize_allocator
( self, config: LMCacheEngineConfig, metadata: Optional[LMCacheMetadata] = None, )
lmcache/v1/storage_backend/local_cpu_backend.py:352
↓ 1 callersMethodinitialize_allocator
( self, config: LMCacheEngineConfig, metadata: LMCacheMetadata )
lmcache/v1/storage_backend/gds_backend.py:1084
↓ 1 callersMethodinitialize_allocator
Create CxlMemoryAdapter backed by the connected handler. Args: config: LMCache engine configuration. metadata: LMCach
lmcache/v1/storage_backend/maru_backend.py:150
← previousnext →4,201–4,300 of 13,371, ranked by callers