MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 1 callersFunction_coerce_float
Coerce config values that may arrive as numeric strings.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:959
↓ 1 callersFunction_coerce_jsonable
Recursively coerce ``obj`` into a JSON-friendly value. Anything not natively serializable falls through to ``str(obj)``.
lmcache/v1/mp_observability/trace/recorder.py:279
↓ 1 callersFunction_collect_ec_overrides_from_env
Collect EC-specific overrides from environment variables.
lmcache/v1/config.py:1099
↓ 1 callersFunction_collect_ec_overrides_from_file
Collect EC-specific overrides from the LMCache YAML config file.
lmcache/v1/config.py:1116
↓ 1 callersMethod_collect_metadata_with_lock
Fast metadata collection with lock. Returns metadata for each key, None if key doesn't exist.
lmcache/v1/storage_backend/nixl_storage_backend.py:1108
↓ 1 callersMethod_combine_found
Merge the L1 found indices with an L2 result bitmap into one bitmap over the original key positions. ``l2_local`` is indexed over the
lmcache/v1/distributed/storage_manager.py:554
↓ 1 callersMethod_commit_expansion
Call sbrk in the address manager to commit the expansion.
lmcache/v1/lazy_memory_allocator.py:248
↓ 1 callersMethod_compute_chunk_hashes_hex
Compute prefix hashes for all chunks and return as hex strings. Args: token_ids: List of token IDs to process Returns:
lmcache/v1/lookup_client/record_strategies/base.py:53
↓ 1 callersMethod_compute_hash
Compute rolling hashes up to end_chunk. Uses cached state to skip already-computed chunks. Args: end_chunk: Compute hash
lmcache/v1/multiprocess/session.py:94
↓ 1 callersFunction_compute_required_buffer_gb
Compute the required CPU buffer size in GB for the benchmark. Calculates the total size of all chunks based on KV shape and dtype, then conve
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:188
↓ 1 callersMethod_compute_shm_pool_info
Compute normalized SHM pool metadata from storage config. Returns an empty pool (disabled SHM transport) when ``shm_name`` is empty o
lmcache/v1/multiprocess/engine_context.py:231
↓ 1 callersMethod_connect
(self, server_url: str)
lmcache/v1/distributed/transfer_channel/impl/nixl_impl.py:546
↓ 1 callersMethod_convert_bind_to_connect_url
Convert a bind address to a connectable address. Bind addresses like "0.0.0.0:port" or "*:port" cannot be used by workers to connect.
lmcache/v1/cache_controller/controller_manager.py:273
↓ 1 callersMethod_copy_bytes_from_offsets
Copy bytes from shared memory offsets to destination view.
lmcache/v1/storage_backend/connector/sagemaker_hyperpod_connector.py:571
↓ 1 callersFunction_copy_bytes_with_tensor
Copy raw bytes between pointers using torch tensor semantics. Note: This function only works for CPU-accessible memory. For device memory (CU
lmcache/python_ops_fallback.py:231
↓ 1 callersMethod_count_consecutive_exists
Check how many consecutive keys exist (prefix match). Fans out individual EXISTS checks across the thread pool for parallel round-tri
lmcache/v1/storage_backend/connector/valkey_connector.py:554
↓ 1 callersFunction_counter_delta
Compute the difference between two counter snapshots.
tests/v1/mp_observability/subscribers/metrics/test_l2.py:86
↓ 1 callersFunction_counter_delta
Compute the difference between two counter snapshots.
tests/v1/mp_observability/subscribers/metrics/test_lookup.py:72
↓ 1 callersMethod_create_audit_async_method
Create an audit async method with logging
lmcache/v1/storage_backend/connector/audit_connector.py:109
↓ 1 callersMethod_create_audit_sync_method
Create an audit sync method with logging
lmcache/v1/storage_backend/connector/audit_connector.py:152
↓ 1 callersMethod_create_backend
Create and return the backend instance.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:278
↓ 1 callersMethod_create_external_lookup_client
Create an external lookup client based on the URI format. Args: external_lookup_uri: URI in format <scheme>://<address>
lmcache/v1/lookup_client/factory.py:147
↓ 1 callersMethod_create_handler
Create and connect a MaruHandler. Args: config: LMCache engine configuration. Returns: Connected MaruHandler
lmcache/v1/storage_backend/maru_backend.py:105
↓ 1 callersMethod_create_immediate_empty_future
Create a Future that is already resolved with None.
lmcache/v1/storage_backend/maru_backend.py:254
↓ 1 callersFunction_create_l1_memory_desc
Create an L1 memory descriptor for a contiguous test buffer.
lmcache/v1/check/check_mode_test_l2_adapter.py:77
↓ 1 callersMethod_create_label_view
Reuse registered collectors with a different metadata/label view.
lmcache/observability.py:1920
↓ 1 callersMethod_create_mooncake_lookup_client
Create a MooncakeLookupClient instance.
lmcache/v1/lookup_client/factory.py:187
↓ 1 callersFunction_create_multi_group_metadata
(chunk_size: int = 16)
tests/v1/storage_backend/test_dax_backend.py:53
↓ 1 callersMethod_create_passthrough_async_method
Create a pass-through async method without logging
lmcache/v1/storage_backend/connector/audit_connector.py:83
↓ 1 callersMethod_create_passthrough_sync_method
Create a pass-through sync method without logging
lmcache/v1/storage_backend/connector/audit_connector.py:96
↓ 1 callersMethod_create_recv_primitives
Create asyncio primitives bound to the receiver event loop. Must be called from within the receiver event loop.
lmcache/v1/storage_backend/pd_backend_async.py:1149
↓ 1 callersMethod_create_sessions
Create sessions with synthetic prompts and history.
lmcache/cli/commands/bench/engine_bench/workloads/multi_round_chat.py:199
↓ 1 callersMethod_create_stream
Dispatch the streaming API call (chat or completions). When ``ignore_eos`` is set on the sender, ``{"ignore_eos": true}`` is added to
lmcache/cli/commands/bench/engine_bench/request_sender.py:220
↓ 1 callersFunction_dax_operation_payload
( operation: str, payload: dict[str, object], )
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:242
↓ 1 callersFunction_declare_signatures
Set argtypes/restype on libcufile symbols. Idempotent.
lmcache/v1/gpu_connector/_cufile_async.py:53
↓ 1 callersFunction_declares_slot_compression
Return whether a spec declares slot compression (must not be edited). Covers ``MLAAttentionSpec.compress_ratio > 1`` (DeepSeek-V4 slot packing)
lmcache/integration/vllm/kv_cache_group_edits.py:77
↓ 1 callersMethod_decode_slot_header
Decode a raw-block slot header into identity and payload length.
lmcache/v1/storage_backend/raw_block/core.py:1410
↓ 1 callersMethod_default_do_read
(self, offset: int, memory_obj: MemoryObj, size: int)
lmcache/v1/storage_backend/dax/core.py:1018
↓ 1 callersMethod_default_section
Return the unnamed default section, creating it on first use.
lmcache/cli/metrics/metrics.py:123
↓ 1 callersFunction_default_trace_path
Mint a timestamped path for an unnamed trace file.
lmcache/v1/mp_observability/trace/lifecycle.py:31
↓ 1 callersMethod_delete_request
(self, key_str: str)
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:930
↓ 1 callersMethod_deregister_region_locked
Deregister one region with cuFile (caller holds the lock). Args: buffer: A staging-buffer slot previously registered.
lmcache/v1/gpu_connector/gds_context.py:298
↓ 1 callersMethod_describe_kvcache
(self, args: argparse.Namespace)
lmcache/cli/commands/describe.py:331
↓ 1 callersFunction_detect_block_transfer_accepts_tensor
Return True if lmc_ops.multi_layer_block_kv_transfer accepts list[torch.Tensor] for its lmcache_objects_ptrs parameter.
lmcache/v1/multiprocess/transfer_context/base.py:54
↓ 1 callersFunction_detect_device
Detect the available accelerator and return the corresponding torch device module and device type string. Returns: tuple[Any, st
lmcache/__init__.py:27
↓ 1 callersFunction_detect_device_type
Return the ``torch.device.type`` describing *kv_caches*. All wrappers in *kv_caches* must materialize tensors on the same device type; mixed-
lmcache/v1/platform/cache_context.py:142
↓ 1 callersMethod_detect_object_groups
Detect object groups based on the provided engine group infos. Args: engine_group_infos: LMCache-owned engine KV cache group meta
lmcache/v1/kv_layer_groups.py:556
↓ 1 callersFunction_discover_and_register_handlers
Dynamically discover and register all operation handlers
lmcache/tools/controller_benchmark/handlers/__init__.py:18
↓ 1 callersFunction_discover_commands
Scan direct submodules of this package and collect all concrete :class:`BaseCommand` subclasses, returning one instance per class. Import err
lmcache/cli/commands/__init__.py:15
↓ 1 callersFunction_discover_detectors
Import every detector module in this folder and index it by its engine.
lmcache/v1/gpu_connector/kv_format/detectors/registry.py:20
↓ 1 callersMethod_discover_devices
Discover all available accelerator devices and map their UUIDs to the physical device ordinals.
lmcache/v1/multiprocess/custom_types.py:52
↓ 1 callersMethod_discover_health_checks
Discover all HealthCheck subclasses and instantiate them. This method dynamically scans all modules in the checks package, f
lmcache/v1/health_monitor/base.py:227
↓ 1 callersFunction_discover_platforms
Auto-discover all platform build profiles. Walks ``setup_extensions.build_profiles`` via ``discover_subclasses``, which uses ``pkgutil.iter_m
setup_extensions/policy.py:87
↓ 1 callersFunction_discover_specs
Import every spec module in this folder and index it by its format.
lmcache/v1/gpu_connector/kv_format/specs/registry.py:21
↓ 1 callersFunction_discover_storage_backends
Auto-discover all optional L2 storage backend build profiles. Walks ``setup_extensions.storage_backend_profiles`` via ``discover_subclasses``
setup_extensions/policy.py:105
↓ 1 callersFunction_discover_strategies
()
lmcache/v1/lookup_client/record_strategies/__init__.py:17
↓ 1 callersMethod_dispatch
Send a single benchmark request.
lmcache/cli/commands/bench/engine_bench/workloads/multi_round_chat.py:311
↓ 1 callersMethod_dispatch
Send a single benchmark request, then release the semaphore.
lmcache/cli/commands/bench/engine_bench/workloads/long_doc_qa.py:260
↓ 1 callersMethod_dispatch
Send a single prefill request with max_tokens=1.
lmcache/cli/commands/bench/engine_bench/workloads/random_prefill.py:156
↓ 1 callersMethod_dispatch
Send a single benchmark request, then release the semaphore. Args: messages: Chat messages for the request. perm_idx:
lmcache/cli/commands/bench/engine_bench/workloads/long_doc_permutator.py:381
↓ 1 callersMethod_dispatch_eviction
Send the DELETE and log the outcome. Failures are not retried.
lmcache/v1/mp_coordinator/l2/eviction_manager.py:172
↓ 1 callersMethod_do_load
( self, keys: list[ObjectKey], objects: list[MemoryObj], tid: L2TaskId, )
examples/lmc_external_l2_adapter/src/lmc_external_l2_adapter/adapter.py:325
↓ 1 callersMethod_do_read
(self, offset: int, memory_obj: MemoryObj, size: int)
lmcache/v1/storage_backend/plugins/dax_backend.py:673
↓ 1 callersMethod_do_store
( self, keys: list[ObjectKey], objects: list[MemoryObj], tid: L2TaskId, )
examples/lmc_external_l2_adapter/src/lmc_external_l2_adapter/adapter.py:276
↓ 1 callersMethod_drain_and_send
Drain the queue and send all messages in a batch. This method is called by the consumer thread to collect all pending operations from
lmcache/v1/storage_backend/batched_message_sender.py:154
↓ 1 callersMethod_drain_deserialize
Drain deserialize completions; report inner's load bitmap on success, all-zeros on deserialize failure.
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:543
↓ 1 callersMethod_drain_fingerprints_sync
Sync-drain pending fingerprint registrations (the async drainer races at low max_tokens).
lmcache/v1/multiprocess/modules/blend_v3.py:540
↓ 1 callersMethod_drain_inner_load
Drain inner load completions; on per-key success submit deserialize, otherwise fail the keys immediately.
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:499
↓ 1 callersMethod_drain_inner_store
Drain inner store completions; release temp read locks (auto- delete) and finalize the wrapped tasks.
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:475
↓ 1 callersMethod_drain_l2_store_completions
Deposit each signaled adapter's L2 outcomes onto their in-flight tasks, to be consumed by ``_advance_request``.
lmcache/v1/distributed/storage_controllers/store_controller.py:692
↓ 1 callersMethod_drain_serialize
Poll pending serialize tasks; on success submit inner store.
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:429
↓ 1 callersMethod_drain_submission_queue
Move items from the thread-safe submission queue to the pending queue.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:736
↓ 1 callersMethod_drop_positions
Return the positions in ``keys`` to drop, in ascending order. ``keys`` is the token-ordered chunk batch of one load, so index ``i`` is
lmcache/v1/distributed/l2_adapters/fault_inject_l2_adapter.py:260
↓ 1 callersMethod_dynamic_metrics
Dynamically get value by lambda function while capture
lmcache/observability.py:1495
↓ 1 callersMethod_emit_eviction_metrics
Record histogram observations for an evicted block.
lmcache/v1/mp_observability/subscribers/metrics/l0_lifecycle.py:269
↓ 1 callersFunction_emit_l2_adapter_metrics
Emit L2 adapter benchmark summary using the CLI metrics system.
lmcache/cli/commands/bench/l2_adapter_bench/command.py:474
↓ 1 callersFunction_emit_replay_metrics
Print the replay summary using the shared :class:`Metrics` renderer. Args: stats: The stats collector populated during replay. re
lmcache/cli/commands/trace/replay_command.py:242
↓ 1 callersFunction_emit_server_bench_metrics
Emit server bench summary using the CLI metrics system. Args: command: The owning :class:`BaseCommand` instance. args: Parsed CLI
lmcache/cli/commands/bench/server_bench/command.py:609
↓ 1 callersMethod_encode_header
Encode a fixed-size raw-block slot header.
lmcache/v1/storage_backend/raw_block/core.py:1398
↓ 1 callersFunction_encode_single_channel
Core arithmetic encoding for a single (layer, channel). Returns number of bytes written.
lmcache/python_ops_fallback.py:2134
↓ 1 callersFunction_engine_group_id_per_view
Return, per LMCache group, the engine group it draws block IDs from. Args: groups: The LMCache KV groups, in protocol order. Returns
lmcache/v1/multiprocess/group_view.py:89
↓ 1 callersMethod_ensure_bucket_for_writes
(self)
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:818
↓ 1 callersMethod_ensure_bucket_for_writes
Create the bucket on demand when configured to do so.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:520
↓ 1 callersMethod_ensure_heartbeat_started
Lazily start the heartbeat thread on first use.
lmcache/integration/vllm/vllm_multi_process_adapter.py:683
↓ 1 callersMethod_ensure_http_session
Ensure HTTP session with connection pooling is initialized.
lmcache/v1/storage_backend/connector/sagemaker_hyperpod_connector.py:186
↓ 1 callersMethod_ensure_peer_connection
( self, receiver_id: str, receiver_host: str, receiver_init_port: int,
lmcache/v1/storage_backend/pd_backend_async.py:685
↓ 1 callersMethod_ensure_socket
Ensure socket is created and connected
tests/v1/storage_backend/test_p2p_backend_with_controller.py:87
↓ 1 callersFunction_error_stream
Async generator that raises after yielding some chunks.
tests/cli/commands/bench/engine_bench/test_request_sender.py:80
↓ 1 callersMethod_evict
(self, needed: int)
examples/lmc_external_l2_adapter/src/lmc_external_l2_adapter/adapter.py:265
↓ 1 callersFunction_eviction_loop
Periodically check usage against quotas and dispatch eviction RPCs to any one registered MP server.
lmcache/v1/mp_coordinator/app.py:100
↓ 1 callersFunction_eviction_manager
Return the shared :class:`L2EvictionManager` from ``app.state``.
lmcache/v1/mp_coordinator/http_apis/l2_api.py:60
↓ 1 callersMethod_execute
Execute one cycle of the periodic task. This method should be overridden by subclasses to implement the actual periodic work
lmcache/v1/periodic_thread.py:352
↓ 1 callersMethod_execute_benchmark
Execute the benchmark with concurrent writes. Supports two modes: - write_bench=True: Single-phase write-only benchmark - wri
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:467
↓ 1 callersMethod_execute_delete
Run DELETE for each key and drop its size-tracking entry. Returns parallel lists of successfully deleted keys and their stored sizes,
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:1246
↓ 1 callersMethod_execute_delete
( self, keys: list[ObjectKey], )
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:611
↓ 1 callersMethod_execute_list
Issue one ``ListObjectsV2`` call and parse the response.
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:1233
↓ 1 callersMethod_execute_load
( self, keys: list[ObjectKey], objects: list[MemoryObj], task_id: L2TaskId,
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:1194
↓ 1 callersMethod_execute_load
( self, keys: list[ObjectKey], objects: list[MemoryObj], task_id: L2TaskId,
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:590
↓ 1 callersMethod_execute_load
( self, keys: list[ObjectKey], objects: list[MemoryObj], task_id: L2TaskId,
lmcache/v1/distributed/l2_adapters/fs_l2_adapter.py:665
↓ 1 callersMethod_execute_load_in_loop
Load each found key from its file via dynamic DMA read.
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:729
← previousnext →3,201–3,300 of 13,371, ranked by callers