MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 2 callersFunction_open_and_mmap
Open (or create) a POSIX SHM segment and ``mmap`` it. Returns a ``(mmap_obj, base_addr)`` pair. The fd is always closed before returning so w
lmcache/v1/multiprocess/posix_shm.py:73
↓ 2 callersMethod_open_arena
(self)
lmcache/v1/storage_backend/dax/core.py:1025
↓ 2 callersMethod_open_flags
Return os.open flags for storage files.
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:146
↓ 2 callersFunction_pack_slot_mapping
( slot_mapping: torch.Tensor, starts: list[int], ends: list[int] )
tests/v1/test_xpu_sglang_connector.py:37
↓ 2 callersFunction_pack_slot_mapping
( slot_mapping: torch.Tensor, starts: list[int], ends: list[int] )
tests/v1/test_musa_connector.py:41
↓ 2 callersFunction_pack_str
(s: str)
lmcache/v1/kv_codec/encoded_kv.py:100
↓ 2 callersMethod_paged_byte_allocator
Build a paged uint8 allocator with ``n_pages`` pages of ``page_bytes`` each, suitable for testing block-recycle resets.
tests/v1/test_memory_management.py:1048
↓ 2 callersMethod_parse_persist_config
Parse optional ``"persist_enabled"`` key from an adapter JSON spec. Defaults to ``True``. Expected format:: {
lmcache/v1/distributed/l2_adapters/config.py:264
↓ 2 callersFunction_parse_positive_float
(value: object, field_name: str)
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:118
↓ 2 callersMethod_parse_serde_config
Parse an optional ``"serde"`` sub-dict from an adapter JSON spec. Expected format:: { "type": "fs",
lmcache/v1/distributed/l2_adapters/config.py:198
↓ 2 callersFunction_parse_url
Parse a ``host:port`` (optionally ``tcp://host:port``) into (host, port).
lmcache/v1/distributed/transfer_channel/impl/nixl_impl.py:60
↓ 2 callersFunction_patch_transfer_context_factory
Patch ``create_transfer_context`` to mint recorded MagicMocks, returning the list every created context is appended to.
tests/v1/test_vllm_mp_adapter.py:125
↓ 2 callersFunction_percentile
Return the percentile *pct* (0..100) using nearest-rank. Returns 0.0 for an empty list.
lmcache/cli/commands/bench/l2_adapter_bench/result.py:15
↓ 2 callersMethod_pin_memory_chunk
Pin a chunk of memory. Args: offset (int): Offset in the buffer to pin. size (int): Size of the memory chunk
lmcache/v1/lazy_memory_allocator.py:227
↓ 2 callersFunction_poll
(fn, timeout_s: float = 10.0)
tests/v1/distributed/l2_adapters/test_p2p_l2_adapter_integration.py:107
↓ 2 callersFunction_poll_prefetch_status
QUERY_PREFETCH_STATUS — poll until done. Returns the hit chunk count, or ``None`` if the polling budget is exhausted. The server keys prefetc
lmcache/cli/commands/bench/server_bench/helpers.py:463
↓ 2 callersMethod_post_non_blocking
Await a nixl transfer until done.
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:285
↓ 2 callersMethod_prepare_params
(self)
lmcache/v1/protocol.py:166
↓ 2 callersMethod_prepare_test_data
Prepare test data by putting chunks into FSConnector.
tests/v1/internal_api_server/test_load_fs_chunks.py:133
↓ 2 callersFunction_presence_cache_metadata
()
tests/v1/test_nixl_storage.py:396
↓ 2 callersMethod_probe_decoder_cache
Query the decoder's cache to check which blocks are already cached. This is the bidirectional NIXL cache probe: the prefiller queries the
lmcache/integration/vllm/vllm_v1_adapter.py:1251
↓ 2 callersMethod_process_finished_stores
Merge LMCache-side and engine-side finished store info.
lmcache/integration/vllm/vllm_multi_process_adapter.py:1535
↓ 2 callersMethod_process_ops
Drain the ops queue and apply register/unregister to the poller.
lmcache/v1/multiprocess/mq.py:212
↓ 2 callersFunction_process_request
Run the full lookup -> retrieve/store flow. When ``client_tensors`` is provided (data-mode self-check), the flow gains two extra steps:
lmcache/cli/commands/bench/server_bench/helpers.py:915
↓ 2 callersFunction_process_tokens_request
Process tokens request and validate parameters. Args: request: FastAPI request object tokens_mock: Mock token range specification
lmcache/v1/internal_api_server/vllm/cache_api.py:379
↓ 2 callersFunction_prompt_gate
Ask the user whether to configure a section or skip with defaults. Returns True if the user wants to configure.
lmcache/cli/commands/bench/engine_bench/interactive/__init__.py:82
↓ 2 callersFunction_public_callables
Return {name: obj} for all public, non-dunder, non-enum callables.
tests/v1/test_c_ops_fallback_parity.py:41
↓ 2 callersFunction_public_data_attrs
(inst: object)
tests/v1/test_c_ops_fallback_parity.py:378
↓ 2 callersFunction_public_descriptor_classes
Return {name: cls} for public classes that are neither enums nor plain functions — e.g. pybind11 ``py::class_<>`` types like ``PageBufferShape
tests/v1/test_c_ops_fallback_parity.py:78
↓ 2 callersFunction_public_enums
Return {name: obj} for all public enum-like classes. Detects both Python enum.Enum subclasses and pybind11 enums. pybind11 enums have a ``__m
tests/v1/test_c_ops_fallback_parity.py:54
↓ 2 callersMethod_put_one
( self, key: KeyT, obj: MemoryObj, *, raise_on_full: bool, rai
lmcache/v1/storage_backend/dax/core.py:849
↓ 2 callersMethod_put_with_metadata
Put using put_parts when metadata is stored remotely. This is used when save_chunk_meta=True (matches _batch_get_buffer).
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:786
↓ 2 callersMethod_query_sync_status
Query sync status from controller
lmcache/v1/cache_controller/full_sync_sender.py:174
↓ 2 callersFunction_raise_json_blob_error
(blob: str)
lmcache/cli/commands/query/_request.py:66
↓ 2 callersFunction_raise_openai_error
(obj: dict[str, Any])
lmcache/cli/commands/query/_request.py:60
↓ 2 callersFunction_raise_server_unreachable
Raise a verbose ConnectionError when the LMCache MP server is unreachable. The message intentionally spells out the most common cause (the
lmcache/integration/vllm/vllm_multi_process_adapter.py:241
↓ 2 callersFunction_random_text
Generate random ASCII text that tokenizes to unique tokens. Uses distinct vocabulary per seed so that no two generated texts share a token-le
examples/kv_cache_reuse/remote_backends/valkey/benchmark_l2.py:46
↓ 2 callersFunction_range
(prefix: str, tokens: list[int])
tests/v1/mp_coordinator/test_blend_client.py:66
↓ 2 callersFunction_read_counters
Snapshot all counter values from the module-level reader, summed across attribute combinations. A counter with multiple labeled data points (
tests/v1/mp_observability/subscribers/metrics/test_l2.py:44
↓ 2 callersFunction_read_counters
Snapshot counter values, summed across attribute combinations.
tests/v1/mp_observability/subscribers/metrics/test_lookup.py:34
↓ 2 callersMethod_read_frame
Read one length-prefixed frame. Returns ``None`` on clean EOF (when ``strict=False``). On truncation in the middle of a frame, logs
lmcache/v1/mp_observability/trace/reader.py:133
↓ 2 callersFunction_read_histograms
Snapshot all histogram data points.
tests/v1/mp_observability/subscribers/metrics/test_l0_lifecycle.py:67
↓ 2 callersMethod_read_metadata
( self, key: CacheEngineKey, filename: str, subdir_key: str, )
lmcache/v1/storage_backend/gds_backend.py:454
↓ 2 callersFunction_read_requirements
(path: Path)
setup.py:26
↓ 2 callersFunction_readinto_full
Loop readinto() until *buf* is full or EOF. A single ``readinto()`` may return fewer bytes than *len(buf)* even when more data is available.
lmcache/v1/distributed/l2_adapters/fs_l2_adapter.py:59
↓ 2 callersMethod_rebuild
Re-create ``slots`` sized to the live entries, dropping tombstones.
lmcache/v1/mp_coordinator/blend_directory.py:186
↓ 2 callersFunction_reconfigure_size_bytes
(payload: dict[str, object])
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:137
↓ 2 callersMethod_record
( cls, event: Event, pending: dict[tuple[str, str], float], hist: Any, )
lmcache/v1/mp_observability/subscribers/metrics/l0_l1_throughput.py:119
↓ 2 callersMethod_record
Record a throughput sample. Args: event: The COMPLETED event, used for timestamp and metadata. correlation_k
lmcache/v1/mp_observability/subscribers/metrics/l2_throughput.py:153
↓ 2 callersMethod_record_submission
Track an in-flight submission so its ctypes storage outlives the DMA. Accumulated per (current) stream; every ``_SUBMISSION_CHECKPOINT_EVERY`
lmcache/v1/gpu_connector/gds_context.py:356
↓ 2 callersMethod_recreate_all_sockets
Recreate all sockets after a failure.
lmcache/v1/rpc/zmq_transport.py:74
↓ 2 callersMethod_recreate_heartbeat_socket
(self)
lmcache/v1/cache_controller/worker.py:326
↓ 2 callersMethod_recreate_req_socket
(self)
lmcache/v1/cache_controller/worker.py:333
↓ 2 callersMethod_recv_handshake
Receive one handshake reply, mapping a timeout to a clear error. Args: socket: The REQ socket awaiting the server's reply.
lmcache/v1/distributed/transfer_channel/impl/nixl_impl.py:522
↓ 2 callersFunction_ref_attention
Naive PyTorch attention (no sparsity).
tests/v1/compute/attention/test_triton_kernels.py:31
↓ 2 callersMethod_register_single_file
Register a single file with nixl and return (reg_descs, xfer_handler). Returns: Tuple of (reg_descs, xfer_handler) for later clea
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:157
↓ 2 callersMethod_release_entry
Release resources for a popped entry (run outside the lock). Sweeps the instance's pending SHM transfers and unregisters its layout d
lmcache/v1/multiprocess/modules/engine_driven_transfer.py:264
↓ 2 callersMethod_release_entry
Release resources for a popped entry (run outside the lock). Args: entry: The entry removed from the registry.
lmcache/v1/multiprocess/modules/lmcache_driven_transfer.py:545
↓ 2 callersFunction_release_memory_objs
Release memory objects.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:171
↓ 2 callersMethod_release_restore_resources
(self)
lmcache/v1/storage_backend/plugins/dax_backend.py:633
↓ 2 callersFunction_render_shape
(fmt: "lmc_ops.EngineKVFormat", token: Callable[[str], str])
lmcache/v1/gpu_connector/kv_format/specs/base.py:46
↓ 2 callersFunction_report_import_error
(full_name: str, exc: Exception)
lmcache/v1/utils/subclass_discovery.py:209
↓ 2 callersMethod_reserve_slot_state_locked
(self, slot_id: int)
lmcache/v1/storage_backend/dax/core.py:950
↓ 2 callersMethod_reset_label_views
Return label views whose children must be restored after clear(). Counter.clear() and Histogram.clear() remove every child from the s
lmcache/observability.py:1931
↓ 2 callersMethod_resolve_cb_layout_desc
Find the CB KV buffer layout for ``(model_name, world_size)``. Reads the thread-safe ``layout_desc_registry`` (populated by ``lmcache
lmcache/v1/multiprocess/modules/blend_v3.py:600
↓ 2 callersFunction_resolve_dtype
(name: str)
lmcache/v1/mp_observability/trace/codecs.py:180
↓ 2 callersFunction_resolve_l2_adapter
Resolve the ``(descriptor, adapter)`` pair the handler targets. ``selector=None`` picks the primary adapter; a string picks the first adapter
lmcache/v1/multiprocess/http_apis/l2_api.py:93
↓ 2 callersFunction_resolve_pinned_alloc_free
Resolve the alloc/free function pair based on memory type. Returns: A PinnedAllocFree with the resolved functions and their extra
lmcache/v1/memory_management.py:436
↓ 2 callersMethod_resolve_single_group_obj_keys
Resolve object keys for the single object group used by non-GPU transfers.
lmcache/v1/multiprocess/modules/engine_driven_transfer.py:295
↓ 2 callersFunction_resolve_size_bytes
(size: SizeRequest)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:133
↓ 2 callersFunction_run_batched_get_prefix_stop
( memory_allocator, loop_in_thread, *, async_get: bool, )
tests/v1/storage_backend/test_rust_raw_block_backend.py:279
↓ 2 callersFunction_run_lookup_phase
Run lookup phase and return (stats, bitmap).
lmcache/v1/check/check_mode_test_l2_adapter.py:115
↓ 2 callersFunction_run_one_mode
Run one benchmark mode.
benchmarks/musa/bench_inprocess_transfer.py:212
↓ 2 callersMethod_s3_download
Download a file from S3.
lmcache/v1/storage_backend/connector/s3_connector.py:297
↓ 2 callersFunction_sanitize_setup_config
Return a copy of *setup_config* with sensitive values redacted, safe for logging.
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:46
↓ 2 callersFunction_scan_module
Yield classes from *module* that match all filters.
lmcache/v1/utils/subclass_discovery.py:146
↓ 2 callersMethod_schedule_slot_reclaim_locked
(self, slot_id: int, generation: int)
lmcache/v1/storage_backend/dax/core.py:963
↓ 2 callersFunction_scheduler_metadata
()
tests/v1/storage_backend/test_nixl_shared_pool.py:647
↓ 2 callersMethod_send_cmd
Send a command and return the response line.
tests/v1/shm_allocator/shmfile_connector.py:170
↓ 2 callersMethod_send_heartbeat_msg
Send heartbeat message via dedicated heartbeat DEALER socket. This is separate from async_put_and_wait_msg to keep heartbeat independ
lmcache/v1/cache_controller/worker.py:408
↓ 2 callersFunction_send_lookup
LOOKUP — submit a prefix lookup. The server-side handler returns ``None`` (void) on success, so we only distinguish RPC timeout from a comple
lmcache/cli/commands/bench/server_bench/helpers.py:450
↓ 2 callersMethod_send_proxy_notif
Encode and send a ProxyNotif to the proxy side channel. :param transfer_spec: Provides the req_id for the notification.
lmcache/v1/storage_backend/pd_backend_async.py:1024
↓ 2 callersMethod_send_register_kv_caches_request
Submit a REGISTER_KV_CACHE request and wait for the response. Shared by the public ``register_kv_caches`` entry point and the heartbe
lmcache/integration/vllm/vllm_multi_process_adapter.py:1265
↓ 2 callersMethod_send_sync_start
Send FullSyncStartMsg and wait for confirmation
lmcache/v1/cache_controller/full_sync_sender.py:125
↓ 2 callersFunction_serve
Start the coordinator in a background thread; return (server, thread).
tests/v1/mp_coordinator/test_integration.py:41
↓ 2 callersMethod_set_setup_key
(self, key: str, value: str)
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:200
↓ 2 callersMethod_setup_heartbeat_socket
Setup heartbeat DEALER socket after getting heartbeat_url from register
lmcache/tools/controller_benchmark/benchmark.py:359
↓ 2 callersMethod_should_enable_l2_eviction
Whether to wire an adapter into the L2 eviction controller. Args: adapter: The adapter to evaluate. eviction_config:
lmcache/v1/distributed/storage_manager.py:1022
↓ 2 callersMethod_should_sample
(self, key: tuple[str, bytes])
lmcache/v1/mp_observability/subscribers/metrics/sm_lifecycle.py:120
↓ 2 callersMethod_should_skip_plugin
Determine if plugin should be skipped based on role/worker ID. The naming convention is ROLE_WORKERID_NAME (e.g. server_0_foo.py). Wh
lmcache/v1/plugin/runtime_plugin_launcher.py:58
↓ 2 callersMethod_shutdown_loop
Cancel all pending tasks on *loop*, stop it, and join the thread. Uses a ``threading.Event`` to synchronize shutdown completion so that
lmcache/v1/storage_backend/pd_backend_async.py:1580
↓ 2 callersMethod_shutdown_native
Shutdown the native client and cleanup resources.
lmcache/v1/storage_backend/resp_client.py:122
↓ 2 callersMethod_shutdown_native
(self, best_effort: bool = False)
lmcache/v1/storage_backend/native_clients/connector_client_base.py:75
↓ 2 callersMethod_signal_store_event
Signal the store event fd to notify completion.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:701
↓ 2 callersFunction_single_group_block_ids
Return the flat block-id list for transports without HMA support.
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:103
↓ 2 callersFunction_slice_by_slot_dim
Slice tensor by slot dimension based on tensor ndim.
lmcache/v1/internal_api_server/vllm/cache_api.py:145
↓ 2 callersMethod_slot_mapping_to_block_ids
(self, slot_mapping: torch.Tensor)
lmcache/integration/sglang/multi_process_adapter.py:214
↓ 2 callersMethod_slot_to_offset
Convert a data-slot index to its byte offset.
lmcache/v1/storage_backend/raw_block/core.py:1459
↓ 2 callersMethod_snapshot_indexed_object_keys
Return decoded ObjectKeys for all indexed raw-block entries.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:636
↓ 2 callersMethod_snapshot_inflight_loads
``{adapter_idx: (count, reserved_bytes)}`` for in-flight L2 -> L1 loads, computed via GIL-atomic ``dict.copy()`` snapshots so the OTel
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:478
← previousnext →2,401–2,500 of 13,371, ranked by callers