MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 1 callersMethod_recover_from_local_cpu_fallback
Recover from LOCAL_CPU fallback when a health check passes again. This will: 1. Disable bypass for the specified backend in
lmcache/v1/health_monitor/base.py:370
↓ 1 callersFunction_redis_available
Check if Redis is reachable.
tests/v1/distributed/test_resp_l2_adapter_integration.py:33
↓ 1 callersFunction_reference_longest_hit
Longest model-wide prefix hit, mirroring vLLM's per-group ``find_longest_cache_hit`` combined across a hybrid model (independent brute force;
tests/v1/distributed/test_bitmap_ops.py:401
↓ 1 callersMethod_register_cpu_buffer
Register CPU buffer for zero-copy operations.
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:441
↓ 1 callersMethod_register_cuda_host_memory
(self)
lmcache/v1/memory_management.py:2690
↓ 1 callersMethod_register_region_locked
cuFile-register one <=16 MiB region (caller holds the lock).
lmcache/v1/gpu_connector/gds_context.py:281
↓ 1 callersMethod_register_self_gauges
Register the two self-monitoring gauges via ``register_gauge``.
lmcache/v1/mp_observability/event_bus.py:124
↓ 1 callersMethod_registered_ctx
Register ``buf``; capture the ``(base, offset)`` that ``transfer_async`` resolves a slice to before handing it to the slab.
tests/v1/gpu_connector/test_gds_context.py:102
↓ 1 callersMethod_relative_object_path
Return the path relative to the configured LMCache prefix.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:717
↓ 1 callersFunction_release_partial_kv_wrappers
Best-effort unlink of SHM segments owned by partially built wrappers. Used by :func:`wrap_kv_caches` to roll back a half-finished batch when
lmcache/integration/vllm/vllm_multi_process_adapter.py:166
↓ 1 callersMethod_remote_adapters_builtin_launcher
Automatically load all builtin remote connector adapters.
lmcache/v1/storage_backend/connector/__init__.py:244
↓ 1 callersMethod_remote_adapters_plugin_launcher
Automatically load all plug and play remote connector adapters.
lmcache/v1/storage_backend/connector/__init__.py:260
↓ 1 callersMethod_remote_allocate
( self, receiver_id: str, alloc_request: AllocRequest )
lmcache/v1/storage_backend/pd_backend.py:455
↓ 1 callersMethod_remove_finished_sessions
(self)
benchmarks/multi_round_qa/multi-round-qa.py:424
↓ 1 callersFunction_render_banner
Render the banner text. Args: colored: Whether to wrap the logo in ANSI color escapes. Returns: The multi-line banner: the L
lmcache/banner.py:62
↓ 1 callersFunction_report
Print best/median/mean latency and throughput for the measured reads.
lmcache/tools/transfer_channel_benchmark/benchmark.py:340
↓ 1 callersMethod_report_block_allocation_deltas
Gather per-request block allocation deltas and report to LMCache. For new requests: all allocated_block_ids and token_ids are new. Fo
lmcache/integration/vllm/lmcache_mp_connector.py:1251
↓ 1 callersMethod_report_block_allocation_deltas
Gather per-request block allocation deltas and report to LMCache. For new requests: all allocated_block_ids and token_ids are new. Fo
lmcache/integration/vllm/lmcache_mp_connector_0201.py:1103
↓ 1 callersFunction_require_full_install
Exit with an install hint if the full LMCache runtime is missing. ``lmcache bench server`` needs torch, zmq and ``lmcache.v1.*`` (MP client,
lmcache/cli/commands/bench/server_bench/helpers.py:90
↓ 1 callersFunction_require_musa
Fail clearly when the benchmark is run away from a MUSA host.
benchmarks/musa/bench_inprocess_transfer.py:156
↓ 1 callersFunction_requires_single_l1_memory_region
( adapter_config: L2AdapterConfigBase, )
lmcache/v1/distributed/config.py:33
↓ 1 callersMethod_resend_missing_batches
Resend missing batches to the controller. Args: sync_id: The sync session ID keys: List of all keys
lmcache/v1/cache_controller/full_sync_sender.py:303
↓ 1 callersMethod_reserve_put
( self, key: KeyT, obj: MemoryObj, size: int, *, raise_on_full
lmcache/v1/storage_backend/dax/core.py:897
↓ 1 callersFunction_resolve_backend
(device_type: str)
lmcache/v1/platform/cache_context.py:107
↓ 1 callersFunction_resolve_backend_adapter_index
( sm: _StorageManagerLike, backend: str, adapter_index: int, )
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:227
↓ 1 callersFunction_resolve_before_export
Resolve tokens_per_gb_kvcache and model before exporting. If the user provided an LMCache URL, query the server to get ``tokens_per_gb_kvcach
lmcache/cli/commands/bench/engine_bench/interactive/__init__.py:140
↓ 1 callersMethod_resolve_buffer
Locate the registered region ``gpu_buffer`` starts in. Returns ``(base_ptr, dev_offset, region_nbytes)``; ``region_nbytes - dev_offse
lmcache/v1/gpu_connector/gds_context.py:314
↓ 1 callersFunction_resolve_config_class
Resolve the config class for a plugin adapter. Discovery order: 1. Explicit ``config.config_class_name`` field. 2. Convention: ``config.c
lmcache/v1/distributed/l2_adapters/plugin_l2_adapter.py:179
↓ 1 callersFunction_resolve_device_id
Derive an integer device index from *dst_device*. Args: dst_device: Device string such as ``"cuda:2"``, ``"cuda"``, or ``"cpu
lmcache/v1/storage_backend/path_sharder.py:21
↓ 1 callersFunction_resolve_hf_token
Resolve Hugging Face token from env-first adapter config.
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:105
↓ 1 callersFunction_resolve_hf_token
Resolve the Hugging Face access token from env-first configuration.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:855
↓ 1 callersMethod_resolve_local_cpu_allocator_alignment
Determine pinned-memory alignment for LocalCPUBackend allocator. Precedence: 1) explicit override: extra_config["local_cpu.p
lmcache/v1/storage_backend/local_cpu_backend.py:555
↓ 1 callersMethod_resolve_max_data_transfer_size
Resolve transfer split size from config or NVMe sysfs queue limits. Args: configured_size: Explicitly configured max data transfe
lmcache/v1/storage_backend/raw_block/core.py:353
↓ 1 callersFunction_resolve_mode
Coerce ``mode`` into :class:`MPTransferMode`, falling back to env.
lmcache/v1/multiprocess/transfer_context/worker_transfer.py:62
↓ 1 callersMethod_resolve_object_sizes
Resolve cached and uncached object sizes while preserving order.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:618
↓ 1 callersMethod_resolve_object_sizes_sync
(self, keys: list[ObjectKey])
lmcache/v1/distributed/l2_adapters/hfbucket_l2_adapter.py:666
↓ 1 callersFunction_resolve_package
(package: Union[ModuleType, str])
lmcache/v1/utils/subclass_discovery.py:32
↓ 1 callersFunction_resolve_per_layer_sw_sizes
Resolve the sliding window size in tokens for each registered KV tensor. Will resolve -1 for non-sliding-window layers. Args: vllm_g
lmcache/integration/vllm/kv_cache_groups.py:31
↓ 1 callersFunction_resolve_sysfs_queue_dir
Resolve sysfs queue directory for NVMe character device paths.
lmcache/v1/storage_backend/raw_block/core.py:110
↓ 1 callersMethod_resource_manager
(self)
lmcache/v1/health_monitor/checks/remote_backend_check.py:269
↓ 1 callersMethod_restore_batch
( self, keys: list[CacheEngineKey], prefix_only: bool, )
lmcache/v1/storage_backend/plugins/dax_backend.py:849
↓ 1 callersMethod_rotate_file
(self)
lmcache/v1/lookup_client/record_strategies/file_hash.py:69
↓ 1 callersMethod_rotate_file
Close current file and open a new one.
lmcache/v1/mp_observability/subscribers/logging/lookup_hash.py:182
↓ 1 callersMethod_run
(self, disk_path: str)
tests/v1/distributed/serde/test_serde_fs_e2e.py:106
↓ 1 callersMethod_run_async
Internal async implementation of the run loop.
lmcache/cli/commands/bench/engine_bench/workloads/base.py:91
↓ 1 callersFunction_run_load_phase
Run load phase and return (stats, bitmap).
lmcache/v1/check/check_mode_test_l2_adapter.py:128
↓ 1 callersMethod_run_peer_request_handler_with_recovery
Wrapper method that runs _handle_peer_requests with exception handling. This ensures the handler keeps running even if unexpected err
lmcache/v1/storage_backend/p2p_backend.py:251
↓ 1 callersMethod_run_performance_test
(self, strategy_name: str, async_preprocess_chunks: bool)
tests/v1/lookup_client/test_chunk_statistics_lookup_client.py:263
↓ 1 callersMethod_run_restore_waves
(self, waves: Sequence[_RestoreWave])
lmcache/v1/storage_backend/plugins/dax_backend.py:829
↓ 1 callersFunction_run_store_phase
Run store phase and return (stats, success).
lmcache/v1/check/check_mode_test_l2_adapter.py:101
↓ 1 callersMethod_run_sync_mem_to_storage
( self, handle: NixlXferHandle, keys: Sequence[CacheEngineKey], reg_descs: nix
lmcache/v1/storage_backend/nixl_storage_backend.py:1831
↓ 1 callersMethod_s3_upload
Upload a file to S3.
lmcache/v1/storage_backend/connector/s3_connector.py:496
↓ 1 callersMethod_safe_config_dict
Best-effort conversion of a StorageManagerConfig to a JSON dict. Delegates to the module-level :func:`safe_storage_config_dic
lmcache/v1/mp_observability/trace/recorder.py:242
↓ 1 callersMethod_scan_metadata
(self)
lmcache/v1/storage_backend/gds_backend.py:352
↓ 1 callersFunction_scatter_flat_chunks_to_paged
Inverse of :func:`_gather_paged_to_flat_chunks`. Writes each ``[2, NL, chunk_size, hidden]`` flat chunk back into the paged client tensors at
lmcache/cli/commands/bench/server_bench/helpers.py:580
↓ 1 callersMethod_schedule
( self, batch_size, cu_query_lens, max_query_len, seqlens, max_seq_len, causal )
lmcache/v1/compute/attention/flash_attn.py:105
↓ 1 callersMethod_secondary_lookup_locked
Check if a data file for ``key`` exists on disk; if so, populate ``_memory_objects`` and return the entry. Caller must hold ``_lock``.
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:695
↓ 1 callersMethod_seed_usage_from_core_snapshot
Seed byte counters for entries recovered by RawBlockCore startup.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:617
↓ 1 callersMethod_select_latest_checkpoint
Return the newest valid checkpoint header and payload.
lmcache/v1/storage_backend/raw_block/core.py:1550
↓ 1 callersMethod_send_cleanup_message
Send cleanup message to workers to release memory objects.
lmcache/v1/lookup_client/lmcache_async_lookup_client.py:277
↓ 1 callersMethod_send_cmd_pipe
Send via stdin/stdout PIPE.
tests/v1/shm_allocator/shmfile_connector.py:177
↓ 1 callersMethod_send_cmd_tcp
Send via TCP socket.
tests/v1/shm_allocator/shmfile_connector.py:186
↓ 1 callersFunction_send_end_session
END_SESSION — clean up server-side session state.
lmcache/cli/commands/bench/server_bench/helpers.py:813
↓ 1 callersMethod_send_key_batches
Send keys in batches to the controller. Args: sync_id: The sync session ID keys: List of all keys to send
lmcache/v1/cache_controller/full_sync_sender.py:263
↓ 1 callersFunction_send_register_kv_cache
Register a KV cache context with the MP server. Dispatches to the correct protocol based on ``use_handle``: * Handle mode: ``REGISTER_KV_CAC
lmcache/cli/commands/bench/server_bench/helpers.py:318
↓ 1 callersFunction_send_retrieve
Retrieve KV cache blocks. Returns status. Handle mode uses the single-shot ``RETRIEVE`` RPC (GPU CUDA-IPC, or CPU SHM with an empty event han
lmcache/cli/commands/bench/server_bench/helpers.py:743
↓ 1 callersFunction_send_store
Store KV cache blocks. Returns status string. Handle mode uses the single-shot ``STORE`` RPC (GPU CUDA-IPC, or CPU SHM with an empty event ha
lmcache/cli/commands/bench/server_bench/helpers.py:675
↓ 1 callersFunction_serve_catalog
Serve the source object catalog over a ZMQ REP socket until timeout.
lmcache/tools/transfer_channel_benchmark/benchmark.py:160
↓ 1 callersMethod_set_sync_info
Set sync info for a worker. Returns True if successful.
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:61
↓ 1 callersMethod_setup_config
Setup backend-specific configuration. Override if needed.
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:365
↓ 1 callersMethod_setup_device
(self)
benchmarks/storage_backend_io/storage_backend_io_benchmark.py:359
↓ 1 callersMethod_setup_metrics
Setup metrics for monitoring data structures in the connector.
lmcache/integration/vllm/vllm_v1_adapter.py:642
↓ 1 callersMethod_setup_metrics
( self, config: "LMCacheEngineConfig", metadata: "LMCacheMetadata", )
lmcache/v1/pin_monitor.py:225
↓ 1 callersMethod_setup_metrics
(self)
lmcache/v1/cache_controller/controllers/registration_controller.py:43
↓ 1 callersMethod_setup_metrics
(self)
lmcache/v1/cache_controller/controllers/kv_controller.py:74
↓ 1 callersMethod_setup_metrics
(self)
lmcache/v1/lookup_client/chunk_statistics_lookup_client.py:195
↓ 1 callersMethod_setup_metrics
Register OTel observable gauges for lookup module metrics.
lmcache/v1/multiprocess/modules/lookup.py:498
↓ 1 callersMethod_setup_metrics
Register OTel observable gauges for P2P controller metrics.
lmcache/v1/multiprocess/modules/p2p_controller.py:566
↓ 1 callersMethod_setup_metrics
(self)
lmcache/v1/storage_backend/local_cpu_backend.py:106
↓ 1 callersMethod_setup_metrics
(self)
lmcache/v1/storage_backend/remote_backend.py:104
↓ 1 callersMethod_setup_metrics
Setup metrics for monitoring queue size.
lmcache/v1/storage_backend/batched_message_sender.py:84
↓ 1 callersMethod_setup_metrics
(self)
lmcache/v1/storage_backend/storage_manager.py:291
↓ 1 callersMethod_setup_socket_metrics
Setup metrics for socket message counts.
lmcache/v1/cache_controller/controller_manager.py:237
↓ 1 callersFunction_short_op_name
Return a compact, human-readable label for a traced qualname. Plain methods collapse to the method name: the table has limited column width a
lmcache/cli/commands/trace/replay_command.py:303
↓ 1 callersMethod_should_drop_key
Return whether ``key`` falls in the rate-based drop bucket. Deterministic in ``(seed, key)``: a stable blake2b hash bucketed by ``rat
lmcache/v1/distributed/l2_adapters/fault_inject_l2_adapter.py:239
↓ 1 callersMethod_should_sample
(self, key: object)
lmcache/v1/mp_observability/subscribers/metrics/l1_lifecycle.py:154
↓ 1 callersMethod_should_sample
(self)
lmcache/v1/mp_observability/subscribers/metrics/l0_lifecycle.py:289
↓ 1 callersFunction_should_use_lazy
Lazy allocator requires CUDA. CPU-only hosts (our primary replay target) must use eager allocation.
tests/cli/commands/trace/test_driver.py:48
↓ 1 callersMethod_shrink_device_locked
( self, entry: DaxDeviceEntry, new_max_slots: int, mode: HotplugResizeMode,
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:1481
↓ 1 callersMethod_signal_load_event
(self)
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:586
↓ 1 callersMethod_signal_load_event
Signal the load event fd to notify completion.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:838
↓ 1 callersMethod_signal_load_event
Signal the load event fd to notify completion.
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:464
↓ 1 callersMethod_signal_lookup_event
(self)
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:583
↓ 1 callersMethod_signal_lookup_event
Signal the lookup event fd to notify completion.
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:809
↓ 1 callersMethod_signal_lookup_event
Signal the lookup event fd to notify completion.
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:447
↓ 1 callersMethod_signal_store_event
(self)
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:580
↓ 1 callersMethod_signal_store_event
Signal the store event fd to notify completion.
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:372
↓ 1 callersFunction_simulate_chain_walk
Longest contiguous prefix where each chunk i has an aligned match. Stand-in for the real chain walker, which probes storage by chained chunk
tests/v1/multiprocess/test_optimized_lookup_v3.py:75
↓ 1 callersFunction_slice_kv_at
( start_idx: int, kv_tensors: torch.Tensor, chunk_size: int, )
tests/v1/test_mem_kernels.py:77
↓ 1 callersMethod_snapshot_state
Build a JSON-serializable checkpoint state snapshot.
lmcache/v1/storage_backend/raw_block/core.py:1568
← previousnext →3,601–3,700 of 13,371, ranked by callers