MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 3 callersFunctiontest_eviction
(new_idx, old_idx)
tests/v1/test_nixl_storage.py:203
↓ 3 callersFunctiontest_long_document_qa
Test long document QA with the given prompts and sampling parameters. Process prompts concurrently with a limit on inflight requests. Ar
benchmarks/long_doc_qa/long_doc_qa.py:292
↓ 3 callersMethodtime_to_transfer
(self)
lmcache/observability.py:239
↓ 3 callersFunctionto_blob
(kv_tuples)
tests/test_serde.py:38
↓ 3 callersMethodto_bytes
Serialize a pytorch tensor to bytes. The serialized bytes should contain both the data and the metadata (shape, dtype, etc.) of the t
lmcache/storage_backend/serde/cachegen_encoder.py:361
↓ 3 callersMethodto_worker_info
Convert to WorkerInfo for backward compatibility.
lmcache/v1/cache_controller/utils.py:230
↓ 3 callersMethodtracked_instance_count
(self)
tests/v1/multiprocess/test_worker_liveness.py:199
↓ 3 callersFunctionunpack_metadata
(buffer: bytes)
lmcache/v1/storage_backend/gds_backend.py:123
↓ 3 callersMethodunpin
(self)
lmcache/v1/memory_management.py:771
↓ 3 callersMethodunpin
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:362
↓ 3 callersMethodunregister_kv_cache
Unregister a non-GPU KV cache context for the given instance ID. Args: instance_id: The worker instance identifier.
lmcache/v1/multiprocess/modules/engine_driven_transfer.py:390
↓ 3 callersFunctionunwrap_kv_cache_tensors
(kv_caches: KVCache)
lmcache/v1/platform/cuda/cache_context.py:45
↓ 3 callersMethodupdate_heartbeat
Record a heartbeat timestamp for an instance. Args: instance_id: Identifier of the instance. timestamp: Monotonic-clo
lmcache/v1/mp_coordinator/registry.py:140
↓ 3 callersMethodvalidateAndSetCondition
validateAndSetCondition runs validation and updates the ConfigValid condition. Returns an error if validation fails (to stop reconciliation).
operator/internal/controller/reconcile_helpers.go:90
↓ 3 callersFunctionvalidate_max_output_length_supported
Validate that a max output length can be specified for ``workload``. Only workloads with a max-output-length parameter (``long-doc-qa``, ``mu
lmcache/cli/commands/bench/engine_bench/workloads/__init__.py:73
↓ 3 callersFunctionvalidate_mla_config
Validate MLA-related configuration.
lmcache/integration/vllm/utils.py:376
↓ 3 callersFunctionvllm_layout_hints
Build layout_hints dict by querying vLLM at runtime.
lmcache/integration/vllm/utils.py:39
↓ 3 callersMethodwait_and_resync
Poll the registry until an MP server registers, then resync. Returns the number of keys recorded; ``0`` if no MP server registered wi
lmcache/v1/mp_coordinator/l2/resync_manager.py:117
↓ 3 callersFunctionwait_eventfd
Block until the eventfd is signalled or *timeout* seconds elapse. Uses ``select.poll`` + ``consume_fd`` for cross-platform compatibility. Re
lmcache/cli/commands/bench/l2_adapter_bench/data.py:159
↓ 3 callersFunctionwait_for_condition
Poll until predicate returns True or timeout.
tests/v1/distributed/serde/test_serde_fs_e2e.py:53
↓ 3 callersMethodwait_prefetch_result
Block until a prefetch request's result is published, or until timeout. Thread-safe. Lets a handler wait for prefetch completion ins
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:422
↓ 3 callersMethodwait_prefetch_status
Block until a prefetch job completes, then return its chunk count. Like query_prefetch_status, but waits for the daemon to publish the
lmcache/v1/multiprocess/modules/lookup.py:381
↓ 3 callersMethodwrite_bytes
(self, offset: int, data: bytes)
tests/v1/shm_allocator/verify_shmfile_tcp.py:94
↓ 3 callersFunctionwrite_keys_to_l1
( l1_manager: L1Manager, keys: list[ObjectKey], layout: MemoryLayoutDesc, )
tests/v1/distributed/test_runtime_adapters.py:105
↓ 3 callersFunctionwrite_keys_to_l1
( l1_manager: L1Manager, keys: list[ObjectKey], layout: MemoryLayoutDesc, )
tests/v1/distributed/test_l1_l2_state_metrics.py:87
↓ 2 callersFunctionBuildConnectionConfigMap
BuildConnectionConfigMap creates the <name>-connection ConfigMap with kv-transfer-config JSON.
operator/internal/resources/configmap.go:41
↓ 2 callersFunctionBuildLookupService
BuildLookupService creates a ClusterIP Service with internalTrafficPolicy=Local for node-local service discovery. vLLM pods connect to this service an
operator/internal/resources/service.go:31
↓ 2 callersFunctionBuildMetricsService
BuildMetricsService creates a headless Service for Prometheus scraping.
operator/internal/resources/service.go:69
↓ 2 callersFunctionCoordinatorEndpoint
CoordinatorEndpoint returns the in-cluster base URL other components use to reach the coordinator.
operator/internal/resources/coordinator_service.go:43
↓ 2 callersFunctionCoordinatorServiceName
CoordinatorServiceName returns the name of the ClusterIP Service that fronts the coordinator. It equals the CR name so the in-cluster endpoint is the
operator/internal/resources/coordinator_service.go:37
↓ 2 callersFunctionCreateEvictionPolicy
Factory method to create an eviction policy based on the provided configuration. Args: eviction_config (EvictionConfig): The configu
lmcache/v1/distributed/eviction_policy/factory.py:13
↓ 2 callersMethodDeepCopy
DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CacheBlendEngine.
operator/api/v1alpha1/zz_generated.deepcopy.go:65
↓ 2 callersMethodGetStoreMetadata
Generate the store metadata for the current request tracker. Args: tracker: The request tracker to generate the metadata
lmcache/integration/vllm/lmcache_mp_connector.py:298
↓ 2 callersMethodGetStoreMetadata
Generate the store metadata for the current request tracker. Args: tracker: The request tracker to generate the metadata
lmcache/integration/vllm/lmcache_mp_connector_0201.py:301
↓ 2 callersMethodGetStoreMetadata
Generate the store metadata for the current request tracker. Args: tracker: The request tracker to generate the metadata
lmcache/integration/vllm/lmcache_mp_connector_0180.py:282
↓ 2 callersFunctionLoadFixture
LoadFixture returns the raw bytes of a fixture file by name (no path).
operator/test/utils/fixtures.go:39
↓ 2 callersFunctionLookupServiceName
LookupServiceName returns the name of the node-local lookup Service for discovery.
operator/internal/resources/configmap.go:31
↓ 2 callersMethodProcessSummary
( df: pd.DataFrame, start_time: Optional[float] = None, end_time: Optional[float] = No
benchmarks/multi_round_qa/multi-round-qa.py:470
↓ 2 callersFunctionRESPAuthSecretName
RESPAuthSecretName returns the name of the managed local copy of the RESP auth secret for a given LMCacheEngine.
operator/internal/resources/helpers.go:53
↓ 2 callersFunctionServiceMonitorEnabled
ServiceMonitorEnabled returns true if the ServiceMonitor should be created.
operator/internal/resources/servicemonitor.go:27
↓ 2 callersMethodWaitLoop
(cls)
benchmarks/multi_round_qa/utils.py:61
↓ 2 callersMethodWaitLoop
(cls)
benchmarks/rag/utils.py:69
↓ 2 callersMethod__exit__
(self, *_exc: object)
lmcache/cli/commands/trace/_driver.py:207
↓ 2 callersMethod__init__
(self, **kwargs)
tests/v1/distributed/test_l2_adapter_factory.py:302
↓ 2 callersMethod__init__
( self, config: Optional[LMCacheEngineConfig] = None, metadata: Optional[LMCacheMetada
lmcache/v1/token_database.py:76
↓ 2 callersMethod__init__
(self, loop: asyncio.AbstractEventLoop)
lmcache/v1/storage_backend/local_disk_backend.py:40
↓ 2 callersMethod_acquire_storage_handle
Open FDs, register the storage handler, and build the transfer handle. On any failure, releases everything already acquired (FDs, NIXL
lmcache/v1/storage_backend/nixl_storage_backend.py:1565
↓ 2 callersFunction_adapter_backend_name
(adapter: dict)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:164
↓ 2 callersMethod_add_adapter
Create and register a peer L2 adapter for ``inst``. Args: inst: The peer to connect to. Returns: ``True`` if
lmcache/v1/multiprocess/modules/p2p_controller.py:509
↓ 2 callersMethod_add_device_entry_locked
( self, *, device_path: str, size_bytes: int, )
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:1222
↓ 2 callersMethod_admit_track
Admit a newly-sampled entry; random-evict on overflow.
lmcache/v1/mp_observability/subscribers/metrics/sm_lifecycle.py:105
↓ 2 callersFunction_alloc
( shape: tuple[int, ...], a_dtype: torch.dtype, )
lmcache/cli/commands/bench/server_bench/helpers.py:237
↓ 2 callersMethod_alloc
(dtype: torch.dtype)
tests/cli/commands/bench/test_server_bench.py:336
↓ 2 callersFunction_alloc_page_aligned_pinned_view
Allocate a pinned CPU buffer whose first usable byte is page-aligned, and return a torch view of ``size`` bytes plus its base pointer. I
lmcache/python_ops_fallback.py:380
↓ 2 callersMethod_alloc_task_id
(self)
lmcache/v1/distributed/serde/async_processor.py:154
↓ 2 callersMethod_alloc_temp_buffers
Reserve one temp byte buffer per input key. All-or-nothing: any single failure releases the partial successes and returns ``(temp_keys
lmcache/v1/distributed/l2_adapters/serde_wrapper.py:579
↓ 2 callersFunction_allocate_kv_obj
( *, num_tokens: int, hidden_dim: int = 8, fill_value: int = 0, )
tests/v1/storage_backend/test_dax_backend.py:99
↓ 2 callersMethod_allocate_slot_locked
(self)
lmcache/v1/storage_backend/dax/core.py:998
↓ 2 callersFunction_api_url
(base: str, path: str)
lmcache/cli/commands/query/_request.py:78
↓ 2 callersFunction_as_flat_sglang_mha
Convert [[k_list], [v_list]] to flat [k0, ..., kN, v0, ..., vN].
tests/v1/test_xpu_sglang_connector.py:75
↓ 2 callersFunction_as_tensor_list
Return ``value`` as a flat tensor list, or ``None`` when unsupported.
lmcache/v1/platform/musa/native_kv_transfer.py:336
↓ 2 callersFunction_assert_disjoint
Assert that no two ``(start, end, label)`` byte ranges overlap.
tests/v1/platform/test_gpu_cache_context.py:164
↓ 2 callersMethod_assert_no_destination_duplicates_locked
( self, source: DaxDeviceEntry, keys: list[ObjectKey], )
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:1353
↓ 2 callersMethod_assert_worker_fields
(self, worker_data)
tests/v1/internal_api_server/test_worker_info_api.py:176
↓ 2 callersMethod_assert_worker_list_response
(self, response, expected_count)
tests/v1/internal_api_server/test_worker_info_api.py:189
↓ 2 callersMethod_async_remote_allocate
Send an allocation or cancellation request to the remote receiver. :param receiver_id: The remote receiver identifier. :param alloc_r
lmcache/v1/storage_backend/pd_backend_async.py:739
↓ 2 callersMethod_async_transfer_task
Perform a single-batch KV transfer: remote alloc → RDMA write → callbacks. Runs as an independent concurrent coroutine on _sender_loop. Multi
lmcache/v1/storage_backend/pd_backend_async.py:796
↓ 2 callersFunction_backend_adapter_entries
(status: dict, backend: str)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:171
↓ 2 callersFunction_bare_non_gpu_module
A EngineDrivenTransferModule with only the liveness state initialized.
tests/v1/multiprocess/test_worker_liveness.py:47
↓ 2 callersMethod_batched_get_prefix
( self, keys: Sequence[CacheEngineKey], )
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:508
↓ 2 callersMethod_batched_memcpy
( self, src_ptrs: Sequence[int], dst_ptrs: Sequence[int], sizes: Sequence[int]
lmcache/v1/storage_backend/plugins/dax_backend.py:680
↓ 2 callersMethod_batched_transfer
( self, memory_objs: List[MemoryObj], starts: List[int], block_ids: List[int],
lmcache/v1/gpu_connector/gpu_connectors.py:2143
↓ 2 callersFunction_best_ms
Best wall-clock time of ``fn`` over ``reps`` runs, in milliseconds.
benchmarks/microbenchmark/bitmap_ops_benchmark.py:33
↓ 2 callersFunction_bitmap_count
Count how many bits are set in *bitmap*. Returns 0 when None.
lmcache/cli/commands/bench/l2_adapter_bench/runner.py:45
↓ 2 callersMethod_block_ids_per_group
(self, op: LoadStoreOp)
lmcache/integration/vllm/vllm_multi_process_adapter.py:1262
↓ 2 callersFunction_block_ids_to_slot_mapping
Expand paged block IDs into token slot IDs on ``device``.
lmcache/v1/platform/musa/native_kv_transfer.py:380
↓ 2 callersMethod_blocked_payload_locked
( self, entry: DaxDeviceEntry, )
lmcache/v1/distributed/l2_adapters/dax_l2_adapter.py:1307
↓ 2 callersFunction_build_app
Create a fresh FastAPI app with all internal API routes registered.
lmcache/v1/internal_api_server/api_server.py:25
↓ 2 callersMethod_build_direct_odirect_view
Build an aligned memoryview for direct O_DIRECT I/O when possible. Args: memory_obj: Memory object whose backing allocation may b
lmcache/v1/storage_backend/raw_block/core.py:1015
↓ 2 callersFunction_build_engine
( *, name: str, cfg, connector, num_layers: int, chunk_size: int, num_heads: int,
tests/benchmarks/test_xpu_layerwise_connector_benchmark.py:267
↓ 2 callersFunction_build_fake_gpu_context
Returns a MagicMock matching the minimal GPUCacheContext surface used by _apply_cb_rope_batched.
tests/v1/multiprocess/test_blend_v3_load_store_opts.py:229
↓ 2 callersMethod_build_global_segments
Convert coordinator matches into chunk-granular retrievable segments. Each coordinator ``object_key`` is the hex of the chunk's content hash
lmcache/v1/multiprocess/modules/blend_v3.py:1316
↓ 2 callersFunction_build_huggingface_hub_error
Create a consistent runtime error for unsupported ``huggingface_hub``.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:899
↓ 2 callersMethod_build_l2_adapter
Create a L2 adapter instance based on the config. Args: config: The adapter configuration. Returns: A ``(ada
lmcache/v1/distributed/storage_manager.py:996
↓ 2 callersMethod_build_messages
Build chat messages for one request. The breaker is freshly randomized on every call, so two requests for the same prefix produce dif
lmcache/cli/commands/bench/engine_bench/workloads/prefix_suffix_tuner.py:517
↓ 2 callersMethod_build_operation_distribution
Build operation distribution list based on percentages
lmcache/tools/controller_benchmark/benchmark.py:440
↓ 2 callersFunction_build_resource
Build an OTel ``Resource`` from the given attribute dict. Returns an empty ``Resource`` when *resource_attributes* is empty or ``None`` so th
lmcache/v1/mp_observability/otel_init.py:30
↓ 2 callersFunction_build_round_keys
Build per-submit key batches for round *r* (store/load).
lmcache/cli/commands/bench/l2_adapter_bench/command.py:252
↓ 2 callersFunction_build_round_objs
Build per-submit object batches backed by the registered L1 buffer. Some adapters register the L1 buffer passed through ``L1MemoryDesc``
lmcache/cli/commands/bench/l2_adapter_bench/command.py:260
↓ 2 callersFunction_build_server_slot_views
Build zero-copy tensor views over server SHM slot descriptors. Each ``ShmSlotDescriptor`` carries the ``(offset, length, shape, dtype)`` of o
lmcache/cli/commands/bench/server_bench/helpers.py:505
↓ 2 callersMethod_build_setup_dict
Extract mooncake setup keys from *raw* config dict. Rules (evaluated in order): 1. Keys in ``_LMCACHE_ONLY_KEYS`` are skipped.
lmcache/v1/storage_backend/connector/mooncakestore_connector.py:245
↓ 2 callersMethod_build_slot_tensors
(self, slots: list[dict[str, Any]])
lmcache/v1/multiprocess/transfer_context/shm.py:133
↓ 2 callersMethod_bump_salt_counter
Advance and return the per-salt access counter.
lmcache/v1/mp_observability/subscribers/metrics/sm_lifecycle.py:99
↓ 2 callersFunction_byte_region
Return ``(start_ptr, end_ptr)`` covering a tensor's bytes.
tests/v1/platform/test_gpu_cache_context.py:158
↓ 2 callersMethod_calculate_checksum
Calculate SHA-256 checksum for data validation
lmcache/v1/storage_backend/connector/audit_connector.py:235
↓ 2 callersFunction_call_sm_method
Build a handler that forwards a record to ``sm.<method_name>``. The returned callable invokes ``getattr(ctx.sm, method_name)(**args)`` and di
lmcache/cli/commands/trace/_dispatch.py:141
↓ 2 callersMethod_cb_store_gpu_copy
Helper function to perform GPU-to-CPU copy operations for storing chunks. Args: obj_keys: List of object keys to store.
lmcache/v1/multiprocess/modules/blend.py:681
↓ 2 callersFunction_check_lmcache_engine
Check if LMCache engine is available. Args: request: FastAPI request object Returns: Tuple of (lmcache_engine, error_respons
lmcache/v1/internal_api_server/vllm/cache_api.py:82
↓ 2 callersMethod_check_socket_has_pending
Check if socket has pending messages. Returns: 1 if socket has pending messages, 0 otherwise
lmcache/v1/cache_controller/controller_manager.py:311
↓ 2 callersMethod_chunk
Return [(start, end, key)] for token_ids using the engine TDB. The list is materialized so we can iterate it twice (key check and per
lmcache/v1/hidden_state_store.py:310
← previousnext →2,101–2,200 of 13,371, ranked by callers