MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 26 callersFunction_make_result
( request_id: str = "req_0", successful: bool = True, ttft: float = 0.3, request_latency: floa
tests/cli/commands/bench/engine_bench/test_stats.py:25
↓ 26 callersFunction_make_workload
( config: LongDocPermutatorConfig | None = None, seed: int = 42, )
tests/cli/commands/bench/engine_bench/workloads/test_long_doc_permutator.py:261
↓ 26 callersMethodclose
Release restore workers, staging buffers, and DAX resources.
lmcache/v1/storage_backend/plugins/dax_backend.py:599
↓ 26 callersFunctioncreate_memory_obj
( *, shape: torch.Size = _DEFAULT_SHAPE, dtype: torch.dtype = torch.bfloat16, fill_value: floa
tests/v1/distributed/test_dax_l2_adapter.py:83
↓ 26 callersFunctioncreate_object_key
Create a test ObjectKey with the given chunk ID.
tests/v1/distributed/test_nixl_store_dynamic_l2_adapter.py:74
↓ 26 callersMethodis_available
(self)
tests/v1/test_musa_support.py:67
↓ 26 callersMethodmemcheck
(self)
tests/v1/storage_backend/test_gds_backend.py:508
↓ 26 callersMethodregister
Register a check mode function
lmcache/v1/check/__init__.py:23
↓ 26 callersMethodsubmit_prefetch_task
Prefetch objects into L1 asynchronously. Args: keys: Object keys to prefetch. layout_desc: Memory layout description.
lmcache/v1/distributed/storage_manager.py:395
↓ 26 callersMethodupdate_on_hit
( self, key: KeyType, cache_dict: dict[KeyType, Any], )
lmcache/v1/storage_backend/cache_policy/lfu.py:38
↓ 25 callersMethod_create_dynamic_gauge
( self, name: str, documentation: str, labelnames: List[str], multipro
lmcache/observability.py:1467
↓ 25 callersFunction_l1_read
(keys: list)
tests/v1/mp_observability/subscribers/metrics/test_sm_lifecycle.py:59
↓ 25 callersFunction_make_event
( event_type: EventType = EventType.L1_READ_FINISHED, session_id: str = "s1", **metadata, )
tests/v1/mp_observability/test_event_bus.py:28
↓ 25 callersMethodbatched_submit_put_task
Submit a batched put task.
tests/v1/storage_backend/test_storage_plugin.py:81
↓ 25 callersFunctioncreate_memory_obj
(size: int = 1024, fill_value: float = 1.0)
tests/v1/distributed/test_native_connector_l2_adapter.py:171
↓ 25 callersFunctioncreate_object_key
Create an ObjectKey for testing.
tests/v1/distributed/test_object_key_parallel.py:95
↓ 25 callersFunctionnormalize_kv_and_discover_format
Normalize ``kv_caches`` into canonical form and discover its Engine KV format. Thin wrapper over :func:`lmcache.v1.gpu_connector.kv_format.de
lmcache/v1/gpu_connector/utils.py:202
↓ 25 callersMethodput
Put a value.
tests/v1/storage_backend/test_remote_storage_plugin.py:58
↓ 25 callersMethodremove
Remove a session by request_id. Args: request_id: Unique request identifier. Returns: The removed session, o
lmcache/v1/multiprocess/session.py:148
↓ 25 callersMethodstart
Start the background drain thread. No-op when disabled or already running.
lmcache/v1/mp_observability/event_bus.py:223
↓ 25 callersFunctionwait_for_prefetch_result
Poll query_prefetch_result until it returns a non-None value.
tests/v1/distributed/test_prefetch_controller.py:92
↓ 24 callersMethod_log_operation
Helper method to log operation results.
lmcache/v1/storage_backend/audit_backend.py:29
↓ 24 callersFunction_make_full_state
Build a state with all required fields set.
tests/cli/commands/bench/engine_bench/interactive/test_state.py:18
↓ 24 callersFunction_make_key
(salt: str, model: str = "m", rank: int = 0, h: str = "aa")
tests/v1/mp_coordinator/test_eviction_manager.py:21
↓ 24 callersFunction_make_workload_config
(**overrides)
tests/cli/commands/bench/engine_bench/workloads/test_prefix_suffix_tuner.py:197
↓ 24 callersMethodallocate
( self, shapes: Union[torch.Size, list[torch.Size]], dtypes: Union[torch.dtype, list[t
lmcache/v1/storage_backend/pd_backend.py:349
↓ 24 callersFunctioncreate_object_key
( chunk_id: int, model_name: str = "test_model", cache_salt: str = "", )
tests/v1/distributed/test_dax_l2_adapter.py:70
↓ 24 callersMethodfileno
(self)
lmcache/v1/platform/event_notifier.py:190
↓ 24 callersFunctionnewEngine
newEngine builds an LMCacheEngine fixture and Creates it in the cluster (which populates APIVersion / Kind / UID — required for ctrl.SetControllerRefe
operator/internal/controller/reconcile_helpers_test.go:74
↓ 24 callersMethodpublish_on_stream
Schedule event recording as a CUDA host function on *stream*. Uses a C++ callback via ``cudaLaunchHostFunc`` so the callback never to
lmcache/v1/mp_observability/event_bus.py:167
↓ 24 callersMethodquery_prefetch_status
Poll the status of a prefetch job by request_id. Returns the chunk count when the prefetch is complete, or None if it is still in pro
lmcache/v1/multiprocess/modules/lookup.py:324
↓ 23 callersMethod_create_counter
Create a Counter and register it for reset_counters().
lmcache/observability.py:861
↓ 23 callersMethod_log_counter
(self, counter, data: Union[int, float])
lmcache/observability.py:1681
↓ 23 callersFunction_make_mem_obj
(idx: int = 0)
tests/v1/storage_backend/test_pd_backend_async.py:62
↓ 23 callersMethod_notify_keys_stored
Update byte accounting and notify listeners that ``keys`` were stored. ``sizes[i]`` is the byte size of ``keys[i]``. Accounting is he
lmcache/v1/distributed/l2_adapters/base.py:357
↓ 23 callersFunction_store
Helper: record the bytes against the usage ledger and register the key in the eviction LRU — the two calls that the production ``/l2/events``
tests/v1/mp_coordinator/test_eviction_manager.py:45
↓ 23 callersMethodbatched_submit_put_task
Store a batch of memory objects in the DAX arena. Args: keys: Cache keys to store. objs: Memory objects whose content
lmcache/v1/storage_backend/plugins/dax_backend.py:320
↓ 23 callersMethodclose
Close the memory manager and release all resources.
lmcache/v1/distributed/memory_manager/l1_memory_manager.py:230
↓ 23 callersFunctionconsume_fd
Consume a pending signal from a raw file descriptor. This is a convenience function for code that only has a raw fd (e.g., obtained from ``ad
lmcache/v1/platform/event_notifier.py:235
↓ 23 callersMethodcurrent_stream
Return the current stream for the given device. Args: device: The device to query. Ignored in the stub. Returns:
lmcache/v1/platform/cpu/stub_cpu_device.py:240
↓ 23 callersMethodfrom_metadata
Create a connector from :class:`LMCacheMetadata`. Args: metadata: Metadata carrying ``kv_shape`` ``(num_layers, 2
lmcache/v1/gpu_connector/gpu_connectors.py:1990
↓ 23 callersMethodget
Return the current byte usage for ``cache_salt``.
lmcache/v1/mp_coordinator/l2/usage_manager.py:95
↓ 23 callersMethodget_subscriptions
(self)
tests/v1/mp_observability/subscribers/metrics/test_event_bus_self_metrics.py:63
↓ 23 callersMethodis_valid
Check if the MemoryObj is valid.
lmcache/v1/memory_management.py:215
↓ 23 callersMethodset_tokens
Update the token sequence (idempotent, replaces not extends). Args: full_token_ids: Complete token sequence.
lmcache/v1/multiprocess/session.py:40
↓ 22 callersFunction_make_app
Build a FastAPI app with only the l2_keys router mounted and the fake engine bolted onto ``app.state``.
tests/v1/multiprocess/http_apis/test_l2_api.py:84
↓ 22 callersFunctioncheck_mem_obj_equal
check whether two memory objects are the same
tests/v1/utils.py:405
↓ 22 callersMethodclose
(self)
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:659
↓ 22 callersFunctioncreate_object_key
(chunk_id: int, model_name: str = "test_model")
tests/v1/distributed/test_s3_l2_adapter.py:331
↓ 22 callersFunctiondevice_sync
Synchronize device operations. Args: device: Device string ("cuda", "xpu", or "cpu") This function synchronizes operations for GPU d
tests/v1/test_python_ops_fallback.py:26
↓ 22 callersMethodget_stats_and_clear
This function should be called with by prometheus adapter with a specific interval. The function will return the latest state
lmcache/observability.py:666
↓ 22 callersMethodinsert
Insert a new entry, or refresh if already present. O(log n).
lmcache/tools/cache_simulator/lru_cache.py:109
↓ 22 callersFunctionlog
(msg: str)
lmcache/cli/commands/bench/l2_adapter_bench/command.py:187
↓ 22 callersMethodon_request_finished
Record a completed request. Thread-safe.
lmcache/cli/commands/bench/engine_bench/stats.py:92
↓ 22 callersMethodrecord_stored
Record that ``key`` is resident on L2 at ``num_bytes``. Re-storing the same key replaces its size (delta-adjusts the per-salt and glo
lmcache/v1/mp_coordinator/l2/usage_manager.py:37
↓ 22 callersMethodstop
Stop the drain thread, flush remaining events, and shut down all registered subscribers. Safe to call when not started.
lmcache/v1/mp_observability/event_bus.py:240
↓ 22 callersFunctionwait_for_event_fd
(event_fd: int, timeout: float = 10.0)
tests/v1/distributed/test_mooncake_store_l2_adapter.py:120
↓ 21 callersFunction_build_manager
Build a manager using the per-layer NHD format. Tensors in these tests have shape ``[2, NB, BS, NH, HS]`` — the canonical vLLM flash-attentio
tests/v1/test_kv_layer_groups_manager.py:28
↓ 21 callersFunction_key
(chunk: int = 0, salt: str = "a")
tests/v1/mp_coordinator/test_usage_manager.py:12
↓ 21 callersFunction_make_controller
Build a P2P-disabled controller (no thread / transfer channel).
tests/v1/multiprocess/test_p2p_controller.py:140
↓ 21 callersFunction_workload_is
Return a condition that checks the workload value.
lmcache/cli/commands/bench/engine_bench/interactive/schema.py:74
↓ 21 callersMethodcontains
(self, key: CacheEngineKey, pin: bool = False)
lmcache/v1/storage_backend/remote_backend.py:162
↓ 21 callersMethodcreate
Create health check instance(s) from a LMCacheManager. This method should extract the necessary components from the manager
lmcache/v1/health_monitor/base.py:129
↓ 21 callersFunctioncreate_test_key
(key_id: int = 0)
tests/v1/storage_backend/test_gds_backend.py:54
↓ 21 callersFunctioncreate_test_key
Create a test CacheEngineKey.
tests/v1/storage_backend/test_local_cpu_backend.py:58
↓ 21 callersMethodfrom_cache_engine_config
Convert the LMCacheEngineConfig to PDConfig
lmcache/v1/storage_backend/pd_backend.py:104
↓ 21 callersMethodfrom_string
(s)
lmcache/utils.py:489
↓ 21 callersMethodget_eviction_actions
Get the eviction actions to evict objects from L1 cache. Returns keys in LRU order (least recently used first). Args:
lmcache/v1/distributed/eviction_policy/lru.py:128
↓ 21 callersFunctionlog
Print progress messages; suppressed by --quiet.
lmcache/cli/commands/bench/server_bench/command.py:229
↓ 21 callersFunctionmake_descriptor
Create an AdapterDescriptor for testing.
tests/v1/distributed/test_prefetch_policy.py:35
↓ 21 callersFunctionmake_object_key
Create a test ObjectKey with the given chunk ID.
tests/v1/distributed/test_store_controller.py:53
↓ 21 callersMethodreg_worker
Register a worker.
tests/v1/cache_controller/conftest.py:162
↓ 21 callersMethodretrieve
Retrieve the KV caches from the cache engine. And put the retrieved KV cache to the serving engine via the GPU connector. :param torc
lmcache/v1/cache_engine.py:777
↓ 21 callersMethodset_ping_error_code
(self, error_code: int)
tests/v1/test_health_monitor_fallback_recovery.py:41
↓ 21 callersMethodto_bytes
Concatenate header + payload for on-disk write.
lmcache/v1/kv_codec/asym_k16_v8.py:374
↓ 20 callersMethodValidateSpec
ValidateSpec validates the LMCacheEngineSpec and returns any validation errors.
operator/api/v1alpha1/lmcacheengine_validation.go:24
↓ 20 callersFunction_client
()
tests/v1/mp_coordinator/test_l2_api.py:12
↓ 20 callersMethod_create_histogram
Create a Histogram and register it for reset_histograms(). If the ``config`` object's extra_config contains a key ``histogram_bucket_
lmcache/observability.py:871
↓ 20 callersMethod_log_histogram
(self, histogram, data: Union[List[int], List[float]])
lmcache/observability.py:1688
↓ 20 callersMethod_run_all_checks
Run all health checks. Returns: bool: True if all checks pass (considering fallback policies), False i
lmcache/v1/health_monitor/base.py:437
↓ 20 callersFunction_total_count
(name: str)
tests/v1/mp_observability/subscribers/metrics/test_l2_throughput.py:136
↓ 20 callersFunctionderefString
derefString returns the value pointed to by p, or def if p is nil.
operator/internal/resources/helpers.go:28
↓ 20 callersMethodexists
(self, key)
tests/v1/test_remote_mla_worker_id_as0.py:26
↓ 20 callersMethodget_limit_bytes
Return the effective limit for ``cache_salt``. Unregistered salts resolve to ``0`` (allowlist semantics) — callers use this value to
lmcache/v1/distributed/quota_manager.py:64
↓ 20 callersFunctionhex_hash_to_int16
Convert a hash identifier into a 16-bit integer. Historically, LMCache expected multimodal identifiers to be hex strings. In practice (e
lmcache/integration/vllm/utils.py:143
↓ 20 callersFunctionipc_key_to_object_keys
Convert a single IPCCacheServerKey and its chunk hashes to per-object-group lists of ObjectKey. When the ipc_key's worker_id is None, ea
lmcache/v1/distributed/api.py:273
↓ 20 callersMethodis_running
Check if the thread is currently running.
lmcache/v1/periodic_thread.py:188
↓ 20 callersFunctionmake_bitmap
Create a Bitmap with specific bits set.
tests/v1/distributed/test_prefetch_policy.py:41
↓ 20 callersFunctionmake_object_key
Helper to create ObjectKey instances.
tests/v1/distributed/test_distributed_storage_manager.py:138
↓ 20 callersFunctionminimalEngine
()
operator/internal/resources/resources_test.go:52
↓ 20 callersMethodnotify
Wake the polling loop to process outbound tasks.
lmcache/v1/multiprocess/mq.py:208
↓ 20 callersMethodput
( self, key: CacheEngineKey, mem_obj: MemoryObj, )
lmcache/v1/storage_backend/pd_backend.py:977
↓ 20 callersMethodremove
Remove a key from storage.
tests/v1/storage_backend/test_storage_plugin.py:108
↓ 20 callersMethodset
(self, key: str, value: Any)
lmcache/cli/commands/bench/engine_bench/interactive/state.py:74
↓ 20 callersMethodstart
(self, operation_name)
lmcache/v1/storage_backend/connector/eic_connector.py:46
↓ 20 callersMethodstart_sync
Start sync for a worker. Returns: True if sync started successfully, False otherwise
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:129
↓ 20 callersMethodstop
Signal the loop to stop, wait for the thread to join. Releases all in-flight read locks on shutdown so that L1 objects are n
lmcache/v1/distributed/storage_controllers/store_controller.py:308
↓ 20 callersMethodstore_hidden_states
Store hidden_states chunked under the same keys as KV. Args: token_ids: 1-D int tensor or list of the **full** token-ID prefix
lmcache/v1/hidden_state_store.py:115
↓ 19 callersMethodValidateSpec
ValidateSpec validates the CacheBlendEngineSpec and returns any validation errors. It mirrors LMCacheEngine.ValidateSpec and additionally enforces the
operator/api/v1alpha1/cacheblendengine_validation.go:27
↓ 19 callersFunction_make_config
(**overrides)
tests/cli/commands/bench/engine_bench/workloads/test_long_doc_permutator.py:223
← previousnext →201–300 of 13,371, ranked by callers