MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 11 callersFunctionwait_for_event_fd
(event_fd: int, timeout: float = 10.0)
tests/v1/distributed/test_resp_l2_adapter_integration.py:90
↓ 10 callersMethodDestroyInstance
Destroy the singleton instance and stop monitoring. This is mainly used for testing to ensure clean state between tests.
lmcache/v1/pin_monitor.py:253
↓ 10 callersMethodSetDefaults
SetDefaults applies defaults that cannot be expressed purely via kubebuilder markers.
operator/api/v1alpha1/lmcacheengine_defaults.go:28
↓ 10 callersFunction_auto_alloc
Allocator that returns a fresh MemoryObj stub on every call.
tests/v1/storage_backend/test_pd_backend_async.py:127
↓ 10 callersMethod_create_server
Helper to create a lookup server with transport.
tests/v1/lookup_client/test_lmcache_lookup_client.py:97
↓ 10 callersFunction_end_event
( event_type: EventType, session_id: str, t: float, total_bytes: int, engine_id: int = 0,
tests/v1/mp_observability/subscribers/metrics/test_l0_l1_throughput.py:54
↓ 10 callersFunction_entry
Wire shape of one ``KeyListPage`` entry: nested ``key`` plus a sibling ``size_bytes``.
tests/v1/mp_coordinator/test_resync_manager.py:37
↓ 10 callersFunction_get_histogram_count
(name: str)
tests/v1/mp_observability/subscribers/metrics/test_l0_lifecycle.py:80
↓ 10 callersMethod_get_sync_info
Get sync info for a worker from the registry tree.
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:52
↓ 10 callersFunction_key
(chunk_id: int, cache_salt: str)
tests/v1/distributed/test_cache_salt_l2_eviction.py:50
↓ 10 callersMethod_key_to_path
( self, key: CacheEngineKey, base_path: Optional[str] = None, )
lmcache/v1/storage_backend/gds_backend.py:531
↓ 10 callersFunction_make_context
Build a real ``GPUCacheContext`` via its public constructor.
tests/v1/platform/test_gpu_cache_context.py:190
↓ 10 callersFunction_make_key
A CacheEngineKey whose to_string() contains both '/' and '@'.
tests/v1/test_nixl_doca_memos.py:41
↓ 10 callersFunction_make_key
(i: int)
tests/v1/multiprocess/test_p2p_controller.py:35
↓ 10 callersFunction_make_metadata
Create a minimal mock LMCacheMetadata for testing.
tests/test_observability.py:293
↓ 10 callersFunction_make_temp_buffer
Build a ``_TempGPUBuffer`` backed by a real manager.
tests/v1/platform/test_gpu_cache_context.py:109
↓ 10 callersFunction_make_workload
( config: RandomPrefillConfig | None = None, seed: int = 42, )
tests/cli/commands/bench/engine_bench/workloads/test_random_prefill.py:106
↓ 10 callersFunction_memory_obj
Small (~0.5 KB with 128 floats) tensor so quotas fit comfortably inside the mock adapter's 1 MB capacity.
tests/v1/distributed/test_cache_salt_l2_eviction.py:59
↓ 10 callersFunction_poll_fd
(fd, timeout_ms=500)
tests/v1/native_storage_ops/test_periodic_event_notifier.py:64
↓ 10 callersFunction_populate_basic_pkg
A minimal layout used by the majority of the tests: base.py defines Base + AbstractChild, child_a.py & child_b.py define one concrete subclass
tests/v1/test_subclass_discovery.py:50
↓ 10 callersFunction_populate_nested_pkg
Layout exercising multi-level discovery: * depth 1: ``base.py`` (Base + AbstractChild), ``leaf_top.py`` (TopChild) * depth 1 sub-pkg ``mid/``
tests/v1/test_subclass_discovery.py:333
↓ 10 callersFunction_retrieve_end
( retrieved_count: int, engine_id: int = 0, model_name: str = "test-model", cache_salt: str =
tests/v1/mp_observability/subscribers/metrics/test_engine.py:28
↓ 10 callersMethod_send_message
Send message to controller via ZMQ For register: uses DEALER socket (DEALER-ROUTER mode), returns response For heartbeat: uses DEALER
tests/v1/internal_api_server/test_worker_info_api.py:114
↓ 10 callersFunction_start_event
( event_type: EventType, session_id: str, t: float, engine_id: int = 0, device: str = "cud
tests/v1/mp_observability/subscribers/metrics/test_l0_l1_throughput.py:34
↓ 10 callersFunction_store_sync
Submit a store and wait for it to land in the base-class accounting (``_notify_keys_stored`` fires in ``_execute_store_in_the_loop``).
tests/v1/distributed/test_cache_salt_l2_eviction.py:88
↓ 10 callersFunction_sum
(name: str)
tests/v1/mp_observability/subscribers/metrics/test_l2_throughput.py:141
↓ 10 callersMethod_tensor
( self, nb: int = 4, nl: int = 3, kv: int = 2, nh: int = 8, bs
tests/v1/test_trtllm_integration.py:183
↓ 10 callersFunctioncheck_interprocess_event_support
Check if the current backend supports interprocess Events. This function checks if torch_dev.Event exists and exposes the interprocess parame
lmcache/utils.py:49
↓ 10 callersMethodclose
(self)
tests/v1/distributed/test_l2_adapter_factory.py:717
↓ 10 callersMethodclose
(self)
tests/v1/storage_backend/test_rust_raw_block_backend.py:142
↓ 10 callersMethodclose
Close the notifier.
lmcache/v1/distributed/storage_controllers/store_controller.py:155
↓ 10 callersMethodclose
Shut down the thread pool and close event notifiers.
lmcache/v1/distributed/serde/async_processor.py:146
↓ 10 callersMethodconsume
(self)
lmcache/v1/platform/event_notifier.py:202
↓ 10 callersMethodcontains
(self, key: Any)
lmcache/tools/cache_simulator/lru_cache.py:42
↓ 10 callersFunctioncreate_workload
Resolve workload-specific config and create the workload instance. Dispatches on ``config.workload`` to the appropriate workload module, reso
lmcache/cli/commands/bench/engine_bench/workloads/__init__.py:95
↓ 10 callersFunctioncurrent_v3_lookup
Mirror of the V3 lookup body (without the storage-prefetch round).
tests/v1/multiprocess/test_optimized_lookup_v3.py:57
↓ 10 callersFunctiondiscover_api_routers
Scan *search_path* for modules whose name ends with *suffix* and return every ``router`` attribute that is an :class:`~fastapi.APIRouter`.
lmcache/v1/utils/router_discovery.py:17
↓ 10 callersFunctionencode_object_key
Encode an MP ObjectKey for raw-block storage. Args: key: Object key supplied by the MP storage layer. Returns: Encoded key a
lmcache/v1/storage_backend/raw_block/key_codec.py:84
↓ 10 callersMethodfill_defaults
Set all unset items (whose conditions are met) to their defaults.
lmcache/cli/commands/bench/engine_bench/interactive/state.py:161
↓ 10 callersFunctionget_correct_device
Get the correct device based on the given device string. Args: device (str): The device string, could be cpu or the accelerator type
lmcache/v1/transfer_channel/transfer_utils.py:9
↓ 10 callersFunctionget_l1_memory_used
Return current L1 memory usage in bytes via public report_status.
tests/v1/distributed/serde/test_serde_e2e.py:188
↓ 10 callersMethodget_num_connected_clients
(self)
tests/v1/distributed/transfer_channel/test_factory.py:54
↓ 10 callersFunctionget_payload_classes
Get the expected payload classes for a request type. Args: req_type: The request type to look up Returns: List of expec
lmcache/v1/multiprocess/protocol.py:32
↓ 10 callersMethodget_total
Return total bytes tracked across all salts.
lmcache/v1/mp_coordinator/l2/usage_manager.py:105
↓ 10 callersMethodhandle_full_sync_batch
Handle full sync batch message from a worker. This adds the keys from the batch to the registry.
lmcache/v1/cache_controller/controllers/kv_controller.py:249
↓ 10 callersMethodhash_to_bytes
Convert hash value to bytes for ObjectKey.chunk_hash. Handles both bytes (sha256_cbor) and int (builtin hash) return types.
lmcache/v1/multiprocess/token_hasher.py:233
↓ 10 callersMethodinitialize_kvcaches_ptr
Initialize the kvcaches pointers if not already initialized.
lmcache/v1/gpu_connector/gpu_connectors.py:134
↓ 10 callersFunctioninitialize_transfer_channel_context
Create the global transfer channel context. Args: transfer_channel_type: Currently only ``"nixl"`` is supported. l1_memory_desc:
lmcache/v1/distributed/transfer_channel/__init__.py:45
↓ 10 callersFunctionis_mla
Return ``True`` for a Multi-head Latent Attention (MLA) layout.
lmcache/v1/gpu_connector/utils.py:104
↓ 10 callersFunctionis_tracing_enabled
Return whether the trace gate is currently on.
lmcache/v1/mp_observability/trace/decorator.py:44
↓ 10 callersMethodlist_l2_keys
( self, model_name: Optional[str] = None, page_size: int = 500, cursor: Option
tests/v1/multiprocess/http_apis/test_l2_api.py:51
↓ 10 callersFunctionmake_json_safe
Recursively convert ``obj`` into a JSON-serializable value. Dicts, lists and tuples are traversed; primitive JSON types are returned unchange
lmcache/v1/utils/json_utils.py:16
↓ 10 callersFunctionmigrate_to_shm_and_wrap
Re-point ``tensor``'s storage at a POSIX SHM segment, then wrap. Used as the registered ``"cpu"`` KV-wrapper factory: the LMCache mp server c
lmcache/v1/platform/cpu/shm.py:193
↓ 10 callersMethodon_keys_created
(self, keys: list[ObjectKey])
lmcache/v1/distributed/eviction_policy/isolated_lru.py:66
↓ 10 callersFunctionprint_banner_once
Print the LMCache startup banner to ``stream`` at most once. The banner shows the LMCache logo, version, and website, followed by a hint desc
lmcache/banner.py:100
↓ 10 callersMethodreconcileRESPAuthSecret
reconcileRESPAuthSecret ensures a local copy of the RESP auth secret exists in the engine's namespace. If the source secret is in the same namespace,
operator/internal/controller/reconcile_helpers.go:422
↓ 10 callersMethodregister_kv_cache_engine_driven_context
Register non-CUDA KV layout metadata for non-GPU context mode. Args: payload: Struct containing all registration fields
lmcache/v1/multiprocess/modules/engine_driven_transfer.py:300
↓ 10 callersMethodretrieve_hidden_states
Retrieve cached rows for layer_idx as a contiguous prefix. Walks chunks of token_ids in order. Stops at the first chunk where either
lmcache/v1/hidden_state_store.py:227
↓ 10 callersFunctionsafe_get
Walk nested dicts by *keys*, returning *default* on any miss.
lmcache/cli/commands/describe.py:79
↓ 10 callersMethodset
(self, key: str, buf: memoryview)
lmcache/v1/storage_backend/native_clients/connector_client_base.py:100
↓ 10 callersMethodset_backend_bypass
Set bypass mode for a specific backend. When a backend is bypassed: - It will be skipped during contains/put/get operations
lmcache/v1/storage_backend/storage_manager.py:880
↓ 10 callersMethodshutdown
Close the current file handle on EventBus shutdown.
lmcache/v1/mp_observability/subscribers/logging/lookup_hash.py:132
↓ 10 callersFunctionsimulate
Replay lookup events through an LRU cache and compute token hit-rate statistics. Parameters ---------- events: Lookup ev
lmcache/tools/cache_simulator/simulator.py:176
↓ 10 callersMethodto_gpu
Expect a kwarg 'kvcaches' which is a nested tuple of K and V tensors. The kvcaches should correspond to the "WHOLE token sequence". N
lmcache/v1/gpu_connector/gpu_connectors.py:1503
↓ 10 callersMethodtoken_dim
(self)
lmcache/v1/memory_management.py:88
↓ 10 callersMethodupdate_local_cache_usage
(self, usage: int)
lmcache/observability.py:519
↓ 9 callersFunctionCreateStorageBackends
( config: LMCacheEngineConfig, metadata: LMCacheMetadata, loop: asyncio.AbstractEventLoop, dst
lmcache/v1/storage_backend/__init__.py:111
↓ 9 callersFunction_attrs
Build the sorted-tuple attribute key the subscriber emits.
tests/v1/mp_observability/subscribers/metrics/test_engine.py:49
↓ 9 callersMethod_create_client
Helper to create a lookup client with transport.
tests/v1/lookup_client/test_lmcache_lookup_client.py:104
↓ 9 callersFunction_default_key
Build a default IPC cache key with ``tokens`` contiguous token IDs. Args: tokens: Total token count and key end offset. Defaults to ``8``
tests/v1/multiprocess/test_engine_driven_transfer.py:204
↓ 9 callersFunction_default_register_payload
Build a default non-GPU registration payload for server-side tests. Args: instance_id: Worker instance id to register. Defaults to ``1``.
tests/v1/multiprocess/test_engine_driven_transfer.py:177
↓ 9 callersFunction_find_arg
Return the value following ``name`` in ``argv``, or ``None``.
tests/lmcache_frontend/lmcache_mp_plugin/test_lmcache_mp_frontend_plugin.py:18
↓ 9 callersMethod_log_gauge
(self, gauge, data: Union[int, float])
lmcache/observability.py:1677
↓ 9 callersFunction_make_args
(**overrides)
tests/cli/commands/bench/engine_bench/workloads/test_create_workload.py:49
↓ 9 callersFunction_make_config
(**overrides)
tests/cli/commands/bench/engine_bench/workloads/test_create_workload.py:32
↓ 9 callersFunction_make_config
(**overrides)
tests/cli/commands/bench/engine_bench/workloads/test_random_prefill.py:72
↓ 9 callersFunction_make_deps
()
tests/cli/commands/bench/engine_bench/workloads/test_create_workload.py:76
↓ 9 callersFunction_make_event
Helper to create a lookup event with metadata.
tests/v1/mp_observability/subscribers/logging/test_lookup_hash_logger.py:52
↓ 9 callersFunction_make_req
(req_id: str, can_save: bool = True)
tests/v1/test_vllm_layerwise_wait_for_save.py:55
↓ 9 callersFunction_make_stub
(**kwargs)
tests/cli/commands/bench/engine_bench/workloads/test_base_workload.py:47
↓ 9 callersMethod_notify_keys_accessed
(self, keys: list[ObjectKey])
lmcache/v1/distributed/l2_adapters/base.py:383
↓ 9 callersMethod_on_store_start
(self, event: Event)
lmcache/v1/mp_observability/subscribers/logging/mp_server.py:36
↓ 9 callersFunction_parse_local_disk
Parse local disk path configuration. Accepts a single path or comma-separated paths, each optionally prefixed with ``file://``. Returns a co
lmcache/v1/config_base.py:50
↓ 9 callersFunction_peer
( instance_id: str, url: str = "tc-host:9", ip: str = "10.0.0.2", mq_port: int = 5555, )
tests/v1/multiprocess/test_p2p_controller.py:283
↓ 9 callersFunction_percentile
Compute the p-th percentile using linear interpolation. Uses the Tensormesh-Benchmark V1 method: ``k = (len(sorted_data) - 1) * p / 100``
lmcache/cli/commands/bench/engine_bench/stats.py:251
↓ 9 callersFunction_publish_and_drain
Publish an event and synchronously drain it.
tests/v1/mp_observability/subscribers/logging/test_lookup_hash_logger.py:77
↓ 9 callersFunction_run_lookup
(adapter: RawBlockL2Adapter, keys)
tests/v1/distributed/test_raw_block_l2_adapter.py:215
↓ 9 callersFunction_send_request
( mq_client: MessageQueueClient, request_type: RequestType, payloads: list, )
lmcache/integration/tensorrt_llm/tensorrt_mp_adapter.py:58
↓ 9 callersFunction_store
(salt: str, nbytes: int, **kw)
tests/v1/mp_coordinator/test_l2_api.py:26
↓ 9 callersFunction_usage
(prompt: int = 100, completion: int = 2)
tests/cli/commands/bench/engine_bench/test_request_sender.py:88
↓ 9 callersMethodacquire
Acquire the lock with optional timeout.
lmcache/v1/cache_controller/locks.py:127
↓ 9 callersMethodacquire_write
Acquire a write lock with optional timeout.
lmcache/v1/cache_controller/locks.py:59
↓ 9 callersMethodbatch_msg
Create FullSyncBatchMsg.
tests/v1/cache_controller/conftest.py:182
↓ 9 callersFunctioncb
(key)
tests/v1/storage_backend/test_pd_backend_async.py:266
↓ 9 callersFunctioncheck_preemption_assertion
Reproduces the fixed assertion from build_connector_meta for preempted requests. On full cache hit where lmcache is dominant, expected is -1.
tests/v1/test_decode_save_and_preemption.py:63
↓ 9 callersMethodclose
(self)
tests/conftest.py:197
↓ 9 callersMethodclose
(self)
lmcache/v1/storage_backend/gds_backend.py:1201
↓ 9 callersMethodcomplete_sync
Mark sync as completed for a worker. Returns: True if completion was successful, False otherwise
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:237
↓ 9 callersMethodcontains
(self, key, search_range=None, pin: bool = False)
tests/v1/test_hidden_state_store.py:100
↓ 9 callersMethodcontains
Check whether ``key`` exists in the backend. Args: key: The cache key to look up. pin: If ``True`` and the key is pre
lmcache/v1/storage_backend/plugins/dax_backend.py:254
← previousnext →601–700 of 13,371, ranked by callers