MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 19 callersFunction_make_mmap_file
( tmp_path, size: int = 4 * 1024 * 1024, name: str = "l1-devdax-test.bin" )
tests/v1/distributed/test_devdax_l1_allocator.py:44
↓ 19 callersFunction_names
(classes: Iterable[type])
tests/v1/test_subclass_discovery.py:94
↓ 19 callersFunction_sm_with
(*entries: tuple[str, _FakeAdapter])
tests/v1/multiprocess/http_apis/test_l2_api.py:98
↓ 19 callersFunction_value_for
Look up the current value of ``metric`` for the exact attribute set.
tests/v1/distributed/test_l1_l2_state_metrics.py:147
↓ 19 callersFunctioncreate_gpu_connector
(hidden_dim, num_layers)
tests/v1/utils.py:633
↓ 19 callersFunctioncreate_test_config
(extra_overrides=None)
tests/v1/storage_backend/test_bigtable_connector.py:170
↓ 19 callersFunctionmake_key
Build a test ObjectKey with the given model name and chunk id.
tests/v1/mp_observability/subscribers/metrics/counter_helpers.py:22
↓ 19 callersMethodprocess_tokens
Process the tokens and return the corresponding cache engine keys. :param Optional[Union[torch.Tensor, List[int]]] tokens: The tokens to proc
lmcache/v1/token_database.py:197
↓ 19 callersMethodregister
Hash and insert published token ranges (idempotent per chunk). The coordinator chunks each range's tokens, hashes each chunk, and maps
lmcache/v1/mp_coordinator/blend_directory.py:254
↓ 19 callersFunctionshm_unlink
Best-effort segment removal. Idempotent: a missing segment is treated as a successful no-op so callers can blindly call this on shutdown.
lmcache/v1/multiprocess/posix_shm.py:202
↓ 19 callersMethodstart
Start the event loop in a separate thread
lmcache/v1/check/utils.py:536
↓ 19 callersFunctionvllmPod
vllmPod returns a minimal args-only vLLM pod (no command override) bound to the test engine via annotation. mutate may further customize it.
operator/internal/webhook/pod_injector_test.go:144
↓ 18 callersMethodSetDefaults
SetDefaults applies defaults that cannot be expressed purely via kubebuilder markers. It mirrors LMCacheEngine.SetDefaults and additionally pins the b
operator/api/v1alpha1/cacheblendengine_defaults.go:22
↓ 18 callersFunction_create_error_response
Create a standardized error response. Args: error_info: Dictionary containing error information status_code: HTTP status code
lmcache/v1/internal_api_server/vllm/cache_api.py:65
↓ 18 callersFunction_read_counters_by_attrs
Snapshot counter values keyed by (metric_name, frozenset(attrs)).
tests/v1/mp_observability/subscribers/metrics/test_l2.py:69
↓ 18 callersFunction_setup
( eviction_ratio: float = 0.5, trigger_watermark: float = 1.0, )
tests/v1/mp_coordinator/test_eviction_manager.py:30
↓ 18 callersMethodbatched_allocate
(self, *args, **kwargs)
tests/v1/storage_backend/test_gds_backend.py:505
↓ 18 callersMethodclose
(self)
tests/v1/storage_backend/test_gds_backend.py:469
↓ 18 callersMethodcompute_eviction_plan
Select eviction candidates per ``cache_salt``. Salts over ``watermark * quota`` get ``eviction_ratio`` of their LRU keys; salts with
lmcache/v1/mp_coordinator/l2/eviction_manager.py:75
↓ 18 callersMethodcreate
(cls, manager)
tests/v1/test_health_monitor.py:49
↓ 18 callersFunctionderefInt32
derefInt32 returns the value pointed to by p, or def if p is nil.
operator/internal/resources/helpers.go:20
↓ 18 callersMethodexists
(self, key: str)
lmcache/v1/storage_backend/resp_client.py:163
↓ 18 callersMethodfrom_env
Build a config from ``LMCACHE_MP_COORDINATOR_*`` environment variables. Unset variables fall back to the dataclass defaults. Returns
lmcache/v1/mp_coordinator/config.py:99
↓ 18 callersMethodfrom_token_ids
Create a key from token ids. Only used by the tests.
lmcache/v1/multiprocess/custom_types.py:306
↓ 18 callersFunctionget_size_bytes
Calculate the size in bytes with the given shapes and dtypes.
lmcache/integration/vllm/utils.py:328
↓ 18 callersMethodis_healthy
Check if the LMCacheManager is healthy. Returns False if: - Initialization failed - HealthMonitor reports unhealthy
lmcache/v1/manager.py:508
↓ 18 callersMethodregister
Insert or replace an mp server entry atomically. Args: instance: The instance to store. If one with the same ``in
lmcache/v1/mp_coordinator/registry.py:67
↓ 17 callersMethodDeepCopy
DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BlendSpec.
operator/api/v1alpha1/zz_generated.deepcopy.go:46
↓ 17 callersFunction_adapter
(lookup_timeout_s: float = 10.0, load_timeout_s: float = 10.0)
tests/v1/distributed/l2_adapters/test_p2p_l2_adapter.py:49
↓ 17 callersFunction_compute_block_checksums
Compute MD5 checksums over KV cache blocks, grouped ``chunk_size`` blocks per chunk. The gather is performed in block space on ``block_axis``
lmcache/v1/multiprocess/http_apis/cache_api.py:184
↓ 17 callersFunction_create_memory_obj
(size: int = 1024, fill_value: float = 0.0)
tests/v1/distributed/test_raw_block_l2_adapter.py:90
↓ 17 callersFunction_key
(chunk_id: int, cache_salt: str = "")
tests/v1/distributed/test_isolated_lru_eviction_policy.py:17
↓ 17 callersFunction_make_chat_chunk
Build a minimal ``ChatCompletionChunk``.
tests/cli/commands/bench/engine_bench/test_request_sender.py:27
↓ 17 callersFunction_make_key
Create a CacheEngineKey for testing.
tests/v1/test_cache_engine.py:1666
↓ 17 callersFunction_make_key
(chunk_id: int, salt: str = "")
tests/v1/distributed/test_l2_adapter_base.py:26
↓ 17 callersFunction_make_memory_obj
Allocate a TensorMemoryObj from the adapter.
tests/v1/storage_backend/test_maru_backend.py:121
↓ 17 callersFunction_make_mock_factory
Create a mock BaseServiceFactory with sensible defaults.
tests/v1/test_manager.py:18
↓ 17 callersFunction_make_workload
( config: LongDocQAConfig | None = None, seed: int = 42, )
tests/cli/commands/bench/engine_bench/workloads/test_long_doc_qa.py:165
↓ 17 callersFunction_mock_backend
Return a ``Mock(spec=NixlDynamicStorageBackend)`` with sensible defaults. Every test that exercises ``batched_contains`` or ``contains`` needs
tests/v1/test_nixl_batched_contains.py:48
↓ 17 callersFunction_no_overlap
Check that no allocations overlap. Args: allocations: List of (address, size) tuples Returns: True if no overlaps, Fals
tests/v1/test_address_manager.py:1187
↓ 17 callersFunction_object_key_to_string
Serialize an ObjectKey to a deterministic S3 object name. Unsalted:: <model_name>@<kv_rank_hex>@<object_group_id_hex>@<chunk_hash_hex>
lmcache/v1/distributed/l2_adapters/s3_l2_adapter.py:171
↓ 17 callersFunction_salt_counts
(keys: list)
tests/v1/mp_observability/subscribers/metrics/test_l2.py:40
↓ 17 callersMethodadd_kv_op
Add a KV operation to the batch queue. Producer method: Adds operation to queue and notifies consumer when batch size threshold is re
lmcache/v1/storage_backend/batched_message_sender.py:129
↓ 17 callersMethodallocate
Allocate memory objects based on the provided layout description and count. This function should be thread-safe Args:
lmcache/v1/distributed/memory_manager/l1_memory_manager.py:126
↓ 17 callersMethodbatched_submit_put_task
( self, keys: Sequence[CacheEngineKey], objs: List[MemoryObj], transfer_spec:
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:374
↓ 17 callersMethodclear
Clear objects from L1 cache. Args: force: If True, clear ALL objects including locked ones. This may corrupt in-f
lmcache/v1/distributed/l1_manager.py:717
↓ 17 callersMethodclear
Clear all caches in the specified locations. :param Optional[List[str]] locations: The range of storage backends to perform
lmcache/v1/storage_backend/storage_manager.py:1116
↓ 17 callersMethodget_physical_size
Get the physical size of the MemoryObj in bytes.
lmcache/v1/memory_management.py:265
↓ 17 callersMethodget_worker
Simulates sync GET: register → prepXfer → postXfer → deregister.
tests/v1/test_device_id_race.py:152
↓ 17 callersMethodinit
Initialize the device backend. No-op in the stub implementation.
lmcache/v1/platform/cpu/stub_cpu_device.py:222
↓ 17 callersFunctionmake_object_key
Create a test ObjectKey with the given chunk ID.
tests/v1/distributed/test_prefetch_policy.py:26
↓ 17 callersFunctionnewTestEngine
newTestEngine returns a defaulted CacheBlendEngine with the given injection overrides applied via mutate (nil = pure defaults).
operator/internal/webhook/pod_injector_test.go:57
↓ 17 callersFunctionregister_l2_adapter_factory
Register an adapter factory for the given type name. Each adapter module should call this at import time **after** its config class has b
lmcache/v1/distributed/l2_adapters/factory.py:62
↓ 17 callersFunctionsend_lmcache_request
Helper function to send the request to the LMCache multiprocess server Args: mq_client: The LMCache multiprocess mode message queue
lmcache/integration/vllm/vllm_multi_process_adapter.py:199
↓ 17 callersMethodsend_request
Send a single streaming request and return the result. Streams the response via SSE, measures TTFT, decode speed, and total latency.
lmcache/cli/commands/bench/engine_bench/request_sender.py:101
↓ 17 callersFunctionvalidCBInjection
--- ValidateSpec tests --- validCBInjection returns the minimal injection block ValidateSpec requires (injection.payloadImage.repository). Tests that
operator/api/v1alpha1/cacheblendengine_test.go:194
↓ 17 callersFunctionwait_for_event_fd
(event_fd: int, timeout: float = 5.0)
tests/v1/distributed/test_s3_l2_adapter.py:353
↓ 16 callersFunctionBuildContainerArgs
BuildContainerArgs maps CRD spec fields to server.py CLI flags.
operator/internal/resources/compute.go:53
↓ 16 callersMethodHandle
Handle implements admission.Handler. It applies mutations M0–M7 to an opted-in pod whose named CacheBlendEngine connection ConfigMap exists, then retu
operator/internal/webhook/pod_injector.go:119
↓ 16 callersFunction_json_response
Helper to create JSON response.
lmcache/v1/internal_api_server/vllm/lookup_api.py:13
↓ 16 callersFunction_key
(i: int)
tests/v1/distributed/l2_adapters/test_p2p_l2_adapter.py:28
↓ 16 callersFunction_make_keys
(count: int, cache_salt: str = "")
tests/v1/mp_observability/subscribers/metrics/test_l2.py:36
↓ 16 callersFunction_make_workload
( config: MultiRoundChatConfig | None = None, seed: int = 42, )
tests/cli/commands/bench/engine_bench/workloads/test_multi_round_chat.py:263
↓ 16 callersFunction_register
( matcher: BlendTokenRangeMatcherV3, tokens: list[int], chunk_hash_seeds: list[int], position_
tests/v1/multiprocess/test_optimized_lookup_v3.py:349
↓ 16 callersMethodbatched_free
( self, memory_objs: List[MemoryObj], allocator_type: Optional[str] = None, up
lmcache/v1/memory_management.py:2192
↓ 16 callersMethodclear
Used by ``/clear-cache``; delegates to :class:`ManagementModule`.
lmcache/v1/multiprocess/server.py:133
↓ 16 callersMethodclose
Wait for worker pools, close the core, and close eventfds.
lmcache/v1/distributed/l2_adapters/raw_block_l2_adapter.py:564
↓ 16 callersMethodcompute_chunk_hashes
Compute rolling prefix hashes for complete chunks in a token range. The rolling hash is always computed from the beginning of ``token
lmcache/v1/multiprocess/token_hasher.py:192
↓ 16 callersFunctioncreate_object_key
(chunk_id: int, model_name: str = "test/model")
tests/v1/distributed/test_hfbucket_l2_adapter.py:136
↓ 16 callersFunctioncreate_test_metadata
(kv_shape=(28, 2, 256, 8, 128), chunk_size=256)
tests/v1/storage_backend/test_bigtable_connector.py:191
↓ 16 callersMethodcurrent_device
Return the index of the currently selected device. Returns: 0 always for the stub.
lmcache/v1/platform/cpu/stub_cpu_device.py:298
↓ 16 callersMethoddebug_has_key
Check whether a specific key is stored in the mock adapter. This method is intended for testing and debugging only. Args:
lmcache/v1/distributed/l2_adapters/mock_l2_adapter.py:316
↓ 16 callersFunctionengineKey
engineKey is shorthand for the typed namespaced name pair used throughout the smoke specs.
operator/test/e2e/smoke_helpers_test.go:205
↓ 16 callersMethodfind
Look up a layout descriptor by (model_name, world_size). Args: model_name: The model name. world_size: The world size
lmcache/v1/multiprocess/engine_context.py:111
↓ 16 callersMethodfinish_write
Finish writing the objects into the storage manager. Args: keys (list[ObjectKey]): List of object keys that have been wr
lmcache/v1/distributed/storage_manager.py:224
↓ 16 callersMethodfree
Free the provided memory objects. This function should be thread-safe. Args: mem_objs (list[MemoryObj]): List of
lmcache/v1/distributed/memory_manager/l1_memory_manager.py:153
↓ 16 callersMethodfrom_bytes
Parse a byte buffer into an EncodedKV. Cross-config gates: if `expected_hashes` is provided, any non-empty mismatch raises `CodecMism
lmcache/v1/kv_codec/asym_k16_v8.py:385
↓ 16 callersFunctiongenerate_sglang_kv_cache_paged_list_tensors
Instead of Tuple[Tuple[Tensor, Tensor]], return List[Tensor] where KV are in the same tensor For MLA: List[num_layers] of [page_buffer_s
tests/v1/utils.py:333
↓ 16 callersMethodget_blocking
Load one key from DAX into a CPU memory object. Args: key: Cache key to retrieve. Returns: Restored ``Memory
lmcache/v1/storage_backend/plugins/dax_backend.py:412
↓ 16 callersMethodget_or_create
Get existing session or create a new one. Args: request_id: Unique request identifier. Returns: The Session
lmcache/v1/multiprocess/session.py:131
↓ 16 callersMethodget_records
(self)
tests/v1/storage_backend/test_audit_connector.py:134
↓ 16 callersFunctionmakeRequest
makeRequest builds a CREATE admission.Request carrying the given pod as raw JSON in req.Object.
operator/internal/webhook/pod_injector_test.go:100
↓ 16 callersFunctionnewPodInjector
newPodInjector returns a PodInjector backed by a fake client seeded with the given objects, plus the engine's connection ConfigMap when seedConn is tr
operator/internal/webhook/pod_injector_test.go:81
↓ 16 callersFunctionregister_l2_adapter_type
Register an L2 adapter config class under a type name. The type name is used in JSON specs as the "type" field. Each adapter config modu
lmcache/v1/distributed/l2_adapters/config.py:55
↓ 16 callersFunctionrolling_hash_windows_numba
Compute rolling polynomial hashes over a uint64 array. This function computes a polynomial rolling hash over a sliding window of size `k
lmcache/v1/multiprocess/token_hasher.py:247
↓ 16 callersFunctionwait_for_condition
Poll until a predicate returns True or timeout. Args: predicate: Callable returning bool. timeout: Max wait time in seconds.
tests/v1/distributed/test_store_controller.py:75
↓ 16 callersMethodwarmup
(self)
tests/cli/commands/bench/engine_bench/workloads/test_base_workload.py:30
↓ 15 callersMethod__new__
(cls, success: bool, bytes_transferred: int)
lmcache/v1/distributed/internal_api.py:189
↓ 15 callersMethod_apply_state
Record the new state and reconcile adapters against ``upstream``. Args: state: The registration state derived from this cycle.
lmcache/v1/multiprocess/modules/p2p_controller.py:432
↓ 15 callersFunction_delta
( before: dict[str, tuple[float, int]], after: dict[str, tuple[float, int]] )
tests/v1/mp_observability/subscribers/metrics/test_sm_lifecycle.py:94
↓ 15 callersFunction_make_args
(**overrides)
tests/cli/commands/bench/test_bench_command.py:33
↓ 15 callersFunction_store_tensor
( backend: DaxBackend, key: CacheEngineKey, *, num_tokens: int, hidden_dim: int = 8, f
tests/v1/storage_backend/test_dax_backend.py:117
↓ 15 callersMethodadd_section
Add a named section. Args: key: Machine-readable section key (used in JSON output and for ``metrics["key"]`` acce
lmcache/cli/metrics/metrics.py:70
↓ 15 callersMethodbatched_async_contains
( self, lookup_id: str, keys: List[CacheEngineKey], pin: bool = False, )
lmcache/v1/storage_backend/p2p_backend.py:284
↓ 15 callersMethodbatched_put
Non-blocking function to batched put the memory objects into the storage backends. Do not store if the same object is being s
lmcache/v1/storage_backend/storage_manager.py:383
↓ 15 callersMethodclear
(self)
lmcache/v1/utils/bloom_filter.py:85
↓ 15 callersMethodclose
(self)
tests/v1/internal_api_server/test_backend_api.py:36
↓ 15 callersMethodclose
No-op teardown (real GPUCacheContext.close deregisters its GDS buffer).
tests/v1/multiprocess/test_lmcache_driven_layout_registry.py:20
↓ 15 callersFunctioncompress_slot_mapping
Compress a list of slot indices into ranges while preserving order. Consecutive slots (3 or more) are represented as [start, end] ranges. Sin
lmcache/utils.py:108
↓ 15 callersFunctioncompute_retrieve_plan
Reproduce the skip logic from CacheServer.retrieve._retrieve_loop. Returns a list of (chunk_index, skip_in_chunk) for each chunk that wo
tests/v1/multiprocess/test_skip_first_n_tokens.py:17
← previousnext →301–400 of 13,371, ranked by callers