MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 2 callersFunction_split_token2d_kv
Accepts either: - [2, T, D] - [T, 2, D] Returns: - k_tok: [T, D] - v_tok: [T, D]
lmcache/v1/gpu_connector/utils.py:610
↓ 2 callersMethod_stage_block_ids
(self, block_ids: List[int])
lmcache/v1/gpu_connector/gpu_connectors.py:2082
↓ 2 callersMethod_store_key
Build the ``(adapter_index, task_id)`` correlation key. Returns ``None`` if either field is missing.
lmcache/v1/mp_observability/subscribers/metrics/l2_throughput.py:127
↓ 2 callersFunction_stub_static_agent
(monkeypatch)
tests/v1/storage_backend/test_nixl_shared_pool.py:221
↓ 2 callersMethod_sweep_stale_evictions
Report T and discard evicted entries older than max_evict_reuse_wait.
lmcache/v1/mp_observability/subscribers/metrics/l1_lifecycle.py:157
↓ 2 callersFunction_synthetic_attention_shape
Factor a page's element count into the synthetic attention layout. Args: elems_per_page: Total elements in one page (one logical block).
lmcache/integration/vllm/kv_cache_group_edits.py:143
↓ 2 callersFunction_tensor_bytes
(t: torch.Tensor)
tests/v1/distributed/serde/test_multi.py:97
↓ 2 callersFunction_tensors_to_ptrs
Convert a list of tensors to a list of their data_ptr() values.
lmcache/v1/multiprocess/transfer_context/base.py:100
↓ 2 callersFunction_token_db
(config: LMCacheEngineConfig)
tests/v1/test_hidden_state_store.py:61
↓ 2 callersMethod_transfer
( self, tensor_ptr: int, block_ids: List[int], direction: "lmc_ops.TransferDir
lmcache/v1/gpu_connector/gpu_connectors.py:2096
↓ 2 callersFunction_trim_misc_buffer
(misc: list[str], limit: int = _MAX_ERR)
lmcache/cli/commands/query/_request.py:120
↓ 2 callersFunction_unfold_python
Reference (pure-Python) implementation of :func:`unfold`, for testing only. Kept as the oracle the native :func:`unfold` is validated against in
lmcache/v1/distributed/bitmap_ops/fold.py:261
↓ 2 callersFunction_unique_placeholders
(prompt_template: str)
lmcache/cli/commands/query/_prompt.py:125
↓ 2 callersMethod_unlock_all_lookups
Unlock all keys locked during lookup (nothing to load case).
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:1175
↓ 2 callersMethod_unlock_all_plan_keys
Phase 2 unlock: release L2 locks for all keys in the load plan.
lmcache/v1/distributed/storage_controllers/prefetch_controller.py:1169
↓ 2 callersFunction_unpack_str
(buf: memoryview, off: int)
lmcache/v1/kv_codec/encoded_kv.py:111
↓ 2 callersMethod_validate_block_chunk_size_config
Validate the chunk size configuration against the slot and tokens block detected from the serving engine. Raises: ValueEr
lmcache/v1/kv_layer_groups.py:596
↓ 2 callersMethod_validate_blocking_handlers
Validate that all request types are registered BlockingRequestHandlers.
lmcache/v1/multiprocess/mq.py:760
↓ 2 callersMethod_validate_http_url
(self, url: str)
lmcache/connections.py:42
↓ 2 callersMethod_validate_memory_format
Validate that the memory object has the expected format. Args: memory_obj: The memory object to validate. Raises:
lmcache/v1/gpu_connector/musa_connectors.py:308
↓ 2 callersMethod_validate_memory_format
Validate that the memory object has the expected format. Args: memory_obj: The memory object to validate. Raises:
lmcache/v1/gpu_connector/hpu_connector.py:245
↓ 2 callersMethod_validate_supported_kv_format
Validate that this connector implements the discovered engine KV format. Raises: ValueError: If the active KV cache layout is uns
lmcache/v1/gpu_connector/musa_connectors.py:330
↓ 2 callersFunction_validation_error_response
(exc: ValidationError)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:146
↓ 2 callersMethod_verify_hot_cache
Verify hot cache contains expected memory objects with ref count 1.
tests/v1/internal_api_server/test_load_fs_chunks.py:243
↓ 2 callersFunction_wait_finished
Poll query_read_status until the task reports finished (or timeout).
tests/v1/distributed/transfer_channel/test_nixl_impl.py:90
↓ 2 callersMethod_wait_for_workers
Wait for expected number of workers to be registered
tests/v1/internal_api_server/test_worker_info_api.py:152
↓ 2 callersFunction_wait_until_up
(base_url: str, timeout: float = 5.0)
tests/v1/mp_coordinator/test_integration.py:30
↓ 2 callersMethod_write_buffers
Write one or more buffers through the configured Rust I/O path. Args: offsets: Device offsets for each write. buffers
lmcache/v1/storage_backend/raw_block/core.py:1245
↓ 2 callersMethod_write_frame
(self, frame: bytes)
lmcache/v1/mp_observability/trace/recorder.py:180
↓ 2 callersFunction_zmq_endpoint
Return a ZMQ tcp endpoint for a ``host:port`` (or pass-through url).
lmcache/tools/transfer_channel_benchmark/benchmark.py:63
↓ 2 callersMethodacquire
Acquire *slots* from the semaphore, blocking until available. Args: slots: Number of slots to acquire (must be <= capacity).
examples/disagg_prefill/disagg_proxy_server.py:46
↓ 2 callersMethodactive_count
Return the number of active sessions. Returns: Number of currently tracked sessions.
lmcache/v1/multiprocess/session.py:184
↓ 2 callersFunctionadapter_by_id
Fetch an active adapter by stable id via the public ``l2_adapters()`` API (no private-member access).
tests/v1/distributed/test_runtime_adapters.py:91
↓ 2 callersFunctionadd_benchmark_arguments
Add all benchmark arguments to ``parser``. Defined here (not in ``benchmark.py``) so the CLI can build its parser without importing torch or
lmcache/tools/transfer_channel_benchmark/config.py:126
↓ 2 callersMethodadd_blocking_handler
( self, request_type: RequestType, payload_clss: list[Any], handler )
lmcache/v1/multiprocess/mq.py:747
↓ 2 callersFunctionadd_gen_dataset_arguments
Register all gen-dataset CLI flags onto *parser*. Called by both :func:`main` (``python -m`` entry point) and the ``lmcache tool cache-simula
lmcache/tools/cache_simulator/gen_bench_dataset.py:226
↓ 2 callersMethodadd_handler
Register a handler for a specific request type. Args: request_type (RequestType): The type of the request to handle.
lmcache/v1/multiprocess/mq.py:708
↓ 2 callersFunctionadd_http_frontend_args
Add HTTP frontend configuration arguments to an existing parser. Args: parser: The argument parser to add arguments to. Returns
lmcache/v1/multiprocess/config.py:480
↓ 2 callersMethodadd_mm_data
Append one multimodal metadata entry.
lmcache/integration/vllm/vllm_ec_adapter.py:48
↓ 2 callersFunctionadd_p2p_args
Add peer-to-peer configuration arguments to an existing parser. Args: parser: The argument parser to add arguments to. Returns:
lmcache/v1/multiprocess/config.py:411
↓ 2 callersMethodadd_request
Add a request to the pool, evicting LRU entries if necessary.
examples/agents/prefix_analysis.py:119
↓ 2 callersFunctionadd_simulate_arguments
Register all ``simulate`` CLI flags onto *parser*. Called by both the module ``main()`` and by :class:`~lmcache.cli.commands.tool.ToolCommand
lmcache/tools/cache_simulator/simulator.py:664
↓ 2 callersFunctionadd_sweep_arguments
Register all ``sweep`` CLI flags onto *parser*. Called by both the module ``main()`` and by :class:`~lmcache.cli.commands.tool.ToolCommand` s
lmcache/tools/cache_simulator/plot_hit_rate.py:50
↓ 2 callersMethodadd_worker
Add a worker to this instance.
lmcache/v1/cache_controller/utils.py:262
↓ 2 callersMethodaddresses_to_indices
Translate (offset, size) addresses into page indices in the prepped dlist.
lmcache/v1/distributed/transfer_channel/impl/nixl_impl.py:398
↓ 2 callersFunctionalign_to
Align the given size to the nearest multiple of align_size. Args: size (int): The size to align. align_size (int): The align
lmcache/v1/lazy_memory_allocator.py:44
↓ 2 callersMethodall_instances
Return a snapshot list of all registered instances. Returns: A new list containing every currently registered instance.
lmcache/v1/mp_coordinator/registry.py:122
↓ 2 callersMethodall_session_ids
Return the set of session IDs that have at least one open span. Returns: Set of session ID strings currently tracked in the regis
lmcache/v1/mp_observability/subscribers/tracing/span_registry.py:103
↓ 2 callersFunctionalloc_pinned_numa_ptr
Non-CUDA equivalent of allocating pinned memory with NUMA awareness. On XPU, uses pin_memory=True (SYCL USM host allocation) for fast transfers.
lmcache/python_ops_fallback.py:400
↓ 2 callersMethodallocate
( self, shapes: Union[torch.Size, list[torch.Size]], dtypes: Union[torch.dtype, list[t
lmcache/v1/memory_management.py:2831
↓ 2 callersMethodallocate
( self, shapes: Union[torch.Size, list[torch.Size]], dtypes: Union[torch.dtype, list[t
lmcache/v1/memory_management.py:3068
↓ 2 callersMethodallocate
Allocate a memory object of shape and dtype
lmcache/v1/storage_backend/gds_backend.py:1095
↓ 2 callersFunctionanalyze_time_series
Analyze chunk distribution over time. Args: records: List of chunk hash records Returns: Dictionary containing time-series s
examples/chunk_statistics/analyze_chunk_hashes.py:176
↓ 2 callersMethodappend_block_ids
Update the block ids for the current request This function will be called when processing the cached requests.
lmcache/integration/vllm/lmcache_mp_connector.py:252
↓ 2 callersMethodappend_block_ids
Update the block ids for the current request This function will be called when processing the cached requests.
lmcache/integration/vllm/lmcache_mp_connector_0201.py:265
↓ 2 callersMethodappend_block_ids
Update the block ids for the current request This function will be called when processing the cached requests.
lmcache/integration/vllm/lmcache_mp_connector_0180.py:247
↓ 2 callersMethodapply_loaded_state
Validate and apply a raw-block metadata checkpoint payload.
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:233
↓ 2 callersFunctionassertDaemonSetShape
assertDaemonSetShape verifies the operator's auto-injected pod-level settings: hostIPC, runtimeClassName=nvidia, container privileged security context
operator/test/e2e/crd_smoke_test.go:167
↓ 2 callersFunctionassertLookupServiceShape
assertLookupServiceShape checks the node-local discovery Service has internalTrafficPolicy=Local (so kube-proxy routes only to the on-node LMCache pod
operator/test/e2e/crd_smoke_test.go:205
↓ 2 callersFunctionassert_layerwise_gpu_connector
Assert that a GPU Connector is a layerwise connector.
lmcache/v1/gpu_connector/utils.py:68
↓ 2 callersMethodasync_batched_write
Write a batch of data through the channel. :param objects: A list of bytes or MemoryObj to be written. :param transfer_spec:
lmcache/v1/transfer_channel/nixl_channel.py:507
↓ 2 callersMethodasync_lazy_init_peer_connection
( self, local_id: str, peer_id: str, peer_init_url: str, init_side_msg
lmcache/v1/transfer_channel/nixl_channel.py:182
↓ 2 callersMethodasync_send_init_side_msg
Async version of send_init_side_msg.
lmcache/v1/transfer_channel/abstract.py:117
↓ 2 callersMethodbatch_get
(self, keys: list[str], bufs: list[memoryview])
lmcache/v1/storage_backend/native_clients/connector_client_base.py:107
↓ 2 callersMethodbatch_set
(self, keys: list[str], bufs: list[memoryview])
lmcache/v1/storage_backend/native_clients/connector_client_base.py:114
↓ 2 callersMethodbatched_allocate
Allocate a batch of storage slot indices. Args: num_objs: Number of indices to allocate. Returns: l
lmcache/v1/distributed/l2_adapters/nixl_store_l2_adapter.py:94
↓ 2 callersMethodbatched_get
Download multiple chunks while preserving input order.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:317
↓ 2 callersMethodbatched_get_non_blocking
Asynchronously load the leading raw-block hit prefix. Args: lookup_id: Lookup identifier supplied by the storage manager.
lmcache/v1/storage_backend/plugins/rust_raw_block_backend.py:631
↓ 2 callersMethodbatched_nixl_desc_exists
Check if multiple descriptors exist via a single ``query_memory`` call. :param reg_list: List of tuples ``(0, 0, 0, meta_info)`` where
lmcache/v1/storage_backend/nixl_storage_backend.py:712
↓ 2 callersMethodbatched_submit_read_task
Read cached KV blocks from the decoder's GPU memory into local buffers. This is the bidirectional NIXL read path: the prefiller quer
lmcache/v1/storage_backend/pd_backend.py:763
↓ 2 callersMethodbind_storage_manager
Attach the engine's :class:`StorageManager`. Required for coupled eviction. Without it, retrieve falls back to HS-only presence check
lmcache/v1/hidden_state_store.py:103
↓ 2 callersMethodblock_size
Returns the block size (tokens per block).
lmcache/v1/platform/cpu/cache_context.py:260
↓ 2 callersFunctionbuildConnectionConfigMapCore
buildConnectionConfigMapCore is the shared core for the <engine>-connection ConfigMap that both engine controllers emit. It produces the kv-transfer-c
operator/internal/resources/configmap.go:70
↓ 2 callersFunctionbuildDaemonSetCore
buildDaemonSetCore constructs the DaemonSet shared by the LMCacheEngine and CacheBlendEngine controllers. It is the single source of truth for the GPU
operator/internal/resources/daemonset.go:67
↓ 2 callersFunctionbuildLanguageHomeUrl
* Build the home URL for a target language. * * @param {"en" | "zh_CN"} language The target language. * @returns {string} The target language home
docs/source/_static/custom.js:63
↓ 2 callersFunctionbuildLanguageUrl
* Build the target language URL for the current page. * * @param {"en" | "zh_CN"} language The target language. * @returns {string} The target URL.
docs/source/_static/custom.js:43
↓ 2 callersFunctionbuildLongPrompt
buildLongPrompt assembles ~1.2k tokens of prose so the prompt spans at least 8 chunks at chunkSize=128. We use a repeated history passage rather than
operator/test/e2e/vllm_integration_smoke_test.go:276
↓ 2 callersFunctionbuildLookupServiceCore
buildLookupServiceCore is the name/namespace/spec-keyed core shared by the LMCacheEngine and CacheBlendEngine lookup-Service builders. The node-local
operator/internal/resources/service.go:38
↓ 2 callersFunctionbuildMetricsServiceCore
buildMetricsServiceCore is the name/namespace/spec-keyed core shared by the LMCacheEngine and CacheBlendEngine metrics-Service builders.
operator/internal/resources/service.go:75
↓ 2 callersFunctionbuildServiceMonitorCore
buildServiceMonitorCore is the name/namespace/spec-keyed core shared by the LMCacheEngine and CacheBlendEngine ServiceMonitor builders. Callers must e
operator/internal/resources/servicemonitor.go:43
↓ 2 callersFunctionbuild_chat
(system_doc: str, user_prompt: str)
examples/online_session/openai_chat_completion_client.py:90
↓ 2 callersFunctionbuild_fewshot_prompt
(example)
benchmarks/rag/utils.py:187
↓ 2 callersFunctionbuild_qa_prompt
(example, query_prompt)
benchmarks/rag/utils.py:180
↓ 2 callersFunctioncalculate_hit_rate
( token_sequences: List[List[int]], pool_size: Optional[int] = None, token_tensor: Optional[torch.
examples/agents/prefix_analysis.py:188
↓ 2 callersFunctioncalculate_throughput
Calculate throughput in GB/s
tests/disagg/test_nixl_cache_engine.py:91
↓ 2 callersFunctioncalculate_throughput
Calculate throughput in GB/s
tests/disagg/test_nixl_storage_backend.py:56
↓ 2 callersFunctioncalculate_throughput
Calculate throughput in GB/s
tests/disagg/test_nixl_channel.py:47
↓ 2 callersMethodcancel_lookup
Mark lookup as aborted. Cleanup will happen after task finishes.
lmcache/v1/lookup_client/lmcache_async_lookup_client.py:256
↓ 2 callersFunctioncanonicalJSON
(s string)
operator/test/e2e/crd_smoke_test.go:243
↓ 2 callersFunctioncertManagerVersion
certManagerVersion returns the cert-manager release to install, honoring the CERT_MANAGER_VERSION env var and falling back to certManagerDefaultVersio
operator/test/utils/utils.go:47
↓ 2 callersFunctioncheck_allocator
(allocator, max_size)
tests/v1/test_memory_management.py:33
↓ 2 callersFunctioncheck_equal
(gpu_t: torch.Tensor, cpu_t: torch.Tensor)
tests/v1/test_mem_kernels.py:879
↓ 2 callersFunctioncheck_extra_config
(config: "LMCacheEngineConfig")
tests/v1/test_config.py:33
↓ 2 callersMethodcheck_finish
( self, msg: CheckFinishMsg )
lmcache/v1/cache_controller/executor.py:419
↓ 2 callersFunctioncheck_method_signatures
Check if method signatures in implementation class match the base class. Returns a list of mismatches.
tests/v1/utils.py:709
↓ 2 callersFunctioncheck_paged_allocator
(allocator, shape, dtype, fmt, max_num_pages)
tests/v1/test_memory_management.py:88
↓ 2 callersMethodcheck_sync_timeout
Check for sync timeouts and mark failed workers. This should be called periodically (e.g., in health check loop).
lmcache/v1/cache_controller/controllers/full_sync_tracker.py:303
↓ 2 callersMethodcleanup
Cleanup ZMQ sockets
lmcache/tools/controller_benchmark/benchmark.py:155
↓ 2 callersMethodcleanup_expired
Remove sessions that have exceeded their TTL. Returns: Number of sessions removed.
lmcache/v1/multiprocess/session.py:165
↓ 2 callersMethodclear
counts the number of memory objects removed
lmcache/v1/storage_backend/local_cpu_backend.py:913
← previousnext →2,501–2,600 of 13,371, ranked by callers