MCPcopy Create free account

hub / github.com/LMCache/LMCache / functions

Functions13,371 in github.com/LMCache/LMCache

↓ 3 callersFunctioncreate_config
()
tests/benchmarks/test_xpu_layerwise_connector_benchmark.py:233
↓ 3 callersFunctioncreate_config
()
tests/benchmarks/test_xpu_connector_benchmark.py:153
↓ 3 callersMethodcreate_connector
(self)
lmcache/v1/storage_backend/connector/__init__.py:348
↓ 3 callersFunctioncreate_default_config
Create a default config without NIXL storage.
tests/v1/storage_backend/test_local_cpu_backend_nixl_paged.py:53
↓ 3 callersFunctioncreate_engine_driven_context
Factory that returns the appropriate :class:`EngineDrivenContext` implementation. Returns SHM-based implementation when shared-memory pool inform
lmcache/v1/multiprocess/transfer_context/base.py:198
↓ 3 callersMethodcreate_message
( self, benchmark: "ZMQControllerBenchmark", test_data: "TestData" )
lmcache/tools/controller_benchmark/handlers/p2p_lookup.py:31
↓ 3 callersMethodcreate_message
( self, benchmark: "ZMQControllerBenchmark", test_data: "TestData" )
lmcache/tools/controller_benchmark/handlers/register.py:31
↓ 3 callersFunctioncreate_test_config
Create a plugin-based config for the HFBucket connector.
tests/v1/storage_backend/test_hfbucket_connector.py:138
↓ 3 callersFunctioncreate_transfer_channel_context
Create a transfer channel context using the registered factory. Args: transfer_channel_type: The type name of the implementation to creat
lmcache/v1/distributed/transfer_channel/factory.py:63
↓ 3 callersMethoddebug_all_elapsed_times
(self)
lmcache/v1/storage_backend/connector/eic_connector.py:67
↓ 3 callersFunctiondecode_legacy_key
Deserialize a legacy non-MP cache key string. Args: encoded: String produced by ``CacheEngineKey.to_string`` or ``LayerCacheE
lmcache/v1/storage_backend/raw_block/key_codec.py:100
↓ 3 callersMethoddecompress
( self, tokens: Union[torch.Tensor, List[int]], method: str, location: str,
lmcache/v1/cache_engine.py:1478
↓ 3 callersMethodderegister
(worker_id)
tests/v1/cache_controller/test_registry_tree.py:625
↓ 3 callersFunctiondescribe_shape
Symbolic shape of a format, e.g. ``NL_X_NB_BS_HS`` -> ``NL x [NB, BS, HS]``. Named ``describe_shape`` (not ``shape_desc``) to avoid confusion wit
lmcache/v1/gpu_connector/kv_format/specs/base.py:52
↓ 3 callersMethoddeserialize
(self, src, dst: MemoryObjGroup)
tests/v1/distributed/serde/test_multi.py:187
↓ 3 callersMethoddeserialize
(self, src: MemoryObj, dst: MemoryObjGroup)
lmcache/v1/distributed/serde/asym_k16_v8.py:210
↓ 3 callersMethoddeserialize
(self, src: MemoryObj, dst: MemoryObjGroup)
lmcache/v1/distributed/serde/asym_k16_v8.py:471
↓ 3 callersMethoddetect
Detect ROCm by checking for hipcc.
setup_extensions/build_profiles/rocm.py:73
↓ 3 callersMethoddevice_count
(self)
tests/v1/test_musa_support.py:70
↓ 3 callersMethoddrain_completions
(self)
tests/v1/distributed/test_l2_adapter_factory.py:714
↓ 3 callersMethoddropped_events_count
Cumulative count of events dropped because the queue was full.
lmcache/v1/mp_observability/event_bus.py:287
↓ 3 callersFunctiondropped_positions
(seed)
tests/v1/distributed/test_fault_inject_l2_adapter.py:221
↓ 3 callersMethodempty_cache
Release all unoccupied cached memory. No-op in the stub implementation.
lmcache/v1/platform/cpu/stub_cpu_device.py:327
↓ 3 callersFunctionencode_hfbucket_object_name
Encode a serialized LMCache key into a reversible bucket object name.
lmcache/v1/storage_backend/connector/hfbucket_connector.py:823
↓ 3 callersMethodend_session
Notify LMCache server to remove the session for a finished request. Args: request_id: The ID of the finished request.
lmcache/integration/vllm/vllm_multi_process_adapter.py:966
↓ 3 callersMethodestimate_serialized_size
( self, layout_descs: LayoutDescGroup, )
tests/v1/distributed/serde/test_multi.py:158
↓ 3 callersMethodevent_fd
(self)
tests/v1/distributed/test_l2_adapter_factory.py:702
↓ 3 callersFunctionevict_stale
Deregister every instance whose heartbeat is older than the timeout. Args: registry: The shared instance registry. instance_timeo
lmcache/v1/mp_coordinator/app.py:46
↓ 3 callersMethodexists
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/connector/mock_connector.py:56
↓ 3 callersMethodexists_many
Return a full hit bitmap as booleans for encoded keys. Args: encoded_keys: Ordered encoded raw-block keys to check. l
lmcache/v1/storage_backend/raw_block/core.py:703
↓ 3 callersFunctionexpand_engine_block_ids
Expand the engine-side block id list to the list per LMCache kernel group. The serving engine reports block IDs per engine group. LMCache transfe
lmcache/v1/multiprocess/group_view.py:108
↓ 3 callersMethodexport_csv
Write per-request results to a CSV file.
lmcache/cli/commands/bench/engine_bench/stats.py:215
↓ 3 callersFunctionfetch_json
GET *url* and return the parsed JSON body. Raises: DescribeError: On network/HTTP errors.
lmcache/cli/commands/describe.py:36
↓ 3 callersMethodfind_kv
Find a KV chunk across all workers. Args: key: The KV chunk key to find. exclude_instance_id: Instance ID to
lmcache/v1/cache_controller/utils.py:580
↓ 3 callersFunctionfind_remote_backend
Find remote backend from storage manager
lmcache/v1/check/utils.py:156
↓ 3 callersMethodfree_lookup_locks
Release read locks acquired during lookup without a full retrieve. Use this when some chunks matched by lookup overlap with blocks that
lmcache/integration/vllm/vllm_multi_process_adapter.py:920
↓ 3 callersMethodfreeze
Record freeze mode changes.
tests/v1/cache_controller/conftest.py:120
↓ 3 callersMethodfrom_gpu
Expect a kwarg 'kvcaches' which is a nested tuple of K and V tensors. The kvcaches should correspond to the "WHOLE token sequence". W
lmcache/v1/gpu_connector/gpu_connectors.py:332
↓ 3 callersMethodfrom_gpu
Expect a kwarg 'kvcaches' which is a nested tuple of K and V tensors. The kvcaches should correspond to the "WHOLE token sequence". W
lmcache/v1/gpu_connector/xpu_connectors.py:206
↓ 3 callersMethodfrom_request_tracker
Create the request metadata from a request tracker. Args: tracker (RequestTracker): the request tracker. block_size (
lmcache/integration/vllm/vllm_v1_adapter.py:299
↓ 3 callersFunctiongenerate_kv_cache
(num_tokens, device)
tests/test_serde.py:22
↓ 3 callersFunctiongenerate_random_slot_mapping
(num_blocks, block_size, num_tokens, device)
tests/benchmarks/test_benchmark.py:32
↓ 3 callersFunctiongenerate_random_slot_mapping
(num_blocks, block_size, num_tokens, device)
tests/benchmarks/test_xpu_layerwise_connector_benchmark.py:37
↓ 3 callersFunctiongenerate_random_slot_mapping
( num_blocks: int, block_size: int, num_tokens: int, device: torch.device, )
tests/benchmarks/test_xpu_connector_benchmark.py:43
↓ 3 callersMethodget
(self, key: str)
lmcache/v1/storage_backend/connector/bigtable_connector.py:43
↓ 3 callersMethodget
Retrieve KV cache data for the given key with metrics reporting. Flow: 1. Acquire a new lease 2. Read from shared me
lmcache/v1/storage_backend/connector/sagemaker_hyperpod_connector.py:663
↓ 3 callersFunctiongetHTTPPort
(spec *lmcachev1alpha1.LMCacheEngineSpec)
operator/internal/resources/compute.go:196
↓ 3 callersMethodgetWrappedConnector
(self)
lmcache/v1/storage_backend/connector/instrumented_connector.py:85
↓ 3 callersMethodget_all_backend_names
Get the list of all registered backend names. Returns: List[str]: List of all backend names
lmcache/v1/storage_backend/storage_manager.py:926
↓ 3 callersMethodget_allocator_backend
Get the allocator backend.
tests/v1/storage_backend/test_storage_plugin.py:116
↓ 3 callersMethodget_blocking
(self, key: CacheEngineKey)
lmcache/v1/storage_backend/pd_backend.py:1038
↓ 3 callersMethodget_bypass_backend_name
Return the backend name to bypass when this health check fails and fallback_policy is LOCAL_CPU. Override this method to spe
lmcache/v1/health_monitor/base.py:115
↓ 3 callersMethodget_bypassed_backends
Get the list of currently bypassed backend names. Returns: List[str]: List of bypassed backend names
lmcache/v1/storage_backend/storage_manager.py:916
↓ 3 callersFunctionget_env_name
(attr_name: str)
lmcache/v1/config.py:990
↓ 3 callersFunctionget_expected_count
(token_len, save_unfull_chunk, chunk_size)
tests/benchmarks/test_xpu_layerwise_connector_benchmark.py:42
↓ 3 callersFunctionget_expected_count
(token_len: int, save_unfull_chunk: bool, chunk_size: int)
tests/benchmarks/test_xpu_connector_benchmark.py:53
↓ 3 callersFunctionget_formatter
Instantiate a formatter by its registered name. Args: name: Registered format name (e.g. ``"terminal"``, ``"json"``). **kwargs: F
lmcache/cli/metrics/formatter.py:58
↓ 3 callersFunctionget_fused_rope
( head_size: int, rotary_dim: int, max_position: int, base: float, is_neox_style: bool = T
lmcache/v1/compute/positional_encoding.py:150
↓ 3 callersMethodget_head_size
(self)
tests/v1/test_ec_connector.py:51
↓ 3 callersMethodget_instance_id
Get the instance id given an ip address.
lmcache/v1/cache_controller/controllers/registration_controller.py:90
↓ 3 callersFunctionget_ip
Get the local IP address of the machine.
lmcache/v1/rpc_utils.py:89
↓ 3 callersFunctionget_items_by_phase
Return all items belonging to a given phase.
lmcache/cli/commands/bench/engine_bench/interactive/schema.py:404
↓ 3 callersMethodget_key_size
Return the bytes tracked for ``key``, or ``None`` if unknown.
lmcache/v1/mp_coordinator/l2/usage_manager.py:32
↓ 3 callersMethodget_l2_adapter_reconfigure_status
(self)
lmcache/v1/multiprocess/http_apis/reconfigure_api.py:40
↓ 3 callersFunctionget_lmcache_chunk_size
Helper function to get the LMCache chunk size from the server Args: mq_client: The LMCache multiprocess mode message queue client
lmcache/integration/vllm/vllm_multi_process_adapter.py:222
↓ 3 callersMethodget_local_ip
Get the local IP address of the machine using multiple methods. Returns: A non-loopback IPv4 string when discoverable, otherwise
lmcache/lmcache_frontend/heartbeat.py:27
↓ 3 callersMethodget_mem_to_storage_handle
( self, mem_indices, storage_xfer_handler, storage_indices )
lmcache/v1/storage_backend/nixl_storage_backend.py:503
↓ 3 callersMethodget_memory_allocator
(self)
tests/v1/storage_backend/test_raw_block_uring_cmd.py:108
↓ 3 callersMethodget_memory_allocator
(self)
lmcache/v1/storage_backend/pd_backend.py:343
↓ 3 callersMethodget_memory_usage
Return ``(used_bytes, total_bytes)`` of the slab.
lmcache/v1/distributed/memory_manager/gds_l1_memory_manager.py:108
↓ 3 callersMethodget_num_kv_heads
(self, parallel_config)
tests/v1/test_ec_connector.py:48
↓ 3 callersMethodget_num_tracked_keys
Number of tracked keys, optionally scoped to one bucket.
lmcache/v1/distributed/eviction_policy/isolated_lru.py:180
↓ 3 callersMethodget_numa_mapping
Get NUMA mapping.
lmcache/v1/system_detection.py:56
↓ 3 callersMethodget_shape
(self, num_tokens: int)
lmcache/v1/gpu_connector/gpu_connectors.py:415
↓ 3 callersMethodget_shape
(self, num_tokens: int)
lmcache/v1/gpu_connector/gpu_connectors.py:1053
↓ 3 callersMethodget_shape
(self, num_tokens: int)
lmcache/v1/gpu_connector/xpu_connectors.py:287
↓ 3 callersMethodget_shape
(self, num_tokens: int)
lmcache/v1/gpu_connector/xpu_connectors.py:874
↓ 3 callersMethodget_slots_per_chunk_in_sw
Return the per-chunk *transfer* slot count for *kernel_group_idx*. For sub-chunk sliding window groups, the transfer slots is smaller
lmcache/v1/kv_layer_groups.py:474
↓ 3 callersMethodget_storage_to_mem_handle
( self, mem_indices, storage_xfer_handler, storage_indices )
lmcache/v1/storage_backend/nixl_storage_backend.py:514
↓ 3 callersMethodget_store_event_fd
(self)
lmcache/v1/distributed/l2_adapters/nixl_store_dynamic_l2_adapter.py:388
↓ 3 callersMethodget_sw_size_chunks
Return the sliding window size of a given kernel group, The size is measured in lmcache chunks. If the kernel group is non-sliding wi
lmcache/v1/kv_layer_groups.py:507
↓ 3 callersMethodget_temp_kernel_group_buffer
Returns the typed temp buffer for the given batch and kernel group. Mirrors :meth:`GPUCacheContext.get_temp_kernel_group_buffer`. Ar
lmcache/v1/platform/cpu/cache_context.py:338
↓ 3 callersMethodget_temp_object_group_buffer
Returns the flat uint8 temp buffer for the given batch and object group. Mirrors :meth:`GPUCacheContext.get_temp_object_group_buffer`
lmcache/v1/platform/cpu/cache_context.py:371
↓ 3 callersMethodget_tracked_salts
Return the set of cache_salts with at least one tracked key.
lmcache/v1/distributed/eviction_policy/isolated_lru.py:188
↓ 3 callersFunctionget_version
Return a human-readable version string. Returns: ``"<version>-<commit-id>"``, with ``"NA"`` substituted when the package version
lmcache/utils.py:289
↓ 3 callersFunctiongroup_by_salt
Group *keys* by ``cache_salt``, returning salt → count.
lmcache/v1/mp_observability/subscribers/metrics/utils.py:15
↓ 3 callersFunctionhasVolume
(pod *corev1.Pod, name string)
operator/internal/webhook/pod_injector_envtest_test.go:130
↓ 3 callersMethodhelp
Return short help text shown by ``lmcache -h``.
lmcache/cli/commands/base.py:54
↓ 3 callersMethodhit_rate
(self)
lmcache/observability.py:132
↓ 3 callersMethodinit_connection
(self)
lmcache/v1/storage_backend/remote_backend.py:118
↓ 3 callersMethodinit_cpu_memory_allocator
( self, size: int, shapes: list[torch.Size], dtypes: list[torch.dtype],
lmcache/v1/memory_management.py:3286
↓ 3 callersFunctioninit_logger
(name: str, log_level=logging.DEBUG)
benchmarks/multi_round_qa/utils.py:40
↓ 3 callersFunctioninit_remote_metadata_info
(num_groups: int)
lmcache/v1/protocol.py:77
↓ 3 callersMethodinsert_key
(self, key: CacheEngineKey, memory_obj: MemoryObj)
lmcache/v1/storage_backend/gds_backend.py:712
↓ 3 callersMethodis_building_sdist
Return True when building a source distribution.
setup_extensions/build_profiles/__init__.py:48
↓ 3 callersFunctionis_cross_layer_format
Return ``True`` if *engine_kv_format* stores all layers in one tensor. Cross-layer formats -- ``NB_NL_TWO_BS_NH_HS`` (vLLM, NHD) and ``NB_NL_
lmcache/v1/gpu_connector/utils.py:88
↓ 3 callersFunctionis_gpt
(gpt)
benchmarks/multi_round_qa/data_preprocessing.py:27
↓ 3 callersFunctionis_human
(human)
benchmarks/multi_round_qa/data_preprocessing.py:23
↓ 3 callersMethodis_native_ext_disabled
Return True when native extensions are disabled. Controlled by the ``NO_NATIVE_EXT`` environment variable. When ``True``, all native
setup_extensions/build_profiles/__init__.py:56
↓ 3 callersMethodis_pinned
(self)
lmcache/v1/storage_backend/nixl_storage_backend.py:432
← previousnext →1,901–2,000 of 13,371, ranked by callers