MCPcopy Create free account
hub / github.com/LMCache/LMCache / _tensor_from_ptr

Function _tensor_from_ptr

lmcache/python_ops_fallback.py:61–130  ·  view source on GitHub ↗

Create a tensor view over a raw pointer (zero-copy where possible). Supports both CPU (pinned or regular) and CUDA device pointers. Args: ptr: Raw memory pointer as int (must be non-zero). shape: Desired tensor shape. dtype: Desired tensor dtype, must matc

(
    ptr: int,
    shape: tuple[int, ...],
    dtype: torch.dtype,
    device: torch.device | str | None = None,
)

Source from the content-addressed store, hash-verified

59
60
61def _tensor_from_ptr(
62 ptr: int,
63 shape: tuple[int, ...],
64 dtype: torch.dtype,
65 device: torch.device | str | None = None,
66) -> torch.Tensor:
67 """
68 Create a tensor view over a raw pointer (zero-copy where possible).
69
70 Supports both CPU (pinned or regular) and CUDA device pointers.
71
72 Args:
73 ptr: Raw memory pointer as int (must be non-zero).
74 shape: Desired tensor shape.
75 dtype: Desired tensor dtype, must match the memory layout.
76 device: Where the pointer lives.
77 - None / "cpu" / torch.device("cpu") → CPU pointer
78 - "cuda" / "cuda:N" / torch.device("cuda", N) → CUDA pointer
79 If None and ptr looks like a CUDA ptr, pass device explicitly.
80
81 Returns:
82 A tensor that shares memory with the original pointer.
83 For CPU: always zero-copy via ctypes + torch.frombuffer.
84 For CUDA: zero-copy via torch._C._construct_storage_from_data_pointer
85 (PyTorch >= 2.0) or __cuda_array_interface__, with a
86 cudaMemcpy D2D fallback.
87
88 Raises:
89 ValueError: if ptr is 0.
90
91 Warning:
92 The caller is responsible for keeping the underlying memory alive
93 for the entire lifetime of the returned tensor.
94 """
95 if ptr == 0:
96 raise ValueError("Pointer must be non-zero")
97
98 # ------------------------------------------------------------------ #
99 # Normalise device #
100 # ------------------------------------------------------------------ #
101 if device is None:
102 device = torch.device("cpu")
103 elif not isinstance(device, torch.device):
104 device = torch.device(device)
105
106 assert isinstance(device, torch.device)
107 # ------------------------------------------------------------------ #
108 # Compute size #
109 # ------------------------------------------------------------------ #
110 numel = 1
111 for dim in shape:
112 numel *= int(dim)
113 element_size = torch.empty((), dtype=dtype).element_size()
114 total_bytes = numel * element_size
115
116 # ------------------------------------------------------------------ #
117 # CPU path #
118 # ------------------------------------------------------------------ #

Callers 4

multi_layer_kv_transferFunction · 0.85
_normalize_paged_layersFunction · 0.85

Calls 3

_tensor_from_cpu_ptrFunction · 0.85
_tensor_from_cuda_ptrFunction · 0.85
deviceMethod · 0.45

Tested by

no test coverage detected