MCPcopy Create free account
hub / github.com/Pints-AI/1.5-Pints / add_array

Method add_array

lit_gpt/packed_dataset.py:164–173  ·  view source on GitHub ↗
(self, arr: np.ndarray)

Source from the content-addressed store, hash-verified

162 return self._filenames.copy()
163
164 def add_array(self, arr: np.ndarray):
165 while self._idx + arr.shape[0] > self._chunk_size:
166 part_len = self._chunk_size - self._idx
167 self._arr[self._idx : self._idx + part_len] = arr[:part_len]
168 self._write_chunk()
169 arr = arr[part_len:]
170
171 arr_len = arr.shape[0]
172 self._arr[self._idx : self._idx + arr_len] = arr
173 self._idx += arr_len
174
175 def write_remainder(self):
176 self._write_chunk()

Callers 2

_build_dataMethod · 0.80

Calls 1

_write_chunkMethod · 0.95

Tested by 1