MCPcopy Create free account
hub / github.com/Pints-AI/1.5-Pints / _read_header

Method _read_header

lit_gpt/packed_dataset.py:208–217  ·  view source on GitHub ↗
(self, path)

Source from the content-addressed store, hash-verified

206 self._load_n_chunks()
207
208 def _read_header(self, path):
209 with open(path, 'rb') as f:
210 magic = f.read(len(HDR_MAGIC))
211 assert magic == HDR_MAGIC, "File doesn't match expected format."
212 version = struct.unpack('<Q', f.read(8))
213 assert version == (1,)
214 (dtype_code,) = struct.unpack('<B', f.read(1))
215 dtype = dtypes[dtype_code]
216 (chunk_size,) = struct.unpack('<Q', f.read(8))
217 return dtype, chunk_size
218
219 def _close_mmaps(self):
220 for mmap in self._mmaps:

Callers 1

_load_n_chunksMethod · 0.95

Calls

no outgoing calls

Tested by

no test coverage detected