A Versioned HDF5 File This is the main entry-point of the library. To use a versioned HDF5 file, pass a h5py file to constructor. The me

Writeable numpy array-like, a.k.a. virtual array, which wraps around a sequence of read-only array-likes of chunks, known as the base slabs.

Class that looks like a h5py.Dataset but is backed by an array

A proxy class representing the hash table for an array The hash table for an array is a mapping from {sha256_hash: slice}, where slice i

Minimal read-only NumPy array-like implementing the ArrayProtocol

Filters keyword arguments for create_dataset and modify_metadata. Not to be confused with h5py.Dataset._filters, which is a dict in the format

Class that looks like a Dataset that has no data (only the fillvalue)

Special mapper that selects all points on the chunks selected by another mapper. This is used to load the entire chunk for the purpose of caching

Minimal writeable NumPy array-like implementing the ArrayProtocol

IndexChunkMapper for slices

Instructions to transfer data: - from a slab to the return value of __getitem__, or - from the value parameter of __setitem__ to a slab, or

Wrap around AstypeWrapper, which exclusively defined __getitem__ and __len__.

Instructions to execute StagedChangesArray.changes().

4-word SHA256 digest

Instructions to execute StagedChangesArray.commit(). Deduplicates the staged chunks against each other, the base slabs and the full slab, the

Compression or other filter property of a dataset. The data is stored on the parent MemoryGroup.

Instructions to execute StagedChangesArray.__getitem__

Instructions to hash all staged chunks of a StagedChangesArray. Finds every chunk that lies on a staged slab and, grouped by slab, produces the

Container for the plan to a single call to hash_slab

Class that looks like a h5py.Dataset but is backed by a versioned dataset The versioned dataset can be modified, which performs modification

IndexChunkMapper for one-dimensional fancy integer array indices. This is also used for boolean indices (preprocessed with np.flatnonzero()).

IndexChunkMapper for scalar integer indices

Load all chunks that have not been loaded yet from the base slabs.

Instructions to execute StagedChangesArray.resize()

Instructions to execute StagedChangesArray.__setitem__

Minimal read-only NumPy array-like interface. Not to be confused with numpy.typing.ArrayLike, which is any object that can be coerced into a

Abstract IndexChunkMapper for numpy basic indexing (slices and integers)

Common setup and teardown for all versioned-hdf5 benchmarks.

Mixin for all staged datasets, handling dtype conversions. Special handling for NumPy StringDType, a.k.a. NpyStrings ------------------------

Key for the {sha256 -> (slab_idx, slab_offset)} map. A SHA256 digest is 4 x uint64 = 32 bytes. The digest is already a cryptographically strong, unifo

Key for the {(slab_idx, slab_offset) -> (slab_idx, slab_offset)} map.

Superclass for classes that look like h5py.Dataset Subclasses should have the following properties defined (properties starting with an

Sentinel for default argument values.

Add properties for compression and other filters to datasets.

Abstract class that manipulates a numpy fancy index along a single axis of a chunked array Parameters ---------- chunk_indices:

Common ancestor of all plans that mutate StagedChangesArray

Test that astype() does not eagerly convert staged slabs, but various methods do so upon first access.

Test cases for the most common use cases where we encounter when we write data to HDF5. In general all data has multiple columns which a

Benchmark ChangesPlan creation and execution. Chunks that lie on the base slabs are yielded as slices, whereas the staged ones are yielded as

Benchmark HashPlan creation and execution, and the whole of commit()

Benchmark CommitPlan creation, which deduplicates the staged chunks against the hashes of the base and full chunks. Unlike the other benchmarks in

Benchmark StagedChangesArray.from_array(). This is the entry point of the resize() of an InMemoryArrayDataset, which calls from_array(as_base

Benchmark GetItemPlan creation and execution

Benchmark hash_slab with a contiguous NumPy slab.

Benchmark hash_slab with non-C-contiguous slabs. **layout** step_outer strided along axis 0, contiguous along the innermost axis (ha

Benchmark hash_slab with string arrays (object dtype and StringDType)

Benchmark LoadPlan creation and execution. Only the chunks that lie on the base slabs are loaded, so this is a no-op for all the states other

Benchmark read_many_slices between h5py src and a contiguous NumPy array

Benchmark read_many_slices between h5py src and a non-contiguous NumPy array

Benchmark read_many_slices with NumPy src and NumPy dst.

Benchmark ResizePlan creation and execution. For the "from_array" state, enlarging also deep-copies the trimmed staged slabs, as in the resiz

Benchmark SetItemPlan creation and execution

One-element update + commit of a 1e7 float64 dataset, by creation method.

Benchmark for string dtypes

Common settings for benchmarks that alter the state of the StagedChangesArray and so need setup() to run again before every call. asv runs se

Common setup for the benchmarks in this module.