Return a context manager to stage a new version The context manager returns a group, which should be modified in-place to bu

Create a new StagedChangesArray with all chunks already in memory and full of fill_value. It won't consume any significant amounts of

Create a new StagedChangesArray from an array. Parameters ---------- as_base_slabs: True (default)

Return a writeable Copy-on-Write (CoW) copy of self. Staged slabs will be individually deep-copied upon first access on either side.

Consolidate all staged chunks into a single, brand new base slab. This is the act of moving every chunk that lies on a staged slab to a new b

Reference (slow) implementation of a single-chunk hash

Completely delete the given versions from a file This function should be used instead of deleting the version group directly, as this will no

Modify metadata for a versioned dataset in-place. The metadata is modified for all versions containing a dataset. `f` should be the h5p

Hash the whole of ``slab`` (or its ``count``-trimmed prefix) as one chunk.

Assert that the setup was executed just before the test invoking this method. This is currently not the case for reruns, which means that you

Preprocess a numpy fancy index used in __getitem__ or __setitem__ Returns ------- - ndindex.Tuple with the preprocessed index - list

Returns ceil(a/b). Assumes a >= 0 and b > 0. Note ---- This module is compiled with the cython.cdivision flag and the data type is un

Make sure the VersionedHDF5File object is no longer reachable.

Variant of np.asarray(a, dtype=dtype), with some differences: 1. If a is a numpy-like array, don't coerce it to a numpy.ndarray 2. If a has a

Hash a single chunk of ``slab`` with hash_slab and return its digest, double-checking it against the reference implementation. The chunk is `

Create a copy of self with changed fill_value. TODO this method is implemented very naively, as it loads all chunks from the base sla

Fixture which provides a function that creates an hdf5 file, optionally with groups.

Yield all the changed chunks so far, as tuples of - slice index in the base virtual array - index of the slab in the slabs list

Create a new virtual dataset by stitching the chunks of the raw dataset together, as indicated by the slices dict. See Also --------

Load all chunks that are not yet in memory from the base array.

Equivalent of ``isinstance(a, ArrayProtocol)`` or ``isinstance(a, MutableArrayProtocol)`` should these classes have the @runtime_checkable dec

Find all chunks within a selection Parameters ---------- slab_indices: StagedChangesArray.slab_indices slab_offsets:

Assert that the StagedChangesArray.slab_offsets matches the expectations. This is useful to test chunk reuse.

Create a new version. datasets should be a dictionary mapping {path: dataset}, where `dataset` is a *Dataset object. If make_curren

Create the version group for a new version. prev_version should be a pre-existing version name, None, or '' If it is None, it defaults t

Find all dtype='O' data groups and rebuild their hashtables.

Formulate a plan to resize the array in place. See Also -------- _setitem_plan

Return a generator that iterates all versions by name If include_first is True, it will include '__first_version__'. Note that the orde

Delete version `version_name`.

Create a StagedChangesArray with the chunks laid out as described by STATES

Position-indexed hash table for a chunk-aligned 1D base slab, as the on-disk legacy hash table would be inverted into.

Internal stage 1 of commit(): hash all staged chunks (and the full slab) in place.

Fetch a slab by index, or the __getitem__/__setitem__ array if idx is None. If there was a previous call to copy() and the slab is going to b

Generate one or more TransferPlan, one for each pair of source and destination slabs. Parameters ---------- slab_indices: Sta

Test BufferMixin._swaps_counter. Do nothing for plain h5py.Dataset.

Override Dataset.dtype to allow hot-swapping equivalent dtypes, e.g. NpyStrings <-> object strings

Compute hash for `data` array.

Return True if the dtype is a variable length string dtype, either a NpyString (a.k.a. StringDType) or an h5py object string; False otherwise.

Recreate dataset from all versions into `newf` `newf` should be a versioned hdf5 file/group that is already initialized (it may or may n

Create a temporary group in `f` for use with :func:`recreate_dataset`.

Implement common workflow of __setitem__, resize, and load.

Formulate a plan to load all chunks from the base slabs into staged slabs. See Also -------- _setitem_plan

Attempt to convert an integer or boolean array index to a slice

Normalize the parameters of Dataset.resize()

Create a new raw dataset without the chunks from versions_to_delete. Parameters ---------- f : VersionedHDF5File | File File for

Formulate a plan to set a slice of the array. Parameters ---------- idx: First argument of __setitem__. Note that

Return a new StagedChangesArray with a different dtype. Staged slabs will be converted lazily, upon first access. Parameters

Create a copy of self with changed chunk_size. This loads all chunks into memory.

Close the dataset, thus ensuring everything has been flushed to disk, and reopen it.

Change the array shape in place and fill new elements with self.fill_value. When enlarging, edge chunks which are not exactly divisible by ch

Run a command in the project root, raising if it fails. If `tee` is given, stdout is also written to that file.

Create two versions of a 3x3 dataset, the second of which alters a single chunk. Return the expected contents of the second version.

Formulate a plan to export all the staged and unchanged chunks. This is a read-only operation.

Formulate a plan to deduplicate and consolidate all staged chunks. You must run _calc_hashes() before invoking this. See Also

Deep-copy a slab into a new NumPy array, padded to full chunk_size.

Formulate a plan to get a slice of the array. This is a read-only operation.

Formulate a plan to hash all staged chunks. This is a read-only operation.

Recreate the hashtable for the dataset f, with only the new chunks in the raw_data_chunks_map. If tmp=True, a new hashtable called '_tmp

Replace the prefix name1 with name2 in path

Commit a StagedChangesArray into `raw_data` and its on-disk hash table. 1. Load the on-disk hash table dataset that hashes all chunks of `raw_dat

Inverse of stop2count. When count == 0 or when step>1, multiple stops can yield the same count. This function returns the smallest stop >= st

Format a numpy ndindex for pretty-printing. >>> format_ndindex(slice(None)) ':' >>> format_ndindex(slice(None, 10, 2)) ':10:2' >>

Reverse engineer create_dataset kwargs from an h5py.Dataset.

Hash a single chunk via hash_slab (treating it as a one-chunk slab) and return the 32-byte digest.