Length of the first axis.

Skip if StringDType is not supported. To be called by setup() for dtype='T'.

Copy every chunk of `data` into the `raw_data` of `f`, deduplicating it against the chunks already there, and return the `{chunk_index: raw_data s

Swap every dataset in old with the corresponding one in new Datasets in old that aren't in new are ignored.

Return self._buffer as a new dtype. Return a view if possible.

Tile `shape` into blocks of whole chunks, each at most `max_bytes` in size. Blocks are grown one axis at a time, starting from the last one, so t

Load the SHA256 digests from the on-disk hash table and convert it to the layout compatible with `StagedChangesArray.hash_tables`, which is indexe

Find all data groups with dtype='O'.

Load the edge chunks along an axis from the base slabs into a new slab

Rebuild the hashtables in data_groups.

Recreate every virtual dataset `name` in the versions `versions` according to the new raw_data chunks in `raw_data_chunks_map`. Returns

Call `cb` on self and recursively on all parents up to the root. The callable must accept the node being currently visited and the name of the

Transitional hack that converts a StagedChangsArray to a legacy data_dict. This was introduced when replacing the legacy system, which was wholly

Deep-copy trimmed staged slabs into new slabs padded to full chunk_size. Trimmed staged slabs are only ever created by from_array(as_base_sla

Return True if the dtypes are compatible. Compatible dtypes are those that are either equal or both variable length strings.

Assert that actual is an object array of either bytes or str. The exact contents (bytes or str) are allowed to diverge between h5py and versio

Hypothesis draw of slice and integer indexes, with potential ellipsis

Return a writeable buffer of random data, of shape SHAPE. This mimics the _buffer attribute of an InMemoryArrayDataset, the input to StagedCh

Compute the difference between two versions of a dataset. Parameters ---------- name : str Name of the dataset

r""" Return a dictionary mapping Slice: array_of_hash. The Slices are all `reduce()`\d.

Variant of numpy.ix_ with added support for mixing with slices. Given a numpy ndindex where each element could be either a 1D fancy integer a

Return a NumPy array of the given shape, dtype, memory layout, and sample values.

Create a StagedChangesArray with staged changes as described by COMMIT_SCENARIOS, ready to be committed

Normalize the ``chunks`` parameter of create_dataset(), guessing a sensible chunk size when it is not explicitly specified.

Generate a ndarray of random strings

Assemble the parameters for read_many_slices() for a multi-dimensional index for a specific transfer type between two arrays. Parameters

Find the smallest integer y >= x where y = a + k*m for whole k's Assumes 0 <= a <= x and m >= 1. a x y | <-- m --> |

Given a start:stop:step slice or range, return the number of elements yielded. This is functionally identical to:: len(range(start, stop

Call read_many_slices() to transfer slices of data from src to dst

Create a new InMemoryGroup object by binding to a low-level GroupID. Parameters ---------- bind : h5g.GroupID Low

Iterate over the first axis. BEWARE: Modifications to the yielded data are *NOT* written to file.

Check if the given versions of a dataset are empty. Doesn't assume the dataset exists in any version. Parameters ---------- f : Vers

The SHA256 of a chunk, as 4x uint64 (one hash_tables row). Does not cover VLEN dtypes (object strings or NpyStrings).

Return the number of points taken from each chunk within self.chunk_indices. All but the last chunk always contain self.chunk_size points.

Return a StagedChangesArray covering shape, with all of its chunks on a single base slab complete with hash table, built by staging random data an

Delete the given dataset from the versions.

Delete _tmp_raw_data if it exists in the file. Parameters ---------- f : File File in which _tmp_raw_data is to be removed na

Compute the difference between two versions of a dataset. Parameters ---------- v1 : str Version of a dataset

Enlarge along a single axis

Check if a dataset exists in a given version. Parameters ---------- f : VersionedHDF5File File where the dataset may reside n

Break down a one-dimensional integer array into a (greedy) minimal list of slices. Populate input of read_many_slices(). Parameters -----

Find all groups containing datasets that are descendents of the given node. Parameters ---------- node : h5py.Group

Get the Dict which maps virtual dataset sources to raw data slices. Parameters ---------- name : str Name of the

Get the fillvalue for an empty dataset. See InMemoryDataset.fillvalue. In h5py3 variable length strings use None for the h5py fillvalue, but

Convert a one-dimensional index, preprocessed by ndindex, to a mapper

Forget everything that was read from the file, both in the live instances and in the instances registry, so that it is read again on next acce

Return True if the dataset at the given version is empty, False otherwise. Assumes the dataset exists in the given version. Parameters -

Recreate a VirtualLayout for d, replacing name1 with name2 in the source dset name

Return `raw_data`, to be used as a base slab of a StagedChangesArray of the given dtype. Variable-width strings are always stored as object d

Return the parameters to read_many_slices() for a single chunk. Returns tuple of: - chunk_sub_start - value_sub_start

Relocate the edge chunks that lie on staged slabs trimmed along axis 0. Trimmed staged slabs are only ever created by from_array(as_base_slab

Replace the prefix name1 with name2 in every string attribute of dataset that holds an absolute path, e.g. the 'raw_data' attribute of a version d

Return a string representation of the i-th row

Inverse conversion of `_data_v4_to_sc_hash_table`

Shrink along a single axis. Load partial edge chunks into memory to avoid ending up with partially overlapping chunks on disk, e.g. [

Yield a handful of strided/broadcast views of ``a``.

Traverse the object tree, returning all `raw_data` datasets. We use this instead of version_data.visit(delete_dataset) because visit() has tr

Convert to kwargs for create_dataset.

Benchmark whatever is installed in the current environment and store the results under commit_hash

Return (slab, hash_table, slab_indices, slab_offsets) of a single base slab that covers the whole of SHAPE with unique chunks, optionally preceded

Hypothesis draw of a slice object to slice an array of <arr_size> points along an axis, returning a view of <view_size> points:: arr = np

Raise if the dtypes are not compatible. Compatible dtypes are those that are either equal or both variable length strings.