Chunks tuple from nested list of arrays >>> x = np.array([1, 2]) >>> chunks_from_arrays([x, x]) ((2, 2),) >>> x = np.array([[1, 2]])

Constrain datetimes to be valid for pandas Since pandas works in ns precision and arrow / parquet defaults to ms precision we

Dask command line interface.

Coarsen array by applying reduction to fixed size neighborhoods Parameters ---------- reduction: function Reduction function (for

Coerce dataframe to dtypes safely Operates in place Parameters ---------- df: Pandas DataFrame dtypes: dict like {'x': float}

Collect partitions from disk and yield k,v group pairs.

Find the common block dimensions from the list of block dimensions Currently only implements the simplest possible heuristic: the common bloc

Compression level to use in svd_compressed Given the size ``n`` of a space, compress that that to one of size ``q`` plus n_oversamples.

Randomly sample matrix to find most active subspace This compression matrix returned by this algorithm can be used to compute both the QR dec

Compute the position of the first nonzero bit for each int in an array.

For each partition, returns the first row of the next nonempty partition.

Returns the dependencies between layers

For each partition, returns the last row of the most recent nonempty partition.

Recursively call np.concatenate along axes

Make nested iterators concrete lists >>> data = [[1, 2], [3, 4]] >>> seq = iter(map(iter, data)) >>> concrete(seq) [[1, 2], [3, 4]]

Add constant slice to either side of array

Get the available CPU count for this system. Takes the minimum value from the following locations: - Total system cpus available on the host

Create a task tree that merges all the keys with a reduction function. Parameters ---------- func: callable Reduction function th

Natural looking dask with some inter-connections

Remove unnecessary tasks from the layer In other words, return a new Layer with only the tasks required to calculate `keys` and a map

Return new HighLevelGraph with only the tasks required to calculate keys. In other words, remove unnecessary tasks from dask. Parame

Return a new HighLevelGraph with only the given layers and their dependencies. Internally, layers are not modified. This is a variant

Dask added an additional keyword-only argument ``method``. method : {'sequential', 'blelloch'}, optional Choose which method to use to pe

Dask added an additional keyword-only argument ``method``. method : {'sequential', 'blelloch'}, optional Choose which method to use to pe

Create new keys for ``fuse`` tasks. The optional parameter `max_fused_key_length` is used to limit the maximum string length for each renamed key

Construct a new Generator with the default BitGenerator (PCG64). Parameters ---------- seed : {None, int, array_like[ints], SeedSequ

Dask added an additional keyword-only argument ``split_every``. split_every: int >= 2 or dict(axis: int), optional Determines the depth o

Return a new array with the same shape and type as a given array. Parameters ---------- a : array_like The shape and data-ty

Apply a function using args / kwargs. If arguments contain dd.DataFrame / dd.Series, using internal cache (``_meta``) for calculation

Get a context manager to safely set environment variables All changes will be undone on close, hence environment variables set within thi

Convert a generic Set into a set. Parameters ---------- s : Set copy : bool If True, guarantee that the return value is alway

Compute task and handle all administration See Also -------- _execute_task : actually execute task

Expand environment variables in a nested config dictionary This function will recursively search through any nested dictionaries and/or lists

Return a 2-D Array with ones on the diagonal and zeros elsewhere. Parameters ---------- N : int Number of rows in the output.

A super docstring An explanation Parameters ---------- a: int an explanation of a

Find an intermediate rechunk that would merge some adjacent blocks together in order to get us nearer the *new_chunks* target, without vi

Find an intermediate rechunk that would split some chunks to get us nearer *new_chunks*, without violating the *graph_size_limit*.

Fire off a task to the thread pool

Adjust end point to ensure span of at least `min_span`

Pretty-format *blocks*. >>> format_blocks((10, 10, 10)) 3*[10] >>> format_blocks((2, 3, 4)) [2, 3, 4] >>> format_blocks((10,

>>> format_chunks((10 * (3,), 3 * (10,))) (10*[3], 3*[10])