Create a Dask Bag from a Series

Create Dask Dataframe from a Dask Bag. Bag should contain tuples, dict records, or scalars. Index will not be particularly meaningfu

K largest elements in collection Optionally ordered by some key function >>> import dask.bag as db >>> b = db.from_sequence(

Extracts dask values from kwargs. Currently only ``dask.bag.Item`` and ``dask.delayed.Delayed`` are supported. Returns a merged dask graph a

Add a new set of defaults to the configuration It does two things: 1. Add the defaults to a global collection to be used by refresh later

Are the provided divisions valid? Examples -------- >>> valid_divisions([1, 2, 3]) True >>> valid_divisions([3, 2, 1]) False

Ensure that all chunks of all input collections have been computed before computing the dependents of any of the chunks. The following exampl

Balance the chunk sizes Parameters ---------- chunks : tuple[int, ...] Chunk sizes for Dask array. Returns -------

>>> new = cumdims_label(((2, 3), (2, 2, 1)), 'n') >>> old = cumdims_label(((2, 2, 1), (5,)), 'o') >>> _breakpoints(new[0], old[0])

Create transformation functions for a normalized aggregate spec. Parameters ---------- spec: a list of (result-column, aggregation-f

Apply func to all keys of collection. Create a Blockwise layer whenever possible; fall back to MaterializedLayer otherwise. Parameters --

Maybe calculate new divisions by periods of size freq Used to shift the divisions for the `shift` method. If freq isn't a fixed size (not anc

Check that graph is well named and non-overlapping

Check if the chunks are regular "Regular" in this context means that along every axis, the chunks all have the same size, except the last one

Private function to perform Cholesky decomposition, which returns both lower and upper triangulars.

Combine multiple file-level statistics into a single dict of metrics that represent the average values of the parquet statistics

Depth of the expression tree Returns ------- depth: int

TODO: In pandas 2.0, mean is implemented for datetimes, but Dask returns None.

TODO: In pandas 2.0, mean is implemented for datetimes, but Dask returns None.

Extract internal cache data (``_meta``) from dd.DataFrame / dd.Series

Take the raw file statistics as returned by pyarrow (as a dict) and filter it to what we care about. The full stats are a bit too verbose and we

Get axis along which point-wise slicing results lie This is mostly a hack because I can't figure out NumPy's rule on this and can't be bother

Calculate coordinate mapping for graph construction. This function handles the high-level logic behind Blockwise graph construction. The outp

Convert a pyarrow.Schema object to pandas dtype dict

Somehow get a unique name for a Layer from a non-HighLevelGraph dask mapping

Group a dataframe and apply multiple aggregation functions. Parameters ---------- df: pandas.DataFrame The dataframe to work

Internal utility to intersect chunks for 1d after preprocessing. >>> new = cumdims_label(((2, 3), (2, 2, 1)), 'n') >>> old = cumdims_lab

List of list of tuple keys, similar to `itertools.product`. Parameters ---------- head : tuple Prefix prepended to all results.