Apply a function elementwise across one or more bags. Note that all ``Bag`` arguments must be partitioned identically. Parameters

Concatenate many bags together, unioning all elements. >>> import dask.bag as db >>> a = db.from_sequence([1, 2, 3]) >>> b = db.from_sequ

Read a Parquet file into a Dask DataFrame This reads a directory of Parquet data into a Dask.dataframe, one file per partition. It sele

See dd.to_csv docstring for more information

Create bag item from a dask.delayed value. See ``dask.bag.from_delayed`` for details

Threaded cached implementation of dask.get Parameters ---------- dsk: dict A dask dictionary specifying a workflow keys: key

Remove elements in collection that match predicate. >>> def iseven(x): ... return x % 2 == 0 >>> import dask.bag as db

Provides rolling transformations. .. note:: Since MultiIndexes are not well supported in Dask, this method returns a

Return the sum of the array elements over the given axis. Refer to :func:`dask.array.sum` for full documentation. See Also

Return the backend-specific RandomState class For example, the 'numpy' backend simply returns ``numpy.random.RandomState``.

Convert the input to a dask array. Subclasses of ``np.ndarray`` will be passed through as chunks unchanged. Parameters ---------- a

Tensor operation: Generalized inner and outer products A broad class of blocked algorithms and patterns can be specified with a concise multi

Apply a function to every partition across one or more bags. Note that all ``Bag`` arguments must be partitioned identically. Parame

Given a path or paths, return delayed objects that read from those paths. The path may be a filename like ``'2015-01-01.csv'`` or a globstring

Convert blocks in dask array x for new chunks. Refer to :func:`dask.array.rechunk` for full documentation. See Also --------

Skip over all subterms of the current level in the traversal

Create an empty array Returns an uninitialized array having a specified shape.

Concatenate nested lists into one long list. >>> import dask.bag as db >>> b = db.from_sequence([[1], [2, 3]]) >>> list(b)

Is this object a numpy array or something similar? This function tests specifically for an object that already has array attributes (e.g. np.

Looks like a Pandas Index

Create dask array from something that looks like an array. Input must have a ``.shape``, ``.ndim``, ``.dtype`` and support numpy-style slicing.

Returns True if any of the elements evaluate to True. Refer to :func:`dask.array.any` for full documentation. See Also -----

Clear out old results from profiler

Rearrange DataFrame into new partitions Uses hashing of `on` to map rows to output partitions. After this operation, rows with the sa

Convert many collections into a single dask expression. Typically, users should not be required to interact with this function. Paramet

Count the number of elements. Examples -------- >>> import dask.bag as db >>> numbers = db.from_sequence([1, 2, 3])

Return the function implementation for the given ``cls``

Create a uniformly filled array Returns a new array having a specified shape and filled with fill_value.

Concatenate arrays along an existing axis Given a sequence of dask Arrays form a new dask Array by stacking them along an existing dimen

Map a function over blocks of the array with some overlap Refer to :func:`dask.array.map_overlap` for full documentation. See Also

Sum all elements

Iterate over x, yielding (index, value, entering), where * ``index``: a tuple of indices up to this point * ``value``: equal

Read HDF5 files into a DataFrame collection Parameters ---------- pattern : str or list Source path(s). k

See dd.to_hdf docstring for more information

Reorders one dimensions of a Dask Array based on an indexer. Refer to :func:`dask.array.shuffle` for full documentation. See Also

Save array to the zarr storage format See https://zarr.readthedocs.io for details about the format. Refer to :func:`dask.array.to_za

Convert the input to a dask array. Parameters ---------- a : array-like Input data, in any form that can be converted to a dask a

Search the expression graph for a specific operation type Parameters ---------- operation The operation type to s

Group collection by key function This requires a full dataset read, serialization and shuffle. This is expensive. If possible you sh

Return a flattened array. Refer to :func:`dask.array.ravel` for full documentation. See Also -------- dask.array.rav

General version of reductions Parameters ---------- x: Array Data being reduced along one or more axes chunk: callable(x_chun

Reduce collection with reduction operators. Parameters ---------- perpartition: function reduction to apply to ea

Parallel version of pandas GroupBy.shift This mimics the pandas version except for the following: If the grouper does not align with

.. note:: Pandas currently uses an ``object``-dtype column to represent boolean data with missing values. This can cau

Read SQL database table into a DataFrame. If neither ``divisions`` or ``npartitions`` is given, the memory footprint of the first few ro

Return the name of a type Examples -------- >>> typename(int) 'int' >>> from dask.core import literal >>> typename(lite

Returns the average of the array elements along given axis. Refer to :func:`dask.array.mean` for full documentation. See Also

Return the product of the array elements over the given axis Refer to :func:`dask.array.prod` for full documentation. See Also

Take the first k elements. Parameters ---------- k : int The number of elements to return npartitions : i

This fixture returns a function that, when called inside the test, XFAILs the currently running test. You should prefer using `@pytest.m

Simulate runtime metrics as though running tasks one at a time in order. These diagnostics can help reveal behaviors of and issues with ``order``

Get dependencies and dependents from dask dask graph >>> inc = lambda x: x + 1 >>> dsk = {'a': 1, 'b': (inc, 'a'), 'c': (inc, 'b')} >>> d

Attempt to import a required dependency. Raises a RuntimeError if the requested module is not available.

Note that this method clears any known divisions. If your mapping function is monotonically increasing then use `is_monotonic`

Register dispatch of `func` on arguments of type `type`

Random sample of items Parameters ---------- n : int, optional Number of items to return is not supported by dask

If a is dask.array, return dask.array.asarray(a, **kwargs), otherwise return np.asarray(a, like=like, **kwargs), dispatching the call to

Compute :meth:`toolz.accumulate` with caching. Caching is by the identify of `seq` rather than the value. It is thus important that `seq` is