MCPcopy
hub / github.com/dask/dask / test_from_delayed_to_dask_array

Function test_from_delayed_to_dask_array

dask/dataframe/io/tests/test_io.py:678–699  ·  view source on GitHub ↗
()

Source from the content-addressed store, hash-verified

676
677
678def test_from_delayed_to_dask_array():
679 # Check that `from_delayed`` can be followed
680 # by `to_dask_array` without breaking
681 # optimization behavior
682 # See: https://github.com/dask-contrib/dask-sql/issues/497
683 from dask.blockwise import optimize_blockwise
684
685 dfs = [delayed(pd.DataFrame)(np.ones((3, 2))) for i in range(3)]
686 ddf = dd.from_delayed(dfs)
687 arr = ddf.to_dask_array()
688
689 # If we optimize this graph without calling
690 # `fuse_roots`, the underlying `BlockwiseDep`
691 # `mapping` keys will be 1-D (e.g. `(4,)`),
692 # while the collection keys will be 2-D
693 # (e.g. `(4, 0)`)
694 keys = [k[0] for k in arr.__dask_keys__()]
695 dsk = optimize_blockwise(arr.dask, keys=keys)
696 dsk.cull(keys)
697
698 result = arr.compute()
699 assert result.shape == (9, 2)
700
701
702def test_from_delayed_misordered_meta():

Callers

nothing calls this directly

Calls 8

delayedFunction · 0.90
optimize_blockwiseFunction · 0.90
from_delayedMethod · 0.80
to_dask_arrayMethod · 0.80
onesMethod · 0.45
__dask_keys__Method · 0.45
cullMethod · 0.45
computeMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…