MCPcopy Create free account
hub / github.com/dask/dask / compute_chunk_sizes

Method compute_chunk_sizes

dask/array/core.py:1465–1512  ·  view source on GitHub ↗

Compute the chunk sizes for a Dask array. This is especially useful when the chunk sizes are unknown (e.g., when indexing one Dask array with another). Notes ----- This function modifies the Dask array in-place. Examples --------

(self)

Source from the content-addressed store, hash-verified

1463 return reduce(mul, self.numblocks, 1)
1464
1465 def compute_chunk_sizes(self):
1466 """
1467 Compute the chunk sizes for a Dask array. This is especially useful
1468 when the chunk sizes are unknown (e.g., when indexing one Dask array
1469 with another).
1470
1471 Notes
1472 -----
1473 This function modifies the Dask array in-place.
1474
1475 Examples
1476 --------
1477 >>> import dask.array as da
1478 >>> import numpy as np
1479 >>> x = da.from_array([-2, -1, 0, 1, 2], chunks=2)
1480 >>> x.chunks
1481 ((2, 2, 1),)
1482 >>> y = x[x <= 0]
1483 >>> y.chunks
1484 ((nan, nan, nan),)
1485 >>> y.compute_chunk_sizes() # in-place computation
1486 dask.array<getitem, shape=(3,), dtype=int64, chunksize=(2,), chunktype=numpy.ndarray>
1487 >>> y.chunks
1488 ((2, 1, 0),)
1489
1490 """
1491 x = self
1492 chunk_shapes = x.map_blocks(
1493 _get_chunk_shape,
1494 dtype=int,
1495 chunks=tuple(len(c) * (1,) for c in x.chunks) + ((x.ndim,),),
1496 new_axis=x.ndim,
1497 )
1498
1499 c = []
1500 for i in range(x.ndim):
1501 s = x.ndim * [0] + [i]
1502 s[i] = slice(None)
1503 s = tuple(s)
1504
1505 c.append(tuple(chunk_shapes[s]))
1506
1507 # `map_blocks` assigns numpy dtypes
1508 # cast chunk dimensions back to python int before returning
1509 x._chunks = tuple(
1510 tuple(int(chunk) for chunk in chunks) for chunks in compute(tuple(c))[0]
1511 )
1512 return x
1513
1514 @cached_property
1515 def shape(self) -> tuple[T_IntOrNaN, ...]:

Calls 2

computeFunction · 0.90
map_blocksMethod · 0.45