Clone dask collections, returning equivalent collections that are generated from independent calculations. Examples -------- (tokens have been simplified for the sake of brevity) >>> import dask.array as da >>> x_i = da.asarray([1, 1, 1, 1], chunks=2) >>> y_i = x_i + 1
(*collections, omit=None, seed: Hashable = None, assume_layers: bool = True)
| 409 | |
| 410 | |
| 411 | def clone(*collections, omit=None, seed: Hashable = None, assume_layers: bool = True): |
| 412 | """Clone dask collections, returning equivalent collections that are generated from |
| 413 | independent calculations. |
| 414 | |
| 415 | Examples |
| 416 | -------- |
| 417 | (tokens have been simplified for the sake of brevity) |
| 418 | |
| 419 | >>> import dask.array as da |
| 420 | >>> x_i = da.asarray([1, 1, 1, 1], chunks=2) |
| 421 | >>> y_i = x_i + 1 |
| 422 | >>> z_i = y_i + 2 |
| 423 | >>> dict(z_i.dask) # doctest: +SKIP |
| 424 | {('array-1', 0): array([1, 1]), |
| 425 | ('array-1', 1): array([1, 1]), |
| 426 | ('add-2', 0): (<function operator.add>, ('array-1', 0), 1), |
| 427 | ('add-2', 1): (<function operator.add>, ('array-1', 1), 1), |
| 428 | ('add-3', 0): (<function operator.add>, ('add-2', 0), 1), |
| 429 | ('add-3', 1): (<function operator.add>, ('add-2', 1), 1)} |
| 430 | >>> w_i = clone(z_i, omit=x_i) |
| 431 | >>> w_i.compute() |
| 432 | array([4, 4, 4, 4]) |
| 433 | >>> dict(w_i.dask) # doctest: +SKIP |
| 434 | {('array-1', 0): array([1, 1]), |
| 435 | ('array-1', 1): array([1, 1]), |
| 436 | ('add-4', 0): (<function operator.add>, ('array-1', 0), 1), |
| 437 | ('add-4', 1): (<function operator.add>, ('array-1', 1), 1), |
| 438 | ('add-5', 0): (<function operator.add>, ('add-4', 0), 1), |
| 439 | ('add-5', 1): (<function operator.add>, ('add-4', 1), 1)} |
| 440 | |
| 441 | The typical usage pattern for clone() is the following: |
| 442 | |
| 443 | >>> x = cheap_computation_with_large_output() # doctest: +SKIP |
| 444 | >>> y = expensive_and_long_computation(x) # doctest: +SKIP |
| 445 | >>> z = wrap_up(clone(x), y) # doctest: +SKIP |
| 446 | |
| 447 | In the above code, the chunks of x will be forgotten as soon as they are consumed by |
| 448 | the chunks of y, and then they'll be regenerated from scratch at the very end of the |
| 449 | computation. Without clone(), x would only be computed once and then kept in memory |
| 450 | throughout the whole computation of y, needlessly consuming memory. |
| 451 | |
| 452 | Parameters |
| 453 | ---------- |
| 454 | collections |
| 455 | Zero or more Dask collections or nested structures of Dask collections |
| 456 | omit |
| 457 | Dask collection or nested structure of Dask collections which will not be cloned |
| 458 | seed |
| 459 | See :func:`bind` |
| 460 | assume_layers |
| 461 | See :func:`bind` |
| 462 | |
| 463 | Returns |
| 464 | ------- |
| 465 | Same as ``collections`` |
| 466 | Dask collections of the same type as the inputs, which compute to the same |
| 467 | value, or nested structures equivalent to the inputs, where the original |
| 468 | collections have been replaced. |
searching dependent graphs…