MCPcopy Create free account
hub / github.com/dask/dask / _unpack

Function _unpack

dask/base.py:547–582  ·  view source on GitHub ↗
(expr)

Source from the content-addressed store, hash-verified

545 collections_token = uuid.uuid4().hex
546
547 def _unpack(expr):
548 if is_dask_collection(expr):
549 tok = tokenize(expr)
550 if tok not in repack_dsk:
551 repack_dsk[tok] = Task(
552 tok, getitem, TaskRef(collections_token), len(collections)
553 )
554 collections.append(expr)
555 return TaskRef(tok)
556
557 tok = uuid.uuid4().hex
558 tsk: DataNode | Task # type: ignore[annotation-unchecked]
559 if not traverse:
560 tsk = DataNode(None, expr)
561 else:
562 # Treat iterators like lists
563 typ = list if isinstance(expr, Iterator) else type(expr)
564 if typ in (list, tuple, set):
565 tsk = Task(tok, typ, List(*[_unpack(i) for i in expr]))
566 elif typ in (dict, OrderedDict):
567 tsk = Task(
568 tok, typ, Dict({_unpack(k): _unpack(v) for k, v in expr.items()})
569 )
570 elif dataclasses.is_dataclass(expr) and not isinstance(expr, type):
571 tsk = Task(
572 tok,
573 typ,
574 *[_unpack(getattr(expr, f.name)) for f in dataclasses.fields(expr)],
575 )
576 elif is_namedtuple_instance(expr):
577 tsk = Task(tok, typ, *[_unpack(i) for i in expr])
578 else:
579 return expr
580
581 repack_dsk[tok] = tsk
582 return TaskRef(tok)
583
584 out = uuid.uuid4().hex
585 repack_dsk[out] = Task(out, tuple, List(*[_unpack(i) for i in args]))

Callers 1

unpack_collectionsFunction · 0.85

Calls 9

tokenizeFunction · 0.90
TaskClass · 0.90
TaskRefClass · 0.90
DataNodeClass · 0.90
ListClass · 0.90
DictClass · 0.90
is_namedtuple_instanceFunction · 0.90
is_dask_collectionFunction · 0.85
itemsMethod · 0.45

Tested by

no test coverage detected