MCPcopy Create free account
hub / github.com/dask/dask / _check_dependents_are_predicates

Function _check_dependents_are_predicates

dask/dataframe/dask_expr/_expr.py:4019–4057  ·  view source on GitHub ↗
(
    expr, other_names, parent: Expr, dependents, allow_reduction=True
)

Source from the content-addressed store, hash-verified

4017
4018
4019def _check_dependents_are_predicates(
4020 expr, other_names, parent: Expr, dependents, allow_reduction=True
4021):
4022 # singleton approach should make this easier
4023
4024 # Walk down the predicate side from the filter to see if we can arrive at
4025 # other_names without hitting an expression that has other dependents that
4026 # are not part of the predicate, see test_filter_pushdown_unavailable
4027 allowed_expressions = {parent._name}
4028 stack = parent.dependencies()
4029 seen = set()
4030 all_dependents = set()
4031
4032 while stack:
4033 e = stack.pop()
4034 if expr._name == e._name:
4035 continue
4036
4037 if e._name in seen:
4038 continue
4039 seen.add(e._name)
4040
4041 if isinstance(e, _DelayedExpr):
4042 continue
4043
4044 all_dependents.update(
4045 {x()._name for x in dependents[e._name] if x() is not None}
4046 )
4047
4048 if not allow_reduction:
4049 if isinstance(e, (ApplyConcatApply, TreeReduce, ShuffleReduce)):
4050 return False
4051
4052 allowed_expressions.add(e._name)
4053 stack.extend(e.dependencies())
4054
4055 return all_dependents.issubset(allowed_expressions) and other_names.issubset(
4056 allowed_expressions
4057 )
4058
4059
4060def calc_divisions_for_align(*exprs, allow_shuffle=True):

Callers 1

Calls 4

setClass · 0.85
popMethod · 0.80
dependenciesMethod · 0.45
addMethod · 0.45

Tested by

no test coverage detected