MCPcopy Create free account
hub / github.com/dask/dask / _check_divisions

Function _check_divisions

dask/dataframe/dask_expr/_expr.py:1672–1699  ·  view source on GitHub ↗
(df, i, division_min, division_max, last)

Source from the content-addressed store, hash-verified

1670
1671
1672def _check_divisions(df, i, division_min, division_max, last):
1673 if not len(df):
1674 return df
1675 if is_index_like(df):
1676 index = df
1677 else:
1678 try:
1679 index = df.index.get_level_values(0)
1680 except AttributeError:
1681 index = df.index
1682 # Check divisions
1683 real_min = index.min()
1684 real_max = index.max()
1685 # Upper division of the last partition is often set to
1686 # the max value. For all other partitions, the upper
1687 # division should be greater than the maximum value.
1688 valid_min = valid_max = True
1689 if not pd.isna(division_min):
1690 valid_min = real_min >= division_min
1691 if not pd.isna(division_max):
1692 valid_max = (real_max <= division_max) if last else (real_max < division_max)
1693 if not (valid_min and valid_max):
1694 raise RuntimeError(
1695 f"`enforce_runtime_divisions` failed for partition {i}."
1696 f" Expected a range of [{division_min}, {division_max}), "
1697 f" but the real range was [{real_min}, {real_max}]."
1698 )
1699 return df
1700
1701
1702class EnforceRuntimeDivisions(Blockwise):

Callers

nothing calls this directly

Calls 4

is_index_likeFunction · 0.85
minMethod · 0.45
maxMethod · 0.45
isnaMethod · 0.45

Tested by

no test coverage detected