MCPcopy Create free account
hub / github.com/dask/dask / _check_divisions

Function _check_divisions

dask/dataframe/dask_expr/_expr.py:1676–1703  ·  view source on GitHub ↗
(df, i, division_min, division_max, last)

Source from the content-addressed store, hash-verified

1674
1675
1676def _check_divisions(df, i, division_min, division_max, last):
1677 if not len(df):
1678 return df
1679 if is_index_like(df):
1680 index = df
1681 else:
1682 try:
1683 index = df.index.get_level_values(0)
1684 except AttributeError:
1685 index = df.index
1686 # Check divisions
1687 real_min = index.min()
1688 real_max = index.max()
1689 # Upper division of the last partition is often set to
1690 # the max value. For all other partitions, the upper
1691 # division should be greater than the maximum value.
1692 valid_min = valid_max = True
1693 if not pd.isna(division_min):
1694 valid_min = real_min >= division_min
1695 if not pd.isna(division_max):
1696 valid_max = (real_max <= division_max) if last else (real_max < division_max)
1697 if not (valid_min and valid_max):
1698 raise RuntimeError(
1699 f"`enforce_runtime_divisions` failed for partition {i}."
1700 f" Expected a range of [{division_min}, {division_max}), "
1701 f" but the real range was [{real_min}, {real_max}]."
1702 )
1703 return df
1704
1705
1706class EnforceRuntimeDivisions(Blockwise):

Callers

nothing calls this directly

Calls 4

is_index_likeFunction · 0.85
minMethod · 0.45
maxMethod · 0.45
isnaMethod · 0.45

Tested by

no test coverage detected