MCPcopy Create free account
hub / github.com/apache/datafusion / wrap_null_count_check_expr

Function wrap_null_count_check_expr

datafusion/pruning/src/pruning_predicate.rs:2022–2031  ·  view source on GitHub ↗

Wrap the statistics expression in a check that skips the expression if the column is all nulls. This is important not only as an optimization but also because statistics may not be accurate for columns that are all nulls. For example, for an `int` column `x` with all nulls, the min/max/null_count statistics might be set to 0 and evaluating `x = 0` would incorrectly include the column. For exampl

(
    statistics_expr: Arc<dyn PhysicalExpr>,
    expr_builder: &mut PruningExpressionBuilder,
)

Source from the content-addressed store, hash-verified

2020/// `x_null_count = x_row_count` will be true, which will cause the
2021/// boolean expression to return false. Therefore, prune out the container.
2022fn wrap_null_count_check_expr(
2023 statistics_expr: Arc<dyn PhysicalExpr>,
2024 expr_builder: &mut PruningExpressionBuilder,
2025) -> Result<Arc<dyn PhysicalExpr>> {
2026 // (x_null_count != x_row_count) AND (<statistics_expr>)
2027 Ok(and_expr(
2028 column_has_non_nulls_expr(expr_builder)?,
2029 statistics_expr,
2030 ))
2031}
2032
2033#[derive(Debug, Copy, Clone, PartialEq, Eq)]
2034pub(crate) enum StatisticsType {

Callers 1

build_statistics_exprFunction · 0.85

Calls 2

and_exprFunction · 0.85

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…