Method try_pushdown_sort

datafusion/datasource/src/file_scan_config/mod.rs:956–1023

Push sort requirements into file-based data sources. # Sort Pushdown Architecture When a partition (file group) contains multiple files in wrong order, `validated_output_ordering()` strips the ordering and `EnforceSorting` inserts a `SortExec`. This optimizer fixes the file order by sorting files within each group by min/max statistics, enabling sort elimination. This applies to both single-par

(
        &self,
        order: &[PhysicalSortExpr],
    )

    ///              NO  → Inexact (files reordered, Sort stays)
    /// ```
    fn try_pushdown_sort(
        &self,
        order: &[PhysicalSortExpr],
    ) -> Result<SortOrderPushdownResult<Arc<dyn DataSource>>> {
        let pushdown_result = self
            .file_source
            .try_pushdown_sort(order, &self.eq_properties())?;

        match pushdown_result {
            SortOrderPushdownResult::Exact { inner } => {
                let config = self.rebuild_with_source(inner, true, order)?;
                // rebuild_with_source keeps output_ordering only when all groups
                // are non-overlapping. If output_ordering was cleared, files
                // overlap despite within-file ordering → downgrade to Inexact.
                if config.output_ordering.is_empty() {
                    Ok(SortOrderPushdownResult::Inexact {
                        inner: Arc::new(config),
                    })
                } else {
                    Ok(SortOrderPushdownResult::Exact {
                        inner: Arc::new(config),
                    })
                }
            }
            SortOrderPushdownResult::Inexact { inner } => {
                let mut config = self.rebuild_with_source(inner, false, order)?;
                // `rebuild_with_source` reorders files by stats; if the
                // post-sort files are non-overlapping AND the request now
                // validates against the new file groups, `output_ordering`
                // is preserved and we can upgrade back to Exact. This
                // restores the sort-elimination behaviour that lived in
                // the `Unsupported` → `try_sort_file_groups_by_statistics`
                // path before #21956 routed `column_in_file_schema` cases
                // here.
                if config.output_ordering.is_empty() {
                    return Ok(SortOrderPushdownResult::Inexact {
                        inner: Arc::new(config),
                    });
                }
                // Upgrading to Exact: the post-sort file groups are
                // non-overlapping and each file's declared ordering
                // re-validates, so reading the files in their natural
                // (declared-sorted) order already yields the requested
                // ordering — exactly like the `Unsupported` → Exact path,
                // which reads files in natural order too.
                //
                // Drop the runtime row-group reorder hints the Inexact
                // source carried (`sort_order_for_reorder` /
                // `reverse_row_groups`) by restoring the original,
                // hint-free source. With the `SortExec` removed those
                // hints are not just redundant but unsafe: for a DESC
                // request the opener sorts row groups ASC-by-min and then
                // reverses them, which reorders two row groups within a
                // single file that share the same `min` incorrectly
                // (e.g. a file `[10,8,8,8]` whose row groups are
                // `[10,8]` and `[8,8]` would stream as `8,8,10,8`).
                // The `SortExec` used to mask this; once it is gone the
                // reordered stream is the final, wrong answer.