Class OptimizeStmt

fe/src/main/java/org/apache/impala/analysis/OptimizeStmt.java:70–355

Representation of an OPTIMIZE TABLE statement used to execute table maintenance tasks in Iceberg tables. It operates in 2+1 modes: full table compaction, partial compaction and no-operation. The mode is decided during analysis. 1. Full table compaction: It rewrites the entire table - compacting smal

 * - there were no files that qualified for the selection criteria in partial mode.
 */
public class OptimizeStmt extends DmlStatementBase {
  // Target table name as seen by the parser.
  private final TableName originalTableName_;
  // Data files larger than the given value will only be compacted if they are referenced
  // from delete files.
  // The value comes from FILE_SIZE_THRESHOLD_MB, but is converted to bytes.
  private final long fileSizeThreshold_;

  /////////////////////////////////////////
  // BEGIN: Members that need to be reset()

  // Target table that should be compacted. May be qualified by analyze().
  private TableName tableName_;
  private TableRef tableRef_;
  // SELECT statement that reads the data that we want to compact.
  private SelectStmt sourceStmt_;
  // Output expressions that produce the final results to write to the table.
  // Set in prepareExpressions().
  // It will contain one Expr for each column of the table.
  // The i'th expr produces the i'th column of the table.
  private List<Expr> resultExprs_ = new ArrayList<>();
  // For every column of the target table that is referenced in the optional
  // 'sort.columns' table property, this list will contain the corresponding
  // result expr from 'resultExprs_'. Before insertion, all rows
  // will be sorted by these exprs. If the list is empty, no additional sorting by
  // non-partitioning columns will be performed. The column list must not contain
  // partition columns.
  private List<Expr> sortExprs_ = new ArrayList<>();
  private List<Integer> sortColumns_;
  private TSortingOrder sortingOrder_ = TSortingOrder.LEXICAL;;
  // Exprs corresponding to the partition fields of the table.
  protected List<Expr> partitionKeyExprs_ = new ArrayList<>();
  // File paths of data files without deletes selected for compaction after file size
  // filtering.
  private Set<String> selectedIcebergFilePaths_ = new HashSet<>();
  // Describes the mode of this OPTIMIZE operation. Decided during analysis.
  // NOOP: The table was empty or no files were selected for compaction. This means that
  // the operation has no effect.
  // PARTIAL: In this mode only the selected files are compacted, all others will remain
  // unchanged. Files that will be selected:
  // - data files without deletes that are smaller than fileSizeThreshold_,
  // - all delete files,
  // - all data files with deletes.
  // Possible only if FILE_SIZE_THRESHOLD_MB is set.
  // REWRITE_ALL: Rewrite all files of the table. This will ensure that the optimized
  // table has the latest schema and partition spec. Possible if FILE_SIZE_THRESHOLD_MB is
  // not set in the SQL command or large enough that all files get selected.
  private TIcebergOptimizationMode mode_;

  // END: Members that need to be reset()
  /////////////////////////////////////////

  public OptimizeStmt(TableName tableName) {
    tableName_ = tableName;
    originalTableName_ = tableName_;
    fileSizeThreshold_ = -1;
  }