MCPcopy Create free account
hub / github.com/apache/paimon-rust / assign_row_tracking_meta

Method assign_row_tracking_meta

crates/paimon/src/table/table_commit.rs:672–705  ·  view source on GitHub ↗

Assign row tracking metadata: snapshot ID as sequence number, and first_row_id for new APPEND files that don't already have one. Normal files advance the main counter. Blob files (identified by file name) use per-column counters starting from the same base, since each blob column rolls independently.

(
        &self,
        snapshot_id: i64,
        first_row_id_start: i64,
        entries: Vec<ManifestEntry>,
    )

Source from the content-addressed store, hash-verified

670 /// use per-column counters starting from the same base, since each blob column
671 /// rolls independently.
672 fn assign_row_tracking_meta(
673 &self,
674 snapshot_id: i64,
675 first_row_id_start: i64,
676 entries: Vec<ManifestEntry>,
677 ) -> (Vec<ManifestEntry>, i64) {
678 let mut result = Vec::with_capacity(entries.len());
679 let mut start = first_row_id_start;
680 // Per blob column (write_cols key) counter, each starts from first_row_id_start.
681 let mut blob_starts: HashMap<Vec<String>, i64> = HashMap::new();
682
683 for entry in entries {
684 let mut entry = entry.with_sequence_number(snapshot_id, snapshot_id);
685 if *entry.kind() == FileKind::Add
686 && entry.file().file_source == Some(0) // APPEND
687 && entry.file().first_row_id.is_none()
688 {
689 let is_blob_file =
690 crate::table::blob_file_writer::is_blob_file_name(&entry.file().file_name);
691 if is_blob_file {
692 let key = entry.file().write_cols.clone().unwrap_or_default();
693 let blob_start = blob_starts.entry(key).or_insert(first_row_id_start);
694 entry = entry.with_first_row_id(*blob_start);
695 *blob_start += entry.file().row_count;
696 } else {
697 entry = entry.with_first_row_id(start);
698 start += entry.file().row_count;
699 }
700 }
701 result.push(entry);
702 }
703
704 (result, start)
705 }
706
707 /// Validate that files with pre-assigned `first_row_id` (e.g. partial-column
708 /// files from MERGE INTO) still match existing files in the current snapshot.

Callers 1

try_commit_onceMethod · 0.80

Calls 7

is_blob_file_nameFunction · 0.85
with_sequence_numberMethod · 0.80
kindMethod · 0.80
fileMethod · 0.80
entryMethod · 0.80
with_first_row_idMethod · 0.80
lenMethod · 0.45

Tested by

no test coverage detected