MCPcopy Create free account
hub / github.com/MaterializeInc/materialize / write_rows_spread

Function write_rows_spread

src/clusterd-test-driver/src/data.rs:185–219  ·  view source on GitHub ↗

Writes `rows` spread across timestamps `0..n_ts` (row `i` at time `i % n_ts`) in a single append that seals `[0, n_ts)`. All rows are inserted with diff `+1`. This is one `compare_and_append` regardless of `n_ts` — persist accepts updates at any timestamp within the sealed range — so it stays fast even for very large `n_ts` (a per-timestamp append would be `n_ts` consensus round-trips).

(
    client: &PersistClient,
    shard: ShardId,
    desc: &RelationDesc,
    rows: &[Row],
    n_ts: u64,
)

Source from the content-addressed store, hash-verified

183/// very large `n_ts` (a per-timestamp append would be `n_ts` consensus
184/// round-trips).
185pub async fn write_rows_spread(
186 client: &PersistClient,
187 shard: ShardId,
188 desc: &RelationDesc,
189 rows: &[Row],
190 n_ts: u64,
191) -> anyhow::Result<()> {
192 assert!(n_ts > 0, "n_ts must be positive");
193 let mut writer = client
194 .open_writer::<SourceData, (), Timestamp, StorageDiff>(
195 shard,
196 Arc::new(desc.clone()),
197 Arc::new(UnitSchema),
198 Diagnostics {
199 shard_name: "driver-data".to_string(),
200 handle_purpose: "headless driver spread write".to_string(),
201 },
202 )
203 .await?;
204 let updates: Vec<_> = rows
205 .iter()
206 .enumerate()
207 .map(|(i, r)| {
208 let t = u64::cast_from(i) % n_ts;
209 ((SourceData(Ok(r.clone())), ()), Timestamp::from(t), 1i64)
210 })
211 .collect();
212 let lower = Antichain::from_elem(Timestamp::from(0));
213 let upper = Antichain::from_elem(Timestamp::from(n_ts));
214 writer
215 .compare_and_append(&updates, lower, upper)
216 .await?
217 .map_err(|e| anyhow::anyhow!("{e}"))?;
218 Ok(())
219}
220
221/// Number of rows needed to roughly hit `target_bytes` given `pad`-wide
222/// payloads. Overhead per row is approximate; coarse sizing, not exact.

Callers 2

executeMethod · 0.85

Calls 8

SourceDataClass · 0.85
enumerateMethod · 0.80
cloneMethod · 0.45
to_stringMethod · 0.45
collectMethod · 0.45
mapMethod · 0.45
iterMethod · 0.45
compare_and_appendMethod · 0.45

Tested by

no test coverage detected