MCPcopy Create free account
hub / github.com/apache/datafusion / push_batch

Method push_batch

datafusion/physical-plan/src/recursive_query.rs:311–328  ·  view source on GitHub ↗

Push a clone of the given batch to the in memory buffer, and then return a poll with it.

(
        mut self: std::pin::Pin<&mut Self>,
        mut batch: RecordBatch,
    )

Source from the content-addressed store, hash-verified

309 /// Push a clone of the given batch to the in memory buffer, and then return
310 /// a poll with it.
311 fn push_batch(
312 mut self: std::pin::Pin<&mut Self>,
313 mut batch: RecordBatch,
314 ) -> Poll<Option<Result<RecordBatch>>> {
315 let baseline_metrics = self.baseline_metrics.clone();
316
317 if let Some(deduplicator) = &mut self.distinct_deduplicator {
318 let _timer_guard = baseline_metrics.elapsed_compute().timer();
319 batch = deduplicator.deduplicate(&batch)?;
320 }
321
322 if let Err(e) = self.reservation.try_grow(batch.get_array_memory_size()) {
323 return Poll::Ready(Some(Err(e)));
324 }
325 self.buffer.push(batch.clone());
326 (&batch).record_output(&baseline_metrics);
327 Poll::Ready(Some(Ok(batch)))
328 }
329
330 /// Start polling for the next iteration, will be called either after the static term
331 /// is completed or another term is completed. It will follow the algorithm above on

Callers 1

poll_nextMethod · 0.45

Calls 7

timerMethod · 0.80
deduplicateMethod · 0.80
record_outputMethod · 0.80
cloneMethod · 0.45
elapsed_computeMethod · 0.45
try_growMethod · 0.45
pushMethod · 0.45

Tested by

no test coverage detected