MCPcopy Create free account
hub / github.com/apache/arrow / GetFragmentsAsyncImpl

Method GetFragmentsAsyncImpl

cpp/src/arrow/dataset/dataset.cc:236–244  ·  view source on GitHub ↗

Default impl delegating the work to `GetFragmentsImpl` and wrapping it into BackgroundGenerator/TransferredGenerator, which offloads potentially IO-intensive work to the default IO thread pool and then transfers the control back to the specified executor.

Source from the content-addressed store, hash-verified

234// IO-intensive work to the default IO thread pool and then transfers the control
235// back to the specified executor.
236Result<FragmentGenerator> Dataset::GetFragmentsAsyncImpl(
237 compute::Expression predicate, arrow::internal::Executor* executor) {
238 ARROW_ASSIGN_OR_RAISE(auto iter, GetFragmentsImpl(std::move(predicate)));
239 ARROW_ASSIGN_OR_RAISE(
240 auto background_gen,
241 MakeBackgroundGenerator(std::move(iter), io::default_io_context().executor()));
242 auto transferred_gen = MakeTransferredGenerator(std::move(background_gen), executor);
243 return transferred_gen;
244}
245
246struct VectorRecordBatchGenerator : InMemoryDataset::RecordBatchGenerator {
247 explicit VectorRecordBatchGenerator(RecordBatchVector batches)

Callers

nothing calls this directly

Calls 1

MakeTransferredGeneratorFunction · 0.85

Tested by

no test coverage detected