MCPcopy Create free account
hub / github.com/apache/arrow / GetFragmentsFromDatasets

Function GetFragmentsFromDatasets

cpp/src/arrow/dataset/dataset_internal.h:40–55  ·  view source on GitHub ↗

\brief GetFragmentsFromDatasets transforms a vector into a flattened FragmentIterator.

Source from the content-addressed store, hash-verified

38/// \brief GetFragmentsFromDatasets transforms a vector<Dataset> into a
39/// flattened FragmentIterator.
40inline Result<FragmentIterator> GetFragmentsFromDatasets(const DatasetVector& datasets,
41 compute::Expression predicate) {
42 // Iterator<Dataset>
43 auto datasets_it = MakeVectorIterator(datasets);
44
45 // Dataset -> Iterator<Fragment>
46 auto fn = [predicate](std::shared_ptr<Dataset> dataset) -> Result<FragmentIterator> {
47 return dataset->GetFragments(predicate);
48 };
49
50 // Iterator<Iterator<Fragment>>
51 auto fragments_it = MakeMaybeMapIterator(fn, std::move(datasets_it));
52
53 // Iterator<Fragment>
54 return MakeFlattenIterator(std::move(fragments_it));
55}
56
57inline std::shared_ptr<Schema> SchemaFromColumnNames(
58 const std::shared_ptr<Schema>& input, const std::vector<std::string>& column_names) {

Callers 1

GetFragmentsImplMethod · 0.85

Calls 4

MakeVectorIteratorFunction · 0.85
MakeMaybeMapIteratorFunction · 0.85
MakeFlattenIteratorFunction · 0.85
GetFragmentsMethod · 0.45

Tested by

no test coverage detected