MCPcopy Create free account
hub / github.com/apache/arrow / StreamingReaderImpl

Method StreamingReaderImpl

cpp/src/arrow/json/reader.cc:356–371  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

354class StreamingReaderImpl : public StreamingReader {
355 public:
356 StreamingReaderImpl(DecodedBlock first_block, AsyncGenerator<DecodedBlock> source,
357 const std::shared_ptr<DecodeContext>& context, int max_readahead)
358 : first_block_(std::move(first_block)),
359 schema_(first_block_->record_batch->schema()),
360 bytes_processed_(std::make_shared<std::atomic<int64_t>>(0)) {
361 // Set the final schema for future invocations of the source generator
362 context->SetStrictSchema(schema_);
363 if (max_readahead > 0) {
364 source = MakeReadaheadGenerator(std::move(source), max_readahead);
365 }
366 generator_ = MakeMappedGenerator(
367 std::move(source), [counter = bytes_processed_](const DecodedBlock& out) {
368 counter->fetch_add(out.num_bytes);
369 return out.record_batch;
370 });
371 }
372
373 static Future<std::shared_ptr<StreamingReaderImpl>> MakeAsync(
374 std::shared_ptr<DecodeContext> context, std::shared_ptr<io::InputStream> stream,

Callers

nothing calls this directly

Calls 4

MakeReadaheadGeneratorFunction · 0.85
MakeMappedGeneratorFunction · 0.85
SetStrictSchemaMethod · 0.80
schemaMethod · 0.45

Tested by

no test coverage detected