MCPcopy Create free account
hub / github.com/apache/arrow / GetBatch

Method GetBatch

cpp/src/arrow/util/rle_encoding_internal.h:418–444  ·  view source on GitHub ↗

Get a batch of values return the number of decoded elements. May write fewer elements to the output than requested if there are not enough values left.

Source from the content-addressed store, hash-verified

416 /// May write fewer elements to the output than requested if there are not enough values
417 /// left.
418 [[nodiscard]] rle_size_t GetBatch(value_type* out, rle_size_t batch_size,
419 rle_size_t value_bit_width) {
420 const int64_t bits_read = static_cast<int64_t>(values_read_) * value_bit_width;
421 const int64_t bytes_fully_read = bits_read / 8;
422 // The parser only creates runs whose full payload fits in max_read_bytes_ (see
423 // BitPackedRun), so the max_read_bytes difference below is in [0, max_read_bytes_]
424 // and fits an int. A negative (unbounded) max_read_bytes_ stays negative.
425 ARROW_DCHECK(max_read_bytes_ < 0 || bytes_fully_read <= max_read_bytes_);
426 const uint8_t* unread_data = data_ + bytes_fully_read;
427
428 const ::arrow::internal::UnpackOptions opts{
429 /* .batch_size= */ std::min(batch_size, remaining()),
430 /* .bit_width= */ value_bit_width,
431 /* .bit_offset= */ static_cast<int>(bits_read % 8),
432 /* .max_read_bytes= */ static_cast<int>(max_read_bytes_ - bytes_fully_read),
433 };
434
435 if constexpr (std::is_same_v<T, bool>) {
436 ::arrow::internal::unpack(unread_data, out, opts);
437
438 } else {
439 ::arrow::internal::unpack(
440 unread_data, reinterpret_cast<std::make_unsigned_t<value_type>*>(out), opts);
441 }
442 values_read_ += opts.batch_size;
443 return opts.batch_size;
444 }
445
446 private:
447 /// The pointer to the beginning of the run

Callers

nothing calls this directly

Calls 1

unpackFunction · 0.70

Tested by

no test coverage detected