MCPcopy Create free account
hub / github.com/apache/arrow / MaxBufferSize

Method MaxBufferSize

cpp/src/arrow/util/rle_encoding_internal.h:692–707  ·  view source on GitHub ↗

Returns the maximum byte size it could take to encode 'num_values'. Note: because of the way CheckBufferFull() is called, you have to reserve an extra "RleEncoder::MinBufferSize" bytes. These extra bytes won't be used but not reserving them can cause the encoder to fail.

Source from the content-addressed store, hash-verified

690 /// reserve an extra "RleEncoder::MinBufferSize" bytes. These extra bytes
691 /// won't be used but not reserving them can cause the encoder to fail.
692 static int64_t MaxBufferSize(int bit_width, int64_t num_values) {
693 // For a bit_width > 1, the worst case is the repetition of "literal run of length 8
694 // and then a repeated run of length 8".
695 // 8 values per smallest run, 8 bits per byte
696 int64_t bytes_per_run = bit_width;
697 int64_t num_runs = ::arrow::bit_util::CeilDiv(num_values, 8);
698 int64_t literal_max_size = num_runs + num_runs * bytes_per_run;
699
700 // In the very worst case scenario, the data is a concatenation of repeated
701 // runs of 8 values. Repeated run has a 1 byte varint followed by the
702 // bit-packed repeated value
703 int64_t min_repeated_run_size = 1 + ::arrow::bit_util::BytesForBits(bit_width);
704 int64_t repeated_max_size = num_runs * min_repeated_run_size;
705
706 return std::max(literal_max_size, repeated_max_size);
707 }
708
709 /// Encode value. Returns true if the value fits in buffer, false otherwise.
710 /// This value must be representable with bit_width_ bits.

Callers

nothing calls this directly

Calls 2

CeilDivFunction · 0.85
BytesForBitsFunction · 0.85

Tested by

no test coverage detected