MCPcopy Create free account
hub / github.com/apache/arrow / ReadLevels

Method ReadLevels

cpp/src/parquet/column_reader.cc:1110–1139  ·  view source on GitHub ↗

Read definition and repetition levels. Also return the number of definition levels and number of values to read. This function is called before reading values. ReadLevels will throw exception when any num-levels read is not equal to the number of the levels can be read.

Source from the content-addressed store, hash-verified

1108 // ReadLevels will throw exception when any num-levels read is not equal to the number
1109 // of the levels can be read.
1110 void ReadLevels(int64_t batch_size, int16_t* def_levels, int16_t* rep_levels,
1111 int64_t* num_def_levels, int64_t* non_null_values_to_read) {
1112 batch_size = std::min(batch_size, this->available_values_current_page());
1113
1114 // If the field is required and non-repeated, there are no definition levels
1115 if (this->max_def_level() > 0 && def_levels != nullptr) {
1116 *num_def_levels = this->ReadDefinitionLevels(batch_size, def_levels);
1117 if (ARROW_PREDICT_FALSE(*num_def_levels != batch_size)) {
1118 throw ParquetException(kErrorRepDefLevelNotMatchesNumValues);
1119 }
1120 // TODO(wesm): this tallying of values-to-decode can be performed with better
1121 // cache-efficiency if fused with the level decoding.
1122 *non_null_values_to_read +=
1123 std::count(def_levels, def_levels + *num_def_levels, this->max_def_level());
1124 } else {
1125 // Required field, read all values
1126 if (num_def_levels != nullptr) {
1127 *num_def_levels = 0;
1128 }
1129 *non_null_values_to_read = batch_size;
1130 }
1131
1132 // Not present for non-repeated fields
1133 if (this->max_rep_level() > 0 && rep_levels != nullptr) {
1134 int64_t num_rep_levels = this->ReadRepetitionLevels(batch_size, rep_levels);
1135 if (batch_size != num_rep_levels) {
1136 throw ParquetException(kErrorRepDefLevelNotMatchesNumValues);
1137 }
1138 }
1139 }
1140};
1141
1142template <typename DType>

Callers

nothing calls this directly

Calls 7

ParquetExceptionFunction · 0.85
countFunction · 0.85
max_def_levelMethod · 0.80
ReadDefinitionLevelsMethod · 0.80
max_rep_levelMethod · 0.80
ReadRepetitionLevelsMethod · 0.80

Tested by

no test coverage detected