MCPcopy Create free account
hub / github.com/apache/datafusion / general_repeat

Function general_repeat

datafusion/functions-nested/src/repeat.rs:174–217  ·  view source on GitHub ↗

For each element of `array[i]` repeat `count_array[i]` times. Assumption for the input: 1. `count[i] >= 0` 2. `array.len() == count_array.len()` For example, ```text array_repeat( [1, 2, 3], [2, 0, 1] => [[1, 1], [], [3]] ) ```

(
    array: &ArrayRef,
    count_array: &Int64Array,
)

Source from the content-addressed store, hash-verified

172/// )
173/// ```
174fn general_repeat<O: OffsetSizeTrait>(
175 array: &ArrayRef,
176 count_array: &Int64Array,
177) -> Result<ArrayRef> {
178 let total_repeated_values: usize = (0..count_array.len())
179 .map(|i| get_count_with_validity(count_array, i))
180 .sum();
181
182 let mut take_indices = Vec::with_capacity(total_repeated_values);
183 let mut offsets = Vec::with_capacity(count_array.len() + 1);
184 offsets.push(O::zero());
185 let mut running_offset = 0usize;
186
187 for idx in 0..count_array.len() {
188 let count = get_count_with_validity(count_array, idx);
189 running_offset = running_offset.checked_add(count).ok_or_else(|| {
190 DataFusionError::Execution(
191 "array_repeat: running_offset overflowed usize".to_string(),
192 )
193 })?;
194 let offset = O::from_usize(running_offset).ok_or_else(|| {
195 DataFusionError::Execution(format!(
196 "array_repeat: offset {running_offset} exceeds the maximum value for offset type"
197 ))
198 })?;
199 offsets.push(offset);
200 take_indices.extend(std::iter::repeat_n(idx as u64, count));
201 }
202
203 // Build the flattened values
204 let repeated_values = compute::take(
205 array.as_ref(),
206 &UInt64Array::from_iter_values(take_indices),
207 None,
208 )?;
209
210 // Construct final ListArray
211 Ok(Arc::new(GenericListArray::<O>::try_new(
212 Arc::new(Field::new_list_field(array.data_type().to_owned(), true)),
213 OffsetBuffer::new(offsets.into()),
214 repeated_values,
215 count_array.nulls().cloned(),
216 )?))
217}
218
219/// Handle List version of `general_repeat`
220///

Callers

nothing calls this directly

Calls 13

get_count_with_validityFunction · 0.85
newFunction · 0.85
sumMethod · 0.80
mapMethod · 0.45
lenMethod · 0.45
pushMethod · 0.45
to_stringMethod · 0.45
extendMethod · 0.45
as_refMethod · 0.45
data_typeMethod · 0.45
intoMethod · 0.45
clonedMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…