MCPcopy Create free account
hub / github.com/apache/datafusion / unnest_list_array

Function unnest_list_array

datafusion/physical-plan/src/unnest.rs:926–957  ·  view source on GitHub ↗

Unnest a list array according the target length array. Consider a list array like this: ```ignore [1], [2, 3, 4], null, [5], [], ``` and the length array is: ```ignore [2, 3, 2, 1, 2] ``` If the length of a certain list is less than the target length, pad with NULLs. So the unnested array will look like this: ```ignore [1, null, 2, 3, 4, null, null, 5, null, null] ```

(
    list_array: &dyn ListArrayType,
    length_array: &PrimitiveArray<Int64Type>,
    capacity: usize,
)

Source from the content-addressed store, hash-verified

924/// [1, null, 2, 3, 4, null, null, 5, null, null]
925/// ```
926fn unnest_list_array(
927 list_array: &dyn ListArrayType,
928 length_array: &PrimitiveArray<Int64Type>,
929 capacity: usize,
930) -> Result<ArrayRef> {
931 let values = list_array.values();
932 let mut take_indices_builder = PrimitiveArray::<Int64Type>::builder(capacity);
933 for row in 0..list_array.len() {
934 let mut value_length = 0;
935 if !list_array.is_null(row) {
936 let (start, end) = list_array.value_offsets(row);
937 value_length = end - start;
938 for i in start..end {
939 take_indices_builder.append_value(i)
940 }
941 }
942 let target_length = length_array.value(row);
943 debug_assert!(
944 value_length <= target_length,
945 "value length is beyond the longest length"
946 );
947 // Pad with NULL values
948 for _ in value_length..target_length {
949 take_indices_builder.append_null();
950 }
951 }
952 Ok(kernels::take::take(
953 &values,
954 &take_indices_builder.finish(),
955 None,
956 )?)
957}
958
959/// Creates take indices that will be used to expand all columns except for the list type
960/// [`columns`](UnnestExec::list_column_indices) that is being unnested.

Callers 2

unnest_list_arraysFunction · 0.85
verify_unnest_list_arrayFunction · 0.85

Calls 8

value_offsetsMethod · 0.80
valuesMethod · 0.45
lenMethod · 0.45
is_nullMethod · 0.45
append_valueMethod · 0.45
valueMethod · 0.45
append_nullMethod · 0.45
finishMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…