MCPcopy Create free account
hub / github.com/apache/arrow-rs / from_bitwise_unary_op

Method from_bitwise_unary_op

arrow-buffer/src/buffer/boolean.rs:228–279  ·  view source on GitHub ↗

Create a new [`BooleanBuffer`] by applying the bitwise operation to `op` to an input buffer. This function is faster than applying the operation bit by bit as it processes input buffers in chunks of 64 bits (8 bytes) at a time # Notes: `op` takes a single `u64` inputs and produces one `u64` output. `op` must only apply bitwise operations on the relevant bits; the input `u64` may contain irreleva

(
        src: impl AsRef<[u8]>,
        offset_in_bits: usize,
        len_in_bits: usize,
        mut op: F,
    )

Source from the content-addressed store, hash-verified

226 /// assert_eq!(result.values(), &expected);
227 /// ```
228 pub fn from_bitwise_unary_op<F>(
229 src: impl AsRef<[u8]>,
230 offset_in_bits: usize,
231 len_in_bits: usize,
232 mut op: F,
233 ) -> Self
234 where
235 F: FnMut(u64) -> u64,
236 {
237 let end = offset_in_bits + len_in_bits;
238 // Align start and end to 64 bit (8 byte) boundaries if possible to allow using the
239 // optimized code path as much as possible.
240 let aligned_offset = offset_in_bits & !63;
241 let aligned_end_bytes = bit_util::ceil(end, 64) * 8;
242 let src_len = src.as_ref().len();
243 let slice_end = aligned_end_bytes.min(src_len);
244
245 let aligned_start = &src.as_ref()[aligned_offset / 8..slice_end];
246
247 let (prefix, aligned_u64s, suffix) = unsafe { aligned_start.as_ref().align_to::<u64>() };
248 match (prefix, suffix) {
249 ([], []) => {
250 // the buffer is word (64 bit) aligned, so use optimized Vec code.
251 let result_u64s: Vec<u64> = aligned_u64s.iter().map(|l| op(*l)).collect();
252 return BooleanBuffer::new(result_u64s.into(), offset_in_bits % 64, len_in_bits);
253 }
254 ([], suffix) => {
255 let suffix = read_u64(suffix);
256 let result_u64s: Vec<u64> = aligned_u64s
257 .iter()
258 .cloned()
259 .chain(std::iter::once(suffix))
260 .map(&mut op)
261 .collect();
262 return BooleanBuffer::new(result_u64s.into(), offset_in_bits % 64, len_in_bits);
263 }
264 _ => {}
265 }
266
267 // align to byte boundaries
268 // Use unaligned code path, handle remainder bytes
269 let chunks = aligned_start.chunks_exact(8);
270 let remainder = chunks.remainder();
271 let iter = chunks.map(|c| u64::from_le_bytes(c.try_into().unwrap()));
272 let vec_u64s: Vec<u64> = if remainder.is_empty() {
273 iter.map(&mut op).collect()
274 } else {
275 iter.chain(Some(read_u64(remainder))).map(&mut op).collect()
276 };
277
278 BooleanBuffer::new(vec_u64s.into(), offset_in_bits % 64, len_in_bits)
279 }
280
281 /// Create a new [`BooleanBuffer`] by applying the bitwise operation `op` to
282 /// the relevant bits from two input buffers.

Callers

nothing calls this directly

Calls 7

collectMethod · 0.80
ceilFunction · 0.50
read_u64Function · 0.50
lenMethod · 0.45
as_refMethod · 0.45
iterMethod · 0.45
is_emptyMethod · 0.45

Tested by

no test coverage detected