MCPcopy Create free account
hub / github.com/apache/arrow / CanUseOnBlock

Method CanUseOnBlock

cpp/src/arrow/csv/lexing_internal.h:180–182  ·  view source on GitHub ↗

_mm_cmpistrc is an implicit-length compare: it treats a NUL byte as a terminator and can miss a real delimiter/quote/newline sharing an 8-byte word with one. Never use this filter on a block that contains a NUL. We could instead use the explicit-length compare _mm_cmpestrc but it comes with a massive performance cost on some CPUs (some benchmarks were measured to be twice slower in https://github.

Source from the content-addressed store, hash-verified

178 // (some benchmarks were measured to be twice slower in
179 // https://github.com/apache/arrow/pull/50483).
180 bool CanUseOnBlock(std::string_view data) const {
181 return data.empty() || std::memchr(data.data(), '\0', data.size()) == nullptr;
182 }
183
184 protected:
185 using BulkFilterType = __m128i;

Callers

nothing calls this directly

Calls 3

emptyMethod · 0.45
dataMethod · 0.45
sizeMethod · 0.45

Tested by

no test coverage detected