Binary Quantization (BQ): sign-bit encoding + Hamming distance. Each dimension is encoded as a single bit: 1 if positive, 0 if negative. 32x compression (D/8 bytes vs 4D bytes for FP32). Best used as a coarse pre-filter: compute Hamming distance to quickly eliminate far candidates before computing exact distances on survivors. Recall loss: 5-10% as a standalone index, but combined with a reranki
(vector: &[f32])
| 15 | /// Bit layout: bit 0 of byte 0 = dimension 0, bit 1 = dimension 1, etc. |
| 16 | /// `output.len() = ceil(dim / 8)`. |
| 17 | pub fn encode(vector: &[f32]) -> Vec<u8> { |
| 18 | let num_bytes = vector.len().div_ceil(8); |
| 19 | let mut bits = vec![0u8; num_bytes]; |
| 20 | for (i, &val) in vector.iter().enumerate() { |
| 21 | if val > 0.0 { |
| 22 | bits[i / 8] |= 1 << (i % 8); |
| 23 | } |
| 24 | } |
| 25 | bits |
| 26 | } |
| 27 | |
| 28 | /// Batch encode: encode all vectors into contiguous binary representation. |
| 29 | /// |