K: (seq_k, head_dim) -> compressed dict.

V: (seq_v, head_dim) -> compressed dict (MSE only, no QJL).

Asymmetric estimator: term1 (MSE dot) + term2 (QJL correction).

Full pipeline: scores -> softmax -> weighted V sum.

indices -> centroids -> un-rotate -> rescale.

Random Gaussian projection matrix for QJL.

Haar-distributed random orthogonal matrix via QR of Gaussian.

Returns (centroids, boundaries) as sorted float32 tensors. centroids: (2^bits,) boundaries: (2^bits - 1,)

Higher bits should produce lower MSE in scores.

Compare our engine against the original cutiledump compressors.py implementation.

Estimated scores should correlate well with true Q·K^T scores.

MSE distortion per vector <= sqrt(3) * pi/2 * (1/4^b) for unit vectors.

More bits → strictly lower reconstruction MSE.

k_mse should be a reasonable approximation of K.

Per-vector MSE should be bounded by paper's theoretical upper bound.

Can we still find the most-attended key after compression?

Vector norms should be approximately preserved after round-trip.

Values close to a centroid should map to that centroid's index.

Cosine similarity between original and reconstructed should be high.

Quantize → dequantize should map each centroid exactly to itself.

Scores should be scaled by 1/√d.

Typical decode scenario: seq_q=1, seq_k=large.

The asymmetric estimator should be unbiased: E[estimated_ip] ≈ true_ip. We test by averaging over many random pairs.

Score-only kernel. One program per query block, streams over KV.

2-bit MSE (4 centroids) + 1-bit QJL. total_bits=3.

3-bit MSE (8 centroids) + 1-bit QJL. total_bits=4.

MSE-only value compression, 2-bit (4 levels).

MSE-only value compression, 3-bit (8 levels). No QJL.

4-level centroid lookup, un-rotate, rescale.

8-level centroid lookup, un-rotate, rescale.

Fused attention + online softmax. V is pre-decompressed FP16.

Fused attention + online softmax, decompresses 2-bit V on-chip.

Fused attention + online softmax, decompresses 3-bit V on-chip.