turboquant.cpp is an open-source C++ implementation of the TurboQuant algorithm, and TurboQuant is a near-optimal online vector quantization algorithm. Its theoretical basis is the paper "TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate". The goal of this library is to compress high-dimensional floating-point vectors into low-bit-width integers (1-4 bits per coordinate) without preprocessing, training, or codebook learning, while preserving inner product and distance information as much as possible. Vectors are quantized independently as they arrive, making it suitable for streaming or online scenarios.
In terms of main capabilities, turboquant.cpp provides two quantizers. QuantizerMSE minimizes reconstruction error (mean squared error); QuantizerProd provides a low-variance unbiased inner product estimate. Both can achieve distortion within a constant factor (about 2.7x) of the information-theoretic lower bound. For unit-norm vectors, when the bit width is b, theoretical guarantees are described, but the materials do not give a specific distortion numerical formula. In terms of compression ratio, comparing the fp32 size with the bit-packed wire format (indices plus norm metadata), in a single-threaded environment on Apple M4 Max, using a Bazel-built optimized version to run benchmarks, a complete sweep covering dimensions d=256/768/1536 and bit widths b=1-4 can be observed. Throughput is limited by the dense d x d rotation matrix multiplication, so the complexity is O(d^2) and is almost independent of bit width.
In terms of usage, the project is built with Bazel and depends on Eigen 3.4, which is automatically resolved through the Bazel central registry and requires no manual installation. To run the example, the benchmark can be executed with bazel run -c opt //:turboquant_benchmark. To use turboquant in other Bazel projects, add the dependency in MODULE.bazel and use @turboquant as the dependency.
In terms of reproducibility, quantizers built with the same (dimension, bit width, seed) produce the same encodings on all supported platforms. Rotation and QJL projections are generated using a portable Box-Muller sampler over integers from std::mt19937, rather than using std::normal_distribution, because the latter's algorithm is not specified in the C++ standard (libc++ and libstdc++ return different sequences from the same engine). tests/determinism_test.cpp pins the output with golden hashes and runs in CI on Linux and macOS. turboquant::kAlgorithmVersion changes when the output changes. If encodings are persisted, the version should be recorded together with (dimension, bit width, seed), and encodings from other versions should be rejected.
Limitations include: only bit widths 1-4 are supported (precomputed codebooks), and extending to higher bit widths requires solving the Lloyd-Max optimization for the Beta distribution; the full d x d rotation matrix is stored in memory, which may use a lot of memory at large dimensions, and a randomized Hadamard transform can reduce this to O(d log d); there is no SIMD optimization yet, and the quantization loop can be vectorized with AVX2/AVX-512.
