SuperAPI AI Directory
Partner with us中文AI newsFavorites
DirectoryNews and tutorials

turbovec - Google's new open-source vector indexing algorithm

Author: AI小集 · Original source: AI工具集

What is turbovec

turbovec is Google Research TurboQuant an open-source implementation of the algorithm, written in Rust with Python bindings, and is a high-performance vector indexing library for RAG scenarios. The tool uses a data-independent quantization strategy, requiring no training phase, and can compress 10 million float32 vectors from 31GB to about 4GB. On ARM and x86 platforms, it achieves faster search speeds than FAISS through hand-written SIMD kernels, and supports filtering during search, persistence, and plug-and-play replacement for mainstream frameworks.

turbovec

Main features of turbovec

  • Online ingestion indexing: Adding vectors automatically completes indexing, with no training step, parameter tuning, or rebuilding as the corpus grows.
  • Fast SIMD search: Hand-written NEON (ARM) and AVX-512BW (x86) kernels, with search speeds faster than FAISS IndexPQFastScan.
  • Filtering during search: Supports passing an ID whitelist or slot bitmask, with filtering logic short-circuiting directly inside the SIMD kernels, with no over-fetching.
  • Stable external IDs and deletion: Via IdMapIndex supports custom uint64 external IDs and O(1) time complexity deletion by ID.
  • Index persistence: Supports write saving to disk and load fast recovery, with no re-encoding required.
  • Framework plug-and-play: Provides official integrations for LangChain, LlamaIndex, Haystack, and Agno, allowing replacement of the existing vector store by changing a few import lines.
  • Purely local operation: No hosted service is required, and data does not leave the local machine or VPC, enabling a fully offline RAG stack.

Technical principles of turbovec

  • Normalization: Strips off and stores the length (norm) of each vector separately, making the remaining part a direction vector on the unit hypersphere.
  • Random orthogonal rotation: Rotates all vectors with the same random orthogonal matrix, making each coordinate independently follow a predictable Beta distribution, independent of the original data content.
  • TQ+ adaptive calibration: When vectors are first added, each coordinate is scaled and shifted using the 5%/95% quantiles to map the empirical distribution to the standard Beta marginal distribution; subsequent vectors reuse this calibration parameter without retraining.
  • Lloyd-Max scalar quantization: Precomputes optimal quantization bucket boundaries and centroids based on the known distribution, with distortion close to 2.7 times the information-theoretic lower bound.
  • Bit packing: Compresses each coordinate into a small integer and packs it tightly, reducing a 1536-dimensional vector from 6144 bytes to 384 bytes, achieving 16x compression.
  • Length-renormalized scoring: An additional scale factor is computed during encoding and multiplied back during search to eliminate the systematic inner product underestimation caused by quantization, correcting the estimator from biased to unbiased and further improving recall.
Dig into GitHub

Follow on WeChat and reply "Open Source" to join theAI Open Source Project Communication Group

How to use turbovec

  • Install the library: Run pip install turbovec to get the Python bindings.
  • Create an index: Instantiate TurboQuantIndex(dim=1536, bit_width=4), specifying the dimension and number of quantization bits.
  • Add vectors: Call index.add(vectors) to ingest data in batches, and the system automatically performs rotation, calibration, and quantization.
  • Run search: Call index.search(query, k=10) to obtain the Top-K similarity scores and indices.
  • Persistent storage: Use index.write("my_index.tv") to save the index, and restore it via TurboQuantIndex.load .
  • External ID management: Switch to IdMapIndex to implement add_with_ids and O(1) deletion.
  • Hybrid retrieval: First use systems such as SQL/BM25 to filter candidate IDs, then pass them to allowlist for dense reranking.

turbovec project address

  • Project website：https://pypi.org/project/turbovec/
  • GitHub repository：https://github.com/RyanCodrai/turbovec

Comparison of turbovec with similar competing products

Dimension turbovec FAISS (IndexPQFastScan)
Quantization training No training required, online ingestion Requires k-means codebook training
Compression ratio 16x（2-bit）/ 8x（4-bit） Similar, depends on training quality
ARM search speed 10–19% faster than FastScan Baseline
x86 search speed 4-bit wins, 2-bit is close Baseline, 2-bit VBMI has an advantage
Search-time filtering Short-circuiting within the SIMD kernel, zero recall loss Compute first, then filter, with over-fetching
Deployment form Pure local embedded library Pure local embedded library
Framework integration Official support for 4 frameworks including LangChain Broad community support
Low-dimensional recall On par or ahead after TQ+ calibration Baseline

turbovec application scenarios

  • Memory-sensitive RAG: For small and medium teams or local deployments that need to index tens of millions of documents within limited memory, compressing 10 million vectors from 31GB to 4GB.
  • Low-latency online services: Online RAG, recommendation, or search systems with stringent requirements on vector retrieval latency, relying on SIMD acceleration.
  • Privacy-first architecture: Government, enterprise, and financial scenarios where data cannot be uploaded to third parties or leave the country, running purely locally without hosted services.
  • Edge and mobile: Strong ARM optimization, suitable for running vector retrieval on phones, IoT devices, or embedded hardware.
  • Hybrid retrieval systems: First use SQL / BM25 / permission systems to roughly filter candidate IDs, then pass them to allowlist for dense vector reranking.