redis-vector-search

Redis Vector Search

Guidance for storing and searching embeddings in Redis. Covers index configuration, algorithm selection, hybrid filtering, and the RAG retrieval pattern with RedisVL.

When to apply

Defining a VECTOR field in FT.CREATE (raw RQE) or a RedisVL IndexSchema.
Choosing HNSW vs FLAT and tuning HNSW parameters.
Adding category, date, or tenant filters to a vector query.
Building a retrieval-augmented generation (RAG) pipeline on top of Redis.

This skill builds on the redis-query-engine skill — vector fields live inside RQE indexes and share the same FT.CREATE / FT.SEARCH machinery.

1. Configure the vector index properly

Three settings must match the embedding model:

DIM — the model's output dimensionality (e.g. 1536 for OpenAI text-embedding-3-small). A mismatch produces silent garbage.
DISTANCE_METRIC — COSINE for normalized text embeddings (the common case), IP for unnormalized inner-product, L2 for raw Euclidean.
TYPE / datatype — usually FLOAT32. Use FLOAT16 or quantized variants only when memory cost is a hard constraint.

Raw RQE:

FT.CREATE idx:docs ON HASH PREFIX 1 doc:
    SCHEMA
        content TEXT
        embedding VECTOR HNSW 6
            TYPE FLOAT32
            DIM 1536
            DISTANCE_METRIC COSINE

RedisVL:

schema = IndexSchema.from_dict({
    "index": {"name": "idx:docs", "prefix": "doc:"},
    "fields": [
        {"name": "content", "type": "text"},
        {"name": "embedding", "type": "vector", "attrs": {
            "dims": 1536, "algorithm": "HNSW",
            "datatype": "FLOAT32", "distance_metric": "COSINE",
        }},
    ]
})

See references/index-creation.md for redis-py and RedisVL variants.

2. HNSW vs FLAT

Algorithm	Speed	Accuracy	Memory	Best for
HNSW	Fast (approximate)	~95%+ recall (tunable)	Higher	Large datasets (>10k vectors), latency-sensitive
FLAT	Slow (exact)	100%	Lower	Small datasets (<10k), accuracy-critical

Default to HNSW for any production-scale workload. Tuning levers:

M — connections per node (16–64). Higher = better recall, more memory.
EF_CONSTRUCTION — build-time graph quality (100–500). Higher = better index, slower build.
EF_RUNTIME — query-time candidate-list size. Higher = better recall, slower queries.

Use FLAT when the corpus is small and you need exact results (e.g. semantic dedup over a few thousand items).

See references/algorithm-choice.md.

3. Hybrid search — filter before vector

Apply attribute filters (TAG / NUMERIC) so the engine narrows the search space before the vector comparison. Don't fetch a wide result set and then filter client-side — that's slower and less accurate.

from redisvl.query import VectorQuery
from redisvl.query.filter import Num, Tag

filters = (Tag("category") == "technology") & (Num("date") >= 2024)

query = VectorQuery(
    vector=query_embedding,
    vector_field_name="embedding",
    return_fields=["content", "category", "date"],
    num_results=10,
    filter_expression=filters,
)
results = index.query(query)

For text + vector fusion (BM25-weighted text scoring combined with vector similarity), use HybridQuery on Redis ≥ 8.4 with redis-py ≥ 7.1, or AggregateHybridQuery on older Redis. That's a different "hybrid" from filtered vector search above.

See references/hybrid-search.md.

4. RAG pattern

Standard pipeline: embed the user query → vector search Redis → pass top-K context to the LLM.

# Index documents with embeddings
records = [{"content": doc.content,
            "embedding": embed_model.encode(doc.content).tolist(),
            "source": doc.source}
           for doc in documents]
index.load(records)

# Retrieve relevant context for a user question
q_emb = embed_model.encode(user_question)
results = index.query(VectorQuery(
    vector=q_emb,
    vector_field_name="embedding",
    return_fields=["content", "source"],
    num_results=5,
))

# Generate with retrieved context
context = "\n".join(r["content"] for r in results)
response = llm.generate(f"Context: {context}\n\nQuestion: {user_question}")

Practical tips:

Match metric to model. Most modern text embedding models pair best with COSINE.
Chunk long documents before indexing — retrieval over 200–500-token chunks usually beats indexing whole pages.
Batch inserts with index.load([...]) instead of one call per record.
Pre-filter with attributes (tenant, recency, document type) before the vector search.

See references/rag-pattern.md.

About this skill