Redis vector search

Redis Vector Search

Guidance for storing and searching embeddings in Redis. Covers index configuration, algorithm selection, hybrid filtering, and the RAG retrieval pattern with RedisVL.

When to apply

• Defining a VECTOR field in FT.CREATE (raw RQE) or a RedisVL IndexSchema.
• Choosing HNSW vs FLAT and tuning HNSW parameters.
• Adding category, date, or tenant filters to a vector query.
• Building a retrieval-augmented generation (RAG) pipeline on top of Redis.

This skill builds on the redis-query-engine skill — vector fields live inside RQE indexes and share the same FT.CREATE / FT.SEARCH machinery.

1. Configure the vector index properly

Three settings must match the embedding model:

• DIM — the model's output dimensionality (e.g. 1536 for OpenAI text-embedding-3-small). A mismatch produces silent garbage.
• DISTANCE_METRIC — COSINE for normalized text embeddings (the common case), IP for unnormalized inner-product, L2 for raw Euclidean.
• TYPE / datatype — usually FLOAT32. Use FLOAT16 or quantized variants only when memory cost is a hard constraint.

Raw RQE:

FT.CREATE idx:docs ON HASH PREFIX 1 doc:
    SCHEMA
        content TEXT
        embedding VECTOR HNSW 6
            TYPE FLOAT32
            DIM 1536
            DISTANCE_METRIC COSINE

RedisVL:

schema = IndexSchema.from_dict({
    "index": {"name": "idx:docs", "prefix": "doc:"},
    "fields": [
        {"name": "content", "type": "text"},
        {"name": "embedding", "type": "vector", "attrs": {
            "dims": 1536, "algorithm": "HNSW",
            "datatype": "FLOAT32", "distance_metric": "COSINE",
        }},
    ]
})

See references/index-creation.md for redis-py and RedisVL variants.

2. HNSW vs FLAT

Algorithm	Speed	Accuracy	Memory	Best for
HNSW	Fast (approximate)	~95%+ recall (tunable)	Higher	Large datasets (>10k vectors), latency-sensitive
FLAT	Slow (exact)	100%	Lower	Small datasets (<10k), accuracy-critical

Default to HNSW for any production-scale workload. Tuning levers:

• M — connections per node (16–64). Higher = better recall, more memory.
• EF_CONSTRUCTION — build-time graph quality (100–500). Higher = better index, slower build.
• EF_RUNTIME — query-time candidate-list size. Higher = better recall, slower queries.

Use FLAT when the corpus is small and you need exact results (e.g. semantic dedup over a few thousand items).

See references/algorithm-choice.md.

3. Hybrid search — filter before vector

Apply attribute filters (TAG / NUMERIC) so the engine narrows the search space before the vector comparison. Don't fetch a wide result set and then filter client-side — that's slower and less accurate.

from redisvl.query import VectorQuery
from redisvl.query.filter import Num, Tag

filters = (Tag("category") == "technology") & (Num("date") >= 2024)

query = VectorQuery(
    vector=query_embedding,
    vector_field_name="embedding",
    return_fields=["content", "category", "date"],
    num_results=10,
    filter_expression=filters,
)
results = index.query(query)

For text + vector fusion (BM25-weighted text scoring combined with vector similarity), use HybridQuery on Redis ≥ 8.4 with redis-py ≥ 7.1, or AggregateHybridQuery on older Redis. That's a different "hybrid" from filtered vector search above.

See references/hybrid-search.md.

4. RAG pattern

Standard pipeline: embed the user query → vector search Redis → pass top-K context to the LLM.

# Index documents with embeddings
records = [{"content": doc.content,
            "embedding": embed_model.encode(doc.content).tolist(),
            "source": doc.source}
           for doc in documents]
index.load(records)

# Retrieve relevant context for a user question
q_emb = embed_model.encode(user_question)
results = index.query(VectorQuery(
    vector=q_emb,
    vector_field_name="embedding",
    return_fields=["content", "source"],
    num_results=5,
))

# Generate with retrieved context
context = "\n".join(r["content"] for r in results)
response = llm.generate(f"Context: {context}\n\nQuestion: {user_question}")

Practical tips:

• Match metric to model. Most modern text embedding models pair best with COSINE.
• Chunk long documents before indexing — retrieval over 200–500-token chunks usually beats indexing whole pages.
• Batch inserts with index.load([...]) instead of one call per record.
• Pre-filter with attributes (tenant, recency, document type) before the vector search.

See references/rag-pattern.md.

References

• Redis: Vectors
• Redis: RAG quickstart
• RedisVL documentation