Vector Databases

When PostgreSQL Breaks and You Don't Know Why

The startup had shipped their first RAG-powered feature - semantic search over their customer support knowledge base. It used PostgreSQL with a simple cosine similarity query in the application layer: load all embeddings, compute distances in Python, sort, return top 5. With 10,000 documents it was fine. With 50,000 documents it slowed to 3 seconds. With 200,000 documents, it was timing out.

The fix seemed obvious: add an index. They added pgvector, Postgres's vector extension, and ran CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100). Query times dropped back to 50ms. Problem solved.

Six months later, they had 2 million documents and 50 different customers with their own knowledge bases. The pgvector index was degrading. Queries that used metadata filters alongside vector search were forcing full scans because the query planner wasn't combining the vector index with the row filter efficiently. Their P99 latency was 800ms under load. They needed namespace isolation between customers. They needed the ability to update individual vectors without index rebuild. They needed horizontal scaling.

They had hit the wall that pgvector hits. Not because pgvector is bad - it's excellent within its design envelope - but because they had grown beyond that envelope. They migrated to Qdrant and the problems went away.

This lesson is about understanding what vector databases actually do, what differentiates them, and how to make the right choice for your scale and requirements.

What Vector Databases Do That PostgreSQL Cannot

The core challenge: given a query vector $q \in \mathbb{R}^d$, find the $k$ vectors in a collection of $n$ vectors that have the highest cosine similarity to $q$. Naively this is $O(n \cdot d)$ - you compute the similarity between $q$ and every stored vector. At $n = 1M$, $d = 1536$, and 100 queries per second, that's $1.536 \times 10^{11}$ floating-point operations per second. No database can do this with a SQL query over raw columns.

Vector databases solve this with Approximate Nearest Neighbor (ANN) indexes - data structures that enable sub-linear retrieval at the cost of small approximation error. The dominant algorithms are HNSW (graph-based) and IVF (cluster-based), covered in depth in the next lesson.

Beyond ANN indexing, purpose-built vector databases also handle:

• Filtered vector search: combine semantic search with structured metadata filters without full scans
• Namespaces/tenants: isolate vectors between customers without separate deployments
• Streaming updates: add, delete, and update vectors in real time without full index rebuilds
• Replication and high availability: survive node failures without losing the index
• Hybrid search: native support for combining dense and sparse retrieval
• Payload storage: store document text alongside vectors so you don't need a separate document store

The Major Players

Pinecone

What it is: Managed vector database, serverless or pod-based. The pioneer of the category - launched 2019, the most widely deployed managed vector DB.

Architecture: Pinecone handles all infrastructure. You interact entirely through an API. Indexes are sharded and replicated automatically.

Strengths: Zero operational overhead. Excellent documentation. Stable API. Strong managed metadata filtering. Native sparse+dense hybrid search support.

Weaknesses: Expensive at scale. Limited customization of the underlying index. Cold start latency on serverless tier. Vendor lock-in.

Pricing (2024): Serverless: $0.096/GB stored/month. Pod-based:$0.096/hour per p1.x1 pod (2M vectors at 1536 dims). At 10M vectors, ~$500/month.

from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key="your-api-key")

# Create index
pc.create_index(
    name="documents",
    dimension=1536,
    metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-east-1")
)

index = pc.Index("documents")

# Upsert vectors with metadata
vectors = [
    {
        "id": "doc_001",
        "values": [0.1, 0.2, ...],  # 1536 floats
        "metadata": {
            "source": "faq.pdf",
            "section": "refunds",
            "page": 3,
        }
    }
]
index.upsert(vectors=vectors, namespace="customer_acme")

# Query with metadata filter
results = index.query(
    vector=[0.1, 0.2, ...],
    top_k=10,
    filter={"section": {"$eq": "refunds"}},
    namespace="customer_acme",
    include_metadata=True,
)

Qdrant

What it is: Open-source vector database written in Rust. Self-hostable or managed cloud (qdrant.tech). The best choice for most production deployments with full control requirements.

Architecture: HNSW index with on-disk payload storage. Supports sparse vectors natively for hybrid search. Strong filtering that pushes down metadata filters into the ANN search, avoiding post-hoc filtering.

Strengths: Excellent performance. Open source (Apache-2.0). Mature filtering with payload indexing. Sparse vector support. Strong API. Low resource footprint. Excellent Rust-based reliability.

Weaknesses: Requires self-hosting operational expertise (if not using managed cloud). Smaller ecosystem than Pinecone.

Pricing: Self-hosted: only server costs. Managed: starts at $25/month.

from qdrant_client import QdrantClient
from qdrant_client.http.models import (
    Distance, VectorParams, PointStruct,
    Filter, FieldCondition, MatchValue
)
import uuid

client = QdrantClient(host="localhost", port=6333)

# Create collection
client.create_collection(
    collection_name="documents",
    vectors_config=VectorParams(
        size=1536,
        distance=Distance.COSINE,
    ),
)

# Create payload index for efficient filtering
client.create_payload_index(
    collection_name="documents",
    field_name="doc_type",
    field_schema="keyword",  # indexed for fast filtering
)

# Upsert with rich metadata payload
points = [
    PointStruct(
        id=str(uuid.uuid4()),
        vector=[0.1, 0.2, ...],  # 1536 floats
        payload={
            "text": "Refund requests must be submitted within 30 days...",
            "source": "faq.pdf",
            "page": 3,
            "doc_type": "faq",
            "section": "refunds",
            "indexed_at": "2024-01-15",
        }
    )
]
client.upsert(collection_name="documents", points=points)

# Query with pre-filtering (filter applied before ANN, not after)
results = client.search(
    collection_name="documents",
    query_vector=[0.1, 0.2, ...],
    query_filter=Filter(
        must=[
            FieldCondition(
                key="doc_type",
                match=MatchValue(value="faq")
            )
        ]
    ),
    limit=10,
    with_payload=True,
)

for result in results:
    print(f"Score: {result.score:.4f}")
    print(f"Text: {result.payload['text'][:100]}")

Weaviate

What it is: Open-source vector database with a GraphQL API and native multi-modal support. Built-in hybrid search (BM25 + vector).

Architecture: HNSW index. Unique feature: can embed data automatically using integrated embedding models (no separate embedding step required in your code). GraphQL and REST APIs.

Strengths: Integrated embedding (vectorizers built in). Native hybrid search. Multi-modal support (text + images). Strong schema enforcement. Good for teams that want an opinionated, batteries-included system.

Weaknesses: More complex to set up than Qdrant. GraphQL API has a learning curve. Heavier resource requirements.

import weaviate
from weaviate.classes.init import Auth
from weaviate.classes.config import Configure, Property, DataType

client = weaviate.connect_to_weaviate_cloud(
    cluster_url="https://your-cluster.weaviate.network",
    auth_credentials=Auth.api_key("your-api-key"),
)

# Create collection with hybrid search enabled
client.collections.create(
    name="Documents",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(
        model="text-embedding-3-small"
    ),
    properties=[
        Property(name="text", data_type=DataType.TEXT),
        Property(name="source", data_type=DataType.TEXT),
        Property(name="doc_type", data_type=DataType.TEXT),
    ]
)

collection = client.collections.get("Documents")

# Insert - Weaviate auto-embeds the text
with collection.batch.dynamic() as batch:
    batch.add_object({
        "text": "Refund requests must be submitted within 30 days...",
        "source": "faq.pdf",
        "doc_type": "faq",
    })

# Hybrid search: combines BM25 and dense vector search
results = collection.query.hybrid(
    query="how do I return a product",
    alpha=0.5,   # 0 = pure BM25, 1 = pure vector
    limit=10,
)

Milvus

What it is: Open-source vector database designed for massive scale. The most production-hardened option for 100M+ vector workloads.

Architecture: Distributed architecture with separate components: proxy, query nodes, data nodes, index nodes. Supports HNSW, IVF-Flat, IVF-PQ, DiskANN, and more. Kubernetes-native.

Strengths: Best throughput at scale. Multiple index types. Strong consistency guarantees. Battle-tested at Alibaba, Baidu, and other large-scale deployments.

Weaknesses: Complex deployment (Kubernetes, etcd, MinIO dependencies). Operational overhead is significant. Overkill for under 10M vectors.

When to choose Milvus: 100M+ vectors, need for massive concurrent query throughput, existing Kubernetes infrastructure, team has ML infrastructure expertise.

Chroma

What it is: Lightweight, Python-first vector store. The easiest to get started with - zero configuration, runs in-process.

Architecture: SQLite + in-memory ANN (hnswlib). Persistent storage on disk. Simple REST API for client-server mode.

Strengths: Zero setup. Excellent for development and testing. Native Python client. Good for small-scale production (under 1M vectors).

Weaknesses: Not designed for high availability or horizontal scaling. No production-grade metadata filtering optimizations. Slower than Qdrant at production scale.

import chromadb

# Ephemeral (in-memory, for testing)
client = chromadb.Client()

# Persistent (saves to disk)
client = chromadb.PersistentClient(path="./chroma_data")

collection = client.create_collection(
    name="documents",
    metadata={"hnsw:space": "cosine"}
)

# Add documents - Chroma handles its own IDs
collection.add(
    documents=["Refund requests must be submitted within 30 days..."],
    embeddings=[[0.1, 0.2, ...]],
    metadatas=[{"source": "faq.pdf", "doc_type": "faq"}],
    ids=["doc_001"],
)

# Query
results = collection.query(
    query_embeddings=[[0.1, 0.2, ...]],
    n_results=5,
    where={"doc_type": "faq"},  # metadata filter
)

pgvector: When Postgres Is Enough

pgvector adds vector column types and ANN indexes to PostgreSQL. If you already have a PostgreSQL deployment and your vector count is under 1-2M, pgvector is often the right choice.

When pgvector makes sense:

• Already running Postgres for your application data
• Under 1M vectors
• Team doesn't want to operate another database
• ACID transactions needed (e.g., atomic vector + metadata updates)
• Familiar SQL tooling for analysis and debugging

When pgvector breaks down:

• Over 2M vectors: IVF-flat index recall degrades without careful tuning
• Complex metadata filter + vector search: query planner may not combine indexes efficiently
• High write throughput: index maintenance becomes a bottleneck
• Multi-tenant isolation: schema-per-tenant doesn't scale past a few hundred tenants

# Using pgvector with SQLAlchemy
from sqlalchemy import create_engine, text
from pgvector.sqlalchemy import Vector
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column

engine = create_engine("postgresql://user:pass@localhost/dbname")

class Base(DeclarativeBase):
    pass

class Document(Base):
    __tablename__ = "documents"

    id: Mapped[int] = mapped_column(primary_key=True)
    content: Mapped[str]
    source: Mapped[str]
    doc_type: Mapped[str]
    embedding: Mapped[list] = mapped_column(Vector(1536))

Base.metadata.create_all(engine)

# Create HNSW index (pgvector 0.5+, faster than IVF for most workloads)
with engine.connect() as conn:
    conn.execute(text("""
        CREATE INDEX ON documents
        USING hnsw (embedding vector_cosine_ops)
        WITH (m = 16, ef_construction = 64)
    """))
    conn.commit()

# Query: cosine similarity search with metadata filter
with engine.connect() as conn:
    query_embedding = [0.1, 0.2, ...]  # 1536 floats
    results = conn.execute(text("""
        SELECT content, source, 1 - (embedding <=> :query_emb) AS similarity
        FROM documents
        WHERE doc_type = :doc_type
        ORDER BY embedding <=> :query_emb
        LIMIT 10
    """), {"query_emb": str(query_embedding), "doc_type": "faq"})

    for row in results:
        print(f"Similarity: {row.similarity:.4f} | {row.content[:80]}")

Vector DB Comparison Table

Feature	Pinecone	Qdrant	Weaviate	Milvus	Chroma	pgvector
Hosting	Managed only	Both	Both	Both	Both	Self-host
License	Proprietary	Apache-2.0	BSD-3	Apache-2.0	Apache-2.0	PostgreSQL
Max scale	10B+	1B+	1B+	10B+	~1M	~2M
Hybrid search	Yes	Yes	Yes (native)	Yes	Limited	With pg_bm25
Filtering	Good	Excellent	Good	Good	Basic	Excellent (SQL)
Multi-tenancy	Namespaces	Collections	Tenants	Partitions	Collections	Schema/table
Setup complexity	None	Low	Medium	High	None	Low (if using PG)
Ops cost	High ($)	Low	Medium	High	None	Low

Metadata Filtering: The Critical Production Feature

Metadata filtering - combining vector similarity with structured field constraints - is where vector databases differentiate significantly. The naive implementation fetches the top-k by vector similarity, then post-filters by metadata. This is dangerous: if only 10% of your vectors match the filter, you need to over-fetch by 10x to get k valid results after filtering.

Production vector databases implement pre-filtering: the ANN search is scoped to the subset of vectors matching the metadata filter before running. This requires payload indexes alongside the vector index.

Qdrant implements pre-filtering via payload indexes. Pinecone implements filter-restricted ANN. Chroma's filtering is post-hoc, which is why it degrades with highly selective filters.

Multi-Tenancy: Organizing Vectors at Scale

Most production RAG systems serve multiple users, customers, or domains with isolated knowledge bases. Vector databases handle this differently:

Namespace approach (Pinecone): A single index with logical partitions. Queries are scoped to a namespace. All namespaces share the same index infrastructure. Efficient but no strong isolation.

Collection approach (Qdrant, Chroma): Separate HNSW index per collection. True isolation, but overhead per collection. Practical up to ~1000 collections.

Tenant metadata filter (all): Store tenant ID as a metadata field, filter every query by tenant. Simplest to implement, good performance if the vector DB does pre-filtering. The right choice for 1000+ tenants.

# Qdrant multi-tenant with metadata filter (preferred for many tenants)
from qdrant_client.http.models import Filter, FieldCondition, MatchValue

def search_for_tenant(client, query_vector: list, tenant_id: str, limit: int = 10):
    return client.search(
        collection_name="all_documents",
        query_vector=query_vector,
        query_filter=Filter(
            must=[
                FieldCondition(
                    key="tenant_id",
                    match=MatchValue(value=tenant_id)
                )
            ]
        ),
        limit=limit,
        with_payload=True,
    )

Complete Production Vector Store with Qdrant

import uuid
from typing import List, Dict, Any, Optional
from qdrant_client import QdrantClient
from qdrant_client.http.models import (
    Distance, VectorParams, PointStruct,
    Filter, FieldCondition, MatchValue,
    PayloadSchemaType, UpdateStatus,
)
from openai import OpenAI

openai_client = OpenAI()

class ProductionVectorStore:
    def __init__(self, host: str = "localhost", port: int = 6333):
        self.client = QdrantClient(host=host, port=port)
        self.collection_name = "rag_documents"
        self.dim = 1536
        self._ensure_collection()

    def _ensure_collection(self):
        """Create collection if it doesn't exist."""
        existing = [c.name for c in self.client.get_collections().collections]
        if self.collection_name not in existing:
            self.client.create_collection(
                collection_name=self.collection_name,
                vectors_config=VectorParams(
                    size=self.dim,
                    distance=Distance.COSINE,
                ),
            )
            # Create payload indexes for fast filtering
            for field in ["doc_type", "source", "tenant_id"]:
                self.client.create_payload_index(
                    collection_name=self.collection_name,
                    field_name=field,
                    field_schema=PayloadSchemaType.KEYWORD,
                )

    def _embed(self, texts: List[str]) -> List[List[float]]:
        response = openai_client.embeddings.create(
            model="text-embedding-3-small",
            input=texts,
        )
        return [item.embedding for item in response.data]

    def upsert(self, documents: List[Dict[str, Any]]) -> int:
        """Batch upsert documents. Each doc must have 'text' and 'metadata' fields."""
        texts = [doc["text"] for doc in documents]
        embeddings = self._embed(texts)

        points = [
            PointStruct(
                id=str(uuid.uuid4()),
                vector=emb,
                payload={
                    "text": doc["text"],
                    **doc.get("metadata", {}),
                }
            )
            for doc, emb in zip(documents, embeddings)
        ]

        result = self.client.upsert(
            collection_name=self.collection_name,
            points=points,
        )
        return len(points)

    def search(
        self,
        query: str,
        limit: int = 10,
        filters: Optional[Dict[str, str]] = None,
    ) -> List[Dict[str, Any]]:
        """Semantic search with optional metadata filtering."""
        query_emb = self._embed([query])[0]

        qdrant_filter = None
        if filters:
            conditions = [
                FieldCondition(key=k, match=MatchValue(value=v))
                for k, v in filters.items()
            ]
            qdrant_filter = Filter(must=conditions)

        results = self.client.search(
            collection_name=self.collection_name,
            query_vector=query_emb,
            query_filter=qdrant_filter,
            limit=limit,
            with_payload=True,
        )

        return [
            {
                "text": r.payload["text"],
                "score": r.score,
                "metadata": {k: v for k, v in r.payload.items() if k != "text"},
            }
            for r in results
        ]

    def delete_by_source(self, source: str):
        """Delete all vectors from a specific document source."""
        self.client.delete(
            collection_name=self.collection_name,
            points_selector=Filter(
                must=[FieldCondition(key="source", match=MatchValue(value=source))]
            ),
        )

    def count(self) -> int:
        return self.client.count(self.collection_name).count


# Usage
store = ProductionVectorStore()

store.upsert([
    {"text": "Refunds accepted within 30 days.", "metadata": {"doc_type": "faq", "source": "help.pdf"}},
    {"text": "Express shipping takes 1-2 days.", "metadata": {"doc_type": "faq", "source": "help.pdf"}},
])

results = store.search(
    "how do I get my money back",
    limit=5,
    filters={"doc_type": "faq"}
)
for r in results:
    print(f"[{r['score']:.4f}] {r['text']}")

Production Engineering Notes

Batch upsert size: Qdrant and Pinecone handle batches of up to 100-1000 points per upsert request efficiently. Use batches of 100-500 for best throughput. Smaller batches add network overhead; larger batches risk timeouts.

Index warming: After loading data, ANN indexes may not be fully warmed in memory. Run a few warm-up queries before accepting production traffic - cold queries can be 5-10x slower than warm queries.

Incremental updates: Adding new documents to an HNSW index is efficient - HNSW supports dynamic insertion without full rebuild. Deletions are more complex: most implementations mark vectors as deleted and periodically compact. Monitor index size growth.

Backup strategy: Vector databases are stateful systems. For Qdrant, use snapshots (/collections/{name}/snapshots). For Pinecone, export vectors periodically (no built-in backup - plan for this before you need it).

Common Mistakes

:::danger Using Chroma in Production at Scale Chroma is excellent for development and small datasets. At 500K+ vectors or under concurrent load, it degrades significantly. Many teams reach production with Chroma because prototypes worked fine, then face a painful migration. Choose Qdrant or Pinecone for production systems from the start. :::

:::danger Not Indexing Metadata Fields Adding filters without payload indexes causes full scans of vector payloads. A filter on doc_type without an index scans every payload record - $O(n)$ regardless of the ANN index. Always create payload indexes for fields you'll filter on. This is the most common performance bug in Qdrant deployments. :::

:::warning Choosing a Vector DB Without Evaluating Filtering Behavior If your RAG system requires metadata filters (customer isolation, document type filters, date ranges), test how each vector DB handles filtered ANN search before committing. Some databases post-filter (slow, unpredictable results at k), some pre-filter (fast, reliable). This is a critical correctness and performance difference. :::

Interview Questions and Answers

Q: What does a vector database provide that you can't get from a standard RDBMS with a vector column?

A: A vector database provides ANN indexing - specifically HNSW or IVF-based structures - that reduce nearest neighbor search from $O(n \cdot d)$ to effectively $O(\log n)$ with a small approximation error. A PostgreSQL table with a vector column and no index does a sequential scan: every query computes similarity with every row. At 1M vectors, this is multiple seconds per query. pgvector adds ANN indexing to PostgreSQL (HNSW and IVF-Flat), which covers many use cases. But purpose-built vector databases also provide: efficient pre-filtering that combines metadata constraints with ANN (not all databases do this well), horizontal sharding across machines, streaming updates without index degradation, native multi-tenancy, and hybrid dense+sparse search. pgvector is the right choice up to roughly 1-2M vectors with an existing PostgreSQL deployment. Beyond that, or with complex filtering requirements, purpose-built systems win.

Q: Describe the difference between post-filtering and pre-filtering in vector search. Why does it matter?

A: Post-filtering: run ANN search over the full collection, retrieve top-k results by vector similarity, then discard results that don't match the metadata filter. Problem: if your filter is selective (only 5% of documents match), you need to retrieve 20x as many candidates to get k valid results after filtering. This makes k unpredictable and requires over-fetching, which hurts both latency and relevance. Pre-filtering: use payload indexes to identify the subset of document IDs matching the filter, then run ANN search restricted to that subset. Result: reliable k results within the filter scope, faster because the ANN search space is smaller. Qdrant implements true pre-filtering with payload indexes. This is a major engineering advantage over systems that do post-hoc filtering.

Q: When would you choose pgvector over Qdrant?

A: pgvector makes sense when: (1) you already operate PostgreSQL and don't want to introduce another database system, (2) your vector count is under 1-2M and query latency requirements are loose (sub-second is acceptable), (3) you need ACID transactions across vector and application data (e.g., atomically insert a document record and its embeddings), (4) your team is more comfortable with SQL than a vector DB API. Choose Qdrant when: (5) you need sub-100ms P99 latency under concurrent load, (6) your vector count exceeds 2M, (7) you need efficient pre-filtering with payload indexes, (8) you need production-grade multi-tenancy, (9) you need horizontal scaling.

Q: Your RAG system serves 100 enterprise customers with isolated knowledge bases. How do you architect multi-tenancy in your vector database?

A: Three options with different trade-offs. Option 1 - separate collection per tenant: strong isolation, each tenant gets its own HNSW index. Practical up to ~100-500 tenants (each collection has overhead). Not scalable to 10K+ tenants. Option 2 - tenant_id as metadata filter: all tenants share one collection, every query includes filter: {tenant_id: "acme"}. Scales to unlimited tenants. Requires payload indexes on tenant_id for efficiency. Slightly weaker isolation (a misconfigured query could leak across tenants). Option 3 - hybrid: separate collections for large tenants (where per-tenant isolation and performance justify overhead), shared collection with metadata filtering for small tenants. For 100 enterprise customers, I'd use Option 1 with separate collections - the isolation guarantee matters for enterprise contracts and 100 collections is well within operational capacity for any mature vector DB.

Incremental Indexing and Update Patterns

Production knowledge bases are not static. Documents are added, updated, and deleted continuously. Your vector database architecture must handle all three operations efficiently.

Adding New Documents

HNSW indexes support efficient dynamic insertion - no full rebuild required. The new vector is inserted into the existing graph structure, which updates its connections at each layer.

from qdrant_client import QdrantClient
from qdrant_client.http.models import PointStruct
import uuid

client = QdrantClient(host="localhost", port=6333)

def add_document(
    text: str,
    metadata: dict,
    embedder,
    collection: str = "documents"
) -> str:
    """Add a single document to the vector store in real-time."""
    embedding = embedder.embed_query(text)
    point_id = str(uuid.uuid4())

    client.upsert(
        collection_name=collection,
        points=[PointStruct(
            id=point_id,
            vector=embedding.tolist(),
            payload={"text": text, **metadata}
        )]
    )
    return point_id


def update_document(
    point_id: str,
    new_text: str,
    new_metadata: dict,
    embedder,
    collection: str = "documents"
):
    """Update an existing document - re-embeds with new content."""
    new_embedding = embedder.embed_query(new_text)

    # Qdrant upsert with existing ID overwrites the vector and payload
    client.upsert(
        collection_name=collection,
        points=[PointStruct(
            id=point_id,
            vector=new_embedding.tolist(),
            payload={"text": new_text, **new_metadata}
        )]
    )

Deleting Documents

Deletion is trickier than insertion. HNSW doesn't support true vector deletions - the graph structure is built around existing connections. Most implementations use tombstoning: mark deleted vectors and exclude them from results.

from qdrant_client.http.models import Filter, FieldCondition, MatchValue

def delete_document_by_id(point_id: str, collection: str = "documents"):
    """Delete a specific document by its vector ID."""
    client.delete(
        collection_name=collection,
        points_selector=[point_id]
    )

def delete_documents_by_source(source: str, collection: str = "documents"):
    """Delete all chunks from a specific source document."""
    client.delete(
        collection_name=collection,
        points_selector=Filter(
            must=[FieldCondition(key="source", match=MatchValue(value=source))]
        )
    )
    print(f"Deleted all chunks from source: {source}")

def reindex_document(
    source: str,
    new_chunks: list,
    embedder,
    collection: str = "documents"
):
    """
    Replace all chunks from a document with a new version.
    Use when a document is updated - delete old chunks, add new.
    """
    # Delete old version
    delete_documents_by_source(source, collection)

    # Add new chunks
    texts = [c["text"] for c in new_chunks]
    embeddings = embedder.embed_documents(texts)

    points = [
        PointStruct(
            id=str(uuid.uuid4()),
            vector=emb.tolist(),
            payload={"text": chunk["text"], "source": source, **chunk.get("metadata", {})}
        )
        for chunk, emb in zip(new_chunks, embeddings)
    ]
    client.upsert(collection_name=collection, points=points)
    print(f"Reindexed {len(points)} chunks from {source}")

Periodic Index Optimization

After many insertions and deletions, HNSW indexes can accumulate "soft-deleted" vectors and suboptimal graph connections. Schedule periodic optimization:

def optimize_collection(collection: str = "documents"):
    """
    Trigger Qdrant's internal optimization to compact the index.
    Run during low-traffic periods (e.g., nightly at 3 AM).
    """
    # Qdrant auto-optimizes but you can trigger manually
    client.update_collection(
        collection_name=collection,
        optimizer_config={"indexing_threshold": 20000}  # trigger immediate indexing
    )
    print(f"Optimization triggered for {collection}")

Vector DB Scaling Patterns

Horizontal Scaling with Qdrant Cluster

For large-scale deployments (100M+ vectors), Qdrant supports distributed mode:

# qdrant-cluster.yaml (Docker Compose example)
services:
  qdrant-node-1:
    image: qdrant/qdrant:latest
    environment:
      - QDRANT__CLUSTER__ENABLED=true
      - QDRANT__CLUSTER__P2P__PORT=6335
    ports:
      - "6333:6333"

  qdrant-node-2:
    image: qdrant/qdrant:latest
    environment:
      - QDRANT__CLUSTER__ENABLED=true
      - QDRANT__CLUSTER__BOOTSTRAP__PEERS=http://qdrant-node-1:6335
      - QDRANT__CLUSTER__P2P__PORT=6335

In distributed mode, collections are sharded across nodes. Each shard holds a subset of vectors. Queries are fanned out to all shards and results are merged.

Read Replicas for High Query Throughput

For read-heavy workloads (many concurrent queries, few writes):

from qdrant_client import QdrantClient
import random

# Multiple read replicas behind a load balancer
READ_REPLICAS = [
    QdrantClient(host="qdrant-replica-1", port=6333),
    QdrantClient(host="qdrant-replica-2", port=6333),
    QdrantClient(host="qdrant-replica-3", port=6333),
]
WRITE_PRIMARY = QdrantClient(host="qdrant-primary", port=6333)

def search_with_load_balancing(query_vector: list, collection: str, limit: int = 10):
    """Distribute read queries across replicas."""
    replica = random.choice(READ_REPLICAS)
    return replica.search(
        collection_name=collection,
        query_vector=query_vector,
        limit=limit,
    )

Choosing the Right Vector Database: A Scoring Matrix

Rate each option (1-5) on the dimensions that matter most to your project:

Dimension	Pinecone	Qdrant	Weaviate	Milvus	Chroma	pgvector
Operational simplicity	5	4	3	2	5	4
Cost at 1M vectors	2	5	4	4	5	5
Filtering quality	4	5	4	4	2	5
Scale to 100M+ vectors	5	4	4	5	1	2
Hybrid search	4	5	5	3	2	3
Self-host / compliance	1	5	5	5	5	5
Ecosystem / docs	5	4	4	3	4	5

Decision heuristics:

• Starting a new project with no compliance requirements: Qdrant (free, excellent, easy)
• Existing Postgres infrastructure, under 2M vectors: pgvector (zero new infra)
• Need managed service, can afford it: Pinecone
• Billion-scale, existing Kubernetes: Milvus
• Want built-in hybrid search with schema: Weaviate
• Prototyping / local development: Chroma

:::tip Benchmark Before Committing Spend one day running your actual query workload against your top two candidates before committing to a vector database. Measure P50, P95, P99 latency and recall@10 under your expected concurrent load. Infrastructure migrations at production scale are painful - a day of benchmarking is cheap insurance. :::

:::tip 🎮 Interactive Playground

Visualize this concept: Try the RAG Pipeline demo on the EngineersOfAI Playground - no code required.

:::