Vector Database Landscape

The intuition

A vector index is a data structure: it answers "which of these millions of embeddings is closest to this query?" A vector database is that index plus everything a product needs around it — durable storage, concurrent writers, metadata filters, backups, replication, and an API clients can call without owning the process.

Think of a library card catalog versus a full library system. The catalog (FAISS, hnswlib) tells you which shelf is nearest your topic. The library system (Qdrant, Weaviate, pgvector, Pinecone, Milvus) also checks out books, tracks who may see restricted stacks, opens branches, and survives a fire. Bolting the catalog alone into a product leaves you rebuilding those operational pieces yourself.

The decision is rarely "which brand is best." It is "when does the second store earn its keep?" Most teams already have Postgres. Adding a vector column is often enough until scale, filtered recall, write churn, or isolation force a dedicated engine.

Why it exists

Semantic search and RAG need nearest-neighbor lookup over embeddings at product latency. Exact scan works for small corpora; beyond roughly a million vectors, ANN indexes become mandatory. That alone does not force a new product category — FAISS in a batch job, or HNSW inside Postgres, can serve many apps.

Hard constraints that create the dedicated-engine market:

1. The index is not the product. Serving needs CRUD, persistence across restarts, concurrent writers, auth, and filters on structured metadata — none of which pure libraries provide.
2. Consistency with the source of truth. Embeddings describe rows that live elsewhere. Failed upserts and missed deletes create silent search drift unless the store and pipeline design for reconciliation.
3. Filtered retrieval is the default. Real queries are "nearest among docs this user may see" — filter composition is where naive post-filtering collapses recall (see filtering and metadata).
4. Memory and mutation dominate ops. HNSW wants vectors in RAM; deletes become tombstones; heavy churn forces compaction and rebuilds (see scaling and operations).
5. Team capacity varies. Some organizations will not run another distributed system; others refuse proprietary lock-in or usage-billed surprises.

Alternatives lose differently. FAISS alone for a multi-tenant product forces you to invent the database. Always start on Pinecone pays managed cost and migration pain before you know you need it. Always stay on pgvector past its ceiling turns every latency incident into a Postgres tuning session under OLTP load. The landscape exists so you can match the engine class to the limit you actually hit.

The core idea

The right framing is not "which vector DB is best" but "when does a dedicated vector DB earn its operational cost." The default is pgvector until it hurts: if the product already runs Postgres, adding a vector column gives ANN search with zero new infrastructure, real transactions (the embedding commits or rolls back with the row it describes), joins against business tables, one backup story, and one access-control model. A dedicated vector DB is a second data store — a second thing to deploy, monitor, secure, back up, and keep consistent with the source of truth. That cost must be paid for by a concrete limit you've actually hit.

The limits that justify graduating: (1) scale — pgvector is a single-node index; once the HNSW working set outgrows RAM, p95 latency degrades sharply, which in practice caps it around the tens-of-millions of vectors on a well-provisioned box; (2) filtered search quality — pgvector filters after the index scan, and although iterative index scans (0.8.0+) patch the recall hole, engines with filter-aware graph traversal (Qdrant, Weaviate) handle selective filters natively; (3) write-heavy churn — constant upserts and deletes bloat a Postgres HNSW index and force reindexing, while dedicated engines have background optimizers built for it; (4) operational isolation — vector search competing with OLTP for the same buffer cache.

Among dedicated options the split is managed vs self-hosted: Pinecone if you want zero infrastructure and accept a proprietary, usage-billed service; Qdrant or Weaviate if you want open-source, strong filtering, and are willing to run it (both also sell managed clouds); Milvus when you genuinely need billion-scale distributed search and have an ops team. Chroma is the prototyping default — embedded, minimal ceremony. FAISS is not a database at all: it's an in-process library, the right tool for offline/batch similarity jobs and research, wrong for a served product that needs CRUD, persistence, and filtering.

How it actually works

How the pieces nest — library, engine, product:

Library vs database. FAISS (and hnswlib) give you the index data structure only: you build it in your process's memory, you serialize it to disk yourself, and there is no metadata filtering, no replication, no concurrent-writer story, no auth. Every vector database wraps an ANN index (almost always HNSW, sometimes IVF or DiskANN variants) with the database parts: a WAL, segment/collection storage, a metadata store with inverted indexes for filters, an optimizer that merges segments and purges deletes, replication, and an API layer. That wrapping is the product — Pinecone's blog "Great Algorithms Are Not Enough" is explicitly about why bolting HNSW onto existing storage engines degrades under churn.

Architecture differences that matter:

• pgvector — extension inside Postgres; HNSW and IVFFlat index types; vector (≤2,000 dims indexed), halfvec (2-byte floats), bit, sparsevec types; index build wants the graph in maintenance_work_mem; queries respect MVCC, so dead tuples from updates bloat the index until vacuum. Version 0.8.x; pgvectorscale adds a DiskANN-style index for larger-than-RAM workloads.
• Qdrant — Rust, single binary; collections split into segments, each with its own HNSW; filterable HNSW (extra graph links + a query planner that picks brute-force vs graph traversal by filter cardinality); scalar/product/binary quantization; mmap or on-disk vectors for RAM relief.
• Weaviate — Go; per-collection shards; native BM25 + vector hybrid search with fusion; ACORN filter strategy (default since v1.34); native multi-tenancy with per-tenant shards that can be offloaded to S3 when cold.
• Milvus — genuinely distributed: separate query, data, and index nodes over object storage and a message log; segments sealed and compacted in the background. Highest scale ceiling, highest ops burden; Zilliz Cloud is the managed form.
• Pinecone — closed-source serverless: storage on object store, compute spun up per query, namespaces as the partitioning primitive, billed in read/write units. You trade control and self-hosting for the lowest ops burden on the list.
• Chroma — embedded-first (runs in your Python process, persists locally), with a client/server mode and a serverless cloud. Optimized for developer experience, not for large-scale ops.

Comparison on the axes that matter:

On benchmarks: almost every public vector-DB benchmark is vendor-run — Qdrant's own benchmark suite openly says the authors know their engine best and may misconfigure competitors. ANN-Benchmarks is neutral but library-level (recall vs QPS for algorithms, no filtering, no CRUD). Treat all headline QPS numbers as directional; benchmark on your own data, your own filters, your own hardware before deciding anything.

Elasticsearch/OpenSearch deserve a mention as a hybrid path: Lucene HNSW, mature BM25, zero new infra if you already run them for search. Caveats are JVM heap pressure with large vector segments, historically slower ANN than purpose-built engines, and segment-merge behavior affecting freshness — same "use what you run" logic as pgvector, with different operational fingerprints.

The flows

On the governed enterprise platform, the usual progression is pgvector (or the firm's existing search stack) for early internal RAG, then a dedicated or managed vector store when entitlement-filtered retrieval over large document sets hits recall or latency limits — always with a rebuild pipeline from the source document store, because the vector index is derived data.

A worked example

Setting. An internal knowledge assistant for ~10–15k daily users over ~10k policy and wiki documents (~200k chunks after parsing). Embeddings are 1536-dim. The product already runs Postgres for documents, ACLs, and audit. Leadership wants semantic search in the first release without a new platform team.

Decision path with real-ish numbers.

What "graduate" would look like later. Corpus grows to tens of millions of chunks (merged ticket history + multi-region policies). Selective ACL filters return sparse top-k even with iterative scans; vector traffic contends with OLTP. Team stands up Qdrant (or Weaviate for native hybrid) with filterable HNSW, migrates via CDC from the document table, keeps Postgres as source of truth, and treats re-embedding as DR.

INSERT INTO chunks (id, doc_id, content, embedding, tenant_id, acl)
VALUES ($id, $doc, $text, $vec, $tenant, $groups);
-- same transaction as the document write when possible

SELECT id, content, 1 - (embedding <=> $q) AS score
FROM chunks
WHERE tenant_id = $tenant AND acl && $user_groups
ORDER BY embedding <=> $q
LIMIT 10;
-- iterative index scans keep scanning until k matches survive the WHERE

Common drill-downs

These are the questions engineers ask one level down when choosing or defending a store.

You're adding RAG to an existing product. Which vector store? pgvector, if Postgres is already there and corpus is under ~10M chunks: no new infra, transactional consistency with source data, joins for metadata. Revisit only when hitting a measured limit — latency at scale, filtered recall, or write churn.

When does a dedicated vector DB beat pgvector? When the HNSW working set exceeds single-node RAM (tens of millions of vectors), when selective metadata filters degrade recall/latency, when heavy upsert/delete churn forces frequent reindexing, or when vector load must be isolated from OLTP. Cite the mechanism, not the brand.

FAISS vs a vector database? FAISS is an in-process ANN library: no persistence, filtering, replication, or concurrent CRUD — you build those or you don't have them. Right for offline batch jobs, dedup, research, and custom-index needs (GPU, exotic quantization). A database earns its keep the moment the index must be served, filtered, and mutated by a product.

Managed (Pinecone) vs self-hosted (Qdrant/Milvus)? Managed: near-zero ops, elastic, usage-billed — cost scales with data+queries and you accept lock-in. Self-hosted: control, predictable infra cost, data residency — you own memory sizing, upgrades, replication, and on-call. Small team + spiky load → managed. Steady high volume + platform team → self-hosted is usually cheaper past break-even.

Why not Elasticsearch/OpenSearch — we already run it for search? Legitimate option: it has HNSW (Lucene), mature BM25, so hybrid is native and it's zero new infra — same "use what you run" logic as pgvector. Caveats: JVM heap pressure with large vector segments, historically slower ANN than dedicated engines, and segment-merge behavior affects freshness.

How would you compare two candidate vector DBs fairly? Own data, own embedding model, own filters. Measure recall@k against exact brute-force ground truth, p95/p99 latency under concurrent load, ingest throughput, and behavior during segment merge/rebuild. Distrust vendor benchmarks — even honest ones misconfigure competitors.

Where does Chroma fit? Prototyping and small production: embedded, zero-setup, pleasant API. Its ceiling is scale and ops maturity — fine for a demo or an internal tool, not for hundreds of millions of vectors under load.

Does a long-context model or "just use grep" remove the need for vector DBs? For small corpora, honestly, often yes — stuffing 50 documents into a 1M-token context or lexical search can beat a badly tuned RAG stack. Vector DBs earn their place when corpus size × query volume makes per-query full-context reading impossible on cost/latency grounds.

Production concerns

• The second-database tax is the real cost. A dedicated vector DB means a second consistency domain: your source of truth and your index can drift (failed upserts, missed deletes). Teams end up building a reconciliation/rebuild pipeline anyway — budget for it in the decision. See reliability for outbox/CDC patterns and production RAG for freshness and ACL coupling.
• Memory is the dominant cost driver for self-hosted HNSW engines: vectors must effectively live in RAM (see scaling and operations for sizing math). Quantization and disk-backed indexes are the levers; managed serverless (Pinecone) converts this to per-query billing instead.
• Managed pricing failure mode: usage-based read units scale with data scanned, so a fat namespace or unfiltered scans make cost grow with corpus size, not just traffic. Model cost at target scale before committing; migrations off a proprietary API are expensive. Tie this to cost accounting.
• Self-hosted failure modes: OOM during index build or segment merge; recall silently dropping as tombstones accumulate; a single-node Qdrant/Weaviate is a SPOF until you configure replication.
• Latency: all engines return single-digit-ms ANN at small scale; differences appear under concurrent load, filters, and larger-than-RAM working sets — which is exactly what vendor benchmarks rarely show. See latency.
• Lock-in gradient: pgvector < Qdrant/Weaviate/Milvus (OSS, exportable) < Pinecone (proprietary API and index format; leaving means re-embedding pipelines and rewriting query code).

Test yourself

Go deeper

• pgvector README — index types, vector types, iterative scans, tuning; the single best document on Postgres vector search.
• Great Algorithms Are Not Enough — Pinecone blog — why a great ANN algorithm bolted onto a storage engine degrades under churn; vendor-written but the mechanisms are real.
• Qdrant vector search benchmarks — open-source benchmark suite with an unusually honest FAQ about vendor bias; read the methodology, not just the charts.
• ANN-Benchmarks — neutral, library-level recall-vs-QPS comparisons (FAISS, hnswlib, ScaNN…); no filtering or CRUD, so it bounds algorithms, not databases.
• Vector databases are so hot right now. WTF are they? — Fireship — 7-minute landscape orientation; the canonical quick explainer.
• What is a Vector Database? — IBM Technology — whiteboard-style conceptual grounding of vectors, indexes, and where a vector DB sits in an AI stack.

Where this connects

• Filtering and metadata — the filtered-search axis in the comparison table is often why teams leave naive post-filter stacks; read this before trusting any engine's "supports filters" claim.
• Scaling and operations — memory formulas, tombstones, multi-tenancy patterns, and backup/rebuild: the operational tax after you pick an engine.
• ANN indexes — HNSW/IVF/quantization mechanics underneath every row of the landscape table.
• Production RAG — how store choice shows up as freshness, ACL, cost, and degradation in a full retrieval product.