Vector Database Landscape

The intuition

A vector index answers one question: which of these millions of embeddings is closest to this query? A vector database is that index plus everything a product needs around it — durable storage, concurrent writers, metadata filters, backups, replication, and an API clients can call without owning the process.

Think of a library card catalog versus a full library system. The catalog (FAISS, hnswlib) tells you which shelf is nearest your topic. The library system (Qdrant, Weaviate, pgvector, Pinecone, Milvus) also checks out books, enforces who may see restricted stacks, opens branches, and survives a fire. Bolting the catalog alone into a product leaves you rebuilding those operational pieces yourself.

The decision is rarely "which brand is best." It is "when does a second store earn its keep?" Most teams already have Postgres. Adding a vector column is often enough until scale, filtered recall, write churn, or isolation force a dedicated engine.

Why it exists

Semantic search and RAG need nearest-neighbor lookup at product latency. Exact scan works for small corpora; past roughly a million vectors, ANN is mandatory. That alone is not a new product category — FAISS in a batch job, or HNSW inside Postgres, serves many apps.

What creates the dedicated-engine market is everything around the index:

1. Serving is not just search. CRUD, persistence, concurrent writers, auth, structured metadata filters — pure libraries provide none of these.
2. Embeddings describe rows elsewhere. Failed upserts and missed deletes create silent search drift unless the pipeline reconciles.
3. Filtered retrieval is the default. Real queries are "nearest among docs this user may see"; naive post-filtering collapses recall (see filtering and metadata).
4. Memory and mutation dominate ops. HNSW wants RAM; deletes become tombstones; heavy churn forces compaction (see scaling and operations).
5. Team capacity varies. Some orgs will not run another distributed system; others refuse proprietary lock-in or usage-billed surprises.

FAISS alone for a multi-tenant product forces you to invent the database. Always start on Pinecone pays managed cost and migration pain before you know you need it. Always stay on pgvector past its ceiling turns every latency incident into a Postgres tuning session under OLTP load.

The core idea

Default to pgvector until it hurts. If Postgres is already there, a vector column gives ANN with zero new infrastructure, real transactions (embedding commits or rolls back with its row), joins against business tables, and one backup and ACL story. A dedicated vector DB is a second store to deploy, monitor, secure, back up, and keep consistent with the source of truth. Pay that only for a limit you have measured.

Four graduation limits:

1. Scale — pgvector is effectively single-node. Once the HNSW working set outgrows RAM, p95 degrades sharply — often tens of millions of vectors on a well-provisioned box (higher with half-precision, quantization, or DiskANN-style extensions such as pgvectorscale; still single-system ops).
2. Filtered search quality — pgvector filters after the index scan. Iterative index scans (0.8.0+) keep pulling until k matches survive, patching recall at a latency cost. Filter-aware engines (Qdrant, Weaviate) handle selective filters more natively.
3. Write-heavy churn — constant upserts/deletes bloat Postgres HNSW; dedicated engines run optimizers built for that workload.
4. Operational isolation — vector search competing with OLTP for the same buffer cache.

Among dedicated options: managed vs self-hosted. Pinecone for zero infra and usage-billed lock-in; Qdrant or Weaviate for open-source strong filtering you can run (or buy managed); Milvus for billion-scale distributed search when you have the ops team. Chroma is the prototyping default. FAISS is an in-process library for offline/batch work — not a product store.

How it actually works

Every vector database wraps an ANN index (usually HNSW, sometimes IVF or DiskANN) with a WAL, segment storage, metadata inverted indexes, an optimizer that merges segments and purges deletes, replication, and an API. FAISS/hnswlib stop at the graph. That wrapping is the product — bolting HNSW onto a storage engine not designed for graph mutation degrades under churn.

What differs is where the graph lives and how filters join it:

• pgvector — Postgres extension; HNSW/IVFFlat; vector (≤2,000 dims indexed), halfvec (≤4,000), bit, sparsevec. Builds want maintenance_work_mem; MVCC dead tuples bloat until vacuum. Iterative scans (0.8+) fix filtered under-recall. pgvectorscale adds StreamingDiskANN-style larger-than-RAM indexes on the same box.
• Qdrant — Rust single binary; segment HNSW; filterable HNSW (extra links + planner switching brute-force vs graph by filter cardinality); quantization and on-disk/mmap vectors.
• Weaviate — Go; shards; native BM25 + vector hybrid; ACORN (default for new collections since v1.34) keeps the graph connected under selective predicates; multi-tenancy with cold-offload.
• Milvus — distributed query/data/index roles over object storage and a message log; highest scale, highest ops; Zilliz Cloud managed.
• Pinecone — serverless: object storage, compute per query, namespaces as hard partitions, billed in RUs/WUs. Query cost tracks namespace size scanned — a fat shared namespace with metadata filters often costs far more than one namespace per tenant.
• Chroma — embedded-first plus client/server and cloud; best DX for prototypes and small corpora, not large-scale ops.

Public vector-DB benchmarks are almost always vendor-run; Qdrant's suite admits authors may misconfigure competitors. ANN-Benchmarks is neutral but library-level (no filters, no CRUD). Treat headline QPS as directional — bake off on your embeddings, filters, and churn.

Elasticsearch/OpenSearch are a hybrid path if you already run them: Lucene HNSW, mature BM25, zero new infra. Same "use what you run" logic as pgvector — JVM heap pressure, segment-merge freshness, and ANN that has improved (Lucene ACORN-style filtered search) but still often lags purpose-built engines under heavy vector load.

The flows

On the governed enterprise platform, early internal RAG usually starts on pgvector (or the firm's search stack), then moves to a dedicated or managed store when entitlement-filtered retrieval hits recall or latency limits — always with a rebuild pipeline from the document source of truth, because the vector index is derived data.

A worked example

Setting. Internal knowledge assistant, ~10–15k daily users, ~10k policy/wiki docs (~200k chunks), 1536-dim embeddings. Postgres already holds documents, ACLs, and audit. First release needs semantic search without a new platform team.

Later graduate. Corpus hits tens of millions of chunks; selective ACLs empty top-k even with iterative scans; vector load fights OLTP. Stand up Qdrant (or Weaviate for native hybrid), CDC from the document table, Postgres remains source of truth, re-embedding is the DR path.

INSERT INTO chunks (id, doc_id, content, embedding, tenant_id, acl)
VALUES ($id, $doc, $text, $vec, $tenant, $groups);
-- same transaction as the document write when possible

SELECT id, content, 1 - (embedding <=> $q) AS score
FROM chunks
WHERE tenant_id = $tenant AND acl && $user_groups
ORDER BY embedding <=> $q
LIMIT 10;
-- iterative scans keep scanning until k matches survive the WHERE

Production concerns

The second-database tax is the cost that matters. Source of truth and index drift through failed upserts and missed deletes — budget a reconciliation or rebuild pipeline from day one (reliability, production RAG).

Self-hosted HNSW is a memory product: vectors effectively live in RAM (scaling and operations). Quantization and disk-backed indexes are the levers; managed serverless turns the same pressure into per-query billing. On Pinecone, RUs scale with namespace size scanned — a shared 100 GB namespace with metadata filters can cost ~100× a 1 GB per-tenant namespace for the same logical query (cost). Model that at target corpus × QPS before committing.

Under real traffic: OOM during index build or segment merge; silent recall decay as tombstones accumulate; single-node SPOF until replication is configured. Latency gaps are tiny at demo scale and large under concurrent load, selective filters, and larger-than-RAM working sets (latency). Lock-in rises from pgvector through OSS engines (exportable) to Pinecone (proprietary API and index format).

Common drill-downs

How do you know you hit the wall rather than mis-tuned Postgres? Rule out cheap knobs first: maintenance_work_mem for builds, hnsw.ef_search, iterative scans, halfvec/binary quantization, whether vector traffic shares a noisy OLTP buffer cache. If the working set still will not fit after quantization, or selective filters still empty top-k at acceptable latency, that is a real graduation signal.

When is FAISS still the right tool? Offline batch clustering, dedup, research, custom GPU or exotic quantization. The moment multi-writer product traffic needs filters and durable serving, wrap it yourself or move to a store.

Managed vs self-hosted at steady high volume? Managed for small teams and spiky load. Past RU/WU break-even at high steady QPS, self-hosted is usually cheaper if you already have on-call — and you keep residency and export.

Why not Elasticsearch if we already run it? Legitimate when hybrid BM25+vector is first-class and the team operates the cluster well. Lose when vector segments stress the JVM, ANN lags under your filters, or merge/freshness fails the retrieval SLO.

How do you bake off two candidates fairly? Own data, own model, own filter distribution. recall@k vs brute force, p95/p99 under load, ingest throughput, metrics during merge/rebuild. Distrust vendor charts even when honest.

Test yourself

Go deeper

• pgvector README — index types, iterative scans, halfvec/binary quantization (v0.8.x as of mid-2026).
• Great Algorithms Are Not Enough — Pinecone — why bolt-on HNSW degrades under churn (historical pod architecture; mutation thesis still applies).
• Qdrant benchmarks — open suite; honest FAQ on vendor bias; read methodology, not charts.
• ANN-Benchmarks — neutral library-level recall vs QPS; no filtering or CRUD.
• Pinecone namespaces / multitenancy — serverless RU cost tracks namespace size (1 RU per 1 GB scanned).
• Weaviate filtering / ACORN — selective filters beyond post-filter; ACORN default since v1.34.

Where this connects

• Filtering and metadata — the filtered-search axis is often why teams leave naive post-filter stacks.
• Scaling and operations — memory formulas, tombstones, multi-tenancy, backup/rebuild after you pick an engine.
• ANN indexes — HNSW/IVF/quantization underneath every row of the landscape table.
• Production RAG — store choice as freshness, ACL, cost, and degradation in a full retrieval product.