Similarity Metrics

The intuition

You have two points on a map. "How similar are they?" can mean three different things.

Cosine asks whether they point the same way — length does not count. Dot product asks whether they are aligned and large — direction and length both score. Euclidean (L2) distance asks how far apart the points sit as locations.

Most text search cares about direction: a short query and a long paragraph on the same topic should match, so cosine (or its unit-vector shortcut) is the default. Dot product keeps magnitude as signal — recommenders often prefer it. Euclidean fits clustering and many image or scientific features. The load-bearing fact: if every vector has length 1, all three order neighbors the same way — so stacks often "choose cosine" and implement it as normalized dot product.

Why it exists

Once text is a vector, you still need a rule that turns two vectors into a ranking. Without it, "nearest neighbor" is undefined.

That rule is sticky. ANN indexes bake a metric into graph edges and clusters — build with L2 and query with unnormalized inner product, and you walk the wrong neighborhood. Magnitude may be noise (document length) or signal (popularity). APIs disagree on polarity: similarity (higher better) vs distance (lower better), so thresholds invert if you mix them. Training shapes the space for one objective; a cosine-trained model is not guaranteed under another geometry.

"Use the DB default" works until the first model card that says inner product, the first recommender that needs MIPS (maximum inner product search), or the first absolute threshold calibrated under another range.

The core idea

Three metrics cover almost all vector search.

Dot product (a·b = Σ aᵢbᵢ) is alignment scaled by both magnitudes — range unbounded. Cosine similarity (a·b / (‖a‖‖b‖)) divides those magnitudes out: direction only, range −1 to 1. Euclidean (L2) distance (‖a−b‖) is straight-line distance between the points.

On unit-normalized vectors they are equivalent for ranking. If ‖a‖ = ‖b‖ = 1, cosine equals the dot product, and squared L2 is 2 − 2·cos(a,b) — a monotonic map, so nearest-neighbor order is identical. Many APIs (including OpenAI's) return unit vectors, so the "choice" is often a no-op and pure inner product wins: no norms per comparison, clean SIMD. When it does matter: use the metric the model was trained with, and keep unnormalized dot product when magnitude is product signal — cosine deliberately discards it.

How it actually works

From raw embedding to ranked neighbors, the path branches on normalization:

Cost almost never justifies a switch — all three are O(d) with SIMD. Cosine costs more only when the store divides by norms every comparison, so "cosine" collections often normalize once at write/query and run pure inner product forever. Unnormalized writes into such a collection corrupt rankings with no error.

Cosine asks direction only — good for text. Dot product also rewards magnitude, so high-norm items become hubs: often a feature in recommenders, where item norm tracks popularity and scores are user·item products (MIPS). Euclidean is point distance — natural for k-means, image descriptors, absolute measurements.

Distance vs similarity. Stores report similarity (higher = closer) or distance (lower = closer). Weaviate's cosine distance is 1 − cosine similarity (range 0–2); cosine mode normalizes then uses dot product. FAISS METRIC_INNER_PRODUCT returns similarities; METRIC_L2 returns squared distances (sqrt skipped — ranking unchanged). Pin polarity and range before thresholds.

Indexes bake the metric. HNSW and IVF encode one neighborhood structure — you cannot build with L2 and query with unnormalized IP. Pure MIPS is not a proper metric (no triangle inequality), so libraries offer specialized MIPS modes or reduce it to cosine/L2 via normalization or an extra dimension.

The flows

On the governed enterprise platform, policy RAG uses unit-normalized vectors and cosine-equivalent IP; absolute cosine cutoffs lose to a reranker when precision matters. A "similar tickets" feature that encodes volume stays on dot product on purpose.

A worked example

Three 2-D toy vectors (same algebra at 1536-d). Query q = (0.6, 0.8), norm 1 — "international travel pre-approval." d₁ = (0.6, 0.8), norm 1 — right chunk. d₂ = (1.2, 1.6), norm 2 — same direction, double magnitude. d₃ = (0.8, −0.6), norm 1 — wrong topic.

Under cosine, d₁ and d₂ tie, then d₃. Under dot product, hub d₂ wins, then d₁, then d₃. Normalize d₂ to (0.6, 0.8) and all three metrics agree again.

In production shape: an OpenAI-style API returns unit vectors; the index is labeled cosine but stores IP after normalize-on-write. Real top hits might show cosine 0.78, 0.74, 0.71 — not transferable as thresholds to a model whose scores all sit above 0.85.

Production concerns

Model–metric mismatch is a silent quality bug: results still return, they are just worse. Read the model card, match the index, and treat metric as part of the embedding version. Normalize once, in the right place: stores that implement cosine as IP need unit vectors at write; assuming missing normalization corrupts rankings, while double-normalizing a unit vector is harmless. Truncating Matryoshka embeddings destroys unit norm — re-L2-normalize after.

Score thresholds are model- and metric-specific (many models never score below ~0.6 for any pair). Calibrate per model and corpus, or skip absolute bi-encoder cutoffs and use a reranker. Metric math is a rounding error next to index and embedding cost — choose for correctness (latency). Watch hubness under unnormalized IP: a few high-norm vectors dominate top-k. Entitlement filters on the platform change the candidate set — raw cosine is not calibrated confidence across ACL slices. See filtering and metadata.

Common drill-downs

Is cosine 0.75 "good"? Unanswerable without the model and corpus. Many models compress all pairs into a narrow high band. Thresholds need held-out calibration per model, or a second-stage score.

When would you deliberately keep magnitude? When it encodes popularity, confidence, or another product signal — recommender matrix factorization is MIPS. Also when the model card specifies inner-product training with unconstrained norms.

Teammate switched cosine → L2 and nothing broke — safe? On unit vectors, rankings are identical (L2² = 2 − 2cos), so top-k is unchanged. Absolute thresholds and "similarity" semantics become wrong (distance, lower-is-better). Safe for ranking only.

Test yourself

Go deeper

• Vector Similarity Explained — Pinecone — three metrics and the "match training" rule of thumb.
• Distance metrics — Weaviate docs — cosine/dot/L2 ranges and normalize-then-dot cosine.
• MetricType and distances — FAISS wiki — squared L2 vs inner product conventions.
• Cosine Similarity, Clearly Explained!!! — StatQuest — short geometry walkthrough.

Where this connects

• Embeddings — unit-length vectors and Matryoshka truncation.
• ANN indexes — index built for one metric; recall assumes that geometry.
• Embeddings beyond RAG — recommenders often want dot product for popularity-in-norm.
• Rerankers — rescore pairs when bi-encoder scores are untrustworthy.