Hybrid & multivector search

The breadth-parity guide (vs VectorChord / Qdrant): how to do multivector / late-interaction (ColBERT) re-ranking, hybrid dense+sparse fusion (RRF), and named vectors (multiple vector columns) with pg_turbovec.

All three are additive SQL surface — they do not change the index wire format or the index AM. pg_turbovec indexes one vector per row; the multivector and hybrid pieces are query-layer constructs on top of that single-vector index.

Phase D (v1.13.x). turbovec.max_sim / turbovec.max_sim_cosine (MaxSim re-rank) and turbovec.rrf_score (reciprocal rank fusion) ship as SQL functions. Named vectors are a documented schema pattern. Index-native late interaction is a documented future phase — see the limitation.

1. Multivector / late interaction (MaxSim)

What it is

Late-interaction models (ColBERT and friends) represent a query and a document each as a set of per-token vectors, not a single pooled vector. The relevance score is MaxSim:

MaxSim(Q, D) = sum over query-token vectors q in Q of
                 ( max over doc-token vectors d in D of sim(q, d) )

Each query token finds its single best-matching document token; the score is the sum of those per-token maxima. This captures fine-grained term-level matches that a single pooled embedding averages away.

The functions

turbovec.max_sim(query turbovec.vector[], doc turbovec.vector[])
  -> double precision    -- dot-product MaxSim

turbovec.max_sim_cosine(query turbovec.vector[], doc turbovec.vector[])
  -> double precision    -- cosine MaxSim

Both take arrays of turbovec.vector — one entry per token.

Conventions (read these):

  • Similarity, not distance. max_sim uses raw dot product as the per-pair similarity (higher = more similar). max_sim_cosine uses cosine similarity = 1 - cosine_distance (range [-1, 1], higher = more similar). ColBERT token vectors are normally L2-normalised, in which case max_sim (dot) already equals cosine — prefer max_sim for normalised tokens (it skips the per-pair norm).
  • Dimension. Every token vector in both arrays must share one dimension. A mismatch raises ERROR: different vector dimensions N and M in 'max_sim'.
  • Empty arrays score 0.0. An empty query (nothing to match) and an empty doc (nothing to match against) both yield 0.0. This is the ColBERT convention and lets you LEFT JOIN candidate docs without NULL surprises.
  • Determinism. The outer sum walks the query tokens in array order; no reassociation, so the f64 result is reproducible.
  • Cost. O(|Q| · |D| · dim) per pair. This is a re-rank primitive for a small candidate set (top-N), not an index scan.
SELECT turbovec.max_sim(
  ARRAY['[1,0]','[0,1]']::turbovec.vector[],          -- query tokens
  ARRAY['[1,0]','[0,1]','[1,1]']::turbovec.vector[]   -- doc tokens
);
-- q1=[1,0]: max dot over doc = 1 ; q2=[0,1]: max dot over doc = 1 ; sum = 2

The re-rank usage pattern

MaxSim over every (query, doc) pair in a corpus is O(corpus) — too slow to scan. The standard ColBERT serving recipe is two-stage retrieve-then-rerank:

  1. Stage 1 — ANN retrieve candidates on a single pooled / centroid vector per document. Store one mean-pooled turbovec.vector per doc, index it with a turbovec index, and pull the top-N (say N = 100–1000) candidate documents with a normal <=> / <#> ANN query.
  2. Stage 2 — MaxSim re-rank the N candidates using their full per-token arrays, then take the final top-k.
-- Schema: one pooled vector (indexed) + the per-token array (re-rank only)
CREATE TABLE docs (
  id           bigint PRIMARY KEY,
  pooled       turbovec.vector,          -- mean of token vectors; indexed
  tokens       turbovec.vector[]         -- per-token ColBERT vectors
);
CREATE INDEX ON docs USING turbovec (pooled vec_cosine_ops);

-- Query: :q_pooled is the pooled query vector; :q_tokens the token array.
WITH candidates AS (
  SELECT id, tokens
  FROM docs
  ORDER BY pooled <=> :q_pooled        -- stage 1: ANN on pooled vector
  LIMIT 200                            -- candidate set N
)
SELECT id,
       turbovec.max_sim(:q_tokens, tokens) AS score   -- stage 2: rerank
FROM candidates
ORDER BY score DESC                                    -- MaxSim is a SIMILARITY
LIMIT 10;

The pooled-vector ANN is sublinear (index-accelerated); the MaxSim re-rank is O(N · |Q| · |D| · dim) over just the N candidates, which is cheap for N in the hundreds.

The honest limitation — and what colbert_search (F-1) adds

pg_turbovec’s persistent index stores one vector per row. max_sim is a re-rank primitive over candidate documents' token arrays; the pooled-vector + max_sim pattern above is not an index-accelerated late-interaction scan, and its recall is capped by the pooled-vector ANN: if the right document’s pooled vector isn’t in the top-N candidates, MaxSim never sees it.

turbovec.colbert_search (v1.16.0, Phase F-1) closes exactly that recall gap — it retrieves candidates by best single token, not pooled mean:

turbovec.colbert_search(
    rel         regclass,
    id_col      text,                 -- bigint doc key
    token_col   text,                 -- a turbovec.vector[] column (per-doc tokens)
    query       turbovec.vector[],    -- the query's token vectors
    k           integer,              -- final top-k docs
    per_token_k integer DEFAULT 64,   -- stage-1 hits per query token
    candidate_n integer DEFAULT 256,  -- max candidate docs into stage 2
    bit_width   integer DEFAULT 4
) RETURNS TABLE(id bigint, score double precision)

-- docs(id bigint, tokens turbovec.vector[])  -- per-doc token arrays
SELECT id, score
FROM turbovec.colbert_search('docs'::regclass, 'id', 'tokens',
                             :q_tokens::turbovec.vector[], 10);

How it works (two stages, same shape as the pattern above but with an indexed stage 1 over all tokens):

  1. Stage 1 (index-accelerated). A backend-cached flat token index — one slot per token across all docs, the slot’s id being its doc id — is batch-searched with all |Q| query tokens; the hit doc-ids are unioned into a candidate set (capped at candidate_n, keeping the docs with the best stage-1 token score). Because every token is indexed, a doc whose pooled vector is far but which has one token near a query token is still retrieved — the recall the pooled-vector pattern cannot get.
  2. Stage 2 (exact). Each candidate’s full token array is read from the heap and scored with the exact max_sim kernel; the top k are returned.

What it is and isn’t. colbert_search is the index-native stage-1 over max_sim’s exact stage-2. It is not the full persistent multivector index AM (per-token relfile, MaxSim-aware scan, PLAID centroid pruning) — the token index lives only in the backend cache (like turbovec.knn), so there is no wire-format change and no CREATE INDEX for it. It rebuilds per backend on a cold cache (fine for moderate corpora; the build is the same quantize-pack as knn). For very large token corpora a persistent token-index AM (Phase F-2) is the next step — gated on a measured recall/latency win over this F-1 path (see ).

Tuning. per_token_k (stage-1 hits per query token) and candidate_n (max docs into stage 2) trade recall for work; under heavy token quantization (2–3 bit) raise per_token_k. bit_width 4 is the safe default for 128-d ColBERT tokens. The crossover where colbert_search beats the pooled-vector pattern is workload- dependent (it wins on entity / rare-term / long-doc queries where the pooled mean washes out the matching token; it ties on pooled-friendly semantic queries).

2. Hybrid dense + sparse (Reciprocal Rank Fusion)

What it is

Hybrid search fuses a dense semantic ranking (vector ANN) with a sparse / lexical ranking (BM25 / full-text ts_rank, or a turbovec.sparsevec inner-product score). The two rankers disagree on scale, so you fuse by rank, not by raw score, using Reciprocal Rank Fusion (RRF):

score(d) = sum over rankers r of  1 / (k + rank_r(d))

k (default 60, from Cormack et al. 2009) damps the contribution of low ranks. A document ranked highly by both rankers wins.

The helper

turbovec.rrf_score(rank integer, k integer DEFAULT 60)
  -> double precision    -- = 1.0 / (k + rank)

The arithmetic is trivial; the helper exists so the formula and its k default live in one tested place instead of being retyped in every query. k + rank must be positive (a non-positive denominator raises ERROR). Use a consistent rank base across rankers (this guide uses 0-based ranks via ROW_NUMBER() - 1, but 1-based works as long as both rankers agree).

SELECT turbovec.rrf_score(0);      -- 1/60  ≈ 0.016667
SELECT turbovec.rrf_score(1, 10);  -- 1/11  ≈ 0.090909

The recipe (dense ANN + full-text)

-- Schema: one dense vector column + a tsvector for lexical search.
CREATE TABLE docs (
  id      bigint PRIMARY KEY,
  body    text,
  emb     turbovec.vector,
  body_tsv tsvector GENERATED ALWAYS AS (to_tsvector('english', body)) STORED
);
CREATE INDEX ON docs USING turbovec (emb vec_cosine_ops);
CREATE INDEX ON docs USING gin (body_tsv);

-- Query: :q_emb is the dense query vector; :q_text the keyword query.
WITH dense AS (
  SELECT id,
         ROW_NUMBER() OVER (ORDER BY emb <=> :q_emb) - 1 AS rk
  FROM docs
  ORDER BY emb <=> :q_emb
  LIMIT 100                                      -- dense candidate pool
),
sparse AS (
  SELECT id,
         ROW_NUMBER() OVER (
           ORDER BY ts_rank(body_tsv, plainto_tsquery('english', :q_text)) DESC
         ) - 1 AS rk
  FROM docs
  WHERE body_tsv @@ plainto_tsquery('english', :q_text)
  LIMIT 100                                      -- lexical candidate pool
),
fused AS (
  SELECT id, SUM(s) AS score FROM (
    SELECT id, turbovec.rrf_score(rk) AS s FROM dense
    UNION ALL
    SELECT id, turbovec.rrf_score(rk) AS s FROM sparse
  ) u
  GROUP BY id
)
SELECT d.id, d.body, fused.score
FROM fused JOIN docs d USING (id)
ORDER BY fused.score DESC
LIMIT 10;

Each ranker produces a 0-based rank via ROW_NUMBER(); rrf_score turns rank into a fusion term; the UNION ALL + GROUP BY ... SUM sums the terms per document; the final ORDER BY score DESC returns the fused ranking. A document present in only one ranker still contributes that ranker’s term (it just doesn’t get the boost of appearing in both).

Sparsevec variant

If you keep a learned sparse / SPLADE vector in a turbovec.sparsevec column, swap the sparse CTE’s ordering for the sparse inner product (<#> is negative inner product, so smaller is more similar — order ascending):

sparse AS (
  SELECT id,
         ROW_NUMBER() OVER (ORDER BY svec <#> :q_svec) - 1 AS rk
  FROM docs
  ORDER BY svec <#> :q_svec
  LIMIT 100
)

Tuning k

k = 60 is the canonical default and a fine starting point. Lower k (e.g. 10–20) sharpens the contribution of the very top ranks (the #1 result dominates); higher k (100+) flattens the curve so deeper ranks matter more. Tune on your own relevance judgments; the default is robust across corpora.

Why a scalar helper, not a server-side fusion operator

The roadmap calls for “server-side fusion only on demand.” The scalar rrf_score + the CTE recipe above is the pragmatic core: it composes with arbitrary rankers, WHERE filters, and JOINs, and it is fully inspectable in EXPLAIN. A bespoke two-array fusion aggregate would be less flexible and is not provided until a concrete need appears.

3. Named vectors (multiple vector columns)

“Named vectors” — Qdrant’s term for multiple distinct embeddings per record (e.g. a title embedding and a body embedding, or a CLIP image embedding and a text embedding) — is a schema pattern in PostgreSQL, not a feature. Each named vector is just another turbovec.vector column with its own index:

CREATE TABLE products (
  id        bigint PRIMARY KEY,
  title_emb turbovec.vector,
  body_emb  turbovec.vector,
  image_emb turbovec.vector
);
CREATE INDEX ON products USING turbovec (title_emb vec_cosine_ops);
CREATE INDEX ON products USING turbovec (body_emb  vec_cosine_ops);
CREATE INDEX ON products USING turbovec (image_emb vec_cosine_ops);

Query a single named vector with a normal ANN query (ORDER BY title_emb <=> :q). To combine multiple named vectors into one ranking, fuse them at query time with RRF — exactly the recipe in §2, with each named-vector ANN as one ranker:

WITH by_title AS (
  SELECT id, ROW_NUMBER() OVER (ORDER BY title_emb <=> :q_title) - 1 AS rk
  FROM products ORDER BY title_emb <=> :q_title LIMIT 100
),
by_body AS (
  SELECT id, ROW_NUMBER() OVER (ORDER BY body_emb <=> :q_body) - 1 AS rk
  FROM products ORDER BY body_emb <=> :q_body LIMIT 100
)
SELECT id, SUM(s) AS score FROM (
  SELECT id, turbovec.rrf_score(rk) AS s FROM by_title
  UNION ALL
  SELECT id, turbovec.rrf_score(rk) AS s FROM by_body
) u
GROUP BY id ORDER BY score DESC LIMIT 10;

Each column has its own index, so each sub-query is index-accelerated; RRF fuses the per-column rankings without any cross-column distance needing to be defined.

See also