Support questions come in two shapes that no single index handles well. "Error E1042 when exporting to CSV" hinges on an exact token that vector search tends to blur. "Can I get my money back?" shares no words with a page titled "Refund policy", so keyword search misses it. LayBuild runs several searches in parallel for every question and merges the results. This post describes what each search does, how the merge works, and the parts we would still like to improve.

The legs, and what each one catches

Each customer message that needs an answer (not a greeting or a "thanks") becomes a search query. If it is a follow-up like "what about the second one?", a model first rewrites it into a standalone query using the last few messages. Then the retrieval legs run at the same time:

text
                         search query
                              |
     +-------------+----------+-----------+-------------+
     |             |          |           |             |
  Qdrant       Postgres    Postgres    knowledge     BM25
  vector       full-text   full-text   graph         (off by default,
  search       (chunks)    (Q&A pairs)               NLP_ENABLE_BM25)
     |             |          |           |             |
     +-------------+----------+-----------+-------------+
                              |
                Reciprocal Rank Fusion (k = 60)
                              |
          per-candidate score, drop anything below 0.4
                              |
                    top 4 passages to the prompt

The vector leg embeds the query with the default model, BAAI/bge-small-en-v1.5 via Hugging Face, and runs a cosine search in the Qdrant collection laybuild_knowledge_docs. The search is filtered on the payload by organization and agent, so one tenant's vectors never compete with another's. Parent chunks, the smaller child chunks and one vector per whole document all live in that collection, and the search returns whichever matches best. If the embedding call fails, LayBuild falls back through Gemini text-embedding-004, OpenAI text-embedding-3-small, Qwen text-embedding-v3 and Cloudflare's hosted bge-small. Qdrant is optional. Without it, this leg returns nothing and the others carry the load.

The full-text leg runs Postgres full-text search with the english configuration over your parent chunks, using the chunk's heading path as its title. It is good at exact product names, error codes and plan names, and it stems English words, so "refunds" matches "refund".

The Q&A leg runs the same kind of full-text search over the question and answer text of your manual Q&A pairs. The knowledge graph leg looks up entities and relationships extracted from your documents at ingestion time and returns a short summary of how they connect, which helps with questions like "which plans include the API?". BM25 is covered below.

Each leg asks for up to 8 candidates (twice the 4 passages we keep, capped at 25). Results are cached in Redis for 300 seconds per organization, agent, provider and normalized question.

Why rank fusion instead of blending scores

The obvious way to merge five result lists is a weighted sum of their scores. It doesn't work well here, because the scores live on different scales. Cosine similarity is bounded, and each embedding model spreads unrelated text across a different part of that range. Postgres ts_rank_cd is unbounded and depends on document length and term density. BM25 scores are also unbounded and shift as the corpus grows. A weighted sum needs calibration per corpus, and the calibration drifts every time a customer uploads a large document.

Reciprocal Rank Fusion ignores the raw scores and uses only positions. Each candidate gets 1 / (k + rank) from every list it appears in, and the contributions add up. We use k = 60, the value from the original 2009 RRF paper by Cormack, Clarke and Büttcher. With k = 60, rank 1 in one list is worth 1/61 (about 0.0164) and rank 8 is worth 1/68 (about 0.0147). A passage that shows up at rank 3 in two legs gets 2/63 (about 0.0317), nearly double a passage that tops only one list. Large k flattens the difference between ranks and rewards agreement between legs, which is what you want when the legs are measuring different things.

What fusion does not decide in our pipeline

Here is the part the previous version of this post left out. RRF values are relative. A fused score of 0.03 tells you two legs agreed. It does not tell you the passage is relevant, and you cannot set a threshold on it. So after fusion, LayBuild computes an absolute score for each candidate and uses that for ordering and for the cut-off:

  • Vector hits take the higher of their cosine similarity and a lexical coverage score.
  • BM25 hits take the higher of their BM25 score and the lexical coverage score.
  • Q&A pairs get at least 0.85, and knowledge graph summaries at least 0.9.
  • Candidates found only by full-text search take the lexical coverage score, and must share at least one stemmed word with the question to survive at all.

Lexical coverage is 0.25 + 0.6 × (share of the question's stemmed words found in the passage), capped at 0.95.

In the current code, fusion merges and de-duplicates the legs, and the absolute score decides the order. That has a cost: agreement between two legs does not lift a passage in the final order, even though agreement is the signal RRF is designed to reward. The absolute score is what makes the 0.4 floor below possible, and a pure rank-fusion score could not support a floor. Using the fused value as a boost or tie-breaker on top of the absolute score is an obvious improvement that the current code does not make.

What the 0.4 floor does

After retrieval, anything scoring below 0.4 is dropped. Self-hosted deployments can change this with RAG_MIN_SCORE.

For a passage found only by full-text search, 0.4 means at least a quarter of the question's stemmed content words appear in it. Suppose a question reduces to four stems: "export", "csv", "error", "e1042". A passage containing one of them scores 0.25 + 0.6 × 0.25 = 0.40 and passes. For a vector hit, the passage passes if either its cosine similarity or its lexical coverage reaches 0.4. Q&A pairs and graph summaries always pass.

If nothing passes, the retrieval result is empty, and strict knowledge-base mode takes over (see how LayBuild limits made-up answers). Raising the floor cuts loosely related passages and increases fixed "I do not have specific information" replies. Lowering it does the opposite. Cosine similarity distributions differ by embedding model, so if you change the embedding model, re-check the floor against your own questions.

Why BM25 is off by default

Postgres full-text search already covers exact-term matching. The BM25 leg adds term-frequency weighting, which can help when many documents mention the same terms. It costs more than it looks:

  • It builds an in-memory index per organization and agent inside each API process, over up to 5,000 parent chunks (NLP_BM25_MAX_DOCS). Past that cap, chunks are left out of the index.
  • The index expires after 600 seconds (NLP_BM25_TTL_SECONDS), and the first question after expiry pays for a rebuild.
  • Each API instance holds its own copy. Adding a document clears the index only in the process that handled the upload, so other instances can serve stale results until their copy expires.
  • Raw BM25 scores are unbounded, so BM25 hits clear the 0.4 floor more easily than hits from the other legs.

Turn it on (NLP_ENABLE_BM25=true) if an evaluation on your own questions shows full-text search missing things BM25 finds. Our guide to benchmarking retrieval on your docs covers how to run that comparison.

Other options that are off by default

Three more options exist, and each adds at least one model call per question. The LLM listwise reranker (RAG_ENABLE_RERANKER) asks the chat model to reorder candidates and keeps the top 5. It is not a cross-encoder. Query expansion (RAG_ENABLE_QUERY_EXPANSION) generates alternative phrasings and runs every leg once per phrasing. GraphRAG global search (RAG_ENABLE_GLOBAL_GRAPH) summarizes clusters of related entities for broad questions. Each extra call adds latency and token cost, and the default chat model is small, so measure them on your own questions before turning them on.

Known limitations

Retrieval is English-first. The default embedding model is English-only and the full-text configuration is english. A question in Spanish against English docs relies on the embedding leg alone, and that leg was not trained for it. There is no translation step.

The Q&A leg matches a pair if any single query word matches as a prefix, and Q&A pairs score at least 0.85. That favours your curated answers, but a loosely related Q&A pair can push a better document passage out of the top 4.

The prompt gets at most 4 passages, each cut to 1,000 characters, within a 4,000-character budget shared with pinned documents. A long passage that ranks first can still lose its last few sentences. The chunking post covers what that means for how you split documents.

Where to go next

If you self-host, the retrieval code lives in apps/api/src/services/retrieval/ in the LayBuild repository, and every setting mentioned here is an environment variable. For the content side, read writing help docs an AI agent can retrieve. For the API, see /developers.