If your support answers have to match your current docs, retrieval is the right default. Fine-tuning changes how a model writes much more reliably than it changes what the model knows, and any facts it does absorb go stale the day your pricing page changes. LayBuild does retrieval. We do not fine-tune models, and this post explains that choice. There are still cases where fine-tuning is worth its cost, and we cover those too.
What each approach changes
Fine-tuning updates a model's weights using example inputs and outputs. It is good at teaching patterns: a house tone, a fixed answer format, a classification scheme, vocabulary the base model handles badly. It is a poor way to store facts. You can't easily see which facts the model absorbed, you can't update one fact without training again, and you can't remove one on request.
Retrieval leaves the model alone and changes its input. At answer time, the system searches your content and puts the relevant passages into the prompt. Update a document and the next answer uses the new text. Because the sources are in the prompt, you can check an answer against them, and when nothing relevant is found you can refuse instead of guessing.
Where fine-tuning helps
Fine-tuning earns its cost when the problem is behaviour, not knowledge:
- You need a strict output format on every reply, such as structured fields for a downstream system, and prompting alone isn't consistent enough.
- You run classification or routing at high volume and want a small, cheap model to do one narrow task well.
- Your domain uses terms the base model misreads, and you have many examples of correct usage.
- Your prompts are long because they carry many worked examples, and training those examples into the model would shorten every request.
In each case you need a good set of labelled examples. Collecting and reviewing those is usually the biggest cost, bigger than the training run.
Where it hurts
Fine-tuning is a bad fit for facts that change (prices, plan limits, feature availability, policies), for anything specific to one customer, and for anything you may need to delete. If a fact is only in the weights, deleting it means retraining. It also ties you to one model from one provider. When a better or cheaper model ships, you train again.
A side-by-side comparison
| Question | Fine-tuning | Retrieval |
|---|---|---|
| How do you update a fact? | Retrain on new examples | Edit the document |
| Can you show where an answer came from? | No | The source passages are in the prompt |
| Can it refuse when it doesn't know? | Only as well as it was trained to | Yes, when retrieval finds nothing or the answer strays from the sources |
| Can you delete one piece of information? | Retrain without it | Delete the document |
| What does it need from you? | Labelled example conversations | Well-written documents |
| What does it change best? | Tone, format, narrow tasks | What the model knows right now |
| Switching models later | Train again | Point at the new model |
Questions to decide with
Answer these for your own situation:
- How often do the facts in your answers change? Monthly or more often points to retrieval.
- Do you need to check answers against a source, or refuse when there isn't one? That requires retrieval.
- Do you have hundreds of reviewed example conversations showing the behaviour you want? Without them, fine-tuning has nothing to learn from.
- Is your problem wrong facts or wrong style? Wrong facts are a retrieval and content problem. Wrong style may be a fine-tuning problem, but try the prompt first.
- Who will maintain it? A fine-tuned model needs retraining, evaluation and version management. A knowledge base needs someone to keep the docs current. Both are ongoing work.
On cost, compare the pricing models rather than guessing numbers. Fine-tuning usually means a training charge plus a separate per-token price for the custom model, or hosting it yourself. Retrieval means embedding your documents once, plus embedding each question and some extra prompt tokens per answer. Check your provider's current pricing pages for real figures.
Why LayBuild is retrieval-only
Support content changes constantly, and customers expect today's answer. Retrieval handles that by editing a document.
Our grounding checks also depend on having sources. In strict knowledge-base mode, which is always on, the model is told to answer only from the supplied knowledge. After generation, if fewer than 25% of the answer's stemmed content words appear in the retrieved sources, LayBuild replaces the answer with a fixed "I do not have specific information about that" reply. A fine-tuned model with nothing in the prompt gives that check nothing to compare against. How LayBuild limits made-up answers explains the check and its weaknesses.
Finally, LayBuild works with many model providers: Hugging Face (the default, running Llama 3.1 8B Instruct), OpenAI, Google Gemini, DeepSeek, Qwen, Cloudflare Workers AI, OpenRouter, NVIDIA and any OpenAI-compatible endpoint, with failover between them. Fine-tuning would lock each agent to one model.
To shape behaviour without training, LayBuild gives you a prompt template per agent, manual Q&A pairs (an exact or near-exact question match returns your stored answer without calling the model), and pinned documents that go into every prompt.
Combining the two
The approaches can be combined. If you already run a fine-tuned model behind an OpenAI-compatible API, it can be configured as the model LayBuild calls, and retrieval still supplies the facts. LayBuild will not train, host or evaluate that model for you. Do this only if the fine-tune solves a behaviour problem you have measured, like format consistency, and keep facts out of the training data so they can't contradict your documents.
The limits of retrieval
Retrieval is only as good as your documents. If the answer isn't written down clearly, no retriever finds it. Our grounding check compares words, not meaning, so it can pass a wrong answer that reuses source vocabulary and block a correct paraphrase. The default embedding model and full-text search are English-only, so non-English questions retrieve less well. And retrieval can't teach tone from examples. For that you have the prompt.
Where to go next
If you are going with retrieval, spend your effort on the documents: read writing help docs an AI agent can retrieve, then how LayBuild runs hybrid search. To see how this fits a real support setup, see /features.