Keeping an AI support agent's knowledge current

LayBuild does not sync from Notion, GitHub, Zendesk, Google Docs or Confluence, and it does not re-fetch web pages on a schedule. Your agent's knowledge changes when you, or a script you run, change it. An earlier version of this post described realtime change-data-capture pipelines into a vector database. We don't ship that, and this version replaces it. What follows applies whatever tooling you use: what goes stale first, what LayBuild does when a document changes, a refresh routine, and how to automate updates through the API.

What goes stale first

Some content changes with almost every release. Check these first:

• Prices, plan limits and what each plan includes.
• UI paths like "Settings > Billing > Invoices", which break silently when a menu moves.
• Feature availability: what's in beta, what was removed, what is limited to certain plans.
• Policies with numbers in them: refund windows, trial terms, response-time commitments.
• Workarounds for known bugs, which become wrong the day the bug is fixed.
• Integration instructions, which change when the other vendor changes their product.

Stored answers go stale too. Manual Q&A pairs and auto-learned Q&A pairs hold an answer as it was written at the time. A stale Q&A pair is worse than a stale document, because a matching question gets the stored answer without the model ever seeing your updated docs.

What LayBuild does when you change a document

When you edit a document's content, LayBuild re-chunks it and hashes each new parent chunk. Chunks whose hash matches a stored chunk keep their existing vectors. Changed chunks and their child chunks are re-embedded, and chunks that no longer exist are deleted from Postgres and Qdrant. An edit to one section of a long document re-embeds that section, not the whole file. LayBuild may also regenerate the stored answers of Q&A pairs whose questions look related to the new content, so check your Q&A pairs after a large change.

Caches can delay what customers see. Retrieval results are cached for 300 seconds per question, fast Q&A lookups for 600 seconds, and generated answers for 15 minutes per exact question text. Editing a document does not clear these caches, so a customer who asks exactly the same question within 15 minutes of the change can get the old answer. When you test an update, wait out the cache or phrase the test question differently.

Web pages have one more step. A URL already in your knowledge base is skipped when you add it again. To refresh a changed page, delete its document and add the URL again. LayBuild fetches only the URLs you list, keeps up to 50,000 characters per page, and doesn't run JavaScript.

A refresh routine that works

Tools matter less than ownership. A routine we'd recommend:

• Give every document an owner, the person who knows when it becomes wrong. For product docs, that is usually whoever owns the feature.
• Tie reviews to your release process. When a release changes behaviour, the release checklist should name the knowledge documents it affects.
• Search before you ship. Search your knowledge base for the terms a release touches (the plan name, the setting, the old menu path) and update every hit, including Q&A pairs.
• Replace documents instead of adding new versions. If you upload "Pricing v2" and leave "Pricing v1", both compete in retrieval and the agent may quote either. Delete the old one, or edit its content in place.
• Test with real questions. After an update, ask the questions customers actually ask about that topic and read the answers.
• Review auto-learned Q&A pairs (category "Learned", tag "auto-learned") after any change to policies or pricing, since they may hold the old answer.

Your conversations can tell you what went stale. The fixed reply "I do not have specific information about that in the knowledge base" marks questions your docs don't answer. Handoffs to humans and low ratings often mark answers that were wrong. LayBuild's outbound webhooks fire on conversation created, message received, handoff and conversation closed, so you can send handoffs to wherever your team tracks doc fixes.

Automating updates through the API

If your docs live in a Git repository or another system you control, you can push changes to LayBuild from a script. These are the relevant endpoints:

JSON content is limited to 50,000 characters per document, so split longer files. Creating documents counts against your plan's document limit.

The limitation you need to plan around is authentication. These routes take a bearer token from a logged-in user with the ADMIN role. There is no long-lived API key for them today, and accounts with two-factor login need an extra step to get a token. On self-hosted installs, tokens last 7 days by default (JWT_EXPIRES_IN). So a script needs a token that a person refreshes, or a dedicated admin account whose login you manage carefully. We'd rather tell you that now than have you find it halfway through building a pipeline.

Here is a script that pushes markdown files to existing documents. Create each document once in the dashboard, record its ID in a map file, and run the script whenever the files change, for example from CI after a merge to your docs branch:

import { readFile } from 'node:fs/promises';

// Reader-side script: these come from your own CI secrets.
const apiUrl = process.env.LAYBUILD_API_URL;
const token = process.env.LAYBUILD_ADMIN_TOKEN;
const MAX_CONTENT_CHARS = 50_000;

if (!apiUrl || !token) {
  throw new Error('Set LAYBUILD_API_URL and LAYBUILD_ADMIN_TOKEN');
}

// docs-map.json maps a file path to a LayBuild knowledge document id.
const docMap = JSON.parse(await readFile('docs-map.json', 'utf8')) as Record<string, string>;

for (const [path, documentId] of Object.entries(docMap)) {
  const content = await readFile(path, 'utf8');
  if (content.length > MAX_CONTENT_CHARS) {
    // The API rejects longer content, so split the file into two documents.
    console.error(`${path}: ${content.length} characters, over the limit, skipped`);
    process.exitCode = 1;
    continue;
  }
  const res = await fetch(`${apiUrl}/api/admin/knowledge/${documentId}`, {
    method: 'PUT',
    headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({ content }),
  });
  if (!res.ok) {
    console.error(`${path}: ${res.status} ${await res.text()}`);
    process.exitCode = 1;
    continue;
  }
  console.log(`${path}: updated`);
}

Run it with bun sync-docs.ts. The request returns once the changed chunks are re-embedded, so a large batch takes a while; run it in CI rather than on a laptop you plan to close.

The same pattern works for other sources. If your help center or wiki can send a webhook when an article changes, a small service you host can receive it, fetch the article as markdown and call PUT. You build and run that glue. LayBuild provides the endpoint at the end of it.

What LayBuild does not do

To be explicit: no connectors to Notion, GitHub, Zendesk, Google Docs, Confluence or any other source; no crawling beyond the URLs you list; no scheduled re-fetch or change detection for URLs; no service API keys for the knowledge endpoints. If any of those is a hard requirement, weigh it now rather than after setup.

Where to go next

Write the documents well before you automate them: writing help docs an AI agent can retrieve. To see what LayBuild sends to your systems, read custom webhooks for support automation. API details are on /developers.