What our PII redaction catches, and what it misses

LayBuild's PII redaction runs on the AI's answer, not on the customer's message. Four regular expressions replace card numbers, US Social Security numbers, email addresses and phone numbers with [REDACTED] before a reply is saved. Whatever the customer types still reaches your LLM provider exactly as typed. This post explains where redaction sits, what the patterns catch when you run real strings through them, and what you should add if your customers share sensitive data in chat.

Where redaction sits in the reply pipeline

Each AI reply goes through a LangGraph state machine. Redaction is one step near the end:

customer message
      |
      v
preflight guardrails  (blocked terms, prompt-injection patterns, off-topic refusals)
      |
      v
retrieval             (Qdrant + Postgres full-text, scoped to your org)
      |
      v
LLM provider          <-- sees the raw customer message, recent history, retrieved docs
      |
      v
grounding check       (lexical overlap with sources)
      |
      v
output guardrail      <-- PII regexes run here, answer capped at 8,000 characters
      |
      v
saved assistant message, webhook, widget / WhatsApp

The preflight step can stop a message from reaching the model at all, but only for blocked terms, prompt-injection patterns and off-topic requests. It does not look for personal data. The PII step is on by default (GUARDRAIL_REDACT_PII), and an org admin can switch it off in the guardrail settings.

The four patterns

These are the exact expressions in packages/agent/src/guardrails.ts:

const PII_PATTERNS: RegExp[] = [
  /\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g, // 16-digit card numbers
  /\b\d{3}-\d{2}-\d{4}\b/g, // US SSN with dashes
  /[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/g, // email addresses
  /\b\+?\d{1,3}[-. ]?\(?\d{2,4}\)?[-. ]?\d{3,4}[-. ]?\d{4}\b/g, // phone numbers
];

Regexes are cheap and predictable, which is why we use them in a step that runs on every reply. They also have no idea what a number means. They match shapes.

What happens when you run real strings through them

We ran the four patterns against a set of strings a support conversation might contain. The results show both kinds of error you get from shape matching.

Three things stand out. The card pattern only knows the 16-digit, 4-4-4-4 layout, so 15-digit Amex numbers and 19-digit cards pass through, while any 16-digit order or tracking number gets masked. Common Indian mobile formatting with a space after the country code slips through. And the classes of data that cause most real incidents (names, postal addresses, dates of birth, non-US government IDs, account numbers) have no pattern at all.

None of this is surprising for regex redaction. It is worth knowing before you tell a compliance reviewer that "PII is redacted".

Where unredacted text still goes

The output check protects one thing: the text of the AI answer. Several other paths carry customer text as typed:

• The LLM provider. The raw message, up to 6 recent messages of history and the retrieved documents are sent to whichever provider you configured. Their retention and training terms apply to that data.
• Your database. Messages are stored in Postgres as written. Conversations and messages are not encrypted at the application level (stored LLM keys, SMTP passwords, widget keys and uploaded files are, with AES-256-GCM).
• The live token stream. When the provider streams, tokens are pushed to the chat window as they arrive, before the output guardrail runs. The saved message is redacted; the preview the customer watched being typed was not. If a model echoes a card number back, it can appear briefly on screen.
• Outbound webhooks. The MESSAGE_RECEIVED event carries the customer's message content unredacted to every endpoint you've subscribed.
• The learning loop. LayBuild auto-publishes "learnable" exchanges as knowledge documents without human approval. The answer has been redacted, but the customer's question becomes the document title as typed. An LLM check skips answers that look personal, but the question text itself is not run through the PII patterns.
• Tracing. If you set Langfuse keys, traces include the query.

What to do about it

Decide what your LLM provider is allowed to see

This is the biggest exposure and redaction does nothing for it. Read the data-use terms for the provider you configure: whether prompts are retained, for how long, and whether they are used for training. Terms differ by provider and by account type, so check your own contract rather than a blog summary. If the answer is unacceptable, the options are a provider whose terms you accept, or a model you run yourself. LayBuild can talk to any OpenAI-compatible endpoint, including one on your own network; see self-hosting LayBuild for what else leaves your network.

Redact input before it reaches LayBuild

If you send messages through the REST API from your own app, you control the text before it leaves your servers. Redact there. A Luhn check removes most of the order-number false positives that a pure shape match produces:

const CARD_CANDIDATE = /\b(?:\d[ -]?){13,19}\b/g;

// A shape match alone masks order numbers too; the Luhn checksum
// keeps false positives down without missing real card numbers.
function passesLuhn(digits: string): boolean {
  let sum = 0;
  let double = false;
  for (let i = digits.length - 1; i >= 0; i--) {
    let d = Number(digits[i]);
    if (double) {
      d *= 2;
      if (d > 9) d -= 9;
    }
    sum += d;
    double = !double;
  }
  return sum % 10 === 0;
}

export function redactCards(text: string): string {
  return text.replace(CARD_CANDIDATE, (match) => {
    const digits = match.replace(/[ -]/g, '');
    return digits.length >= 13 && passesLuhn(digits) ? '[CARD]' : match;
  });
}

For names, addresses and national IDs, regexes won't get you far. A named-entity detector such as Microsoft's open-source Presidio, run as a service in front of your API calls, handles more categories, at the cost of latency and its own false positives. Test it on your real traffic before trusting it.

For the hosted web widget, the text goes from the customer's browser to LayBuild directly, so there is no hook for you to redact in between. What you can do there is set expectations: say in the widget's welcome message that customers should not share card numbers or passwords in chat, and route payment problems to a human or a secure form.

Use the blocked-terms list for hard stops

Admins can add blocked terms. A message containing one is refused in preflight and never reaches the model. It is a substring match, so it can't catch a card number, but it can stop topics you never want the AI to handle ("password reset code", for example) and push those customers towards a human.

Clean up what's stored

Plan for data you didn't want to keep. LayBuild has self-service data export and deletion requests for signed-in accounts (these support GDPR requests; they are not a certification), and org admins can delete learned knowledge documents they don't want. Check the auto-learned docs periodically for questions that carry personal details. And if you subscribe to webhooks, treat the receiving system as holding raw chat text, with the same access controls you'd apply to the chat itself.

Frequently asked questions

Does the redaction make LayBuild suitable for payment card data?

No. Output redaction doesn't keep card data out of the provider, the database or webhooks, and LayBuild holds no PCI DSS attestation. Keep card numbers out of chat entirely.

Can I turn redaction off?

Yes, per org, in the guardrail settings. You might do this if the false positives on 16-digit order numbers break answers your customers need. Weigh that against the cases it does catch.

Related reading

• Self-hosting LayBuild: what runs where, and what leaves your network
• Healthcare support chat: why PHI shouldn't go through LayBuild today
• How LayBuild keeps tenants apart
• Preventing hallucinations in customer support