AI customer support that doesn't invent answers

Stanford researchers tested the AI legal research tools that LexisNexis and Thomson Reuters sell to law firms. These are grounded, retrieval-backed, purpose-built products from billion-dollar vendors. They still produced incorrect or misgro

Stanford researchers tested the AI legal research tools that LexisNexis and Thomson Reuters sell to law firms. These are grounded, retrieval-backed, purpose-built products from billion-dollar vendors. They still produced incorrect or misgrounded answers on 17 to 33 percent of queries.

That is the number to hold in your head before you let an AI answer your customers. AI customer support does not fail because the model is stupid. It fails because someone shipped it assuming grounding meant "solved."

Grounding does not mean solved. It means survivable, if you build the rest of it.

Grounding is the whole product, and it is not a setting

Every support tool on the market will tell you it connects to your knowledge base. Fine. Connecting is the easy part.

The question that matters is what the model does when your documentation does not contain the answer. An ungrounded agent guesses fluently. A properly grounded one is built to say "I don't have that" and hand over.

That difference is not a toggle in a dashboard. It is a chain of decisions: what gets retrieved, how much of it reaches the model, whether the model is required to cite, what happens when it cannot, and who reviews the ones it got wrong. Six decisions, none of them in the vendor's onboarding flow.

This is the same pattern we hit deciding what a small business should automate first. The tool is never the project. The policy around the tool is the project.

Retrieval reduces hallucination. It does not remove it.

The Stanford RegLab and HAI study is the most useful measurement we have, and worth reading properly rather than quoting at.

Across more than 200 pre-registered legal queries, Lexis+ AI produced incorrect or misgrounded responses on over 17 percent of them. Westlaw's AI-Assisted Research came in around 33 percent, roughly double. General-purpose chatbots on the same queries: 58 to 80 percent.

So retrieval works. It cut the error rate by more than half. It also left a one-in-three failure rate in a product built by Thomson Reuters for lawyers who bill by the hour.

Two caveats I will make for you rather than let you discover later. It is a legal-domain study, and legal questions are harder than "where is my order." And it dates from 2024, so current versions have moved. Neither caveat changes the shape of the finding.

Plan for the AI being wrong one time in five. If that plan is "the customer reads it and believes it," you do not have a plan.

— Arthur, Bunny Honey Club

The confidence threshold is where the real money sits

Here is the setting nobody demos, because it makes the product look worse.

Your agent should refuse to answer below a confidence floor. Set the floor high and you deflect fewer tickets, which makes your automation-rate slide less impressive. Set it low and the agent confidently answers questions it has no business touching.

Vendors are structurally incentivised to show you a high deflection rate. You are incentivised to have a low wrong-answer rate. Those are not the same number, and the gap between them is your reputation.

We tune this by starting deliberately conservative, then loosening it category by category once we have a week of logs showing where the agent was right and quiet versus wrong and loud. Shipping ratcheted down and loosening beats shipping open and apologising.

Citations turn a two-hour audit into a ten-second one

The single most useful thing you can require is that every answer carries a pointer to the passage it came from.

Claude's Citations feature does this at the API level: you pass your help docs as source documents, and the response comes back with cited_text plus the document index and character or page location for each claim. Anthropic's docs note that because the API extracts the cited text directly, "citations are guaranteed to contain valid pointers to the provided documents." The model cannot cite a page that does not exist.

Two consequences worth the build time. First, an answer that arrives with no citation is a machine-readable escalation signal, so you get your confidence threshold nearly free. Second, auditing stops being a reading exercise: you check whether the cited passage actually says what the reply claims, which takes seconds.

And the cost objection does not survive contact with the docs. cited_text does not count toward output tokens, and is not counted as input when passed back in later turns. Citations are close to free. There is no good reason to run a customer-facing agent without them.

Choosing where this logic lives is its own decision, and we covered the trade-offs in n8n versus Claude agents.

Per-resolution pricing counts a customer who gives up

If you buy rather than build, read the billing definition. Really read it.

Intercom prices Fin at $0.99 per outcome, charged once per conversation. An outcome is counted when a customer confirms their issue is resolved, or when they do not ask for more help after Fin responds, or when Fin completes a workflow including handoffs.

That middle clause is the one to sit with. A customer who reads an unhelpful answer, sighs, and closes the tab did not ask for more help. That is billed as a resolution.

I am not accusing Intercom of anything sharp. It is a reasonable proxy at scale, it is disclosed plainly on their own pricing page, and Fin is a genuinely strong product that will beat a rushed in-house build for most small teams. We are a competing vendor, so weigh that. But "resolution rate" and "customers we actually helped" are different metrics, and only one of them shows up on your invoice. Track abandonment separately or you will never see the difference.

The four categories that never get an autonomous reply

In every support build we ship, four categories are draft-only. The AI writes, a human sends.

• Money. Refunds, billing disputes, discounts, cancellations.
• Legal or contractual. Anything that could be read as a commitment.
• Safety and health adjacent. Where a wrong answer hurts someone.
• Already angry. Frustration is the worst possible moment to be handled by software.

That last one is judgement, not risk management. Customers do not hate AI. They hate being trapped in it, which is the same thing we found asking whether customers actually hate an AI receptionist. An escape hatch that works on the first ask is worth more than any accuracy gain.

What we build, and what we refuse to automate

The deliverable is never the model. It is the boring scaffolding: retrieval over your real docs, citations required on every claim, a tuned confidence floor, four draft-only categories, structured handoff that carries context so the human is not asking the customer to repeat themselves, and a weekly sample of replies read by an actual person.

That last one is the part clients push back on, and the part we hold firm on. Twenty replies a week, read properly. It is the only thing that catches the failure mode where the agent is confidently wrong in a way no metric flags, because the customer never complained. They just left.

If you want that built and run rather than assembled, that is what our automation work is. And if your honest ticket volume is thirty a month, buy Fin, set the threshold high, and spend your money somewhere it does more.