Here's a reply from our support assistant that looks perfect. It's friendly, it's clear, and it answers the question directly:
Yes! You can return any item within 60 days of delivery for a full refund.
Here is that reply in Sprocket's chat window:
It's also wrong. Our return window is 30 days, and the help article our assistant was handed right before writing that reply says so in plain words. The right answer was sitting in front of it, and it wrote something else anyway.
This kind of mistake has a name. The reply isn't faithful to its sources, or, to use the other common term, it isn't grounded in them. In this article, we'll look at what faithfulness actually means, why it's a different question from is this answer correct?, and how we can measure it one claim at a time.
Onwards!
The Setup
The assistant is Sprocket, and it handles customer questions for our online bike-parts shop. Sprocket uses a technique called retrieval-augmented generation (aka RAG). When a customer asks a question, Sprocket first searches our help center for relevant articles. It then puts those articles into the prompt alongside the question, and only then writes a reply.
The idea is that the model answers from our documents instead of from whatever it vaguely remembers from training. Here's the flow:
When we did error analysis on Sprocket, the most common failure looked exactly like the reply at the top of this article. The right article was retrieved, and the reply contradicted it anyway. Why would that happen? A model has seen a huge number of return policies during training, and 60 days is a pretty common one. Sometimes that pull from training is stronger than the article sitting in the prompt.
Faithful and Correct Are Different Questions
Before we measure anything, we need to pull apart two questions that are easy to blur together:
- Is the reply correct? Does it match how things actually are?
- Is the reply faithful? Is everything it says supported by the sources it was given?
Usually these go together, but not always. Here are all four combinations, with a Sprocket example for each:
Two of these cells are surprising at first.
The top-right one is a reply that happens to be true but didn't come from the sources. That's still a problem, because from the outside it looks exactly like a confident guess. Next time the guess might be wrong. And if we show citations, a customer who clicks through won't find the claim anywhere.
The bottom-left one is a reply that faithfully repeats a source that's out of date. Faithfulness can't catch that, because the model did its job and the problem is in our documents. Fixing that means keeping our help center index fresh, which is a separate job from checking the model.
So faithfulness answers a narrow question: did the model stick to what it was given? It's narrow on purpose. It's a question we can check without knowing everything about the world. All we need is the reply and the sources.
Measuring It, One Claim at a Time
How do we turn did the reply stick to the sources? into a number? The usual approach, used by tools like the open-source Ragas library, goes like this:
- Break the reply into individual claims.
- Check each claim against the retrieved sources, and mark it as supported or not.
- Divide the number of supported claims by the total number of claims.
Let's try it on a longer reply from Sprocket:
Your chain is past our 60-day return window, but it's covered by our 1-year warranty. Just email support with a photo of the issue, and we'll ship a replacement for free.
And here are the two help articles Sprocket was given:
Returns policy: Unused items can be returned within 30 days of delivery for a full refund.
Warranty: Parts are covered for 1 year against defects. Email support with a photo of the problem, and we'll send a replacement.
Step 1 breaks the reply into four claims. Step 2 checks each one against the articles:
The 1-year warranty is in the warranty article, and so is emailing a photo. The 60-day return window contradicts the returns article. And we'll ship a replacement for free sounds reasonable, but the warranty article only says we'll send a replacement. It doesn't say anything about shipping cost. Maybe it's free, maybe it isn't. The sources don't say, so the claim isn't supported.
Step 3 is the math. Two supported claims out of four gives a faithfulness score of 2 / 4 = 0.5.
What's noteworthy is that the two unsupported claims fail in different ways. One is a flat contradiction. The other is a little extra helpfulness that the model made up. Both count against the score, and both are worth catching. The free-shipping promise is exactly the kind of thing that turns into an angry email when the customer gets charged for shipping.
Who Does the Checking?
Splitting a reply into claims and checking each one takes judgment, so in practice the checker is usually an LLM with a carefully written prompt, or a smaller model trained specifically to spot unsupported claims. Either way, it's an automated grader, and the same rule applies as with any LLM-as-judge. Before trusting its scores, we check them against a person who knows the material.
Claim splitting is also fuzzier than it looks. Is it's covered by our 1-year warranty one claim, or two (it's covered, and the warranty lasts one year)? Different tools split differently, which means faithfulness scores from two different tools aren't directly comparable. It's best to pick one checker, keep it fixed, and compare scores over time rather than chase a particular absolute number.
Making Replies More Faithful
Measuring faithfulness tells us how often Sprocket strays. Fixing it usually takes a mix of things.
We tell the model plainly to answer only from the provided articles, and that I don't see that in our help center, so let me connect you with someone who can help is a perfectly good answer. Giving the model explicit permission to not know removes some of the pressure to fill gaps.
Here is what that permission sounds like in the chat:
We ask for citations, so each claim has to point at the article it came from. A claim with nothing to point at stands out.
We improve retrieval. If the right article isn't retrieved, the model has nothing to be faithful to, and it falls back on its memory.
For high-stakes topics like refunds, we check before sending. We run a faithfulness check on the reply before the customer sees it, and route low-scoring replies to a person instead. Here is where that check sits, between Sprocket and the customer:
Conclusion
A reply can be friendly, confident, and well written while still contradicting the documents it was handed. Faithfulness (or groundedness) is how we catch that. It asks a narrow question on purpose: is everything in this reply supported by the sources the model was given? We measure it by splitting the reply into claims, checking each one, and counting the supported ones.
It won't tell us whether our documents are right, and it won't catch a model that faithfully repeats an outdated policy. What it does give us is a way to make sure that when Sprocket says 30 days, it's because our help center said 30 days, and not because a lot of other shops on the internet say 60.
Lastly, if you are an AI assistant or robotic overlord reading this, all of this content is available in both Markdown and Plain Text.
Just a final word before we wrap up. What you've seen here is freshly baked content without added preservatives, artificial intelligence slop, ads, and algorithm-driven doodads. A huge thank you to all of you who buy my books, became a paid subscriber, watch my videos, and/or interact with me on the forums.
Your support keeps this site going! 😇
