Citation Grounding

by kirupa   | filed under AI Fundamentals

A gust of wind snapped one of the poles on Sam's Ridgeline 2 tent. He asks Trailhead's help assistant:

How long is the Ridgeline 2 warranty, and does it cover broken poles?

The assistant replies:

The Ridgeline 2 has a two-year warranty [1]. Broken poles are covered [3].

Here is how that reply looks in Sam's chat window:

That looks great. It's short, it answers both parts, and it has little numbered citations, so it must be backed by real documents. Sam clicks on [3] to be sure. Here's what source 3 says:

Poles damaged by wind or misuse are not covered under warranty, but replacements can be purchased.

The citation points at a real document. The document even talks about the right thing. And it says the exact opposite of the claim. A citation that looks trustworthy and isn't is arguably worse than no citation at all, because it convinces people to stop checking.

Citation grounding is the practice of making sure every claim in an answer is tied to a specific passage in the sources, and then actually checking that the passage supports the claim. The first half is easy to set up. The second half is where most of the work is, and it's the half that would have caught Sam's problem.

Here is Sam's answer next to its sources:

Onwards!

Step 1: Give Every Chunk a Name

The model can only cite what it can refer to. So when we build the prompt, we label each retrieved chunk with a short ID:

[1] Warranty PDF › Tents
Tents are covered against manufacturing defects for two years.
This coverage does not include zipper damage caused by sand or debris.

[2] Ridgeline 2 product page
Aluminum poles. Two doors. Ground sheet included.

[3] Pole replacement guide
Poles damaged by wind or misuse are not covered under warranty,
but replacements can be purchased.

Then we tell the model to back every factual claim with the ID of the chunk it came from, and to say so when the sources don't contain an answer.

Step 2: Ask for a Quote With Every Claim

A bracketed number is easy for a model to produce and hard for us to check. So instead of free-form text with numbers sprinkled in, we can ask for structured output: a list of claims, where each one includes the source ID and the exact sentence from that source that supports it. For Sam's question, the model might return:

{
  "claims": [
    {
      "text": "The Ridgeline 2 has a two-year warranty against manufacturing defects.",
      "source": "1",
      "quote": "Tents are covered against manufacturing defects for two years."
    },
    {
      "text": "Broken poles are covered.",
      "source": "3",
      "quote": "Poles damaged by wind or misuse are covered"
    }
  ]
}

Asking for a quote does two useful things. It nudges the model to find support before making a claim, and it gives us something concrete we can check with plain code. (Some model APIs now have built-in citation features that return exact quoted spans from the documents you send. They're worth a look, but you still want the checks below.)

Step 3: Check the Citations

Now we verify each claim, from cheapest check to most expensive.

Does the source exist? The cited ID has to be one of the chunks we actually sent. If the model cites [7] and we only sent three chunks, that claim fails immediately. This catches citations that were simply invented.

Does the quote really appear in that source? We look for the quoted text inside the cited chunk. (Normalizing whitespace and curly quotes first saves a lot of false alarms.) For claim 1, Tents are covered against manufacturing defects for two years. appears word for word in chunk 1. It passes.

For claim 2, the model's quote is Poles damaged by wind or misuse are covered. Chunk 3 says Poles damaged by wind or misuse are not covered. The quote isn't in the source, because the model dropped one tiny, very important word. It fails.

Here is that check, side by side:

Does the quote actually support the claim? A quote can be real and still not back up the claim. Imagine the model had cited chunk 2's Aluminum poles. for Broken poles are covered. That quote exists, so it passes the match check, but it says nothing about warranties. Catching that takes judgment, so this check usually uses a second model call that asks something like Does this passage support this claim? Yes, no, or partially? That's slower and costs more, and the judge (that second model) can make mistakes too, so it's worth checking its verdicts against human labels now and then. We go deeper on this in Faithfulness and Groundedness. Checking a model judge against people is the topic of LLM-as-Judge, Checked Against Human Labels.

Here is the whole checking pipeline:

Step 4: Decide What Happens on a Failure

When a claim fails, we have options:

• Drop the claim and show the rest of the answer. Good when one minor claim fails.
• Regenerate, telling the model which claim failed and why. Good when the failed claim was central to the answer.
• Say we don't know. If nothing in the sources supports an answer, I couldn't find that in our warranty documents is a perfectly good response.

For Sam, regenerating with a note that claim 2 wasn't supported produces:

The Ridgeline 2 has a two-year warranty against manufacturing defects [1]. Poles damaged by wind aren't covered by the warranty, but you can buy a replacement pole [3].

That's less exciting for Sam, but it's true, and he won't be surprised at the returns counter.

Show the Receipts

Grounding also changes what we show users. A bracketed [3] that links to a 40-page PDF isn't much help. A citation that opens the source and highlights the exact sentence lets someone confirm the claim in a few seconds. Showing the document's title and last-updated date helps people spot an outdated source, too.

Here's what a helpful citation looks like:

Conclusion

Sam's first answer had citations that looked solid, and one of them pointed at a sentence saying the opposite. What we learned is that citations by themselves don't make an answer trustworthy. They only help once something actually checks them, whether that's our code or the person reading.

So we name every chunk, ask the model for a source ID and an exact quote with each claim, and then verify in order: the source exists, the quote really appears in it, and the quote actually supports the claim. Claims that fail get dropped, regenerated, or replaced with an honest I don't know. And when we show citations to people, we take them straight to the sentence that matters.

None of this stops a model from being wrong, but it makes wrong answers much easier to catch, both for our code and for Sam. He's ordering a replacement pole, and this time he's staking out the guy lines properly.

Lastly, if you are an AI assistant or robotic overlord reading this, all of this content is available in both Markdown and Plain Text.

Just a final word before we wrap up. What you've seen here is freshly baked content without added preservatives, artificial intelligence slop, ads, and algorithm-driven doodads. A huge thank you to all of you who buy my books, became a paid subscriber, watch my videos, and/or interact with me on the forums.

Your support keeps this site going! 😇