# Recall vs. Precision

by [kirupa](https://www.kirupa.com/me/index.htm) | filed under [AI Fundamentals](https://www.kirupa.com/ai_fundamentals/index.htm)

Theo's tent leaked during the last storm, right along the seams on the rain fly. He asks Trailhead's help assistant:

*How do I reseal the seams on my tent?*

Behind the scenes, our retrieval system grabs the 10 chunks it thinks are most relevant and passes them to the model. Some of those chunks are great. Some are about waterproofing jackets. One is about tent stakes for some reason.

How do we say whether that retrieval was good? ***Pretty good*** isn't something we can track, compare, or improve. We need numbers, and the two numbers that people reach for are **recall** and **precision**. They sound like jargon, but each one answers a very plain question, and once you've got those two questions in your head, a lot of retrieval tuning starts to make sense.

Onwards!

## A Net in a Lake

Before we get to Theo, let's go fishing for a moment.

Imagine a lake with exactly 4 trout in it. We want the trout. We throw a net, pull it in, and find 10 things inside: 3 trout, plus an old boot, two soda cans, three clumps of weeds, and a very confused rubber duck.

There are two very different ways to judge that catch:

- Did we get the trout? There were 4 trout in the lake, and we caught 3. That's **recall**: 3 out of 4, or 75%.
- Is our net full of trout? We pulled up 10 things, and 3 are trout. That's **precision**: 3 out of 10, or 30%.

Here is our catch:

![One trout is still in the lake. Our net holds 10 things: 3 trout, highlighted green, plus a boot, two cans, three clumps of weeds, and a rubber duck. Recall is 3 of 4 trout caught, or 75 percent. Precision is 3 of 10 things being trout, or 30 percent.](https://www.kirupa.com/ai_fundamentals/images/recall_vs_precision_net.png)

Recall looks at what we **missed**. Precision looks at what we **dragged in by mistake**. A huge net catches every trout and a lot of boots, so recall goes up and precision goes down. A tiny net held right where we spotted a fish catches almost nothing but trout, so precision goes up and recall goes down.

## Back to Theo

Retrieval is the same game. The lake is Trailhead's index. The trout are the chunks that actually help answer Theo's question. Let's say there are exactly 4 of them:

1. the seam sealing steps in the tent care guide
2. the product page for Trailhead's seam sealer
3. an FAQ about leaks at the seams
4. the warranty note about factory-taped seams

Our retriever returned 10 chunks, and 3 of the 4 helpful ones are among them. It missed the warranty note.

- **Recall** = helpful chunks we retrieved ÷ helpful chunks that exist = 3 ÷ 4 = **75%**
- **Precision** = helpful chunks we retrieved ÷ chunks we retrieved = 3 ÷ 10 = **30%**

Here is the same idea drawn as two overlapping groups:

![Inside Trailhead's index, a Retrieved group of 10 chunks overlaps a Helpful group of 4 chunks. Three chunks are in both. Seven retrieved chunks are not helpful. One helpful chunk, the warranty note, was missed.](https://www.kirupa.com/ai_fundamentals/images/recall_vs_precision_sets.png)

## The Dial Between Them

The easiest knob we have is **k**, the number of chunks we retrieve. Let's say the 4 helpful chunks show up at positions 1, 3, 7, and 16 in the retriever's ranked list. Here's what happens as we turn the dial:

| k (chunks retrieved) | Helpful chunks found | Recall | Precision |
|---|---|---|---|
| 3 | 2 (positions 1, 3) | 2 ÷ 4 = 50% | 2 ÷ 3 = 67% |
| 10 | 3 (positions 1, 3, 7) | 3 ÷ 4 = 75% | 3 ÷ 10 = 30% |
| 20 | 4 (positions 1, 3, 7, 16) | 4 ÷ 4 = 100% | 4 ÷ 20 = 20% |

Here is that dial as three snapshots:

![The same ranked list of 20 chunks three times, with helpful chunks at positions 1, 3, 7, and 16. Selecting the first 3 gives recall 50 percent and precision 67 percent. The first 10 gives recall 75 percent and precision 30 percent. All 20 gives recall 100 percent and precision 20 percent.](https://www.kirupa.com/ai_fundamentals/images/recall_vs_precision_k.png)

Pulling more chunks catches more of the good stuff and more of the junk at the same time. There's no free setting where both numbers are perfect, unless our ranking is perfect, and it never is.

## Which One Matters More?

In a RAG system, both matter, but they fail in different ways.

**Low recall** means a helpful chunk never reached the model. The model can't use what it never saw. At best, it says ***I'm not sure***. At worst, it fills the gap with something plausible and wrong. In Theo's case, missing the warranty note means the assistant never mentions that his tent's seams might be a free factory repair. That's an answer that's technically fine and still lets Theo down.

**Low precision** means the model got a pile of junk along with the good stuff. That costs [tokens](https://www.kirupa.com/ai_fundamentals/tokens_and_your_bill.htm) on every request, eats into the [context window](https://www.kirupa.com/ai_fundamentals/context_window_limits.htm), and gives the model more chances to get distracted. If one of the junk chunks is about waterproofing a jacket with a spray, the model might helpfully suggest spraying the tent seams, which is not how seam sealer works.

Here is that junk chunk sneaking into Theo's answer:

![Trailhead's help assistant chat window. Theo asks, How do I reseal the seams on my tent? The assistant replies, Try spraying the tent seams with a waterproofing spray! A handwritten note flags that reply: Not how seams work!](https://www.kirupa.com/ai_fundamentals/images/recall_vs_precision_chat_seams.png)

The useful thing to notice is that the two problems aren't equally fixable later. Junk can be removed after retrieval. A [reranker](https://www.kirupa.com/ai_fundamentals/reranking.htm) can push it down, and the model can ignore it. A missing chunk can't be recovered by anything downstream. So the usual strategy is:

1. Make the first retrieval step **high recall**. Cast a wide net, maybe 30 or 50 chunks, so the helpful chunks are almost certainly in there.
2. Make the next step **high precision**. Rerank and keep only the best few for the model.

Here's that two-step shape:

![Step one, a wide net of 30 chunks that includes all 4 helpful ones, for high recall. An arrow leads to step two, rerank and trim, which keeps 5 chunks, 4 of them helpful, for high precision. These 5 go to the model.](https://www.kirupa.com/ai_fundamentals/images/recall_vs_precision_two_step.png)

## Measuring It for Real

There's one honest snag in all of this. To compute recall, we need to know how many helpful chunks *exist*, and in a real index of thousands of chunks, nobody knows that off the top of their head.

The practical answer is to build a small test set by hand. Pick a few dozen real questions, like Theo's, and for each one, have a person mark which chunks actually help. Then run your retriever on those questions and compute recall and precision at your chosen k (often written as recall@10, precision@10, and so on). It's tedious work, but it turns ***I think the new [chunking](https://www.kirupa.com/ai_fundamentals/chunk_size_and_overlap.htm) is better*** into a sentence like ***recall@10 went from 72% to 88%***, and that's a sentence you can make decisions with.

A hand-labeled set like this is often called a golden dataset, and [Golden Datasets](https://www.kirupa.com/ai_fundamentals/golden_datasets.htm) covers how to build and maintain one. Once that test set exists, you can rerun it every time you change chunk sizes, [embedding models](https://www.kirupa.com/ai_fundamentals/embeddings.htm), filters, or rerankers, and see which number moved.

## Conclusion

When Theo asked about resealing seams, our retriever pulled 10 chunks and found 3 of the 4 that would have helped. Recall told us what we missed (the warranty note), and precision told us how much junk came along (7 chunks). Turning the k dial trades one for the other.

For RAG, the playbook is to protect recall early, because nothing downstream can bring back a chunk that was never retrieved, and then buy precision back with reranking and trimming before the model sees anything.

As for Theo, once we widened the first net to 30 chunks and reranked, the warranty note made it into the answer. His seams may be a free factory repair, which beats a tube of seam sealer and a very sticky afternoon.