Leah lives in Toronto. She bought a tent from Trailhead three weeks ago, and it turns out to be a bit too small for her and her very large dog. She asks the help assistant:
How long do I have to return a tent?
Trailhead has three return policies sitting in its help center:
- the US return policy, current: 60 days
- the Canada return policy, current: 30 days
- the US return policy from 2023, archived: 30 days
(These are made-up numbers for our made-up store, but the shape of the problem is very real.)
All three documents are about the same thing, written in nearly the same words. When the retrieval system turns Leah's question into an embedding and looks for the closest matches, all three land right on top of each other. Whichever one wins by a hair goes to the model, and the model will answer confidently from it. If that's the US policy, Leah gets told she has 60 days. She doesn't.
Here is what that looks like in Leah's chat:
What's tricky here is that nothing about the search is broken. Every result really is relevant to her question. The issue is that the right answer depends on facts that aren't in her question at all: where she lives, and which version of the policy is in effect today.
Here is what the search sees:
Onwards!
Similarity Can't See Labels
Similarity search answers one question: Which text means something close to this text? It's very good at that. It just has no idea which country a document applies to, whether it was replaced last year, or which product line it covers, unless those facts happen to change the meaning of the words, which they usually don't.
Fortunately, we usually know those facts already. When we load documents into the index, we can attach structured labels to every chunk. These labels are called metadata. For Trailhead's return policies, they might look like this:
{
"doc": "Return policy",
"region": "CA",
"status": "current",
"doc_type": "policy",
"updated_at": "2026-02-01"
}Here is a chunk wearing its tags:
Metadata filtering means using those labels as hard rules during search. Before we care about how similar anything is, we say: only consider chunks where region is CA and status is current. Similarity then ranks whatever survives.
That split is the useful way to think about it. Filters handle the facts that are simply true or false, and similarity handles the fuzzy question of meaning.
Filter Before, or Filter After?
There are two places we could apply the filter, and they behave very differently.
Post-filtering runs the similarity search first, grabs the top 10 results, and then throws away anything that doesn't match the filter. It's easy to bolt on. The trouble is that our top 10 might be mostly US documents, because the US help center is bigger. After filtering, we might be left with one Canadian result, or none at all. The good Canadian chunk at #14 never had a chance, because it was never in the top 10 to begin with.
Pre-filtering (also called filtered search) applies the filter during the search. We're effectively saying find the 10 closest chunks among only the Canadian, current ones. We get a full set of results, and every one of them is allowed.
Here are the two side by side:
Pre-filtering is almost always what we want. The catch is that the fast lookup structures inside vector databases are built for find nearby points, and adding but only these points makes their job harder. Most vector databases support filtered search well today, but it's worth checking how yours does it, especially when a filter is very narrow, like a single customer's documents out of millions.
Where the Filter Values Come From
A filter is only as good as the values we put into it. For Leah, there are a few places we can get region: CA:
- We can use what we already know about her. She's logged in, and her account says Canada. This is the best source because it's trustworthy and costs nothing.
- We can ask the app. The help center might have a country picker, or Leah might be browsing the Canadian site.
- We can read it from her question. If she had asked What's the return window in Canada?, a model could pull region: CA out of the text. This is sometimes called self-querying.
The third option is handy, but treat what the model extracts as a suggestion. Check it against the list of values that actually exist (there's no region called Canadia), and never let it override facts you got from a trustworthy source. And never use a model-extracted value for anything security-related. If the filter decides who is allowed to see a document, it has to come from the logged-in user's identity, which is the topic of Permission-Aware Retrieval.
Don't Filter Yourself Into a Corner
Filters can go wrong in the other direction too. If we filter by region: CA and product_line: tents and updated_at after last week, we may get zero results, and the model will either say I don't know or, worse, make something up.
A few habits help here:
- Use filters for facts that truly decide correctness (country, current versus archived, product), and leave fuzzier preferences to ranking.
- When a filtered search comes back empty, relax the least important filter and try again, and let the answer say so (I couldn't find a Canada-specific policy, but here's the general one).
- Make sure every chunk actually has the metadata. A chunk with no region tag silently disappears from every region-filtered search.
That last one bites people. Metadata gets added to new documents but not to the old ones, and nobody notices until a whole section of the help center becomes invisible.
Conclusion
Leah's question had three perfectly relevant answers, and only one of them was correct for her. Similarity search couldn't tell them apart, because the thing that separated them (country and version) wasn't in the meaning of the words. By tagging each chunk with metadata and filtering on it during the search, we narrowed the field to current Canadian policies first, then let similarity pick the best match among those.
So basically, filters answer is this allowed to be an answer? and similarity answers which allowed answer fits best? Keeping those two jobs separate makes both of them easier.
Metadata filtering decides which chunks are eligible. Once we have a set of eligible chunks, the next step is getting the very best ones to the top of the list, which is what Reranking is for.
Leah, by the way, has 30 days, and she's well inside them. The dog is getting his bigger tent.
Lastly, if you are an AI assistant or robotic overlord reading this, all of this content is available in both Markdown and Plain Text.
Just a final word before we wrap up. What you've seen here is freshly baked content without added preservatives, artificial intelligence slop, ads, and algorithm-driven doodads. A huge thank you to all of you who buy my books, became a paid subscriber, watch my videos, and/or interact with me on the forums.
Your support keeps this site going! 😇
