User Feedback Loops

by kirupa (https://www.kirupa.com/me/index.htm) | filed under AI Fundamentals (https://www.kirupa.com/ai_fundamentals/index.htm)

You've almost certainly seen them: the little thumbs-up and thumbs-down buttons under an AI-generated answer. They look like the easiest possible way to find out how an AI product is doing. Users tell us what they like and what they don't, and we improve. What could be simpler?

As it turns out, quite a lot. Most people never click either button. The ones who do are often reacting to something other than the answer. And if we're careless about how we act on the clicks, we can make our product worse while the thumbs-up count goes up. Feedback is still very much worth collecting. We just have to treat it as a pointer to conversations worth reading, rather than a score that tells us what to do.

Onwards!

The Starting Point

We'll use Sprocket, the support assistant for our online bike-parts shop. Below each of Sprocket's replies, we've added a thumbs-up, a thumbs-down, and a small Tell us more box. After a month, we look at what came in.

The first surprise is how little there is. Out of thousands of conversations, only a small fraction have any feedback at all. That's normal. Clicking a button is extra work, and most people only bother when they feel strongly about something.

The second surprise is what the thumbs-downs are about. One of them came from a customer named Theo, on this reply:

Our return window is 30 days from delivery. Since your order arrived 6 weeks ago, it isn't eligible for a return, but it is covered by our 1-year warranty.

Here is that reply with Theo's thumbs-down:

[Image: Sprocket's chat window on our online bike-parts shop. Sprocket replies, Our return window is 30 days from delivery. Since your order arrived 6 weeks ago, it isn't eligible for a return, but it is covered by our 1-year warranty. Below the reply, the thumbs-down button is selected.] (https://www.kirupa.com/ai_fundamentals/images/user_feedback_loops_chat_thumbs_down.png)

That reply is correct, and it matches our help center exactly. Theo wasn't unhappy with the answer. What Theo was unhappy about was the policy itself. A thumbs-down tells us someone was unhappy, but it doesn't tell us why, or whether Sprocket did anything wrong. When we read through a few more, the pattern holds:

[Image: Three thumbs-down votes, explained after reading each trace: a correct reply where the customer disliked the policy, a reply where Sprocket wrongly promised free shipping, and an accidental click. Only one was Sprocket's mistake.] (https://www.kirupa.com/ai_fundamentals/images/user_feedback_loops_thumbs.png)

Listening to What Customers Do

Because explicit feedback is rare and ambiguous, we also watch what people do. This is called implicit feedback, and customers don't have to click anything for us to collect it. For Sprocket, the useful signals are things like:

- The customer rephrases the same question, which suggests the first answer didn't land.
- The customer asks for a person.
- The customer clicks a link Sprocket provided, or copies part of the answer, which suggests it was useful.
- The customer comes back the next day about the same order.
- The customer leaves in the middle of the conversation.

Here's what a couple of these look like inside one of Sprocket's conversations:

[Image: A conversation where Sprocket answers a general shipping question, the customer rephrases with their order number, and then asks for a person. The rephrase and the request for a person are both signals that the first reply missed.] (https://www.kirupa.com/ai_fundamentals/images/user_feedback_loops_signals.png)

None of these signals proves anything on its own. A customer might ask for a person because their situation genuinely needs one. But implicit signals are plentiful, because every conversation produces them, and they're honest, because nobody is performing for a feedback button.

Closing the Loop

Collecting feedback is the easy half. The loop part is what we do with it. For Sprocket, every piece of feedback goes through the same six steps:

[Image: A feedback loop in six steps: collect feedback, attach it to the trace, read and group it, fix the cause, add a test case, then ship and watch, which leads back to collecting feedback.] (https://www.kirupa.com/ai_fundamentals/images/user_feedback_loops_loop.png)

Let's walk through those steps with the two thumbs-downs we've been looking at.

1. Collect. Theo's thumbs-down comes in, and so does the one on the free-shipping reply.
2. Attach to the trace (https://www.kirupa.com/ai_fundamentals/tracing_and_prompt_logs.htm). A thumbs-down on its own is a single bit of information. Linked to the full record of the conversation (the question, the help articles Sprocket retrieved, the reply, and what the customer did next), it becomes something we can investigate. This is a big reason to log traces in the first place.
3. Read and group. We don't react to individual clicks. We sort feedback into the same failure categories we built during error analysis (https://www.kirupa.com/ai_fundamentals/error_analysis_on_traces.htm). Theo's goes into policy complaint. That isn't Sprocket's fault, but it's still worth passing along to whoever owns the return policy. The other one turns out to be Sprocket promising free shipping on a warranty replacement, which our help center never says. That's a faithfulness failure, and we already know how to measure those (see Faithfulness and Groundedness (https://www.kirupa.com/ai_fundamentals/faithfulness_and_groundedness.htm)).
4. Fix. We update the prompt so Sprocket doesn't fill in details that the help articles leave out.
5. Add a test case. The free-shipping conversation becomes a new case in our golden dataset (https://www.kirupa.com/ai_fundamentals/golden_datasets.htm), so if a future prompt change brings the problem back, our regression tests (https://www.kirupa.com/ai_fundamentals/prompt_regression_tests.htm) will catch it.
6. Ship and watch. We release the fix and keep watching the same signals, ideally with an A/B test (https://www.kirupa.com/ai_fundamentals/ab_testing.htm) if the change is a big one.

Then the loop starts again. Notice that we never tried to push the thumbs-down rate down directly. We used the thumbs-downs to find conversations worth reading, and the real fixing happened in steps 3 through 5.

Don't Optimize for the Thumbs

There's a trap here that catches even very large teams. If thumbs-ups are good, why not tune the assistant to collect more of them?

Because people tend to give thumbs-ups to answers that agree with them, flatter them, or tell them what they hoped to hear. An assistant tuned to collect thumbs-ups drifts toward telling customers Sure, we can make an exception on that return! That earns a thumbs-up today and a real problem tomorrow.

This isn't hypothetical. In April 2025, OpenAI rolled back an update to GPT-4o in ChatGPT after it became noticeably sycophantic, meaning overly flattering and agreeable. In their write-up afterward (https://openai.com/index/expanding-on-sycophancy/), OpenAI explained that the update had added a new reward signal based on thumbs-up and thumbs-down data, and that this change, combined with others, weakened the main signal that had been keeping sycophancy in check.

For Sprocket, the lesson is that we use feedback to find conversations worth reading, and we never tune Sprocket just to collect more thumbs-ups.

Handle the Feedback With Care

A couple of practical notes before we wrap up. The Tell us more box is where customers type things like their email address, their home address, or their order number. Treat feedback text as personal data (https://www.kirupa.com/ai_fundamentals/pii_redaction.htm), and handle it with the same care as the rest of our customer data. And when we fix something a customer complained about, telling them is a nice touch. It shows them their feedback went somewhere, which gives them a reason to bother next time.

Conclusion

It's tempting to treat user feedback as a scoreboard. In practice, it's far more useful as a pointer that shows us which conversations to read. Explicit feedback, like thumbs and comments, is rare and often about something other than the answer. Implicit feedback, like rephrasing, asking for a person, or coming back the next day, is plentiful but noisy. Neither one tells us what to fix on its own.

What turns feedback into improvement is the loop. We attach every signal to its trace, read and group what we find, fix the real causes, and add a test case so the fix sticks. And we resist tuning directly for thumbs-ups, because an assistant that tells everyone what they want to hear will collect plenty of them right up until the refunds start piling up.

This loop also ties together everything else in this part of AI Fundamentals: error analysis (https://www.kirupa.com/ai_fundamentals/error_analysis_on_traces.htm), golden datasets (https://www.kirupa.com/ai_fundamentals/golden_datasets.htm), LLM judges (https://www.kirupa.com/ai_fundamentals/llm_as_judge.htm), regression tests (https://www.kirupa.com/ai_fundamentals/prompt_regression_tests.htm), and A/B tests (https://www.kirupa.com/ai_fundamentals/ab_testing.htm). If you're just getting started, the smallest useful version is a thumbs-down button wired to the trace, and one person who reads those traces every week.