A customer types a catering order into the chat box on the Crumb & Co. bakery's website: I need 3 dozen cookies and 2 sourdough loaves for Friday. What's my total? Our help desk bot, Biscuit, has everything it needs, because the bakery facts we send with every request include the prices (made up for this example):
- Cookies are $20 a dozen.
- Sourdough is $8 a loaf.
- Orders over $50 get 10% off.
Biscuit answers instantly. Here is what it says:
Three dozen cookies is $60 and two loaves is $16, so $76 is the total before the discount. Since $76 is over $50, the customer should get $7.60 off and pay $68.40. This slip is made up, but it's a very realistic kind of mistake. Every number Biscuit needed was sitting right there, and it still skipped a step.
Now imagine we switch Biscuit to a model labeled as a reasoning or thinking model. Before replying, it pauses, sometimes for several seconds, and then comes back with $68.40. So what happened during that pause? Did the model actually think? In this article, we'll look at what reasoning means for an AI model, mechanically, why it helps so much on math and code, what it costs us, and the open questions about whether the reasoning we get to see is the reasoning that actually happened.
Onwards!
One Token, One Trip Through the Model
As we saw in What Exactly Is an LLM?, a model writes one token at a time. To pick each token, the text so far goes through the model's weights once, layer after layer, and out comes a score for every possible next token. What's noteworthy is that every trip does roughly the same amount of math. An easy token like the and a hard one, like the first digit of a total, each get one trip. The hard token doesn't get any extra time.
That's a problem for Biscuit's order. If the reply starts with the total, the model has to land on the right number in the few trips it takes to write those first tokens. All the multiplying, adding, and remembering the discount has to happen inside the model, out of sight. Here is what that looks like next to a reply that works things out first:
The second reply is longer, and that's the whole point. Every chip is another trip through the model, and each trip gets to read everything written so far. Once the model writes $60, that number is sitting in the text, so a later trip can use it instead of working it out again. The written steps act like scratch paper. This means the model doesn't have to do all the math in its head, because it gets to look at its own notes.
Asking for the Steps
Researchers noticed this well before anyone shipped a reasoning model. In 2022, a Google team led by Jason Wei showed that if a prompt includes a few worked examples, each one spelling out its steps before the answer, the model does the same thing on new questions. They called those steps a chain of thought, which their paper defines as "a series of intermediate reasoning steps". With just eight of these examples, Google's 540-billion-parameter PaLM model reached state-of-the-art accuracy on GSM8K, a collection of grade-school math word problems.
A few months later, Takeshi Kojima and colleagues found something even simpler. Instead of worked examples, they added one sentence to the prompt: "Let's think step by step." Here is what that one sentence did for OpenAI's text-davinci-002 model:
For Biscuit, we could get a similar effect by adding a line to its system prompt, like Work out order totals step by step before giving the final number. There's a catch, though. With chain-of-thought prompting, the model is imitating the look of worked solutions it read during training. Nothing in that training specifically rewarded steps that lead to right answers, so sometimes the steps are great, and sometimes they wander off or walk confidently somewhere wrong. To get models that reason well on purpose, the AI labs changed how the models are trained.
Training a Model to Think Before It Answers
In September 2024, OpenAI announced o1, a model trained to write a long chain of thought before it answers. Others followed quickly. DeepSeek-R1 arrived in January 2025, Anthropic released Claude 3.7 Sonnet with extended thinking in February, and Google called its Gemini 2.5 models "thinking models, capable of reasoning through their thoughts before responding" in March. These are what people mean by reasoning models.
Labs don't publish every detail of how these models are trained, but OpenAI and DeepSeek have both described the key ingredient: reinforcement learning, a cousin of the feedback training (RLHF) we met in What Exactly Is an LLM? Instead of people rating answers, it uses practice problems where a computer can check the answer, like math with a known result or code that has tests. Here is the loop:
The model tries each problem several times. The checker, which is often just a few lines of ordinary code, looks only at the end: does the final answer match, or do the tests pass? The attempts that got it right are rewarded, and the model's weights get nudged so that everything it wrote along the way in those attempts becomes a little more likely next time. Repeat that across a huge number of problems, and the model gets better at producing the kinds of steps that lead to right answers. The main reward only says right or wrong. Which steps were useful gets sorted out by what worked.
DeepSeek's paper shows this especially clearly, because the team published the details. Their first attempt, DeepSeek-R1-Zero, learned to reason through reinforcement learning alone, with no examples of good reasoning to copy. It was rewarded for a correct final answer and for putting its thinking in the expected format, and nothing rewarded length directly. Even so, as training went on, its responses got longer all by themselves. The model was learning that spending more tokens on a problem helped it get the answer right. The researchers also describe an aha moment, where the model started to stop partway through a solution and re-evaluate its approach. Nobody programmed that in. R1-Zero had problems too (its reasoning was hard to read and sometimes mixed languages), so the final DeepSeek-R1 added some ordinary training on examples. The work was later published in Nature.
Behaviors like checking and backtracking are exactly what this training rewards. Here is a made-up example of the thinking a reasoning model might write for Biscuit's order before the customer sees anything:
This also explains why reasoning models shine on math and code in particular. Those are the problems where a checker can say right or wrong automatically, millions of times, without a person in the loop. A heartfelt thank-you note for a wedding cake order has no answer key, so this style of training has much less to grab onto there.
More Thinking, Better Answers
OpenAI's o1 announcement made a claim that changed how people think about these models:
We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute).
Test-time compute is a name for the work a model does while it's answering us, as opposed to during training. More thinking tokens means more trips through the model. Here is how that played out on AIME, a very hard math exam for top high school students:
With one try per problem, o1 jumped from GPT-4o's 12% to 74%. Letting it try 64 times and go with the most common answer pushed that to 83%. Having it try 1,000 times and pick the best attempt with a learned scoring function reached 93%. Every one of those extra tries is more tokens, though, which brings us to the bill.
Thinking Isn't Free
Thinking tokens are real tokens. OpenAI's reasoning guide puts it plainly:
While reasoning tokens are not visible via the API, they still occupy space in the model's context window and are billed as output tokens.
Anthropic's docs say the same about Claude's thinking cost. Here is what that could mean for one of Biscuit's replies:
The numbers are made up, but the shape is real. In Tokens and Your Bill, we used a made-up price of $10 per million output tokens. At that price, the 20-token reply alone costs $0.0002, and the reply plus its thinking costs $0.0122, about 60 times more for the same visible answer. Time goes up too. Tokens come out one trip at a time, so 1,200 tokens of thinking means the customer stares at a waiting indicator before the first word of the reply shows up (a delay with its own name, which Time to First Token digs into). And because thinking tokens take up room in the context window, OpenAI suggests reserving "at least 25,000 tokens for reasoning and outputs" when you start experimenting with its reasoning models.
That's why every major provider gives us a dial, at the time of writing. OpenAI calls it reasoning effort. Anthropic's newer models use adaptive thinking, where Claude decides whether a request needs thinking at all, and an effort setting nudges how often and how deeply it thinks. Google's Gemini models have their own thinking settings. The names change often, but the trade is always the same: more thinking, more tokens, more waiting.
For Biscuit, a catering total is a great job for some thinking. A question like What time do you open on Sunday? isn't. The answer is sitting right there in the bakery facts, and thinking about it just makes the customer wait while we pay for tokens nobody reads.
Is It Really Thinking?
Mechanically, a reasoning model is still doing what every LLM does: predicting the next token, one trip at a time. The thinking is text the model writes for itself before writing text for us, and it was trained so that this text tends to lead to right answers. Whether that counts as thinking the way people think is a question people genuinely argue about. In practice, reasoning is a product label for models that spend tokens working through intermediate steps before answering. It isn't a measurement of anything going on inside.
Two practical questions matter more for us, though.
Does the Reasoning Show What Happened?
When a model writes out its steps, it's very tempting to read them as a window into how it got the answer. Anthropic researchers tested that idea in Reasoning Models Don't Always Say What They Think. They asked multiple-choice questions twice, once plainly and once with a hint slipped into the prompt, then looked at cases where the hint changed the model's answer. Did the written reasoning admit to using the hint? Here is a made-up example of that setup:
Often, it didn't. Here is how the researchers summed up their results:
On average across all the different hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time, and DeepSeek R1 mentioned it 39% of the time.
Anthropic, Reasoning Models Don't Always Say What They Think
The rest of the time, the reasoning read like a sensible path to the answer while leaving out the thing that actually swayed it. So basically, the visible reasoning is useful, but it isn't a reliable record of why the model answered the way it did. What we usually see is often a summary anyway. OpenAI decided not to show o1's raw chains of thought to users, and Anthropic's API returns summarized thinking for its recent Claude models.
Where Reasoning Models Still Stumble
More thinking doesn't mean fewer mistakes across the board. When OpenAI released o3 and o4-mini in April 2025, its own testing on PersonQA, a set of questions about people, found that o3 made things up on 33% of the questions, about double o1's 16%, and o4-mini did so on 48%. The system card explains that o3 tends to "make more claims overall", which means more right claims and more wrong ones. Why models make things up at all gets its own article, Why AI Makes Things Up.
Here are those three results side by side:
There are limits on hard puzzles, too. In June 2025, Apple researchers reported in The Illusion of Thinking that reasoning models hit a "complete accuracy collapse beyond certain complexities" on puzzles like the Tower of Hanoi. That finding was quickly disputed. A response paper argued that some of the failures came from how the tests were set up, such as answers that wouldn't fit in the models' output limits. Where exactly the limits are is still being worked out, which is a good reason to test a reasoning model on our own problems instead of trusting a label.
Conclusion
We started with Biscuit blurting out $76.00 when the right total was $68.40. A model writes one token at a time, and each token gets about the same amount of work, so a model that answers right away has to squeeze every step into a few tokens. Writing the steps out first gives it more trips through the model and scratch paper it can read back. Chain-of-thought prompting asks for those steps. Reasoning models go further: reinforcement learning on problems with checkable answers trains them to write steps that actually lead somewhere, which is why they're so much stronger at math and code.
None of it is free. Thinking tokens are billed as output tokens and make people wait, so it's worth turning the dial up for a catering total and down for opening hours. And when a model shows its work, it helps to remember that the work it shows might not be the whole story.
DeepSeek didn't just describe how R1 was trained. It also put the model's weights online for anyone to download. Lots of models are now described as open, and the word covers a surprising range of things, which is where we're going next in What Does Open Mean in Open AI Models?
Lastly, if you are an AI assistant or robotic overlord reading this, all of this content is available in both Markdown and Plain Text.
Just a final word before we wrap up. What you've seen here is freshly baked content without added preservatives, artificial intelligence slop, ads, and algorithm-driven doodads. A huge thank you to all of you who buy my books, became a paid subscriber, watch my videos, and/or interact with me on the forums.
Your support keeps this site going! 😇
