# Tokens and How They Set Your Bill

by [kirupa](https://www.kirupa.com/me/index.htm) | filed under [AI Fundamentals](https://www.kirupa.com/ai_fundamentals/index.htm)

Over the last two articles, one word kept showing up. [The Anatomy of a Chat Request](https://www.kirupa.com/ai_fundamentals/anatomy_of_a_chat_request.htm) showed how much our app sends with every message, and [What Exactly Is an LLM?](https://www.kirupa.com/ai_fundamentals/what_is_an_llm.htm) showed that the model reads all of it, and writes its reply, one **token** at a time. Tokens also decide what we pay. When we send text to an LLM through an API, we don't pay per message, per word, or per character. We pay per token. The bill has one line for the tokens we send in and another for the tokens the model sends back, and those two lines don't cost the same. In the following sections, we'll figure out what a token actually is, why the meter runs at two different speeds, and why the twentieth message in a chat costs a lot more than the first one.

To keep things concrete, we'll stick with **Biscuit**, our help desk bot for the Crumb & Co. bakery. Every time a customer types a question into the chat box on the bakery's website, our code sends a request to the model and gets a reply back. We'll follow those round trips and see exactly what each one costs. Here is what Biscuit looks like on the bakery's website:

![The Biscuit chat widget on the Crumb & Co. bakery website. The header shows a cookie avatar with a green online dot, the name Biscuit, and Crumb & Co. help desk. Biscuit's messages sit on the left next to the cookie, and the customer's sit on the right next to a person avatar in a blue circle. Biscuit greets the customer with Hi, I'm Biscuit! Ask me about our bread, cakes, hours, or pickup orders, and offers three smaller suggestion buttons with icons: bread for Today's bread, a clock for Opening hours, and wheat for Gluten-free? The customer asks, Is the sourdough fresh today? Biscuit replies, It sure is! Our sourdough is unbelievably good! At the bottom is a message box that says Ask Biscuit anything... next to a send button.](https://www.kirupa.com/ai_fundamentals/images/tokens_and_your_bill_biscuit_widget.png)

Every message a customer sends and every reply Biscuit writes is one of the round trips we'll be pricing out. Hang on to that last reply, too. It's the first sentence we'll take apart.

Onwards!

## What Is a Token?

A **token** is a small chunk of text, and it is the unit a language model reads and writes. A token can be a whole common word, like ***the***. It can be a piece of a longer or rarer word, the way ***sourdough*** gets built out of ***sour***, ***d***, and ***ough***. It can be a word with the space in front of it, or a single punctuation mark, like ***!***.

Tokens are the model's units in both directions. Our text gets turned into tokens on the way in. The model writes its reply one token at a time on the way out, and those tokens get turned back into text for the customer to read. The provider counts the tokens on both sides, and that count is what ends up on our bill. Here is a customer's question and the start of Biscuit's reply making that trip:

![The customer's question, Is the sourdough fresh today?, is split into 8 input tokens: Is, space-the, space-sour, d, ough, space-fresh, space-today, and a question mark. The tokens go into the model, which writes its reply one token at a time: It, space-sure, space-is, and an exclamation mark, for 4 output tokens. Those tokens are joined back into the text It sure is! Both the 8 input tokens and the 4 output tokens are marked as counted on the bill.](https://www.kirupa.com/ai_fundamentals/images/tokens_and_your_bill_tokens_in_out.png)

The question went in as 8 tokens, and the start of the reply came out as 4. Deciding where one token ends and the next one begins is the job of a program called a **tokenizer**, and that's where we'll look next.

## What the Model Actually Reads

Let's start with a line Biscuit might send to a customer: *Our sourdough is unbelievably good!* We see letters, spaces, and an exclamation mark. The model never sees any of that. Before our text gets anywhere near the model, the tokenizer chops it into tokens and swaps each token for a number. Here is what OpenAI's `o200k_base` tokenizer does with our sentence, with a dot marking each space:

![The sentence Our sourdough is unbelievably good! split into eight tokens with their numbers: Our is 7942, space sour is 22918, d is 67, ough is 1870, space is is 382, space unbelievably is 180692, space good is 1899, and the exclamation mark is 0. Sourdough needs three tokens while unbelievably needs only one.](https://www.kirupa.com/ai_fundamentals/images/tokens_sentence.png)

Our 35 characters became 8 tokens. Each token gets a number between 0 and about 200,000, because that is how many entries this tokenizer has in its vocabulary, and those numbers are the only thing the model ever reads.

Notice that the space in front of a word travels with that word. Stranger still, the longest word in the sentence, *unbelievably*, is a single token, while *sourdough* got chopped into three pieces. Why would a 12-letter word take one token when a 9-letter word takes three?

The answer has to do with how the vocabulary was built. Starting from a huge pile of text, the tokenizer repeatedly glued together the pieces that showed up side by side most often (this approach is known as **byte pair encoding**). A word that appears all the time, like *unbelievably*, ends up with its own entry. A less common word, like *sourdough*, gets spelled out from smaller pieces that did earn entries. This means that what decides how many tokens a word needs is how often that word showed up in the text the tokenizer learned from. Its length has very little to do with it.

Why not feed the model letters or whole words instead? Letters would turn every sentence into a very VERY long sequence, and the model's work grows with the length of what it reads. Here is what our sourdough sentence looks like if every letter gets its own token:

![The sentence Our sourdough is unbelievably good! drawn twice. On top, each of its 35 characters, spaces included, is its own token tile, wrapped over three rows, for 35 tokens. Underneath, the same sentence as the tokenizer's 8 chunks, from Our to the exclamation mark, for 8 tokens. Each letter tile shares the color of the chunk it belongs to, and a handwritten note says Over 4x longer!](https://www.kirupa.com/ai_fundamentals/images/tokens_and_your_bill_letters.png)

Whole words would need a vocabulary entry for every word in every language, every typo, and every variable name ever written, and the model would still be stuck the first time it met a new one. Chunks are the compromise. Common words and word pieces get a token of their own, and rarer words get built out of those smaller pieces, so there is always some way to represent whatever text we throw at the model.

### Counting Tokens

For everyday English, [OpenAI's rule of thumb](https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them) is that one token is about four characters, or about three quarters of a word. A 43-word reply from Biscuit works out to 53 tokens.

That rule of thumb falls apart outside of English prose. Below is the same question, *Do you have gluten-free bread today?*, written in five languages and counted by two of OpenAI's tokenizers, the newer `o200k_base` and the older `cl100k_base`:

| Question | Newer | Older |
|---|---|---|
| **English** Do you have gluten-free bread today? | 8 | 8 |
| **Spanish** ¿Tienen pan sin gluten hoy? | 8 | 8 |
| **Japanese** 今日はグルテンフリーのパンはありますか？ | 12 | 18 |
| **Hindi** क्या आज आपके पास ग्लूटेन-फ्री ब्रेड है? | 14 | 40 |
| **Tamil** இன்று பசையம் இல்லாத ரொட்டி இருக்கிறதா? | 16 | 53 |

A customer asking in Tamil costs twice as much as one asking in English with the newer tokenizer, and more than six times as much with the older one. Languages that show up less often in a tokenizer's training text get chopped into smaller pieces, and different models use different tokenizers, so the same words can have a different count depending on which model reads them.

## We Send a Lot More Than the Question

Say a customer types a 25-token question into Biscuit's chat box. How many input tokens does that request use? It isn't 25. It's 1,525.

The customer's question is only a small part of what we send. The model also needs instructions on how to behave (***you are a friendly help desk assistant for Crumb & Co., keep answers short, never make up prices***), known as the **[system prompt](https://www.kirupa.com/ai_fundamentals/system_prompts_vs_user_prompts.htm)**, and it needs the bakery's facts: opening hours, the menu, and which loaves are gluten-free. All of it goes out with the question, every single time:

![A stacked column showing one request of 1,525 input tokens. The system prompt takes 400 tokens and the bakery facts take 1,100 tokens, filling almost the whole column. The customer's question is a thin 25-token sliver at the bottom.](https://www.kirupa.com/ai_fundamentals/images/request_contents.png)

The part the customer typed is less than 2% of what we pay to send. And Biscuit is a simple bot! If we give the model tools to call, their descriptions go out as input tokens too, and reasoning models think before answering, using reasoning tokens that never appear in the reply but are [billed as output tokens all the same](https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them).

## Reading the Bill

Providers price tokens per million, with one price for input tokens and a different price for output tokens. To keep the math friendly, we'll use made-up prices of **$2 per million input tokens** and **$10 per million output tokens**. Real prices vary a lot between models and change often, but that ratio is in the right neighborhood. At the time of writing, the text models listed on [OpenAI's pricing page](https://developers.openai.com/api/docs/pricing) charge between four and eight times more for an output token than for an input token.

If Biscuit's first reply is 100 tokens long, the bill for that one request looks as follows:

![A receipt for Biscuit's first request. 1,525 input tokens at 2 dollars per million tokens cost 0.00305 dollars. 100 output tokens at 10 dollars per million cost 0.00100 dollars. The total is 0.00405 dollars. A handwritten note points at the output line: 5x the price per token.](https://www.kirupa.com/ai_fundamentals/images/token_receipt.png)

That is less than half a cent, which adds up fast across every message from every customer. We don't have to count any of this by hand, either. Every API response includes a **usage** object with the real counts, which OpenAI's Responses API and Anthropic's Messages API both call `input_tokens` and `output_tokens`. Turning them into dollars takes two multiplications:

```js
function requestCost(usage, prices) {
  const input = usage.input_tokens * prices.inputPerMillion / 1_000_000;
  const output = usage.output_tokens * prices.outputPerMillion / 1_000_000;
  return input + output;
}

// Made-up prices, for illustration only
const prices = { inputPerMillion: 2, outputPerMillion: 10 };

console.log(requestCost({ input_tokens: 1525, output_tokens: 100 }, prices)); // 0.00405
```

### Why Output Costs More

Why would a token the model writes cost more than a token it reads? The answer is in how the model does the work. When our request arrives, the model can look at all 1,525 input tokens at once, in one big parallel pass. Writing is a different story. The model produces its reply one token at a time, and every new token takes another full pass through the model, one that considers everything that came before it, including the tokens it just wrote:

![Two panels. In the top panel, a block of 1,525 input tokens flows into the model in a single pass. In the bottom panel, the model writes its reply one token per pass: Yes on pass 1, a comma on pass 2, we on pass 3, and bake on pass 4, with an arrow looping each new token back into the model.](https://www.kirupa.com/ai_fundamentals/images/read_vs_write.png)

Reading is like taking in a page someone hands us. Writing is like producing a page one word at a time, glancing back over everything so far before adding the next word. A 100-token reply means 100 trips through the model, and they can't happen in parallel because each one needs the token before it. That keeps expensive hardware busy for longer per token, and the price reflects it. It is also why a long reply takes noticeably longer to finish than a short one.

## Why Every Turn Costs More Than the Last

Biscuit's first reply cost less than half a cent. What happens as the conversation keeps going?

As we saw in [The Anatomy of a Chat Request](https://www.kirupa.com/ai_fundamentals/anatomy_of_a_chat_request.htm), the model doesn't remember anything between requests. Each API call starts from a blank slate, so Biscuit has to send the entire history every time: the system prompt, the bakery facts, every earlier question and answer, and then the new question. Some APIs can store the conversation for us, but the earlier turns are still billed as input tokens, as OpenAI's [conversation state guide](https://developers.openai.com/api/docs/guides/conversation-state) points out.

Let's say a typical question is about 25 tokens and a typical reply is about 100. Each turn then adds about 125 tokens of history, and every one of them gets sent again on every turn that follows. Here is what the input looks like over a 20-turn conversation:

![A bar chart of input tokens per request across 20 turns of a conversation. Every bar has the same 1,500 tokens of system prompt and bakery facts at the bottom, topped by a band of earlier questions and replies that grows by 125 tokens each turn. Turn 1 sends 1,525 tokens, turn 10 sends 2,650 tokens, and turn 20 sends 3,900 tokens. A handwritten note says piling up!](https://www.kirupa.com/ai_fundamentals/images/conversation_growth.png)

Turn 20 sends 3,900 input tokens and costs a bit more than twice as much as turn 1. Added up, the whole 20-turn chat costs about 13 cents. Here is that same math as code, reusing our `requestCost` function:

```js
const SENT_EVERY_TIME = 1500; // system prompt + bakery facts
const QUESTION = 25;          // typical customer question
const REPLY = 100;            // typical reply from Biscuit

let history = 0;
let total = 0;

for (let turn = 1; turn <= 20; turn++) {
  const input = SENT_EVERY_TIME + history + QUESTION;
  total += requestCost({ input_tokens: input, output_tokens: REPLY }, prices);
  history += QUESTION + REPLY;
}

console.log(total.toFixed(4)); // 0.1285
```

What's noteworthy is where that money went. Over 20 turns, we paid for 54,250 input tokens and only 2,000 output tokens. Output tokens are five times pricier, yet they account for just 2 cents of the 13. The input side is where the volume is, and it keeps growing for as long as the conversation does.

## What This Means for What We Build

Once we can see where the tokens come from, a handful of practical habits fall out of it:

- Log the usage object on every response instead of guessing token counts.
- Cap the output length (`max_output_tokens` at OpenAI, `max_tokens` at Anthropic) so a runaway reply can't run up the bill.
- Keep the parts we send every time short. Every token trimmed from Biscuit's system prompt and bakery facts is saved on every single turn.
- Drop or summarize old turns so the history can't pile up forever.
- Put that fixed text at the very start of every request, unchanged. Providers charge much less for a recently seen prefix, a trick called [prompt caching](https://www.kirupa.com/ai_fundamentals/prompt_caching.htm).

## Conclusion

When we started, a token was just a mysterious unit on a bill. Now we know it is a chunk of text, usually a piece of a word, picked by a tokenizer based on how common that chunk is. We pay for every token we send and every token we get back, and the ones coming back cost more because the model writes them one at a time. Because the model forgets everything between calls, the tokens we send keep piling up as a conversation grows, which is why Biscuit's bill depends more on how we structure our requests than on how chatty the model is.

There is one detail we skipped over. If every turn sends more tokens than the last, at some point the request gets too big for the model to read at all. That limit is called the context window, and it's where we'll go next in [Context Window Limits](https://www.kirupa.com/ai_fundamentals/context_window_limits.htm).