How AI answers are made
Tokens (LLM)
Also called: LLM tokens, Tokens
Tokens are the small pieces of text a language model reads and writes: whole words, parts of words, single characters or even bytes.
In plain words
A model doesn’t read letter by letter or word by word. It reads tokens.
Anthropic notes that how many characters a token stands for depends on the model and the language.
Example
Anthropic says its newer Claude models use a tokenizer that produces about 30 percent more tokens for the same text than earlier models. So the same page can count as a different number of tokens, depending on the model.
Why it matters for AI search
Context windows and output limits are counted in tokens.
When a system can only pass a limited number of tokens to a model, it has to choose which passages to include.
What you can do
- Say important things early and plainly, instead of after a long introduction.
- Keep repeated boilerplate text out of your main content.
- Don’t count tokens yourself: write clearly for people.
Often confused with
- Words
- A token is not the same as a word. A long or rare word can be split into several tokens.
Sources
Facts and sources last checked on 19 October 2026.
Related terms
Go deeper on GEO-Score
How does your page do in AI search?
GEO-Score checks one page on 22 signals, from AI crawler access to structured data, and shows what to fix first.
Analyze a page