Tokens — how AI reads text

You broke a sentence into tokens — the small pieces an LLM actually counts and predicts. A word might be one token, or several; punctuation and spaces count too.

Think of tokens as LEGO bricks of text. The model’s job is: given the bricks so far, what’s a good next brick?

• Longer prompts use more tokens
• Responses also spend tokens
• Every model has a maximum it can hold at once (that’s the next lesson: the context window)

• Trim a long dump of notes before pasting into chat
• Prefer bullet goals over giant copy-pastes
• Estimate whether a document will “fit” before you ask questions about all of it

Token count ≠ word count. Fancy formatting and repeated boilerplate burn tokens without helping the answer.

Take one chat message and rewrite it half as long without losing the goal. Same ask, fewer tokens.