AI models do not read text word by word. They split everything into smaller pieces called tokens. A token is usually a short word, part of a longer word, or a piece of punctuation. In English, one token is roughly three quarters of a word, so 1,000 tokens is about 750 words.
Tokens matter for two practical reasons. First, they are the pricing unit: using an AI model through its API is billed per million tokens, with a price for what the model reads (input) and a price for what it writes (output). Second, they are the limit unit: a model can only hold so many tokens in its head at once. That limit is called the context window.
Why you keep hearing it
Because both cost and capacity conversations happen in tokens. "This model has a 200k context window" means it can hold roughly 150,000 words of your material at once. "We burned through a lot of tokens" means the AI did a lot of reading and writing, and someone paid for it.
What it means for you
Longer documents and longer conversations cost more and eventually hit limits. If an AI tool starts forgetting what you said earlier in a long session, you have usually run out of context window, not patience.