06Tokens and context

Two words you'll use every day.

A token is a chunk of text. Not a letter, not a word, something in between - usually part of a word. The model counts everything in tokens, and tokens are what you pay for. Send a long file, spend a lot. Get a long answer, same. By the way, the same text in most other languages costs more tokens than in English.

Context is how much it's holding in its head right now. Your whole conversation, every file you showed it, all of its replies - all of that sits in context and takes up room. On modern models that room is huge, hundreds of thousands of tokens, a million on some. But not infinite.

Two effects follow, and you will definitely notice both.

First: a long chat gradually starts going dumb. Not because the AI got tired: the context is full, older parts get compressed or lost, and the important stuff drowns in everything else. The fix is simple: start a new chat and restate the gist in a couple of sentences. Surprisingly often that fixes "it stopped understanding me".

Second: don't dump everything into the chat "just in case". Extra material doesn't help, it hurts: the more junk in the context, the worse it thinks. Give it what's relevant.