06Tokens and context

Two words worth knowing. There will be almost no more jargon on this page.

A token is a chunk of text. Not a letter, not a word, something in between - usually part of a word. The model counts everything in tokens, and tokens are what you pay for. Send a long file, spend a lot. Get a long answer, same.

Context is how much it is holding in its head right now. Your whole conversation, every file you showed it, all of its replies - all of that sits in context and takes up room. The room is finite.

Two effects follow, and you will definitely notice both.

First: a long chat starts going dumb. Not because the AI got tired, but because context filled up and the beginning is being pushed out. The fix is simple - start a new chat and restate the gist in two paragraphs. That magically fixes "it stopped understanding me" more often than you'd expect.

Second: don't dump everything into the chat "just in case". Extra material doesn't help, it hurts - the important part drowns in the noise. Give it what's relevant.