Context window

The window is shared between input and output. A 200,000-token window filled with 199,000 tokens of documents leaves room for a very short answer. When a conversation grows past the limit something has to give, which is why chat products summarise or drop older turns — and why a long session can appear to forget what you agreed twenty messages ago. Using the rough English ratio of about four characters to a token, 100,000 tokens is roughly 75,000 words — around a 300-page book — which is a more useful mental model than the raw number.

The window is the entire working memory of the system. A model with no memory of your codebase, your policies or your last conversation is not being unhelpful; it was never given them. Almost every technique for making AI useful on real work — retrieval, memory, tool results, examples — is ultimately a strategy for deciding what deserves the space.

Treating a large window as a reason to stop being selective. Models attend unevenly across a long context, and material buried in the middle of a very long prompt is measurably more likely to be overlooked than the same material near the start or the end. Filling a million-token window with everything you have is usually worse, slower and more expensive than retrieving the right ten thousand tokens.

The related mistake is assuming a bigger window replaces retrieval. It changes the economics of retrieval; it does not remove the need to choose what goes in.