Context Window

A context window is the maximum amount of text a large language model can consider in a single pass, measured in tokens rather than words — for Claude, one token is roughly 3.5 characters of English text, so the token count always runs higher than the word count (Anthropic). Everything the model needs shares that one budget: the system prompt, the running conversation, any knowledge-base articles pulled in by RAG, and the reply it is generating. When the total exceeds the window, something has to be dropped — usually the oldest turns — so a long support chat can quietly forget what the customer said at the start.

The ceilings have risen fast. Anthropic's flagship Claude models now ship a 1-million-token context window (Anthropic); Google's Gemini 1.5 Pro opened a 2-million-token window in 2024 (Google) — twice the 1-million-token window it launched with, which Google described as enough to hold about an hour of video or 700,000 words (Google). For a support bot, a bigger window means you can hand the model more of the knowledge base and more conversation history without truncating — which is why vendors quote the number as a headline capability.

But capacity is not the same as attention. In a widely replicated study, Liu et al. found model performance follows a U-shaped curve across the window: accuracy is highest when the relevant fact sits at the very beginning or end of the input, and degrades sharply when the model has to retrieve it from the middle (Liu et al., TACL 2024). Stuffing your entire help centre into a 2-million-token prompt does not guarantee the model reads the article that answers the ticket.

What it hides: a large advertised window sells the fantasy that you can pour in everything and the model will use all of it. In practice, retrieval quality and position still decide the answer — the middle of a long context is where facts go to be ignored — and every extra token you load adds latency and cost on every call. Worse, overflow is silent: the model does not warn you it dropped the earliest turns, so a bot that looked reliable in a short demo can lose the thread on a real, drawn-out conversation. Treat the window as a hard ceiling to budget against, not a guarantee that anything inside it will be attended to.