Two years ago, mainstream models had context windows in the 4K to 8K range. Now, million-token windows are standard for flagship models.
Current landscape
- Gemini 2.5 Pro: 1 million tokens (expandable to 2 million)
- Claude Opus 4.6: 1 million tokens (beta)
- GPT-5.4: 1 million tokens (experimental support)
- Llama 4 Scout: 10 million tokens
- Kimi K2: 128K → 256K
Llama 4 Scout's 10 million tokens sounds impressive, but how well it can maintain information retrieval accuracy over such a long context remains a big question.
Is longer always better?
Context length and practical usability are two different things. The key metric is long-context retrieval accuracy — whether a model can accurately find the needed information in extremely long text.
Opus 4.6 data: 8-needle retrieval accuracy is 93% at 256K context, dropping to 76% at 1 million tokens. 76% may not sound low, but it means nearly a quarter of the information you put in could be "forgotten."
Another real-world issue is cost. Inputting 1 million tokens is not free — under current API pricing, filling the context window can cost anywhere from a few dollars to tens of dollars. Without controls, an agent running dozens of conversation rounds could rack up a frightening token bill.
Compaction technology
Anthropic's Compaction API and GPT-5.4's auto_compact mechanism are trying to address this: when the context is nearly full, they automatically compress earlier content while retaining key information. In theory, this supports "infinite-length conversations," but the actual effect depends on how much information is lost during compression.
A longer context window is not an end in itself; maintaining accurate information recall over long contexts is what matters. The finish line of this arms race is not "who has the bigger number," but "who remains reliable with extremely long contexts."
Sources: Anthropic official documentation, CocoLoop, model release reports