How Claude's Context Window Works: Limits, Costs, and Overflow

June 23, 2026 · updated June 25, 2026 · How Claude Actually Works (part 4)

▶ Watch on YouTube & subscribe to The Stack Underflow

Every developer who builds with Claude eventually hits the same wall: the model starts missing facts it was explicitly given, or the API returns a context-exceeded error, or the bill jumps in a way that is hard to explain. Almost always, the root cause is the same thing — a misunderstanding of what the context window actually is, what fills it, and what the model does with it once it is full.

The context window is not “the memory limit for user messages.” It is the model’s entire working reality for one request. System prompt, tool definitions, conversation history, retrieved documents, your new message, and the model’s output all compete for space inside a single finite buffer. Understanding the anatomy of that buffer — and the documented phenomenon that degrades recall in its middle — is the foundation for building reliable, cost-predictable Claude applications.

The one-sentence version: The context window is Claude’s total working memory for one API call — every byte of input and output shares one finite strip of tokens, the strip size varies by model, and the model’s recall of information placed in the middle of a very long strip is measurably worse than its recall of information at the edges.

The anatomy of one API call

Visualize the context window as a single horizontal bar that fills left to right. When you send a request, Claude does not see “a system prompt and a question.” It sees one continuous token stream assembled in a fixed order:

|-- system prompt --|-- tools --|---- history ----|-- docs --|-- msg --|- output -|
^                                                                        ^          ^
left edge                                                          200K mark   hard limit

Each segment of that bar is a real cost:

System prompt tokens are the sneakiest cost. You write the prompt once, but it is counted — and billed — on every single API call in the session. A 4,000-token system prompt with 10 tool schemas attached can silently consume 10,000 tokens before the user types a single character.

Output tokens come out of the same total budget on most models. If a model has a 200K context window and you request up to 16K output tokens, your effective input budget is roughly 184K, not 200K. Always check the model card.

What happens when you overflow

When the bar fills to the hard right edge, there is no grace period. One of three things happens:

Overflow scenario A — API error:
  [your 220K request] --> server enforces 200K limit --> HTTP 400
  "model context window exceeded"

Overflow scenario B — silent client truncation:
  SDK drops oldest conversation turns until the request fits
  Model never sees the dropped history (no warning to user)

Overflow scenario C — explicit compaction:
  You summarise / truncate history before sending
  Controlled data loss, visible to your application logic

Scenario B is the most dangerous because it is invisible. The model appears to respond normally, but it is working from an amputated version of the conversation. Always instrument your usage.input_tokens field on every response to know where you actually are.

Model-by-model limits (June 2026)

“Claude has a 1M context window” is a headline that papers over significant per-model variation. Here is the accurate picture as of June 2026 (docs.anthropic.com):

A critical correction from earlier in 2025: Sonnet 4.6’s extended context window no longer requires an anthropic-beta header. The 1M window is the production default for that model. If your code still sends anthropic-beta: extended-context-window-2025-05, it will still work, but the header is now redundant.

Haiku 4.5 has no upgrade path to 1M. If you need 1M tokens and you are using Haiku, you need a different model.

Pricing in 2026: the surcharge is gone

Earlier versions of Claude’s pricing applied a step-change rate to input tokens above 200K. As of the Opus 4.6/Sonnet 4.6 generation, Anthropic removed that surcharge. Input tokens are now billed at a flat per-token rate regardless of how deep into the context window you go (platform.claude.com/docs/en/about-claude/pricing, 2026):

Standard per-MTok rates (input / output), June 2026:

  Opus 4.8   ·  $5.00  / $25.00
  Sonnet 4.6 ·  $3.00  / $15.00
  Haiku 4.5  ·  $1.00  /  $5.00

The cost lever that matters now is not a 200K cliff — it is the sheer volume of tokens you send. A 900K-token Opus 4.8 request costs roughly $4.50 in input alone. Plan accordingly.

Prompt caching is still the most effective cost control tool. Cached input tokens are billed at approximately 0.1x the standard input rate. For a large, stable system prompt or a fixed document corpus that you send on every call, caching cuts that portion of your input bill by 90%. The usage field in the response will show cache_read_input_tokens separately so you can confirm the cache is hitting. See Prompt Caching: Cut Your AI Bill for the mechanics.

The middle is dangerous: “lost in the middle”

Fitting inside the context window does not mean every token is equally useful. This is the second — and less obvious — half of the context window story.

A well-documented positional attention effect occurs in transformer-based language models: recall accuracy follows a U-shaped curve across the context window. Information placed near the beginning and end of the context is recalled reliably. Information buried in the middle is recalled significantly less reliably.

Recall
accuracy
  ^
  |  *                                        *
  |     *                                  *
  |        *                            *
  |           *         DIPS         *
  |              *                *
  |                 *          *
  |                    * ---- *
  +---------------------------------------------->
  start          middle of context           end

                "lost in the middle"

The architectural root cause, per 2024–2025 research (Liu et al., 2023; MIT/Google, 2024), is the interaction between Rotary Position Embeddings (RoPE) and softmax attention. RoPE introduces a long-term decay in dot-product similarity between distant tokens, which systematically reduces attention weight on mid-context material. Softmax normalisation then amplifies the effect by concentrating probability mass on the highest-scoring tokens — those at the edges.

Empirically, the effect is most pronounced for needle-in-a-haystack retrieval in very long contexts (100K+ tokens). For typical RAG workloads under 50K tokens, the degradation is less dramatic — but the principle holds at any scale, and the mitigation strategies are cheap to implement.

The three practical mitigations

The same fact, placed at the start of a 500K context, is answered correctly. Placed in the exact middle of that same context, it is missed or hallucinated. Position is a feature. Design for it.

How to read the usage field

Every Claude API response includes a usage object. Reading it on every call is not optional if you care about cost or correctness:

{
  "usage": {
    "input_tokens": 4821,
    "output_tokens": 312,
    "cache_creation_input_tokens": 8500,
    "cache_read_input_tokens": 6200
  }
}

input_tokens is the count of uncached input tokens in this call — the tokens after your last cache breakpoint. cache_read_input_tokens is how many tokens were served from cache (billed at 0.1x). cache_creation_input_tokens is how many tokens were written to a new cache entry (billed at 1.25x, amortised over subsequent reads). Total input tokens consumed = input_tokens + cache_read_input_tokens + cache_creation_input_tokens.

Log this object to your observability layer on every call. It is the only reliable way to detect a context growth problem before it becomes a billing shock.

How to apply this: a decision checklist

Before you start a new Claude integration, run through these:

1. Pick your model first, then verify its context limit. Do not assume 1M. Check whether your target deployment environment (Bedrock, Vertex, Microsoft Foundry) supports it for that model.
2. Count your baseline token cost. Estimate system prompt + tool schemas + average history. Use the token-counting endpoint (POST /v1/messages/count_tokens) to measure before you send a single real request.
3. Set a context budget and enforce it. Decide the maximum input tokens you want to send, and build a history truncation or summarisation strategy for when you approach it.
4. Put critical content at the edges. System prompt for standing instructions. Close to the user’s message for the specific documents or context most relevant to the current turn.
5. Enable prompt caching for stable content. System prompts, fixed tool schemas, and large reference documents are good candidates. Verify with cache_read_input_tokens in the response.
6. Instrument every response. Log usage.input_tokens + usage.output_tokens. Alert when you approach 80% of your chosen budget.

Common misconceptions

“The context window is only for user messages.” Every segment shares the same space: system prompt, tool definitions, all prior turns, retrieved documents, the new message, and the output. A verbose system prompt and eight tool schemas can consume 15,000 tokens before a user types a word. Treat all of them as token budget.

“1M tokens means 1M tokens of user input.” The 1M figure is the total context size, which includes the output space. Output tokens are reserved from the same pool. On Sonnet 4.6 with a 64K output limit, your maximum input is 1M minus 64K, not 1M flat.

“Overflow is automatically handled.” The API does not auto-compact. On a hard limit violation it returns an error. On soft violations, client SDKs may silently drop history, which produces a model that appears to work but is responding to an amputated conversation. History management is your responsibility.

“Long context is always better than RAG.” A 1M window that you fill with an entire document corpus is slower, more expensive, and subject to the lost-in-the-middle degradation. A well-designed RAG pipeline that retrieves 3–5 relevant chunks into a 20K context is faster, cheaper, and often more accurate. Reach for the big window when you need it; reach for retrieval when you need precision.

Frequently asked questions

How do I know how many tokens my request is using before I send it? Use the token-counting endpoint: POST /v1/messages/count_tokens. It takes the same messages array as a normal request and returns an estimated token count without consuming output budget or incurring inference cost. This is useful for enforcing a budget before a large request is submitted.

Does Sonnet 4.6 still need the beta header for 1M context? No, as of mid-2026. The anthropic-beta: extended-context-window-2025-05 header was required during the beta period but is now redundant. Sonnet 4.6’s 1M window is production-GA at standard pricing on the Claude API, Bedrock, and Vertex AI.

Is the “lost in the middle” effect real or just theoretical? It is empirically documented across multiple model families in peer-reviewed research (Liu et al., 2023, Stanford/UCSB; MIT/Google Cloud AI, 2024). The architectural mechanism — RoPE long-term decay combined with softmax attention amplification — is well understood. It is most pronounced for needle-in-a-haystack retrieval at very long contexts, but the mitigation strategies (pinning, summarising, RAG) are worth applying at any context length.

If I use prompt caching, does it interact with the context window budget? Yes. Cached tokens still count toward your context window. Prompt caching changes the price of those tokens (0.1x for reads), not their size. A 50,000-token cached system prompt costs 90% less to send, but it still occupies 50,000 tokens of your context budget on every call.

Should I use Haiku 4.5 for cost and just manage the 200K limit? That is a legitimate strategy for many workloads. Haiku 4.5 at $1/$5 per MTok is significantly cheaper than Sonnet or Opus. If your use case fits in 200K tokens, Haiku’s ceiling is not a constraint. The tradeoff is capability: for complex reasoning, long-document synthesis, and agentic tasks, Sonnet or Opus will outperform Haiku at the same token budget.

Where this fits in the series

Understanding the context window is the load-bearing prerequisite for almost everything else in this course. The token primer (How LLM Tokens Work and Your AI Bill) explains how text becomes tokens in the first place — read that first if the token counts here feel abstract. The stop-reason tutorial (Understanding stop_reason in the Claude API) covers what the API tells you when the model finishes — including max_tokens stop events that indicate you are hitting output-space constraints. Context engineering (Context Engineering: Pin, Summarize, Prune, Compact) goes deeper on the mitigation strategies introduced here. And Prompt Caching: Cut Your AI Bill is the practical guide for the 90%-savings lever mentioned in the pricing section. Browse all tutorials to follow the full series.