Back to the index
10/ 55

USING MODELS

Context window.

The amount of information a model can work with in a single inference context.

In plain words

A context window is the limit on the input and, depending on the model’s accounting, generated tokens available during a model call. It sets a working-space limit, usually measured in tokens.

A closer look

Context can include instructions, conversation history, documents, tool descriptions, and tool results. An application chooses what to send; the model does not automatically see every past conversation or file. Output and reasoning tokens may consume part of the available budget, with separate limits depending on the service.

When a conversation grows too long, software may remove older messages, summarize them, retrieve selected passages, or start a new context. These choices affect what the model can use. A large context window helps with lengthy material but does not guarantee that every detail will be correctly retrieved or applied.

In practice

AN EXAMPLE

You supply a long manual and ask about a clause. The clause must be included in the model’s active context, or retrieved into it, for the model to ground its answer in that text.

A useful distinction

Context is not the same as training or permanent memory. Adding a document to a conversation usually changes the input to a call, not the model’s learned weights.

Sources & further reading

Anthropic — Effective context engineering for AI agents (opens in a new tab)