What's in an agent's context?

One block of text

An agent has no session, no memory of yesterday, and no standing view of your machine. Every turn it sends the model one block of text and gets one block back. That block is the context, and everything the agent knows at that moment is inside it.

Two words are worth separating before anything else. The context window is the maximum amount of text a model will accept, measured in tokens - a token being roughly four characters of English, so a 200,000-token window is a few hundred pages. The context is what is actually sitting in that space right now. The window is the shelf; the context is what you put on it.

The model does not do the putting. A program called the harness - the coding assistant, the chat product, or your own script running the loop - gathers everything it wants the model to know, joins it into one request, and sends it. Next turn it does the whole thing again from scratch, because the model kept nothing: it is stateless, in the strict sense described in What is an LLM?

Six kinds of thing go into that block. The rest of this article is each one, what it costs, and what happens when they stop fitting.

The system prompt

The system prompt is the instruction text the product's authors wrote, and it goes in first. It sets the role the model plays, the rules it follows, the shape its answers take, and when it is allowed to reach for a tool.

You do not see it, and you pay for it on every single turn. It is also the honest answer to a question beginners ask constantly: why do two products built on the same model behave so differently? A support bot and a coding agent can run identical weights and share nothing but a completely different first thousand tokens.

The practical version: when an agent does something you never asked for - refuses a request, appends a summary you did not want, always writes a test before the code - the instruction is usually sitting in the system prompt rather than in the model.

Tools and their schemas

A tool is a function the agent can call: read a file, run a query, search the web, open a pull request. The model cannot execute any of it. What the model gets is a description.

const readFile = {
  name: "read_file",
  description: "Read a UTF-8 text file from the project. Call before editing.",
  input_schema: {
    type: "object",
    properties: {
      path: { type: "string", description: "Path relative to the project root" },
      maxLines: { type: "integer", description: "Stop after this many lines" }
    },
    required: ["path"]
  }
};   // the description is not documentation - it is the only
     // thing the model has when deciding whether to call this

The model answers with a request to call read_file and the arguments it wants. The harness runs the function, and the result goes back into the context as the next thing the model reads. The model never touched your filesystem; it asked, and something else acted.

Every connected tool is described in the window every turn, whether it gets used or not. Ten tools is a few thousand tokens of overhead before the conversation has started. This is what surprises people who connect several MCP servers at once: each server brings its whole catalogue, and the catalogue is rent, paid every turn.

The conversation so far

Everything said up to now goes back in: your messages, the model's replies, and the part people forget - every tool call the model made and every result that came back.

That last part is why an agent's context grows so much faster than a chat's. A chat turn adds a few hundred tokens. An agent turn that reads a 600-line file adds the entire file, and it is still sitting there twenty turns later, long after it stopped being relevant. Ten file reads and a couple of searches can outweigh the actual conversation by an order of magnitude.

The useful picture: the conversation is a transcript that gets re-read from the top on every turn. Nothing in it expires on its own, and nothing shrinks unless the harness deliberately shrinks it.

What it read along the way

Retrieval is the practice of going and finding text, then putting it in the window so the model can read instead of recall. Search results, file contents, a fetched page, rows from a database, the three relevant sections of a handbook.

It is the most effective single fix for a model inventing things, because it changes the task. Producing a plausible-looking answer is what a model does when it has nothing to work from; reading is what it does when the answer is on the page in front of it.

It is also, in most real runs, the largest thing in the window. Retrieved text arrives in blocks of thousands of tokens and it arrives repeatedly, which makes precision worth real effort: fetching the one relevant section rather than the whole document is not tidiness, it is the difference between a run that finishes and a run that fills up.

Skills, loaded on demand

A skill is a packaged set of instructions - how this team writes a changelog, the steps in a release, the house rules for a chart - that the agent loads only when the job calls for it. What are Agent Skills? covers the format in full.

The mechanism only makes sense in terms of the window. The agent is told, in about a line per skill, that the skill exists and when it applies. The body stays on disk. Forty skills therefore cost roughly forty lines until one of them fires, at which point that one is pasted in whole and the other thirty-nine still cost a line each.

So a skill is not merely packaged instructions. It is a decision about what does not have to be in the window.

Memory

Memory is anything the harness stores outside the model and inserts back into the context later: a project instructions file, preferences you stated once, a summary written at the end of the last session.

It is not the model remembering. Nothing inside the model changed. A file was read and pasted into the block, exactly like a retrieved document, and the result is a convincing impression of continuity built entirely out of re-sending text.

Which sets up the tradeoff. Memory that always loads is a permanent tax on every turn, and it competes with the work for the same space. Keep the always-on part short and let the rest be fetched when it is actually relevant.

What it all costs

Here is one run, twenty minutes into a coding job, on a 200,000-token window. These figures are illustrative - the shape of a typical run, not a measurement of any product - but the proportions are the point.

Two observations, and they are most of the reason this article exists.

First, the parts you would think about first are not the parts that matter. System prompt, tools, skills, and memory together are about 13% of what is used. The conversation and the documents are the other 87%, and unlike the first four they grow with every turn. Trimming the system prompt while a run reads whole files is optimising the wrong end.

Second, the whole block is re-sent every turn. Not just the new message - all of it. A forty-turn run pays for its transcript forty times, which is why what an agent actually costs is a question about context rather than about the number of requests.

What falls out first

Windows fill. When the context no longer fits, one of three things happens and only one of them is graceful.

• Compaction. The harness summarises the oldest turns into a paragraph and drops the originals. The run continues, having quietly traded detail for room.
• Truncation. Old messages or oversized tool results are cut outright, usually oldest first, with no summary standing in for them.
• Failure. The request goes over the limit, the provider rejects it, and the run stops where it is.

The symptom you actually notice is an agent that forgets something you established early. You said "do not touch the migrations" at turn three; by turn forty that sentence is gone, or it is one clause inside a summary of a summary. Nothing malfunctioned. The instruction simply is not in the block any more, and the block is all there is.

Position matters too. Text at the very start and the very end of a long context tends to be used more reliably than text buried in the middle - which is exactly where compaction leaves its lossiest work.

Two habits follow, and they cost nothing. Restate the constraint that actually matters, late and in your own words, rather than trusting that turn three survived. And when the job changes, start a fresh run instead of dragging a full transcript into unrelated work. Why agents fail follows this failure mode further into a long run.

Why Skills, MCP, and Hooks exist

Read the agent tooling landscape with the window in mind and it stops looking like a feature list. Every one of these mechanisms is an answer to the same question: what gets to be in there?

• MCP is a standard way to put tools into the window. It solves discovery and plumbing, and it charges schema tokens on every turn - which is the argument for connecting the servers a job needs rather than every server you own.
• Skills keep instructions out of the window until the moment they apply, so expertise can be large without being expensive.
• Hooks run ordinary code at fixed points in the loop, outside the window entirely. A formatter that always runs after an edit does not need to be a rule the model is told, has to remember, and might not follow.
• Sub-agents are a second window. The child burns its own context on file reads and dead ends, then hands the parent a short summary instead of the mess that produced it.
• Retrieval puts the answer in the window rather than hoping it is somewhere in the model.

Skills, MCP, Hooks, and Plugins sets those mechanisms side by side and shows where each one belongs. This article is the thing underneath all four of them.

One block of text, reassembled and re-sent on every turn, with a hard ceiling and no memory behind it. Once you can see the window, the rest of agent engineering reads as one long argument about what deserves to be in it.