Context engineering: assembling what your agent sees

Every turn, before your model generates a single token, something decides what the model gets to see. The system prompt, the conversation so far, retrieved memories, tool definitions, the user's latest message — all of it competes for a finite window, and the quality of that assembly puts a hard ceiling on the quality of everything downstream. That assembly discipline is context engineering, and most production agent failures we debug trace back to it, not to the model.

This post walks the whole pipeline: where context comes from, how retrieval decides what gets in, how token budgets force trade-offs, and what the assembly code actually looks like. It is also, deliberately, a showcase — it exercises every interactive primitive in our blog system, including the new scroll-driven walkthroughs, code walkthroughs, glossary terms, and comprehension checks. Expect a higher component density than a normal post; that is the point of this one.

Prompts are static; context is a system

"Prompt engineering" suggests a craft of wording. But in a production agent the system prompt is the only part of the input that stands still. Everything else — history, memories, retrieved documents, tool results — is selected at runtime, per turn, by code you wrote. The context window is a budgeted surface, and something in your stack is already doing context engineering. The only question is whether it does it deliberately or by accident.

“

Context engineering is the delicate art and science of filling the context window with just the right information for the next step.

The failure mode that makes this concrete: an agent that answers perfectly in turn three and confidently contradicts itself in turn thirty. The window was never full. What happened is attention dilution — every irrelevant transcript fragment competes with the relevant ones, and the model's effective precision drops as the noise grows. Relevance beats volume, every time.

The context assembly pipeline

Context assembly is a pipeline, and naming its stages gives you places to put instrumentation, budgets, and tests. Tap a stage for what it owns:

To see the pipeline run, follow one turn of a support agent as its context window gets built. Scroll through the four states:

Where memories actually live

"Give the agent memory" is four different storage systems wearing one name. The shape that works in production separates them by lifetime and access pattern. Tap a node for its role:

The distinction that matters most is write policy. Session state is written unconditionally; vector memory is written through a promotion gate ("is this worth remembering across sessions?"); the knowledge base is not written by the agent at all. Collapsing those three write policies into one store is how agents end up confidently recalling things that were never true.

Retrieval is a ranking problem and a policy problem

Recall from vector memory works by semantic similarity: the query is embedded, candidate memories are ranked by distance, and the top-K survive. Try it — switch the query and watch the ranking change:

Similarity ranks candidates, but it cannot decide whether to retrieve at all. That is a policy question, and encoding it explicitly beats retrieving on every turn. Explore how the decision shifts:

The query side of this is a few lines. The discipline is in the threshold and the budget, not the API call:

export async function recallMemories(
  query: string,
  budgetTokens: number
): Promise<MemoryChunk[]> {
  const embedded = await embed(query);
  const hits = await vectorStore.search(embedded, {
    topK: 8,
    minSimilarity: 0.75, // below this, "similar" is coincidence
  });
  return packToBudget(hits, budgetTokens); // whole chunks only, never split
}

The window is a budget — treat it like one

Allocate the input window per source, and enforce the allocation in code. Here is a working budget for an 8K-input agent — drag the sliders and watch what overruns cost you:

When the budget forces compression, you have four levers, each with a different fidelity/cost profile. Expand a row for when to use it:

What the reader feels: memory on vs. off

The difference context engineering makes is easiest to see from the user's side of the chat. Same agent, same questions — toggle the memory knob and step through:

Behind the memory-on script, one turn of the agent looks like this — note that recall is a tool call with a budget, not an ambient superpower:

That last step carries a principle: surface what you recalled. Citing remembered facts back to the user ("for your May 12–19 Tokyo trip") turns stale memory from a silent failure into a correctable one.

The assembly function, line by line

Strip away the vendor SDKs and context assembly is one honest function. Step through how each stage maps to code:

1export async function assembleContext(2  session: Session,3  userMessage: string,4  budget: TokenBudget5): Promise<ModelInput> {6  // Collect: every candidate source, no filtering yet7  const candidates = {8    memories: await recallMemories(userMessage, budget.memory),9    retrieval: await maybeRetrieve(userMessage, session, budget.retrieval),10    history: session.turns,11  };12 13// Select + compress history: recent turns verbatim, the rest summarized14const recent = candidates.history.slice(-RECENT_TURNS);15const summary = await rollingSummary(16candidates.history.slice(0, -RECENT_TURNS),17session.summary,18budget.summary19);20 21// Order: stable blocks first (cacheable), live task last22return render([23block('system', SYSTEM_PROMPT),24block('tools', toolDefinitions(session.workflow)),25block('memory', candidates.memories, { labeled: true }),26block('retrieval', candidates.retrieval, { labeled: true }),27block('summary', summary),28block('history', recent),29block('user', userMessage),30]);31}

Zoom out one level and the same function sits inside a per-turn lifecycle:

The contrast with the naive approach is stark enough to put side by side:

Adopting this in an existing agent is incremental, not a rewrite:

1. 1

   Instrument the window you already build

   Log per-source token counts for every turn. Most teams discover one source (usually history or tool schemas) eating over half the window for no measured benefit.

2. 2

   Set budgets and enforce them

   Write down the allocation, then make the assembly code enforce it. The first enforcement run usually exposes the silent overruns.

3. 3

   Add a retrieval policy

   Replace retrieve-always with the explicit policy decision: skip on chitchat, skip when the answer is in-window, scope by workflow.

4. 4

   Compress, then measure

   Introduce rolling summaries and structured extraction, and verify with evals that answer quality held while tokens dropped. Compression you cannot measure is just deletion with extra steps.

How this maps onto specific stacks differs mainly in vocabulary:

And the failure modes to expect, because every team hits the same four:

Check your understanding

The discipline in one card

Context engineering is not a model trick; it is an engineering discipline with a pipeline, budgets, policies, and logs. The teams whose agents stay coherent at turn fifty are the ones treating the window as a system.

This post focused on a single agent's window. The moment you decompose into multiple agents, context becomes a routing problem too — who sees what, and what crosses the boundary between specialists. That story is the Agent Orchestration series, and the two disciplines compound: good decomposition bounds what each window must hold.

References

• Anthropic, Effective context engineering for AI agents
• Anthropic, Building Effective Agents
• Liu et al., Lost in the Middle: How Language Models Use Long Contexts
• OpenAI, A Practical Guide to Building Agents