The Context Engineering Playbook for More Dependable AI Apps

Context engineering is a systems discipline

A dependable AI application is rarely the product of one brilliant prompt. It is the product of a pipeline that assembles the right instructions, examples, evidence, user state and tool results at the moment a model must act. That assembly process is context engineering. It treats the model’s input as a designed runtime environment rather than a text box. The practical question is not simply what to ask, but what the model needs to know, which source should outrank another, how much material it can reliably use and what must happen when the evidence is incomplete.

This distinction matters because modern context windows can create false confidence. A model that accepts 128,000 tokens does not necessarily reason equally well over every token. Long inputs increase cost and latency, while critical details can be diluted by duplicated policies, irrelevant search results or contradictory customer records. If a support assistant receives 30 policy pages when only two paragraphs govern a refund, more context may produce a less dependable answer. Strong systems maximise signal density: each included item should change, constrain or verify the output.

The work therefore spans architecture, information retrieval, product design and evaluation. Teams must define authority levels, standardise data formats, choose retrieval thresholds, budget tokens and record exactly what the model saw. Prompt wording still matters, but it sits inside a broader control system. The best context stack is legible enough that an engineer can explain why a response occurred and reproduce it from logs.

Build a hierarchy before writing instructions

Start by establishing an explicit order of authority. A useful hierarchy is platform policy, application rules, task instructions, verified business data, retrieved reference material, user preferences and untrusted user-supplied content. The exact labels vary, but the precedence must not. If a customer asks a travel assistant to ignore the cancellation policy, the system should recognise that request as lower authority than the verified policy. Without a hierarchy, the model is left to resolve conflicts through linguistic cues, and the most forceful or recent sentence may win.

Keep permanent instructions short, operational and testable. Replace vague directions such as “be helpful” with rules such as “quote prices in the customer’s selected currency”, “do not claim a booking is confirmed until the booking tool returns a confirmation ID” and “ask one clarifying question when the departure date is missing”. Separate behavioural rules from domain facts. Behaviour belongs in stable instructions; volatile facts such as inventory, interest rates or delivery times belong in retrieval or tools.

Structure also reduces accidental conflict. Use labelled blocks such as TASK, CONSTRAINTS, USER STATE, EVIDENCE and TOOL RESULTS, ideally with machine-readable fields where possible. Delimit untrusted text and state that it is data, not an instruction source. A retrieved webpage containing “ignore previous instructions” should be treated like quoted material in a newspaper article, not a command. This defence is not sufficient on its own, but it gives the model and downstream validators a clear boundary.

Use examples as behavioural test cases

Examples are most valuable when they demonstrate decisions that prose instructions leave ambiguous. A lending assistant may be told to explain a declined application without revealing protected fraud signals. One carefully designed example can show the desired balance: cite eligible public factors, avoid internal risk scores and direct the customer to an appeal route. The example teaches tone, scope and information boundaries simultaneously. A dozen near-identical examples, by contrast, consume tokens and encourage imitation of surface phrasing rather than the underlying rule.

Select examples by coverage, not volume. For a classification workflow, include the common case, a boundary case, an insufficient-information case and a case requiring refusal or escalation. Four diverse examples of 100 tokens each can outperform 20 repetitive examples totalling 2,000 tokens. Examples should mirror production input and output formats exactly; if the application expects JSON, every demonstration should use valid JSON with the same schema. Otherwise the model learns inconsistency from the very material intended to stabilise it.

Do not let examples become stale policy. Store them with owners, version numbers and review dates, especially in regulated or fast-changing domains. Test whether each example improves measurable outcomes through ablation: run the evaluation set with and without it, then compare accuracy, format compliance and unnecessary refusals. If removing an example has no effect across representative tasks, it is occupying budget without earning its place.

Retrieve evidence for the question, not the topic

Retrieval-augmented generation fails when search returns broadly related documents rather than answer-bearing passages. A query about whether a 14-day return window starts at purchase or delivery does not need the entire retail handbook. It needs the clause defining the start date, the applicable jurisdiction and perhaps an exception for personalised goods. Chunk documents around semantic units such as sections, tables or procedures, preserve titles and dates, and attach metadata that supports filtering by product, country, account tier and effective period.

A practical retrieval pipeline often combines keyword search for exact identifiers with vector search for semantic similarity, then reranks the top candidates. It may retrieve 30 passages, rerank 10 and provide the best three to five to the model. Those numbers are not universal; they are tunable controls. The key is to measure answer coverage against distraction. Increasing from five passages to 15 might raise source recall by four percentage points while doubling input tokens and reducing citation precision. That is a tradeoff, not an automatic improvement.

Evidence should carry provenance and authority. Include source name, document version, effective date and a stable link or identifier. When sources conflict, prefer the designated system of record rather than asking the model to average them. Require the model to state when the supplied evidence does not support an answer, and create a fallback such as broader retrieval, a database lookup or human escalation. Dependability grows when uncertainty changes system behaviour rather than merely changing the wording of a guess.

Treat user state as scoped, expiring data

Personalisation requires more than appending a conversation transcript. Useful user state includes durable preferences, current-session facts and workflow status, each with different lifetimes. A preferred language may persist for years; a delivery address may apply to one order; a claim that the user is “travelling tomorrow” may expire within hours. Label state with its source, timestamp and scope. The model should not have to infer whether a fact from 40 messages ago remains current.

Summarise long conversations around decisions and unresolved questions, not merely prose. A good state object might record: selected plan, £49 per month; billing cycle, annual; identity verification, pending; unresolved issue, postcode mismatch. This is more dependable than feeding 12,000 tokens of chat history containing abandoned options and corrected details. Preserve the recent turns verbatim when wording matters, but compress older interaction into structured state with links to the underlying record for audit.

Privacy and correctness impose limits. Include only state required for the current task, and avoid exposing sensitive fields to the model when a deterministic service can use them instead. An assistant may need to know that identity verification passed, not the customer’s passport number. Provide controls to correct or forget stored preferences, because personalised errors compound: an incorrect dietary preference reused across ten restaurant searches feels less like a single mistake and more like a broken product.

Make tool use a verified transaction loop

Tools turn a language model from a commentator into an operator, but every call adds failure modes. Define narrow tools with typed parameters, clear descriptions and explicit error responses. “Manage booking” is too broad; “search_flights”, “hold_itinerary” and “confirm_purchase” create inspectable stages. Validate arguments before execution, enforce permissions outside the model and use idempotency keys for actions that must not be repeated. A timeout must never tempt the application to charge a card twice.

Feed tool results back in compact, canonical form. A search service returning 200 raw records can overwhelm the model; return ranked options with the fields needed for comparison. Distinguish success, partial success, no result and system error. If a booking tool returns status “pending”, the model must not translate that into “confirmed”. For high-impact actions, insert a confirmation gate that restates the amount, recipient and consequence before execution. The model can prepare a £2,400 transfer, but the user or a policy engine should authorise it.

Use deterministic code after generation as well as before it. Validate JSON schemas, check totals, confirm citations and reject impossible dates. The model might draft an invoice explanation, while software calculates tax and verifies that line items sum to the charged amount. This division of labour is central to dependable design: language models handle interpretation and communication; conventional systems enforce arithmetic, permissions and irreversible state changes.

Budget context and test the assembled system

Assign a token budget by component rather than filling the context window opportunistically. In a 16,000-token production request, a team might reserve 1,500 tokens for instructions, 800 for examples, 4,000 for retrieved evidence, 1,500 for user state, 2,000 for tool results and 3,000 for the response, leaving headroom for formatting and variance. The allocations should reflect task risk. A legal research assistant may spend more on evidence; a transactional agent may reserve more for tool traces and validation feedback.

Ordering deserves deliberate testing. Put critical rules where they remain salient, group related evidence and remove duplicate passages. When the context exceeds budget, trim by priority rather than truncating the tail. Preserve authoritative constraints and answer-bearing evidence; compress history, drop redundant examples and summarise verbose tool payloads. Cache stable prefixes where the model provider supports it, but version them carefully so a policy update does not coexist with an old cached instruction block.

Evaluate complete context assemblies, not prompts in isolation. Build a suite containing routine requests, ambiguous inputs, conflicting sources, prompt-injection attempts, stale user state, tool failures and long-context stress cases. Track task success, groundedness, citation accuracy, schema compliance, latency, cost and escalation quality. Then inspect failures by component: was the fact absent, retrieved but buried, contradicted by stale state, or ignored after a tool error? That diagnosis turns an unreliable answer into an actionable engineering change.

Operate context as a versioned product

Production context should be observable and reproducible. Log the instruction version, retrieved document IDs, state snapshot, tool calls, validator results, model version and token counts, subject to privacy and retention rules. A response that looked correct yesterday may change after an embedding upgrade, policy edit or model release. Without an execution trace, teams debate anecdotes; with one, they can replay the request and identify the changed dependency.

Use staged releases and component-level ownership. Retrieval changes can be shadow-tested against live queries without affecting users, while new instruction sets can be deployed to 5 per cent of traffic and compared with a control. Define rollback thresholds before launch: for example, revert if grounded answer accuracy falls by two percentage points or p95 latency rises above three seconds. Human review should focus on high-risk samples and novel failure clusters rather than random output alone.

The playbook is ultimately selective. Include rules that constrain behaviour, examples that clarify boundaries, evidence that answers the question, state that remains valid and tool results that reflect reality. Exclude material added merely because the window permits it. Dependable AI apps do not ask models to reconcile a digital attic; they present a compact case file, establish which facts govern and verify the resulting action.