Context Engineering

Context engineering refers to the systematic design, testing, and iterative improvement of all information provided to an AI agent as context. It goes beyond classic prompt engineering: while prompting optimises individual queries, context engineering addresses the entire information architecture of an agent-based system.

The term was coined to distinguish the practice from simpler prompt writing and to make the growing complexity explicit: an agent with multiple tools, persistent memory, and subagents requires far more than a well-crafted opening prompt.

What Context Engineering Covers

The interplay of these layers determines whether an agent acts reliably and predictably. Missing or unclear context engineering leads to the agent aborting, selecting wrong tools, skipping steps, or hallucinating outputs.

A Case Study: The Deep Research Agent

Problem 1: Incomplete task execution: The agent creates three search tasks but only executes two of them, without explanation. Cause: the system prompt contained no explicit requirement to execute all planned tasks.

Problem 2: Inconsistent status values: The agent spontaneously used various status labels (pending, to-do, done, completed, finished), making automated downstream processing unreliable. Cause: allowed status values had not been defined.

Both problems are not model errors but context errors. The model did exactly what its instructions permitted.

EXECUTION RULES:
- Allowed status values: todo | in_progress | done | failed
- Every planned search task must either be executed or explicitly
  justified as unnecessary before being marked complete.
- Do not skip tasks silently.

Core Principles of Effective Context Engineering

1. Eliminate Ambiguity

Insufficient:

Research the given topic.

Better:

Research the topic by:
1. Breaking the query into 3 to 5 specific search tasks
2. Using the search tool for EACH task
3. Documenting findings in the task tracker
4. Synthesising all findings into a structured report

2. Make Expectations Explicit

Agents do not infer implicit expectations. Requirements for quality, format, and decision logic must be stated explicitly as rules, not as comments or suggestions.

3. Define Error Behaviour

ERROR HANDLING:
- If a search fails: retry once with a rephrased query.
- If the second attempt fails: mark the task as "failed" and continue.
- Inform the user about failed tasks at the end of the run.

4. Work Iteratively

5. Balance Flexibility and Control

Common Mistakes in Context Engineering

Over-specification: Too many rigid rules make the agent inflexible and unable to handle unforeseen situations. The goal is precise guidance, not an exhaustive rulebook.

Under-specification: Vague instructions lead to unpredictable behaviour, particularly dangerous when the agent faces critical decisions such as status values or stop conditions.

Missing error handling: Without explicit instructions for failure cases, the agent either aborts or continues silently; both outcomes are unacceptable in production systems.

Ignored observations: Context engineering only works if deviations are captured systematically and used as the basis for refinement. An agent that is not observed is not improved.

Measuring Effectiveness