By late 2024, chants proclaiming “RAG is dead” reached a fever pitch. Many developers believed that stuffing entire codebases and documents directly into the prompt window would eliminate the need for complex retrieval architectures. Yet production environments soon delivered a sobering reality check: the more data you feed into a model, the easier it derails from its intended path.
In October 2026, technology writer Drew Breunig took the stage at AGNTCon to unpack this pivotal architectural shift. The real evolution of context engineering has not been about expanding capacity, but about surrendering context assembly back to deterministic code. Facing subtle failure modes exposed in long-running tasks, systems must implement decisive, programmatic forgetting mechanisms.
Million-Token Windows Didn’t Kill RAG: Shoveling Everything into LLMs Invites Hallucination
When Gemini 1.5 kicked off the era of massive context windows, a million tokens rapidly became table stakes across the industry. Meta subsequently pushed open-source boundaries with Llama 4 Scout, offering tens of millions of tokens in context. Yet that capacity never translated into corresponding production adoption—or as Breunig put it, the community spent months waiting for a successor that never arrived.
Enormous context windows did not eliminate retrieval; they laid bare the fragility of brute-force prompting. As mountains of uncurated documentation flood the context window, attention mechanisms disperse and recall accuracy degrades precipitously. The premise that a foundation model can effortlessly sift signal from noise on its own collapses under real-world workloads. Long context is a powerful low-level capability, but it is far from an out-of-the-box panacea.
Furthermore, computational resource consumption scales non-linearly over prolonged agent execution. Every subsequent tool call must repeatedly re-read the ever-growing token history, incurring prohibitive latency and GPU memory overhead. Flooding prompts without upstream filtering sends enterprise infrastructure costs spiraling out of control. Engineering harnesses had to reclaim control of the information pipeline.
From Obsessing Over Tea to Negative Example Poisoning: Four Classic Failure Modes
Faced with runaway contexts, Breunig identified four distinct failure modes in 2025: context poisoning, context distraction, context confusion, and context clash. If an autonomous system falls into any of these traps, its downstream logic quickly deteriorates.
Figure: The four classic failure modes of long context: Poisoning, Distraction, Confusion, and Clash. Source: Drew Breunig’s talk “More Context, More Problems”
A striking example of context poisoning was documented in a Google DeepMind technical report. In the original Pokémon Red, players bribe a guard with lemonade. In the remake FireRed, the requisite item was changed to tea. When Gemini 2.5 Pro played the game autonomously, a prior knowledge trigger convinced the model that it absolutely had to find tea.
The model logged “find tea” onto its internal task list and spent hours scouring the game world in vain. Once an erroneous assumption enters context history, the agent can no longer escape it. The appendix revealed an even colder truth: after context expanded past 100,000 tokens, the agent began mechanically repeating past actions, completely losing the ability to formulate new paths.
Figure: Gemini 2.5 Pro stubbornly searching for tea due to misleading prior knowledge, illustrating context poisoning. Source: Google DeepMind technical report, cited by Drew Breunig
Context distraction was rigorously demonstrated by Carolina Camassa and Derek Shiller in their paper Do as I Say, Not as I Do. Researchers tasked models with translating user inputs into Russian, while seeding the context with between 1 and 50 French negative examples. The results showed that Gemma 3 12B suffered a 50% drop in instruction following after seeing just a handful of counter-examples. Even frontier models like Opus 4.6 saw steep performance degradation as counter-examples accumulated.
Figure: As counter-examples accumulate in the context, instruction following across frontier models degrades steadily. Source: Camassa & Shiller paper, cited by Drew Breunig
This experiment mirrors a lethal everyday pitfall for coding agents. While troubleshooting frontend styles, an engineer might iterate and feed a dozen failed CSS snippets back into the model. Those debugging scraps are negative examples manually injected by the developer, directly eroding adherence to core system instructions.
Context confusion and clash are equally pervasive. Models frequently jump to ungrounded assumptions based on user profile snippets—prompting Breunig to disable commercial memory features altogether to keep casual banter from corrupting technical reasoning. Meanwhile, loading hundreds of tools indiscriminately into an agent harness creates namespace overlaps and schema clashes that paralyze decision pipelines.
Reclaiming Control in 16 Lines: Rebuilding Assembly with Deterministic Logic
Terminology has shifted rapidly over the past three years: from prompt engineering in 2023, to context engineering in 2025, and now harness engineering in 2026. While the buzzwords change, the core engineering imperative remains identical: strip context assembly away from probabilistic model calls and return it to deterministic code.
The architectural toolkit for reigning in context bloat has remained remarkably stable over the last six years, built on six foundational mechanisms: augmenting, loading, isolating, pruning, offloading, and summarizing. These techniques correspond to targeted retrieval, schema shrinking, sandboxed execution, dead-weight removal, external persistence, and semantic compression.
Exposing overly broad interfaces is a prime trigger for system collapse. One enterprise platform loaded 600 tool definitions by default on startup, immediately overwhelming the agent. By introducing a simple filtering function in code that loaded tools strictly on-demand, the engineering team instantly restored system reliability.
Architectures built on DSPy neatly illustrate this shift in authority. The core loop of a production agent is essentially a standard while loop wrapped around a ReAct decision module. The primary reasoning chain runs on a frontier LLM, while tool discovery and schema pruning are delegated to lightweight, specialized models.
Isolation spawns dedicated sub-agents in clean sessions, while pruning trims redundant JSON fields after every interaction round. Wiring augmentation, dynamic loading, pruning, and summarization into the main loop required only about 16 lines of code. This deterministic control flow not only slashes token overhead, but also maximizes prefix prompt caching hit rates.
Behind Seven Consecutive ALL-CAPS Commands: Adversarial Weights and Context Inertia
Entering 2026, long-running agents encountered two novel failure modes: adversarial weights and context inertia. These systemic hurdles prove that prompt tuning alone has hit an inescapable ceiling.
Adversarial weights stem from deep behavioral biases baked in during base model pretraining. When a lightweight model controls a browser via MCP tools like Playwright, its first invocation frequently fails and requires self-correction—simply because the tool signatures diverge from the browser APIs seen during training.
The telltale sign of fighting adversarial weights is prompt shouting: repeated, all-caps instructions. In the Claude Code system prompt, the directive “run independent tool calls in parallel” is repeated seven times in uppercase, alongside separate reminders repeated four, three, and two times elsewhere. Because underlying model weights naturally bias toward serial tool calling, external prompts must bludgeon the model with dense repetition to counteract pre-trained priors.
Context inertia poses an even stiffer challenge for multi-hour autonomous runs. When an agent hits an obstacle, false hypotheses recorded in earlier turns act as anchors, locking the model into unproductive dead ends. Concurrent research from UC Berkeley and Microsoft Research highlighted the severe cost of this fixation.
In Microsoft’s Fork-and-Flush paper, agent performance curves consistently hit stepped plateaus. Mid-to-late run, the agent’s actions devolve into trivial code churning, utterly paralyzed by initial assumptions preserved in history. The very perseverance cultivated via reinforcement learning becomes a straitjacket in multi-step exploratory workflows.
The engineering remedy is surprisingly blunt. Anthropic discovered during extensive benchmarks that the single most effective intervention was a complete context flush every 90 minutes. By persisting the file system and artifacts while wiping conversation history clean, the system forces the model to survey the project environment with fresh eyes.
UC Berkeley’s paper Scaling Discovery Requires Forgetting reached the exact same conclusion: introducing scheduled forgetting into agent pipelines is necessary to sustain upward progress curves. When developers complain that an agent has turned “sluggish,” context has typically swelled to 800,000 tokens. Breunig sets his hard ceiling at 200,000 tokens, beyond which conversation history should be aggressively purged.
Surviving 12-Hour Horizons: Knowing Exactly When to Discard the Past
The time horizons used to evaluate AI agents are expanding exponentially. Under METR benchmarks, GPT-5 sustained 3.4 hours of continuous autonomous execution; Gemini 3.1 reached 6.4 hours; and Opus 4.6 crossed the 12-hour threshold. As autonomous tasks stretch into multi-day distributed swarms, context management faces unprecedented scrutiny.
Always-on personal agents have already shown early signs of “memory rot.” Tracking an instance of the Hermes agent running continuously for six months, Breunig observed that its autonomously generated skill library degraded over time. Ad-hoc patches contradicted one another, generic utilities were repeatedly duplicated, and entropy in the memory layer steadily degraded execution efficiency.
Unconstrained accumulation of history eventually causes decision networks to collapse under their own weight. Systems require strategic forgetting. Mastering when to flush stale tokens is the defining unsolved challenge of long-horizon autonomy.
An LLM’s output can be formalized as the product of its context and internal weights. When facing reliability challenges, teams can either wait for the next foundation model release or systematically optimize their context assembly pipelines. The latter costs an order of magnitude less and remains the one variable engineering teams directly control.
Context engineering has matured past the naive pursuit of ever-longer prompt windows. Resilient architectures refuse to bet the farm on probabilistic reasoning alone; they construct rigid execution guardrails out of deterministic code. Knowing precisely when to take over assembly—and having the courage to discard history—is what keeps long-running agents standing.
References:
- Drew Breunig’s Blog & AGNTCon Keynote: The State of Context Engineering
- Google DeepMind Technical Report: Gemini 2.5 Pro
- Carolina Camassa & Derek Shiller: Do as I Say, Not as I Do
- Microsoft Research: Fork-and-Flush
- UC Berkeley: Scaling Discovery Requires Forgetting
