Scaling your coding agent’s context beyond a single AGENTS.md-file
How to help your AI agents handle large codebases with structured, multi-layered context
TL;DR LLM coding agents forget context and repeat mistakes across sessions. Small files like AGENTS.md help in small projects but don’t scale.
A new paper (arXiv 2602.20478) shows a three-layer context system for a 108k-line C# project:
1️⃣ Hot memory – always-loaded rules and conventions
2️⃣ Specialized agents – 19 domain-specific agents
3️⃣ Cold memory – 34 retrievable specification documents
This setup keeps agents consistent, reduces errors, and allows them to coordinate effectively.
Key takeaway: Treat project knowledge as infrastructure, separate rules from detailed docs, encode domain knowledge, and record debugging knowledge. First step: improve your AGENTS.md (guide).
Why a single AGENTS.md isn’t enough for large projects
One of the core limitations of LLM-based coding agents is that they lack persistent memory. Across sessions they lose coherence, forget project conventions, and often repeat the same mistakes. This is one reason developers started introducing files like AGENTS.md or CLAUDE.md: a small set of instructions that every agent run loads before starting work.
If you are interested in how to write a effective AGENTS.md file itself - I wrote about that here: https://ursula8sciform.substack.com/p/how-to-improve-your-agentsmd-file.
For small repositories this works surprisingly well. But once a codebase grows, a single file becomes too limited. Architectural intent, subsystem behavior, and failure modes simply do not fit into one document.
How to scale the context of coding agents
A recent paper proposes a practical way to address this problem:
“Codified Context: Infrastructure for AI Agents in a Complex Codebase.”
https://arxiv.org/pdf/2602.20478
The paper describes the development of a 108,000-line distributed C# system built with extensive use of AI agents. Instead of relying on a single instruction file, the authors created what they call a codified context infrastructure: a structured layer of machine-readable knowledge designed to support multi-agent AI-assisted development.
The system is organized into three layers.
1️⃣ Hot memory — the “constitution”
A compact always-loaded layer containing coding conventions, architecture rules, retrieval hooks, and orchestration protocols.
The first layer is “hot memory,” sometimes described as a constitution. This is a compact document that is always loaded when the agent runs. It contains coding conventions, architectural rules, instructions for retrieving additional information, and protocols for how different agents coordinate their work. Because this layer is small and stable, it provides a consistent starting point across sessions.
2️⃣ Specialized domain agents
19 expert agents responsible for different parts of the system.
The second layer consists of specialized domain agents. In the project described in the paper, the developers created 19 expert agents, each responsible for a particular subsystem or concern. Instead of expecting a single general-purpose agent to understand the entire system, tasks are routed to agents that carry focused knowledge about specific parts of the architecture.
3️⃣ Cold memory - retrievable knowledge
A knowledge base of 34 specification documents that agents can retrieve on demand.
The third layer is cold memory, a knowledge base of documents that can be retrieved when needed. In the reported project this consisted of 34 specification documents describing subsystems, interfaces, and design decisions. These documents are not always loaded into context but can be fetched on demand when a task requires deeper understanding.
The authors tracked the evolution of this context infrastructure over 283 development sessions, analyzing how agents interacted with these context artifacts. Their observations suggest that codified context helps propagate knowledge across sessions: architectural constraints remain visible, previously discovered bugs are less likely to reappear, and agents can coordinate work across different subsystems more consistently.
Practical lessons for developers working with agent-based workflows
1️⃣ Treat project knowledge as infrastructure rather than documentation
Traditional documentation is often optional and quickly becomes outdated. In an agent-driven workflow, these artifacts become operational components that directly influence the quality of generated code. Instructions, architectural constraints, and conventions should therefore be maintained with the same care as the code itself.
2️⃣ Separate different kinds of context
A small, stable “constitution” should encode conventions, architectural boundaries, and general rules. Larger explanations, design notes, and subsystem descriptions should live in retrievable documents instead of being forced into one large instruction file. This keeps the always-loaded context small while still making deeper knowledge accessible when needed.
3️⃣ Capture domain knowledge explicitly
If a subsystem has complex rules or recurring tasks, it can be helpful to encode this knowledge in a dedicated agent specification or specialized skill. This allows the system to route tasks to an agent that already understands the relevant domain instead of forcing a general-purpose agent to rediscover the logic each time.
4️⃣ Record failure modes and debugging knowledge
Agent runs often reveal subtle architectural constraints or recurring failure patterns. When such issues appear, documenting them once — for example as rules, checklists, or debugging notes — can prevent the same mistakes from appearing again in later sessions.
This approach does not eliminate the need for good software engineering practices. In fact, it reinforces them. Clear architecture, modular subsystems, and well-written specifications all make it easier for agents to reason about a project.
In other words, moving beyond a single AGENTS.md file is less about adding more prompts and more about designing the knowledge layer around your codebase.
If you are interested in improving the first step - the AGENTS.md file itself - I wrote about that here: https://ursula8sciform.substack.com/p/how-to-improve-your-agentsmd-file!
Enjoy the read, and let me know in the comments which improvements you were able to implement in your projects!
Hi Ursula, thanks for sharing - not my world but I can see this would be incredibly useful for anyone working with AI coding tools at scale. 🙌🏼
This is really helpful🫡 Funny thing is i just created an agents. Md on my personal project because i got tired of repeating myself , and i still had to repeat myself on certain occasions. Now i have a new challenge