The Evolution of Context Compression in AI Agents

AI agents are a cycle of message exchange between a user and a language model (LLM), where the model can call tools or MCPs. Each interaction appends to the conversation history, which is typically sent in full to the next model call. However, context windows, while large, are not unlimited, and performance degrades with large, poorly structured histories. The classic 'Lost in the Middle' paper shows a U-shaped curve: models use the beginning and end of long inputs better than the middle. The article traces the evolution of approaches to compress context: simple tail trimming (rolling window) in LangChain via trim_messages; summarizing old messages into a summary block using SummarizationMiddleware in create_agent; then protecting the first N messages and the fresh tail while compressing the middle, as seen in Hermes Agent and OpenRouter's middle-out compression. The newest approach separates the stored history from the working context: the full history is kept externally (e.g., in a file), the model call receives a view with the fresh tail plus a summary, and the agent is given tools like read_file or grep to retrieve details from the full history. LangChain's Deep Agents library implements this in its SummarizationMiddleware, though the name is confusingly shared with the basic middleware. This evolution is part of the broader trend of context engineering, distinguishing what the agent knows (full history, tool results, skills) from what goes into the next model call.