Context Compression, RAG and Long-Running Agent Evaluation

Researchers and practitioners stress context compression and RAG improvements for long-lived agents. - Single-session compression cannot preserve task-critical details over million+ token histories. - FactoryAI anchored iterative summarization outperformed OpenAI and Anthropic on probe-based retention tests. - Probe evaluation uses Recall, Artifact, Continuation, Decision and six scoring dimensions via GPT-5.2 judge. - Efficiency should optimize per-task token consumption, not maximal compression ratio. - Box CEO Levie argues context engineering becomes the primary system-level barrier for agents. - Key infrastructure: retrieval, heuristic ranking, system prompts, and state/space optimization. - Agentic RAG for Dummies project adds dialogue memory, hierarchical indexes, and production features for RAG.
Key insights
Context compression must preserve actionable details
Evaluation shows conventional high-compression approaches drop low-entropy yet task-critical details (e.g., file paths, error logs). A probe-based framework measures recall, artifact tracking, continuation, and decision support; it uses GPT-5.2 as an evaluator and scores six dimensions from 0–5. FactoryAI's anchored iterative summarization preserves intent, file edits, decisions, and next-step plans better than OpenAI's opaque compact responses and Anthropic's full-regeneration summaries. The reporting emphasizes optimizing token use per ongoing task rather than maximizing compression percentage.
Context engineering becomes a core system architecture problem
Box CEO Levie argues agent failures will shift from logical deficits to information asymmetry. He outlines an information supply chain requiring large-scale, low-latency retrieval; heuristic sorting to prioritize critical context; precise system prompts; and state/space optimizations to fit complex reasoning into limited windows. He highlights AI's computation advantage for scanning and reorganizing data and positions context engineering as a competitive barrier for production agents.
Production RAG requires dialog memory and hierarchical indexing
The Agentic RAG for Dummies project demonstrates production-grade RAG by adding dialog memory, hierarchical index slicing, and a small-to-large retrieval strategy: search with small shards then callback with large-context slices. The system supports multi-agent orchestration, clarifying questions to avoid hallucination, Gradio UI, document management, dialog persistence, human intervention hooks, PDF-to-Markdown pipelines, Docker deployment, and notebook tutorials for local deployment.
Policy-level implications for token-efficiency and probe-based evaluation
The practical recommendation is that compression metrics must reflect task-continuity effectiveness: per-task token consumption and artifact traceability matter more than raw compression ratios. The probe-based evaluation—testing specific retained facts after compression—outperforms ROUGE/embedding similarity for assessing real agent usability. The analysis reports Factory's higher accuracy and context awareness despite lower raw compression than OpenAI's method.