XNews

Agent Retrieval & Memory: Low‑latency search and context compression for long tasks

Agent Retrieval & Memory: Low‑latency search and context compression for long tasks

New evaluations show structured, anchored summarization and millisecond-level retrieval are critical for long-running agent tasks.

Key insights

Compression quality must preserve actionable artifacts, not maximize ratio

Evaluations of agent context compression found extreme compression (high ratio) often discards low-entropy yet actionable details like file paths, error traces and decision artifacts. FactoryAI’s 'anchored iterative summarization' maintained structured persistent summaries (intents, file changes, decisions) and outperformed generic compacting approaches by preserving continuity and enabling agents to continue tasks without rework.

Probe-based evaluation better measures compression utility for agents

Researchers proposed probe-based evaluation to test compressed context for recall, artifact retention, continuation and decision traceability. The framework uses targeted queries post-compression to validate whether key elements required for task continuation remain accessible, rather than relying on ROUGE or embedding similarity alone.

Opennote’s Index emphasizes millisecond retrieval and hybrid filtering for agents

Opennote described Index: a retrieval system built on Postgres (pgvector + pg_trgm + tsvector) and Cloudflare edge runtime to deliver P99 <300ms for multi-tenant personal knowledge search. Index combines metadata filtering, hybrid semantic+keyword search, fine-grained caching, chunk-level invalidation and minimal re-embedding to meet agent requirements.

Production retrieval requires global distribution, dynamic updates and precise re-ranking

Opennote’s pipeline supports global Workers-based ingestion, chunk-level hashing to avoid reprocessing, and a dynamic alpha blend of keyword and vector scores. The architecture supports multi-tenant filtering and immediate cache invalidation to maintain correctness for agents operating on evolving documents.

Agent memory design should optimize per-task token cost over raw compression

Authors recommended optimizing for lower end-to-end token consumption across the agent’s workflow rather than maximizing per-call compression percentage; structured anchored summaries and artifact indexing reduce repeated retrieval and generation costs.

Practical indexing choices favor relational engines for dynamic filters

Opennote chose Postgres-based combined retrieval over vector-only stores because relational metadata filtering effectively reduces candidate sets before embedding similarity, improving latency and multi-tenant isolation for agent lookups.

Source posts