Raw-RAG Outperforms Agentic Memory on Long-Term Conversation Benchmarks

A study by Zhengye Bao and co-authors found that Raw-RAG, a naive retrieval baseline using dense retrieval and reranking over raw conversation history, ranked first in six of seven benchmark-model settings across LoCoMo, LongMemEval, MEME and MemoryAgentBench. It outperformed the strongest agentic memory system by 0.13 to 5.53 points, with BEAM-1M ingestion taking 0.3 hours versus 3.2 to 27.0 hours for compared memory systems.
Event version 1.
Key insights
Raw-RAG leads most benchmark settings
Raw-RAG ranked first among compared memory systems in six of seven benchmark-model settings across LoCoMo, LongMemEval, MEME and MemoryAgentBench. It outperformed the strongest compared agentic memory system by 0.13 to 5.53 points.
Ingestion cost is far lower
On BEAM-1M ingestion, Raw-RAG took 0.3 hours, compared with 3.2 to 27.0 hours for MemMachine, MemOS, EMem, Mem0 and Memora. The authors describe this as roughly 10x to 83x cheaper.
Evidence coverage explains the result
Raw history covers more of the evidence a question needs, including relations and dependency paths behind an answer, which offsets lower utilization. Transformation into memory entries tends to drop those elements.
Oracle test favors raw evidence
Even complete access to transformed memory stores left an accuracy gap relative to raw evidence. The authors state that preserving raw information matters more than the structure built on top of it.
Ablations do not surpass the default
Removing reranking reduced performance by 3.92 points, and returning only one passage reduced it by 11.73 points. Adding atomic fact extraction, summary augmentation or graph expansion did not exceed the default Raw-RAG score of 48.32.