ChatGPT memory system reverse-engineering

Analyses reverse-engineered ChatGPT memory as a four-layer system rather than vector DB + RAG. - Layers: session metadata, user memory (persistent facts), recent conversation summaries (~15 items), current sliding-window conversation. - Session metadata: device, location, time, subscription level and usage patterns. - User memory: long-term stored facts saved explicitly or auto-identified. - Recent summaries: lightweight summaries of the last ~15 dialogues with timestamps and snippets. - Current window: full immediate conversation with sliding eviction. - Efficiency: approach reduces retrieval costs and latency compared to full RAG searches while preserving coherence.
Key insights
ChatGPT uses a four-tier memory architecture instead of RAG over large vectors
Reverse-engineering suggests ChatGPT’s memory comprises four layers: session metadata (environmental signals), a persistent user memory of explicit facts, a recent-conversation summary buffer (about 15 items), and the full current-session sliding-window context. The system reportedly avoids a global vector DB and RAG for every query, favoring this layered structure to balance personalization and engineering cost.
Design trades off exhaustive recall for efficiency and perceived continuity
The architecture stores core persistent facts for personalization, light summaries to track recent topics, and a sliding window for immediate detail. This ‘remember the important, summarize the recent, focus on now’ strategy aims to provide coherent multi-session experiences while keeping latency and compute manageable for large user bases.