XNews

Retrieval and Memory Systems for Agents: Index and Memori

Retrieval and Memory Systems for Agents: Index and Memori

Two engineering projects—Index (Opennote) and Memori—present production approaches for agent retrieval and persistent memory. - Index is a millisecond-level dynamic document retrieval system tailored for agent contexts with P99 <300ms goals. - Index uses Postgres (pgvector+pg_trgm+tsvector) and Cloudflare Workers to enable hybrid metadata and vector filtering, achieving ~209ms P99 in tests. - Index ingestion pipeline supports chunking, hash-based delta embedding, and precise cache invalidation for updates. - Index mixes keyword and vector scores with an alpha blending parameter and uses local re-ranking with Workers AI. - Memori is an open-source SQL-native memory engine exposing one-line integration (memori.enable()) for LLMs and agents. - Memori stores memories in standard SQL DBs, extracts entities, categorizes memory types (facts, preferences, skills), and supports conscious/auto/combined modes. - Memori runs periodic 'awareness' jobs to distill long-term memory into short-term working memory and integrates with popular agent frameworks. - Both projects prioritize ownership, auditability and low latency for personal agent contexts.

Key insights

Index opts for Postgres-based hybrid retrieval to serve agent needs with low latency

Opennote’s Index uses Postgres with pgvector, pg_trgm and tsvector to implement hybrid semantic and lexical retrieval optimized by metadata filtering. The architecture leverages Postgres’ efficient metadata filtering to reduce candidate sets before vector scoring, enabling millisecond-level latency. Global distribution is achieved via Cloudflare Workers + Hyperdrive and an ingestion pipeline that delta-embeds only changed chunks. Index implements a three-layer cache (exact query cache, semantic cache, and reverse-index invalidation) to keep P99 latency consistent while supporting dynamic updates and multi-tenant filters. (supporting_posts: [13530])

Memori provides SQL-native persistent memory with entity extraction and periodic distillation

Memori is an open-source memory engine that plugs into standard SQL databases and provides LLMs and agents with persistent, queryable memories. Integration is minimal (memori.enable()) and Memori automatically extracts entities, classifies memory types (facts, preferences, skills) and exposes conscious/auto/combined modes. It runs periodic 'awareness' background processes that distill long-term storage into compact short-term context to improve retrieval relevance and continuity across sessions. Memori targets auditability, data ownership and compatibility with common agent frameworks. (supporting_posts: [13406])

Engineering patterns: delta embedding, alpha blending and re-ranking improve production relevance

Index’s pipeline emphasizes delta embedding (only recompute embeddings for changed chunks), alpha blending to fuse lexical and semantic scores, and optional light re-ranking at edge using Workers AI. These techniques reduce compute and storage cost while preserving recall and enabling strong artifact tracking for agents. Both Index and Memori focus on guaranteeing up-to-date results despite high edit rates by supporting UPSERT/DELETE semantics and fine-grained cache invalidation. (supporting_posts: [13530, 13406])

Personal agent contexts demand ownership, auditability and millisecond responsiveness

Both Index and Memori prioritize user data ownership (SQL-backed persistence), fine-grained permissioning, verifiable update pipelines and low-latency retrieval tailored to agents that cannot tolerate noisy or stale context. Opennote targets P99 <300ms, while Memori integrates with existing DBs to preserve audit logs and enable user control. These design choices reflect production requirements for consumer-facing agents that handle private, high-sensitivity documents and expect real-time responsiveness. (supporting_posts: [13530, 13406])

Source posts