XNews

Context engineering: file-as-context and dynamic context discovery

Context engineering: file-as-context and dynamic context discovery

Cursor published 'Dynamic Context Discovery', promoting a files-as-context model and five optimizations to reduce token use while improving agent reliability.

Key insights

Cursor's dynamic context discovery reduces token costs and preserves detail

Cursor presented Dynamic Context Discovery: agents should not preload all possible context; instead, convert long outputs and histories into files and let agents fetch or tail them on demand. Cursor detailed five optimizations: write long tool outputs to files for on-demand reading; store full chat history as searchable files while summarizing the active prompt; load Agent Skills by directory index then fetch full docs when needed; slim MCP tool descriptions to names with on-demand fetch; sync terminal output to files for grep and history queries. Cursor reported a 46.9% token reduction in MCP tool scenarios using this method.

Files-as-context abstraction aligns with Unix 'Everything is a file' philosophy

A separate paper and engineering proposals recommend abstracting all agent context (memories, tools, chat logs, docs) as files managed via a unified agent file system. The AIGNE framework defines a Context Constructor (manifest selection and prioritization), Context Loader (reads/streams content into prompts), and Context Evaluator (validates outputs and writes new memory artifacts back to the file system). This 'everything-as-file' approach aims to standardize context handling and improve traceability and maintainability.

Multiple open-source tools target long-term agent memory and context stores

Projects include memU, an agent memory framework with hierarchical Resource→Item→Category layers, dual retrieval (RAG + LLM), multimodal support and evolving memory folders; and Acontext, a context data platform offering unified store, observation and learning modules for artifact management, agent monitoring, and experience-driven skill extraction. Both projects provide APIs and local/cloud deployment options.

Dynamic loading of skills and tools improves robustness and reduces context bloat

Cursor and Anthropic-related engineering posts recommend keeping only a directory (index) of skills and tool names in system prompts and dynamically fetching full skill/tool docs when the agent chooses to use them. This preserves prompt space, reduces token costs, avoids stale authentications causing silent failures, and lets agents proactively surface re-authentication needs.

File abstraction trade-offs and limits noted

Authors note limitations: heavy reliance on file abstraction, potential information loss in summaries despite files as backup, need for extra adapters for third-party tools, and variable returns depending on project/tool counts.

Source posts