XNews

AI Agent Research: Comparison of PRAXIS and Web Interaction Interfaces

AI Agent Research: Comparison of PRAXIS and Web Interaction Interfaces

Two studies propose a real-time memory mechanism for agents and compare the efficiency of web interaction interfaces. - PRAXIS (Procedural Recall) records the actions and consequences of an agent, retrieving experiences indexed by both external environment and internal state. - PRAXIS is a lightweight plugin that does not require retraining large models, improving task completion rates on web tasks while reducing token usage and operational costs (performing better on the REAL Web Browsing Benchmark). - The Mannheim team compared four web interaction methods: HTML, RAG, MCP, and NLWeb, constructing a WebMall to test four simulated e-commerce websites. - Experimental results: HTML F1=0.67, average F1 for RAG/MCP/NLWeb ≈0.75–0.77; GPT-5+RAG F1=0.87, with a task completion rate of nearly 80%. - Cost differences: HTML consumes about 240,000 tokens on average and takes 291 seconds; other methods require only 470,000 to 140,000 tokens and 50 to 60 seconds. - Cost-effectiveness conclusion: GPT-5-mini + RAG is identified as the best balanced choice in terms of cost and performance. - The research suggests that future websites should provide MCP/NLWeb or optimize to adapt to RAG to reduce agent parsing costs and improve efficiency.

Key insights

PRAXIS Addresses Agent 'Amnesia' with State Indexing

PRAXIS proposes a mechanism to record every action and consequence of the agent in the background during its operation, using 'state' (external environment + internal agent state) to establish an index. When encountering similar scenarios, the system retrieves past experiences to assist in decision-making. This mechanism is a lightweight plugin that does not require retraining large models, significantly improving the completion rate of web browsing tasks, reducing ineffective attempts, and thus lowering token consumption and operational costs (based on PRAXIS descriptions and REAL Web Browsing Benchmark experimental reports).

Dual Indexing (Environment + Internal State) Enhances Experience Matching Accuracy

The key technology of PRAXIS is dual indexing: matching both external interface features and the agent's internal plans and states, ensuring that the retrieved experiences are more contextually and executionally aligned, avoiding erroneous reproductions caused by rough surface matching. This design allows the agent to adjust its strategy based on analogy rather than repeated failures when facing updated interfaces or unseen tasks.

Mannheim Comparison Shows Direct HTML Parsing is the Least Efficient

The Mannheim team compared four interaction methods on the WebMall platform: HTML (directly reading source code), RAG (retrieving relevant snippets), MCP (standard API calls), and NLWeb (natural language Q&A/JSON responses). The results showed that HTML performed the worst, with F1=0.67, and had significantly higher token consumption and time compared to other methods (HTML averaging 240,000 tokens and 291 seconds; other methods requiring 470,000 to 140,000 tokens and 50 to 60 seconds).

Structured Interfaces (RAG/MCP/NLWeb) Significantly Improve Efficiency and Cost-Effectiveness

The experiments indicated that the average F1 for RAG, MCP, and NLWeb was between 0.75 and 0.77, combined with lower token usage and faster execution times. GPT-5 + RAG achieved F1=0.87, with a task completion rate close to 80%; in terms of cost-effectiveness, GPT-5-mini + RAG was identified as the best balanced combination, suitable for scenarios aiming for high completion rates within budget constraints.

Research Indicates Future Websites Should Build 'AI-Friendly' Interfaces

The paper concludes that for websites to be efficiently utilized by AI agents, they should provide structured interfaces like MCP or NLWeb, or optimize to adapt to RAG indexing. Allowing agents to 'hard chew' HTML will lead to high costs, slow speeds, and low accuracy; the research emphasizes the necessity of reconstructing internet interaction channels to cater to agents.

Engineering Practices and Community Guidelines Supplement Methodology for Building Agents

In addition to academic comparisons, another practical article summarizes engineering insights that integrate Anthropic, GitHub, and Docker, emphasizing starting with simple workflows and avoiding over-design, providing practical guidelines for building robust agents (supplementing research perspectives).

Source posts