XNews

Warp's Ben Holmes Presents Self-Improving Agent Loop at AI Engineer

Warp's Ben Holmes Presents Self-Improving Agent Loop at AI Engineer

Ben Holmes of Warp presented a self-improvement loop for AI agents at AI Engineer, in which agents review their own work and refine their skills and configuration over time. The talk described a workflow where a reviewer agent scores task records against team-defined criteria, and a second agent aggregates recurring problems into proposed skill or configuration changes for team review. Holmes cited a case in which an outdated port instruction in a skill caused repeated trial and error that went unreported.

Event version 1.

Key insights

Successful completion can hide process problems

An agent can complete a task after bypassing faulty instructions and repeated attempts while reporting only the final result. In one case, an outdated port instruction in a skill caused trial and error that the team saw only as slight slowness.

Review requires full task records

Review should combine the final answer or code with conversation, tool calls and code changes. Otherwise tasks appear successful while the same problem is rediscovered on later runs.

Scoring criteria are defined by the team

Teams define explicit criteria such as repeated testing, tool-call efficiency and verbosity. A reviewer agent checks task records and explains its deductions.

Recurring causes become reviewable changes

A second agent aggregates review records to find recurring problems and turns fixes into pull requests modifying skills or configuration for team review. The focus is on removing shared causes rather than making individual agents temporarily smarter.

Two practices demonstrated

Skill Doctor performs one-time checks of local history. Warp Factories demonstrates continuous scoring with periodic fix proposals.

Source posts