Microsoft's ParetoSaddler Optimizes Agent Harnesses for Quality and Cost

Microsoft researchers introduced ParetoSaddler, a harness optimizer that separates intermediate candidate promotion from final deployment selection to jointly improve agent quality and cost. On ClawMark, quality rose from 70.86% to 81.35% while cost per task fell from $1.17 to $1.01; on GAIA2, quality rose from 56.22% to 62.89% while cost fell from $0.78 to $0.45.
Event version 1.
Key insights
Deployment preference applied too early
Applying the deployment preference during intermediate optimization steps prematurely rejects candidate harnesses that are valuable for later improvements. ParetoSaddler separates intermediate candidate promotion from final deployment selection to address this mismatch.
Bayesian bootstrap Pareto promotion
At each iteration, an optimizer edits a parent harness into a candidate, runs both repeatedly, and uses Bayesian bootstrap Pareto support to promote candidates showing credible progress in either quality or cost. Promoted candidates receive broader development evaluation and remain available as parents.
Quality and cost gains on two benchmarks
On ClawMark, quality improved from 70.86% to 81.35% while cost per task dropped from $1.17 to $1.01. On GAIA2, quality improved from 56.22% to 62.89% while cost per task dropped from $0.78 to $0.45, using domain-disjoint splits.
Outperforms quality-only optimization
Pareto-selected harnesses achieve higher quality at lower cost than harnesses from quality-only optimization. Analysis shows Pareto-aware optimization retains intermediate candidates that deployment preference would discard but that support later improvements.