HarnessPro improves agent success at lower cost

When an AI agent succeeds more often after adding reasoning or checks, does each task also have to cost more? Cutting those steps indiscriminately can hurt success, so the challenge is to find which work helps and which adds avoidable expense.

HarnessPro uses execution feedback and generated tests to evolve agent harnesses, selecting options that balance task success and execution cost.

On Terminal-Bench 2, a test of terminal tasks, pass rate measures how many tasks pass all official tests. With GPT-5.5 and Codex, HarnessPro’s performance-focused option averaged an 80.8% pass rate and $0.72 per task across three runs. RHO recorded 76.3% and $1.09 per task in the reported comparison.

This result shows that HarnessPro achieved higher task success at lower mean cost in this setup.

The wider evaluation also covers SWE-Bench Pro, which tests repository-level issue resolution, and ProgramBench, which tests program reconstruction, and uses a second model-and-agent setup; those results provide additional tested settings rather than extending the Terminal-Bench 2 figures to every system.

These results cover three tests and two model-and-agent setups, with updates developed using 10 training tasks per test. The Terminal-Bench 2 figures are means across three HarnessPro runs; they do not establish the same differences in other settings.