Inherited software work exposes a gap in coding agents

When a coding agent inherits work that another agent marked complete, can it identify what is still wrong or missing? Clean-start tests do not answer that question, because they give the agent a task without earlier work on it.

SWE-Residual is a benchmark that tests whether coding agents can finish tasks from earlier work that was submitted as complete but remains faulty.

The authors created SWE-Residual, a benchmark of earlier attempts that were submitted as complete but still failed the original hidden tests. Evaluated agents received the original request and inherited state without an explicit failure verdict; matched Clean Starts used the same task conditions.

All five agents had lower success from inherited states on both repository tasks and machine-environment tasks. Since the request, verifier, harness, prompt, and budget were matched, the results show that success on clean starts did not carry over reliably to these inherited states.

The measurements cover 198 states from 111 tasks, drawn from two source tests and three predecessor model families. The acceptance-bias analysis is limited to DeepSWE, where the needed per-test outcomes and final code changes were available; other kinds of handoffs were not tested.