In plain words
Coding agents can get feedback from running tests but still make edits as if their earlier guess were right. This paper introduces RIPPLE, a way to train one agent to notice when it should rethink, explain what went wrong using the evidence so far, and make a matching correction.
Across four software-engineering tests, the trained agent averaged higher scores than the base agent. One test’s result was uncertain, and the study covered one model and coding setup.
The research
When a coding agent receives evidence that an earlier assumption was wrong, what helps it change its next action? A critique may identify the failure without guiding a repair. This paper asks whether training can connect recognizing a deviation, diagnosing it, and acting on that diagnosis.
RIPPLE trains one coding-agent policy to spot recoverable deviations, diagnose them from available evidence, and make corrective edits.
Researchers trained one policy to trigger reflection, diagnose deviations from the task history, and take corrective actions. They compared it with a base policy on four software-engineering tests that score repository-task resolution.
RIPPLE averaged 4.8 points higher across the four test-specific score differences. Three test intervals excluded no difference, supporting an average gain while leaving the Verified test uncertain.
The average is across four benchmark-specific score differences, and the gain on SWE-bench Verified has an uncertainty interval that includes no gain. The study tests one model in one coding scaffold; it does not establish transfer to other models or domains.
Analysis
Deep
- Original paper authors
- Jingxuan Xu, Zihang Xu, Yuechen Luo, Ziqi Miao, Wenqiang Zhu
- Submitted to arXiv
- License
- arXiv non-exclusive