In plain words
A task can require two things to go right, while a checker scores just one of them. The paper studies what happens when that checker is judged by whether the whole task succeeded, especially after a policy changes which failures it produces.
The authors show that this whole-task score can become misleading. In an ECG pipeline test, refreshing a checker with whole-task labels weakened its ability to catch the component failures it was meant to find, while keeping examples of those failures in refresh data preserved performance in the tested settings.
The research
If a checker is judged by whether the entire task succeeds, can its score still tell us whether it catches the particular failure it was built to detect? This matters when a policy changes the mix of failures in the candidates used to audit or refresh that checker.
The paper derives how full-task audits can misjudge a component verifier, then tests refresh damage and a support-preserving repair.
The authors analyze a verifier meant to check one condition when it is scored against the whole task. Their decomposition shows why the score can shift toward failures of another condition as policy outputs change.
An exact engine oracle's full-task score fell across policy pools despite exact component judgments. In ECG tests, full-outcome refresh weakened off-policy component checking and downstream selection; retaining base-policy failures preserved performance in those settings.
The operational refresh results come from CPU-only tests of one ECG signal-processing task, two verifier types, and ten overlapping subject splits per setting.
The paper did not test refreshing a large language model verifier or running online reinforcement learning with a refreshed verifier, and the ECG results are not clinical validation.
Analysis
Deep
- Original paper authors
- Daniel Cieślak, Jacek Łubiński
- Submitted to arXiv
- License
- CC BY 4.0