Canary contracts reveal where safety evidence gets lost
In plain words
This paper introduces canary contracts, a way to check whether an AI application’s safety monitor receives the information it is supposed to use. The test puts a known signal only where a framework changed, then checks whether the monitor sees it and whether the intended safety action follows.
The authors reproduced 18 cases where the parent version missed the signal and a fixed version restored the expected behavior. Fourteen reached a demonstrated enforcement boundary, such as blocking output or interrupting a tool. These cases do not measure how common such failures are.
The research
When a safety monitor is present, how can we tell whether the framework still delivers the evidence it needs to stop or route an action? A functioning detector cannot answer that alone: the evidence or its positive result may be lost between the framework interface and the decision boundary.
Canary contracts place a known signal on a changed framework surface and check whether the same monitor sees it and makes the expected decision before and after repair.
Each contract compares a parent revision with its fixed revision, placing the same payload only on the changed surface. Researchers check whether the monitor sees it and, where the case involves an action, whether the intended decision follows.
Across 18 confirmed cases in 11 framework project groups, 14 reached a demonstrated enforcement boundary. The paired miss-and-repair evidence shows how a live safety path can lose evidence before enforcement.
The 18 cases were selected for feasibility and reproducibility, so their counts do not estimate how often these failures occur. They show failing-parent and fixed-child contrasts, not that a parent version once worked before it failed.
Analysis
Deep
- Original paper authors
- Junchi Liao
- Submitted to arXiv
- License
- arXiv non-exclusive