A crisis-support test shows recognition is not enough
In parole semplici
DistressBench is an English test for whether general-purpose language models can support people in simulated suicide and self-harm crises. It checks more than refusal: it looks for recognizing distress, reducing escalation, offering useful resources, and avoiding harmful or moralizing replies.
Clinicians wrote the situations and the helpful actions to look for. A three-judge panel marks each action present or absent, and clinicians assign more important actions more weight.
Across 25 models, the strongest delivered 88.4% of the required weighted actions, versus 98.7% for clinician reference replies; this is an evaluation result, not a clinical safety guarantee.
La ricerca
When a model recognizes that someone may be in a suicide or self-harm crisis, is avoiding harmful language enough? A refusal can leave the central work undone: helping reduce immediate escalation and connecting the person with human support.
The unresolved issue is whether current safety checks measure that substantive support or mainly measure what a model avoids saying.
DistressBench replaces refusal-only checks with clinician-authored criteria for the actions a model takes after recognizing a crisis.
DistressBench tests 718 simulated English tasks with clinician-written criteria and a three-judge panel. Each criterion has a clinician-assigned importance weight; researchers scored 25 models three times and averaged the share each response satisfied.
The best model scored 88.4%, versus 71.4% for the median and 98.7% for clinician references. In 35.3% of recognized conversations, models still failed to direct users to help, showing why refusal or recognition alone misses substantive crisis support.
These results describe English, expert-written simulations with U.S.-centric resource criteria, not live crisis chats or real-world harm outcomes.
The clinician reference score shows that the rubric can be satisfied, but it is not a human performance baseline because references were written with the rubric visible.
Analisi
Dettagli
- Licenza
- CC BY 4.0