A benchmark measures when reassurance leaves advice goals unmet

When someone asks for advice, can a response sound supportive yet leave the practical goal unresolved? And can a model make the opposite mistake by offering solutions when the person wants emotional support? These questions call for measuring whether an answer fits the goal stated in its scenario.

The Goal Displacement Benchmark tests whether practical help remains after reassurance is removed, while separately checking when problem-solving crowds out requested support.

The authors tested GDB on 18 conversational models, prompting each once with 230 scenarios and having two study authors score the answers. For each model, they measured the share of 130 hard-constraint scenarios coded as displaced under the averaged-rater threshold.

The range, from 0.0% to 37.7%, shows that the test distinguished models on this response failure in the evaluated set.

The distress-only cases captured the reverse problem: unsolicited problem-solving when support was the stated goal. Together, the separate measures show that practical neglect and unwanted advice are distinct behaviors the benchmark can record.

The scenarios were written in English by the authors, include many US institutional contexts in high-stakes cases, and test only a model’s first response. A low displacement score does not establish that the advice is correct or safe.