A clause-based test exposes conflicts with professional ethics rules

Can an answer follow a professional request in fluent language yet still breach the specific rule governing that work? Broad harm categories do not identify the clause at issue, and tests made only of improper requests cannot distinguish conformance from a model that refuses everything.

TenetBench pairs clause-conflicting requests with benign twins and checks model answers against the named professional rule.

TenetBench tests whether language-model answers conflict with a named professional-code clause while also checking whether models serve legitimate requests.

It pairs each conflicting request with a benign twin on the same scenario, then uses a frozen language-model judge to label action, clause conflict, and refusal or warning separately.

Against adjudicated human labels, the judge showed high agreement on the annotated sets. In the campaign, 2,622 of 3,906 answers to conflicting tasks were judged to conflict with their clause, and eleven of fourteen model-and-reasoning rows conflicted on more than 60% of the 279 tasks.

This supports a diagnostic finding about the tested answer text, not a general grade of professional conduct.

These findings concern generated answer-text tasks from Russian professional codes in one legal system, and the model analysis covers nine of ten harm domains. The test does not assess answer quality or duties breached only by a specialist’s later actions.