In plain words
A safety guardrail may judge a user prompt or answer correctly but struggle when the same safety situation is shown as a conversation or a record of an agent using tools. Its confidence can also become unreliable when the format changes.
UniGuard trains a guardrail using several versions of each safety case, teaching it to recognize the case across formats and make confidence better match accuracy.
On three tests of unfamiliar agent or rule formats, the trained model improved both classification and confidence reliability over its pretrained base model.
The research
A guardrail may work on prompts or responses but struggle when the same safety case appears in a conversation, an agent’s tool-use record, or under a changing rule. Can training on several formats of the same case help it keep both its safety judgments and confidence reliable?
UniGuard trains a guardrail on aligned formats of the same safety case, aiming to improve judgments and confidence when the interface changes.
Can a guardrail keep making reliable safety judgments when the same case is presented in a different format, or under a changing safety rule? UniGuard addresses this by training on multiple aligned views of each case and using agreement among those views to shape both classification and confidence.
On three tests of unfamiliar agent-trace formats and externally specified rules, UniGuard-0.6B improved average binary classification quality and confidence reliability compared with its pretrained base guardrail. The results support the approach on these tested interface shifts.
The results cover three studied interface shifts, not new safety domains, tools, policies, or agent failures. UniGuard-FT uses generated views from 1,000 base cases, which may not capture the variety of real systems.
Analysis
Deep
- Original paper authors
- Viet-Thanh Pham, Thuy-Trang Vu, Junae Kim, Rollin Omari, Dinh Phung
- Submitted to arXiv
- License
- CC BY 4.0