A calibrated reward helps models follow rules and answer well

When a model satisfies an output rule, such as a required format, how can we tell whether it still answered the request? A rule checker can give full credit even to an inadequate answer, so training against that score alone may reward the wrong behavior.

CaLIF combines rule-check rewards with a bounded check against the model’s earlier answers, aiming to improve constraint following while preserving answer quality.

The authors compared five model configurations trained with CaLIF or verifier-only IF-RLVR on IFBench, which scores prompt-level compliance with added output rules.

CaLIF averaged 34.64/100 for rule satisfaction, above the base models' 26.98, while answer quality averaged 4.59/10, near the base average of 4.51.

IF-RLVR scored higher on rule satisfaction, 44.93, but lower on answer quality, 2.96. This pattern supports the authors' conclusion that CaLIF can improve rule following while preserving answer quality better than verifier-only training in these configurations.

The evidence covers five model configurations, the reported training data, and these evaluation settings. It does not establish that CaLIF will preserve answer quality for other models, training data, or tests.