In plain words
Safety researchers use probes to check whether a language model’s internal signals distinguish safer from riskier examples. But training a probe takes labeled examples, and its performance can be hard to know in advance.
This paper fits simple rules that estimate probe error from model size and the number of labels.
Across the tested tasks and model families, larger models generally had lower probe error, and the rules predicted results on a next, larger model better than a simple straight-line estimate in the reported comparison.
The research
When a new model or safety task needs monitoring, researchers must choose how many examples to label and which probe to train. Can they estimate the probe’s performance before doing that work, especially when public datasets do not show how performance changes from few labels through saturation?
Wu and Wei fit scaling laws that estimate safety-probe error from model size and label count, helping researchers plan probe training before measuring a new model.
Wu and Wei ask whether researchers can estimate a safety probe’s performance before collecting labels and training it on a new model. Public safety datasets often start near the performance ceiling or lack enough labels to show where a probe’s performance levels off.
They measure probe error across model sizes and label counts, then fit relationships to predict how error changes. In 72 next-model predictions at the largest label count, 72% were within 0.02 of measured error, compared with 54% for a straight-line reference. This supports using the fitted relationships to estimate probe performance in the tested settings.
The main results use 20 dense base checkpoints from five open model families and six tasks. The cost comparison counts parameters multiplied by labels, so it does not include label prices or other practical constraints; the laws also need further evaluation on post-trained models.
Analysis
Deep
- Submitted to arXiv
- License
- arXiv non-exclusive