Model weight spectra rank robustness within model groups

Can a model's robustness to unfamiliar data be compared before examples from that target domain are available? Many diagnostics need source or target data, and a single weight statistic across architectures has not reliably separated robust from fragile models. The paper asks whether model weights can support a useful ranking without target data.

The authors build a method that reads pretrained weight spectra within each architecture-and-training group to rank models' out-of-domain robustness.

The authors tested this approach on 116 models across seven modalities and 15 architecture-and-strategy groups.

Each group's out-of-distribution (OOD) accuracy drop, the source-domain accuracy minus target-domain accuracy, was measured on one source-to-target task, with a frozen encoder and a probe trained on source data.

A cell-specific statistic agreed more closely with model rankings than a single pooled statistic: mean absolute leave-one-out Spearman correlation was 0.824 versus 0.245. The comparison supports choosing measures by group, though the rankings were calibrated using OOD outcomes.

The rankings cover one selected shift per architecture-and-strategy group, and the group-specific measures were selected using OOD outcomes. The theorem depends on stated assumptions, and two measured repair cases do not show that changing feature log stable rank mediates the improvement.