Perturbation consistency adds little to clean image error checks
In plain words
The paper introduces the Decision Consistency Framework (DCF), a way to check whether an image classifier keeps its prediction when an image is changed slightly. It tracks whether the label changes and how much the classifier’s confidence moves, then combines those checks into a score.
Across the tested model families, that score was worse than the classifier’s ordinary confidence at spotting wrong predictions on clean images. It helped in some noise and attack tests, but those results had specific limits.
The research
When an image classifier gives an answer, can checking whether small changes to the image alter that answer reveal errors that its original confidence misses?
The authors compare consistency checks with the clean output directly, then test a mathematical explanation for when stability can add information and where it does.
The authors introduce DCF, which scores label and confidence stability under six mild image changes, then test when that score adds error information beyond clean confidence.
Across 201 evaluations, the consistency score ranked clean-data errors below maximum softmax probability (MSP), which uses the largest probability in the clean output. AUROC measures how well a score separates correct from incorrect predictions, with 0.5 as chance.
For frozen CNNs, consistency averaged 0.057 AUROC points below MSP across 16 configurations on four datasets.
A fixed-margin diagnostic found that changes in classifier output scores added little separation between correct and incorrect images. This supports the conclusion that, on clean data, consistency often repeats information already in the clean output.
Under four noise corruptions, CS beat the logit margin in a direct score comparison. Separately, gains from adding DCF features to confidence features depended on labelled related shifts.
Consistency also distinguished clean from attacked images under a non-adaptive attack, but an adaptive attack could evade it in the tested settings.
These findings concern the tested image datasets, models, and six mild operators. Gains from adding DCF features to confidence features on noise depended on labelled examples from related shifts; the direct CS-versus-margin comparison on four noise corruptions was separate. The adaptive-attack result covers only one tested attack objective.
What the evidence shows
- What consistency can add
The Decision Consistency Framework (DCF) is a post-hoc method for checking whether image-classifier predictions remain stable under small image changes. It applies six mild operators, including rotation and blur, and records how often the label stays the same and how much top-class confidence changes.
The authors ask whether those observations help identify wrong predictions beyond what the classifier’s clean output already shows.
- The clean-data comparison
For clean-data error detection, the authors compared DCF’s consistency score with maximum softmax probability (MSP), the largest probability in the classifier’s clean output. They measured AUROC, which describes how well a score separates correct from incorrect predictions; 0.5 is chance.
Across the reported model families, consistency scored below MSP. For frozen CNNs, its mean AUROC was 0.057 points lower across 16 configurations on four image datasets.
- A test of new information
A mathematical result explains why stability may repeat information in the clean output: a changed label requires the clean logit margin to be no greater than √2 times the size of the change in the classifier’s output scores.
The margin is already in the clean output. In a diagnostic across 84 frozen CNN and transformer runs, score changes separated correct from incorrect predictions poorly among images with similar margins: mean AUROC was 0.515, close to chance.
- A limited noise exception
Consistency helped in selected settings where clean confidence was misleading. On four noise corruption types in CIFAR-10-C, the consistency score exceeded the logit margin by 0.05 to 0.11 AUROC points.
This is a direct comparison of the two scores. Separately, the authors tested whether adding DCF features to confidence features improved error detection using a fitted feature combiner.
When trained on clean data or other corruption families, that combiner’s gain on noise fell to 0.009 or 0.002 AUROC points, respectively; the latter was not statistically significant.
- Exceptions and boundaries
Against a non-adaptive projected-gradient attack, the consistency score distinguished clean from attacked images better than MSP at the tested budget of 2/255. An adaptive attack that penalized instability could evade that detector, though few attacks then changed the prediction.
Separately, correcting for OCTMNIST’s training-to-test class-proportion difference improved the consistency-accuracy gap by about 40%, leaving part unexplained. These results bound when consistency helps and what the model-level gap can explain.
Deep
- Original paper authors
- Amit Kalitaa, Moirangthem Tiken Singh
- Submitted to arXiv
- License
- CC BY 4.0