In plain words
Training-data selectors choose a fixed number of records they score as useful. This paper studies whether an outside supplier can shape harmful records so they score well and still affect a model after training.
The authors’ attack generates and scores candidate records, then submits the best-scoring one that keeps its harmful content. In one over-refusal test using IFD, a difficulty-based selector, the adapted records became more concentrated in the selected set and the reported refusal rate rose. The result is specific to that test setting.
The research
When a model owner selects only a fixed number of training records, can a supplier make harmful records look useful enough to pass the filter and affect the trained model?
The supplier cannot change the selector or see the final selected set, so the question is whether adapting the records themselves can change what gets through.
The authors introduce an adaptive attack that shapes harmful training records to survive data selection, and a two-part defense that audits suppliers and combines selector rankings.
The authors test whether a supplier can make harmful training records survive a selector that keeps only its highest-scoring records. The supplier generates payload-bearing candidates, scores them, and submits the best-scoring one that retains the payload.
In a paired over-refusal test using IFD, a difficulty-based selector, adaptation raised poison enrichment from 1.21x to 6.07x and the reported refusal rate from 0% to 94%. This shows concentration and a behavior change in that tested cell; a separate random-selection control also indicates that concentration depended on selection in its LESS setting.
The evaluations cover particular selector–payload pairs and model settings. CONCERT’s certificate limits which records can be selected; it does not guarantee a poison-free set or bound downstream harm. The code outputs were audited on existing checkpoints, not after retraining on a reduced pool.
Analysis
Deep
- Original paper authors
- Yahao Ren, Jian Lou, Huadi Zheng, Jiawei Duan
- Submitted to arXiv
- License
- arXiv non-exclusive