In plain words
Some language models store useful knowledge and hazardous knowledge in the same parts, so removing one can affect the other. REX, a method for mixture-of-experts models, moves hazardous knowledge into extra parts that can be removed.
The researchers tested three models in biosecurity and cybersecurity. After the same further-training procedure, REX’s released model had the lowest scores on the hazardous-knowledge test among the usable methods in all six tested settings.
The research
Can a pretrained model lose hazardous capability without losing general ability, and can that separation persist when the model is trained further? Existing methods change parameters that support both kinds of knowledge, while later training can restore what they suppressed.
REX relocates hazardous knowledge into removable experts, then trains the recombined model so those experts can be removed at delivery.
Researchers compared paired models before further training: each pair differed only in whether the added experts were present. On WMDP, a four-choice hazardous-knowledge test scored by the share of correct answers, excision lowered accuracy by 9.5 to 48.4 points across six settings. General-capability scores changed little overall.
After a shared 250-million-token training run, REX had the lowest WMDP accuracy among usable unlearning methods in all six settings.
The routes shared data, order and seed, but had different starting checkpoints and prior procedures, so this supports REX under the tested comparison rather than a universal advantage.
The evidence covers three mixture-of-experts models, biosecurity and cybersecurity, and one shared 250-million-token training procedure. It does not establish how REX performs on other model architectures, hazardous domains or further-training procedures.
Analysis
Deep
- Original paper authors
- Cheng Jin, Zhongtian Ma, Xiao Wang, Han Qi, Xiaoyu Wen
- Submitted to arXiv
- License
- CC BY 4.0