In plain words
This paper compares two pairs of open-weight language models, each with an original version and a community edit designed to reduce refusals. The researchers asked the models paired yes/no questions about Taiwan and 1989 Tiananmen in three scripts, and collected open answers.
The edited Qwen stopped refusing the questions in this test, but still answered “yes” to “Taiwan is part of the PRC” in Traditional Chinese, Simplified Chinese, and English. So fewer refusals did not mean the political answers had disappeared.
The research
People may choose an “uncensored” model expecting it to answer political questions without political framing. The paper asks a narrower, testable question: when a community edit reduces refusals, do models’ answers about Taiwan also shift across Traditional Chinese, Simplified Chinese, and English?
The study compares original and community-edited model pairs to test whether reducing refusals also changes political answers across Chinese scripts.
For yes/no tests, Index S summarizes which side a model says “yes” to: it subtracts the average probability for PRC-side statements from that for ROC-side statements, then averages across pairs. The scale runs from −1, consistently favoring PRC-side statements, to +1, consistently favoring ROC-side statements; zero means no separation.
Original Bolt averaged +0.58 on 16 sovereignty pairs in Traditional Chinese and +0.05 in Simplified Chinese.
Across 19 sovereignty and 1989 pairs, Qwen’s average score rose after abliteration in all three scripts. But its edited build still gave “yes” probabilities of 0.77, 0.98, and 0.80 to “Taiwan is part of the PRC” in Traditional Chinese, Simplified Chinese, and English. The average shift therefore did not mean that this PRC-side answer disappeared.
These findings cover two model families at one 2-bit quantization, one community edit per family, and a small set of statement pairs. The edits were not recomputed by the researchers, and the Simplified Chinese intervals were wide.
Analysis
Deep
- Original paper authors
- Ju-Chun Ko
- Submitted to arXiv
- License
- CC BY 4.0