A trainable expert blend helps a student recover domain skills
P$^2$-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors · arXiv:2611.03953
In plain words
Combining specialist language models into one can leave the student weak in a domain, even after more training. P²-MOPD is a method that keeps each specialist’s learned direction available while also letting the student’s own weights change.
The specialists score answers the student writes, and that feedback trains both parts. On four domains with the 1.7B Qwen3 model, average recovery was 94.74 points, versus 80.93 for a version with fixed expert-mixing weights. Recovery runs from the base model at 0 to the experts at 100.
The research
A single model made from specialists can look balanced overall while leaving one skill far behind.
In this test, letting the student adjust how much it follows each expert direction alongside learning its own weights changes the expectation that a fixed merge plus continued distillation is enough to recover the lagging domain.
P²-MOPD trains a student’s weights and expert-mixing coefficients together, testing whether an adaptable expert blend improves multi-domain distillation.
Starting from the same uniform expert merge, P²-MOPD and its main comparison use the same teacher-feedback objective; only the comparison fixes the expert-mixing coefficients.
The evaluation averages three generated answers per item, then measures recovery from the base model toward each expert and equally averages the four domains.
At step 100 on Qwen3-1.7B, P²-MOPD averaged 94.74 ± 0.91 recovery points across three training runs, against 80.93 ± 3.02 for the fixed-coefficient version. This supports a benefit from learning the coefficients, though the average does not describe every domain.
The evidence covers Qwen3 models at 1.7B and 4B scales on four domains. The fifth domain was omitted because its user simulator requires a paid commercial language-model API.
What the evidence shows
- A method for expert consolidation
P²-MOPD is a method for combining domain expert language models into one student while continuing to train it. The evaluation covered medical questions, legal case reasoning, finance problems, and instruction following.
Recovery measures how much of the gap from a base model to its expert teacher the student closes; the headline score averages that recovery equally across the four domains, with the base at 0 and teachers at 100.
- The deciding comparison
Both methods start from the same uniform expert merge and use the same teacher-feedback objective; the key difference is whether the expert-mixing coefficients can change during training.
On the four-domain Qwen3-1.7B evaluation, the jointly trained method finished with higher average recovery, supporting a benefit from learning those coefficients.
- The gain varies by domain
The advantage was concentrated in the medical test, which measures answers to medical questions.
Differences were smaller in finance, where the fixed-coefficient method already recovered nearly all of the teacher gap, and varied in both directions on instruction following; these comparisons average checkpoints and training runs.
- Why the two channels matter
A diagnostic found that P²-MOPD moved farther along the medical expert’s task direction, alongside stronger medical recovery; that association does not show the movement caused the gain.
In a separate control, learning coefficients while freezing student weights remained well below joint training, supporting the view that both channels contribute.
Deep
- Original paper authors
- Sinuo Wang, Jiacheng Lu, Zeyu Zhang, Maojia Song, Rui Sun
- Submitted to arXiv
- License
- arXiv non-exclusive