PLatour links larger prompt models to smaller generators
On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer · arXiv:2611.06976
In plain words
A large language model can read a question and pass information about it to a smaller model, which writes the answer. PLatour is a framework that connects the two during question processing and trains the connection while keeping both models fixed.
The smaller model scored higher with PLatour than alone on three tested generation tasks. But checks found that the sender’s information helped beyond tested alternatives on Chemistry, while on other tasks a control matched or beat it.
The research
A higher score does not by itself show that one model’s information helped another; the receiver might improve through adaptation alone.
That distinction changes how to interpret cross-model gains: PACT-Check finds evidence beyond tested alternatives on generation Chemistry, but not a consistent sender advantage across the tasks.
PLatour connects a larger prompt-reading model to a smaller answer-generating model, while PACT-Check tests whether gains depend on the sender’s information.
In generation tests, the Qwen3-1.7B receiver scored higher with PLatour than alone across three tasks and two sender sizes. Because training the bridge can also change the receiver’s computation, those gains alone cannot show that the sender’s prompt information was useful.
PACT-Check compares PLatour with controls that remove or replace sender information and with receiver-adaptation alternatives. The accuracy margin over the strongest applicable control is positive on Chemistry, near zero on GSM8K, and negative on MATH500, supporting a task-dependent answer.
The controls test specific ways a receiver might improve without useful information from an external sender. They do not rule out every alternative explanation, and a training collapse on one MATH configuration shows that performance was not stable in every reported setup.
What the evidence shows
- A gain has two explanations
PLatour is an open-source framework for prefill-to-autoregressive cross-model transfer: a larger model processes a prompt, and a smaller model then generates the answer. Its trainable bridge passes prompt representations between the models.
The central question is whether better accuracy reflects useful information from the sender, or whether the bridge can improve the receiver by changing its computation.
- Higher accuracy leaves a question
On generation tests involving Chemistry questions, GSM8K grade-school math problems, and MATH500 mathematical reasoning problems, PLatour scored above the Qwen3-1.7B receiver alone with both tested sender sizes. Accuracy measures correct answers under each test’s scoring rules.
This establishes a performance gain in those settings, but the comparison does not identify what produced it: the bridge is trained and can alter receiver computation.
- Controls change the interpretation
PACT-Check asks whether the gains remain when input-specific sender features are removed or replaced, or when the receiver is adapted without an external sender.
PLatour’s advantage over the strongest tested control is clear on Chemistry, close to none on GSM8K, and reversed on MATH500, so evidence for a sender-specific benefit depends on the task.
- Strong results have boundaries
On shared mathematics and coding tests, PLatour performed strongly against Mostik despite using a smaller sender. The comparison is not fully controlled: Mostik’s training data are undisclosed, and it is unclear whether Mostik uses a general bridge across task domains; PLatour uses separate bridges trained on domain-specific data. This comparison does not establish a controlled advantage.
Deep
- Original paper authors
- Zhenyi Shen, Junru Lu, Wei Liu, Jiangnan Ye, Jiazheng Li
- Submitted to arXiv
- License
- CC BY 4.0