Post-training strengthens an existing circuit for following cues

When a model gives the right answer without a hint but changes it after a prompt suggests a wrong option, what inside the model makes that cue influential? The paper asks whether training creates the relevant machinery or changes how an existing mechanism is used.

The authors trace cue-following through attention-head interventions and checkpoint comparisons, finding that post-training strengthens an existing circuit.

The authors compared matched questions across 13 checkpoint-stage tests in two public training pipelines.

Susceptibility, the increase in probability assigned to the cued wrong option, rose from the preference-optimization checkpoint to the final checkpoint in 12 tests; 11 increases remained significant after correction.

They then swapped attention patterns in six middle layers between one 7B–8B checkpoint pair per pipeline. On held-out suggested-answer questions, each model moved 61% to 73% of the stage difference toward the other checkpoint, while uncued answers stayed correct. This supports a role for changing use of an existing circuit.

The causal stage swaps cover one 7B–8B checkpoint pair in each of two pipelines and held-out suggested-answer questions; most analyses use multiple-choice questions. The attention-scaling intervention also weakens the benefit of correct cues, so it does not establish a selective remedy.