Dia calls other models only when reasoning answers disagree

When a language model gives a wrong answer, does asking it to reason longer or sample more answers provide genuinely new evidence?

The paper examines whether its errors persist across repeated answers, continued reasoning, and self-judgment, and whether disagreement during reasoning can signal when another source of evidence is useful.

Dia uses a model’s intermediate and final answers to decide when three independent models should solve the question and arbitrate.

The authors test whether disagreement between a model’s intermediate and final answers can identify cases where another source of evidence may help. Repeated sampling, further reasoning, and self-evaluation can preserve or favor the same model-specific errors.

Dia sends only disagreements to three independent models that solve the original question without seeing the candidate answers. Across seven test families and two base models, it is more accurate than full chain-of-thought in 12 of 14 settings.

That pattern supports selective arbitration on these tests; the MATH exceptions had little room for improvement over the already strong baseline.

The main comparison covers fixed subsets from seven test families and two base models. On the two MATH settings, full chain-of-thought did not leave much room for improvement; the findings do not establish how Dia performs on other models or tasks.