Training helps desktop agents coordinate clicks and code

If a desktop agent can click, type and run code, does simply giving it all three options help it complete tasks? In a diagnostic, code access alone did not improve an unadapted model's score. The study tests whether training on examples that combine these actions helps the agent use them together.

OS-Helix trains one desktop agent on examples that mix clicks, typing and code, so it can select either action within a single task.

Researchers tested OS-Helix on 361 Ubuntu desktop tasks, where success rate means the average share of tasks completed. The agent's single model can choose GUI actions or code.

After supervised fine-tuning on hybrid desktop examples, it achieved 62.75% mean success, compared with 46.50% for a Qwen3.5-27B hybrid baseline.

The comparison shares a backbone and action space, but the baseline's training procedure and identical starting weights are unspecified. The result shows higher average success for OS-Helix in this test set; it does not establish that training alone accounts for the difference.

The main comparison covers 361 Ubuntu desktop tasks, with 8 Google Drive tasks excluded. The paper does not specify whether the baseline and OS-Helix started from identical checkpoint weights, so the result does not isolate training as the only difference.