Training UI agents on live apps helps them transfer beyond iOS
Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS Apps · arXiv:2610.39264
In plain words
The paper introduces a way to train an agent that navigates phone screens. One model plans what to do, while another turns that plan into a tap or other screen action. The planner learns by trying tasks in live iOS apps and receiving feedback about both its steps and whether it finished.
On a test of tasks in Android apps, the reported trained planner completed a larger share of tasks than a supervised-training baseline. That result applies to the tested tasks and checkpoint.
The research
Can a smaller interface agent learn to recover and adapt, rather than simply repeat successful action sequences?
The paper asks whether training through live app interactions, with feedback about intermediate progress as well as task completion, can produce navigation that transfers beyond the apps used for training.
The framework trains a planner through live iOS interactions, using step-by-step and task-finish feedback to improve navigation beyond demonstrations.
On AndroidWorld, a test of task completion in Android apps, a vision-language model checked whether each output trajectory met the task requirements. The reported PPO Step 336 agent succeeded on 32.8% of 116 tasks, versus 4.7% for the supervised-training baseline, after training on iOS apps. This supports transfer beyond the training platform in this test.
Transfer was uneven: on iOSWorld, most reinforcement learning checkpoints made modest gains, although GiGPPO Step 336 reached 9.8% versus 4.5% for the baseline.
These results cover the reported agents, checkpoints, tasks, and evaluation setups. Most reinforcement learning checkpoints showed only modest gains on iOSWorld, and the authors’ proposed explanations for that result were not tested separately.
What the evidence shows
- Train through live interaction
The paper presents an online reinforcement learning framework for navigating phone interfaces. It addresses a problem the authors identify with earlier approaches: supervised training can encourage agents to memorize successful action sequences, while feedback only at the end of a task gives little credit for useful intermediate progress.
The framework trains a planner by letting it interact with live iOS apps, so its evidence comes from trying actions in changing app screens.
- Give each component a role
The framework divides navigation into planning and grounding. A planner reads the instruction, current screen, and action history, then selects a next step; a separate grounding model translates that step into an action and screen coordinates.
Reinforcement learning trains the planner, while supervised training teaches the grounding model. During planner training, judges provide feedback about progress at each step and task completion at the end.
- Measure transfer to Android
AndroidWorld tests task completion in real Android apps. A vision-language model checks whether each output trajectory meets the task requirements, and the reported task-success rate is the share of tasks judged successful.
The PPO Step 336 planner, trained online on iOS apps, succeeded on 32.8% of 116 AndroidWorld tasks, compared with 4.7% for the supervised-training baseline. This is evidence of transfer to the tested Android tasks, with Gemini-3.5-Flash serving as the grounding model in this evaluation.
- Check the reward signal
The authors compared standard PPO, with step-level and task-completion rewards, against PPO with task-completion rewards alone. At Step 168, standard PPO scored higher than terminal-only PPO at Step 336 on UINavBench (25.5% versus 21.6%) and AndroidWorld (20.7% versus 19.8%), but lower on iOSWorld (6.0% versus 7.5%).
The runs differ in training duration, and the paper does not say whether every other training condition was held constant, so this comparison does not isolate the effect of adding step-level rewards.
- Qualify the broader result
Transfer was not uniform across the tests. iOSWorld evaluates tasks in synthetic iOS apps, including personalization and multi-app workflows; most reinforcement learning checkpoints showed modest gains there. GiGPPO Step 336 was an exception, succeeding on 9.8% of tasks versus 4.5% for the supervised baseline.
The authors suggest differences in app interfaces and task types may contribute, but those explanations were not isolated experimentally.
Deep
- Original paper authors
- Peitong Duan, Khashayar Gatmiry, Jeffrey Nichols, Samy Bengio, Emmanuel Abbe
- Submitted to arXiv
- License
- CC BY 4.0