In plain words
Trace2World is a method for turning recordings of people using real apps into resettable browser versions for training software agents. It combines traces to infer shared pages, app state, data, and when actions have different effects. Behavior the traces leave ambiguous stays unsupported.
After building the browser app, a separate checker replays the recorded workflows and reports mismatches for repair. On 200 workflows from ten apps, 91.1% passed all six checks, compared with 59.8% for a task-local builder.
The research
Can partial records of people using an app support a training environment that follows the app’s recorded behavior, rather than merely reaching the same final screen? A route can end correctly while taking a state change the real app does not.
Trace2World turns real-app traces into resettable browser environments by reconciling shared behavior first, then compiling and checking it through source-workflow replay.
Trace2World combines traces into a shared, source-linked app model before compiling a browser environment. A checker replays original workflows against that fixed model, comparing screens, state, action effects, outcomes, and data; package repairs cannot change the model.
On the same 200 workflows from ten apps, 91.1% passed all six checks, versus 59.8% for a task-local builder. Because every check must pass, this measures preserved workflows rather than endpoint reachability alone.
The evidence comes from recorded workflows and browser reimplementations of mobile apps. The real-device comparison covers policy-training apps, and the study does not measure transfer to unseen apps on real devices.
Analysis
Deep
- Original paper authors
- Zezhou Wang, Hanzhang Zhou, Panrong Tong, Quyu Kong, Pengxiang Li
- Submitted to arXiv
- License
- arXiv non-exclusive