GTAGym turns GTA V missions into checkable agent tasks
GTAGym: Real AAA Game Tasks Environment for Evaluation and Reinforcement Learning · arXiv:2611.13028
In plain words
GTAGym is a framework for testing whether game-playing agents actually finish tasks based on GTA V missions. The authors turned mission stages into tasks with goals and rules, then checked completion from game data or marked target regions instead of trusting an agent’s claim.
On its tests, agents still struggled with aerial control, evasion and interception. Training one model on generated walking routes improved its success on separate benchmark walking tasks, but that result covers one skill area.
The research
A game-playing agent can look convincing without actually completing a long task.
GTAGym makes success depend on whether mission goals and required steps are met, and its results show that strong performance in some game skills can coexist with persistent trouble in aerial control, evasion and interception.
GTAGym turns GTA V mission stages into verifiable agent tasks and generates training environments, revealing persistent gaps in dynamic control.
The framework evaluates six models across 150 mission-derived tasks, checking completion against game states or annotated image regions. On the GTAVBench-AGI protocol, GPT-6-Astra averaged 70.9% task accuracy across three seeds, while three experienced players averaged 89.8% across the same tasks.
Category results show the models still struggled with aerial navigation, evasion and interception. Training Qwen3.8-27B on generated on-foot routes raised its accuracy on the benchmark’s separate on-foot tasks, supporting a bounded case for generated-task training.
The benchmark and transfer results cover selected tasks from GTA V missions. The native-game comparison used 100 navigation and interaction tasks, and the human and model comparison does not establish identical interaction protocols.
What the evidence shows
- A framework for game tasks
GTAGym is an environment and framework for evaluating agents on tasks drawn from GTA V missions, and for generating reinforcement-learning environments. It addresses a testing problem: long interactive tasks need checks that actions reach a goal under the required conditions. The framework turns selected mission stages into tasks that can be run and scored.
- Completion depends on outcomes
The tasks specify goals, interaction order and completion conditions. Agents act through screenshots and keyboard-and-mouse inputs, while evaluators check game state or marked image regions.
Success is binary, so an unfinished route or interaction does not earn partial credit. That makes task completion depend on observable conditions rather than an agent’s declaration.
- The task mix exposes gaps
On GTAVBench-AGI, the strongest reported model configuration completed a smaller share of the shared tasks than the experienced players did, though the interaction protocols were not established as identical.
The category pattern also shows uneven capabilities: aerial control, evasion and interception remained difficult across evaluated models.
- Generated routes help on foot
Training Qwen3.8-27B on generated on-foot routes improved its mean accuracy on benchmark on-foot tasks that were excluded from training. This supports a skill-specific benefit in this setup, while results did not improve across every task category.
- A separate native-game check
The authors also tested the trained model on native GTA V navigation and interaction tasks, with reviewers checking recordings for task completion. This offers limited evidence that the improvement extended beyond reconstructed benchmark routes on a selected set that excluded shooting and sniping.
Deep
- Original paper authors
- Jiajun Chen, Shicheng Fang, Jiahao Lu, Yuxin Wang, Xipeng Qiu
- Submitted to arXiv
- License
- arXiv non-exclusive