EN
A rendering-free simulator trains drone navigation policies in minutes
Rendering-Free Event Frame Simulation for Fast Reinforcement Learning of Autonomous Robot Navigation · arXiv:2610.37543
In plain words
Training a drone to avoid obstacles with an event camera takes many practice runs in simulation. Existing simulators make images before converting them into the camera’s event data, which slows those runs.
The paper introduces a simulator that makes event observations directly from scene edges and camera motion. Its trained controller succeeded more often than a rendered-event-trained controller on the same simulated episodes, though it had received more training. The controller also flew successfully in three lab courses.
The research
Can reinforcement learning train a drone to avoid obstacles with event-camera data without waiting for a simulator to render images before creating each observation?
The answer matters because training requires many repeated interactions with a simulated environment, and that rendering step can slow the process.
The authors introduce an analytic event simulator that trains drone obstacle-avoidance controllers directly from scene edges and camera motion, without rendering images first.
The authors train a drone navigation controller using reinforcement learning and the analytic simulator, then compare it with a controller trained on rendered events. Both face the same 30 simulated layouts in shadow and texture conditions.
The proxy-trained controller reaches the goal more often in both conditions, although it received substantially more training.
The same controller also succeeds in unseen ray-traced simulations and onboard laboratory flights without adaptation. Together, these results show that training from the proxy can produce a controller that transfers across the tested simulated and real settings.
The shared simulation comparison gives the proxy-trained policy more training than the rendered-event policy, so it does not isolate simulator choice from training amount. Real-flight results cover 17 flights across three laboratory courses, not every setting a drone might encounter.
What the evidence shows
- Train without rendering
The paper introduces an analytic event simulator, a method for training drone navigation controllers without first rendering images. Existing event simulators generally produce images and convert them into events, slowing the repeated environment interactions that reinforcement learning needs.
The paper asks whether scene geometry and camera motion can generate useful event observations directly, allowing the controller to train faster while still navigating clutter.
- Generate event observations
The simulator builds observations from projected 3D scene edges as the camera moves, then accumulates signed event counts into the format used by the target sensor. It represents edge motion and event density with simplified rules rather than modeling full images.
The authors calibrate five parameters against one short camera recording, using sensor-output statistics rather than flight or task data from a target environment.
- Check the training bottleneck
A speed comparison on one GPU and identical scenes measured both event generation and closed-loop training. The analytic proxy generated an observation in 17 microseconds, compared with 6.9 milliseconds for Isaac Sim with its event renderer; it also supported many more policy steps per second.
This establishes that the proxy reduces the reported simulation cost, but speed alone does not show that a trained controller can navigate successfully.
- Compare navigation outcomes
The authors evaluated the proxy-trained controller and a controller trained on rendered events on the same Isaac Sim episodes. Success means reaching the goal without failure.
The proxy-trained controller had higher success rates in both shadow and texture appearances, but it received 40 million training steps while the rendered-event controller collected 195,000 transitions. The comparison supports the result for these two training procedures, not a conclusion that simulator choice alone caused the difference.
- Test transfer across settings
The same proxy-trained controller succeeded in unseen ray-traced simulation scenes and in three real laboratory courses, without adaptation between settings. Success was lower in the denser texture condition than in shadow scenes, and the real flights covered 17 attempts across three arenas.
These results support transfer across the tested simulations and courses; the proxy’s simplified scene model and the limited flight evaluation bound that conclusion.
Deep
- Original paper authors
- Dequan Ou, Austin Lopez, Quentin Missinne, Christophe De Wagter, Guido C. H. E. de Croon
- Submitted to arXiv
- License
- CC BY 4.0