A driving test’s comfort score can reward staying still
Cette page n’a pas encore de traduction vérifiée. La version anglaise est affichée. Lire la version anglaise
En termes simples
This paper checks whether a simulator driving test gives a fair picture of how models drive. Its comfort score counted comfortable moments by time, so a vehicle could earn credit while stopped. The authors fixed how motion was measured and counted comfort by distance instead.
Across eight models, the corrected score was no longer linked to how long they stayed stopped. The paper also found that controller steering and route directions changed results for two models, but those findings come from simulator tests, not roads.
La recherche
When a simulator test reports that one driving model is more comfortable than another, does that reflect smoother motion or simply more time spent stopped?
Bench2Drive’s time-based score and controller can influence what gets measured, so the study asks whether correcting those parts changes the interpretation.
The authors correct Bench2Drive’s comfort score and navigation inputs to check whether the test measures driving behavior faithfully.
Bench2Drive is a simulated driving test where models act on a changing route; its comfort score was meant to reflect smooth driving. The authors found that the original score rose with time stopped, so they corrected its signal calculations and counted comfort by distance.
Across eight models, the link to stopped time nearly disappeared, and comfort no longer tracked driving score after accounting for speed.
Separately, tests on UniAD and VAD showed that the test controller and the route point supplied to a model can change results. The authors’ fixes affected the two models differently, showing that protocol details can shape apparent performance.
These findings come from simulator tests. The comfort analysis covers eight models; the steering and target-point tests cover UniAD and VAD on a 10-route development subset. The authors did not audit the other six models’ training data for target-point mismatch.
Analyse
Détails
- Auteurs de l’article original
- Thomas Schwarzfischer, Emad Aghajanzadeh, Moritz Thoma, Nael Fasfous, Manoj Rohit Vemparala
- Soumis à arXiv
- Licence
- arXiv non-exclusive