A driving test’s comfort score can reward staying still

When a simulator test reports that one driving model is more comfortable than another, does that reflect smoother motion or simply more time spent stopped?

Bench2Drive’s time-based score and controller can influence what gets measured, so the study asks whether correcting those parts changes the interpretation.

The authors correct Bench2Drive’s comfort score and navigation inputs to check whether the test measures driving behavior faithfully.

Bench2Drive is a simulated driving test where models act on a changing route; its comfort score was meant to reflect smooth driving. The authors found that the original score rose with time stopped, so they corrected its signal calculations and counted comfort by distance.

Across eight models, the link to stopped time nearly disappeared, and comfort no longer tracked driving score after accounting for speed.

Separately, tests on UniAD and VAD showed that the test controller and the route point supplied to a model can change results. The authors’ fixes affected the two models differently, showing that protocol details can shape apparent performance.

These findings come from simulator tests. The comfort analysis covers eight models; the steering and target-point tests cover UniAD and VAD on a 10-route development subset. The authors did not audit the other six models’ training data for target-point mismatch.