Reflex helps a humanoid catch thrown boxes with moving-camera vision

A humanoid that catches an ordinary thrown box must coordinate balance and impact handling while predicting fast motion from sensors attached to its own moving body.

It remains unclear whether these demands can be combined on real hardware without external cameras or marked objects, rather than solving only hand interception or body control separately.

Reflex links whole-body control, box-motion prediction, and moving-camera vision through one latent representation for humanoid catching.

Reflex trains in stages: a state-based controller learns body placement and retention; a second module compresses delayed box observations into a motion representation; a visual encoder then predicts it from RGB-D history and camera motion while the controller stays fixed. This separates visual learning from motor-skill learning.

Across six controllers and 640 test throws each, visual input reached 85.7% catch success, requiring support and low speed for 0.6 seconds, versus 87.9% with the same decoder given direct box information. Keeping the decoder and evaluation fixed makes this a test of the latent source.

The main comparison is from simulation and does not guarantee performance on every throw or hardware condition.

Long-range visual grounding remains weak: at 5 m, simulation reached 43.0% with visual input versus 79.7% with direct box information; adding far-throw training raised simulated success to 75.8%, but hardware was not retested.