Researchers at the University of Washington presented Reflex, a three-stage system for humanoid robots that catches thrown boxes using visual input and coordinated whole-body movement. The paper was published on October 8 and submitted to Hugging Face Papers on October 12. Catching a box requires a robot to coordinate its body, respond to the box’s motion in real time, and interpret what it sees from a camera on the robot. Existing systems addressed only parts of this task, relying on end-effectors, motion capture, or marked objects.

Reflex trains the capabilities in three stages. First, a state-based policy learns whole-body catching through reinforcement learning. Second, it learns to infer a moving box’s dynamics from delayed and incomplete observations. Third, an egocentric RGB-D encoder (a camera-based depth and color vision system) learns to recover that motion information from visual observations, while the whole-body controller stays fixed. A compact representation of object dynamics connects the stages, and the authors say the decomposition gives each capability a direct training signal and makes visual learning more scalable.

In simulation, Reflex caught boxes successfully in 85.7% of trials using egocentric RGB-D, compared with 87.9% when it received privileged box observations. Removing the history of visual observations lowered success by 35.6 percentage points for centered throws and 39.8 points for throws offset to the side. On a real Unitree G1 humanoid, the system caught human-thrown boxes with 65% success at 2.5 m and 40% at 5 m. At those distances, those rates were 76% and 59% of the success rates achieved by human catchers, respectively. The study used box catching as a task requiring the three capabilities the system was designed to combine.

Researchers at the University of Washington presented Reflex, a three-stage system for humanoid robots that catches thrown boxes using visual input and coordinated whole-body movement. The paper was published on October 8 and submitted to Hugging Face Papers on October 12. Catching a box requires a robot to coordinate its body, respond to the box’s motion in real time, and interpret what it sees from a camera on the robot. Existing systems addressed only parts of this task, relying on end-effectors, motion capture, or marked objects.

Reflex trains the capabilities in three stages. First, a state-based policy learns whole-body catching through reinforcement learning. Second, it learns to infer a moving box’s dynamics from delayed and incomplete observations. Third, an egocentric RGB-D encoder (a camera-based depth and color vision system) learns to recover that motion information from visual observations, while the whole-body controller stays fixed. A compact representation of object dynamics connects the stages, and the authors say the decomposition gives each capability a direct training signal and makes visual learning more scalable.

In simulation, Reflex caught boxes successfully in 85.7% of trials using egocentric RGB-D, compared with 87.9% when it received privileged box observations. Removing the history of visual observations lowered success by 35.6 percentage points for centered throws and 39.8 points for throws offset to the side. On a real Unitree G1 humanoid, the system caught human-thrown boxes with 65% success at 2.5 m and 40% at 5 m. At those distances, those rates were 76% and 59% of the success rates achieved by human catchers, respectively. The study used box catching as a task requiring the three capabilities the system was designed to combine.