Knock Until You Can See It: Cross-Modal Transfer from Sound to Vision
This project explores how robots can leverage self-generated sounds from physical interactions to improve visual perception and manipulation. By actively knocking on objects and learning cross-modal representations between audio and vision, the robot can infer object properties that may be difficult to determine from visual observations alone. The work aims to enable more robust, adaptive, and efficient robotic manipulation through active multimodal perception.