Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V Demonstration
This work addresses the problem of reliable and high-data-rate wireless communication for mobile vehicles, representing an incremental improvement by integrating vision with existing mmWave systems.
The paper tackles the challenge of aligning millimeter-wave beams in vehicle-to-vehicle communications by using a deep learning model that predicts future beams from 360-degree camera images, achieving approximately 85% top-5 beam prediction accuracy and reducing beam training overhead.
Accurately aligning millimeter-wave (mmWave) and terahertz (THz) narrow beams is essential to satisfy reliability and high data rates of 5G and beyond wireless communication systems. However, achieving this objective is difficult, especially in vehicle-to-vehicle (V2V) communication scenarios, where both transmitter and receiver are constantly mobile. Recently, additional sensing modalities, such as visual sensors, have attracted significant interest due to their capability to provide accurate information about the wireless environment. To that end, in this paper, we develop a deep learning solution for V2V scenarios to predict future beams using images from a 360 camera attached to the vehicle. The developed solution is evaluated on a real-world multi-modal mmWave V2V communication dataset comprising co-existing 360 camera and mmWave beam training data. The proposed vision-aided solution achieves $\approx 85\%$ top-5 beam prediction accuracy while significantly reducing the beam training overhead. This highlights the potential of utilizing vision for enabling highly-mobile V2V communications.