Vision-Aided Beam Tracking: Explore the Proper Use of Camera Images with Deep Learning
This work addresses beam tracking challenges in mmWave communication for users in varying signal conditions, but it is incremental as it builds on existing datasets and methods.
The paper tackles wireless beam tracking on mmWave bands by using camera images and deep learning to predict optimal beam indices, showing that image assistance improves tracking, especially in serious non-line-of-sight (NLOS) conditions, with performance gains demonstrated through dataset clustering.
We investigate the problem of wireless beam tracking on mmWave bands with the assistance of camera images. In particular, based on the user's beam indices used and camera images taken in the trajectory, we predict the optimal beam indices in the next few time spots. To resolve this problem, we first reformulate the "ViWi" dataset in [1] to get rid of the image repetition problem. Then we develop a deep learning approach and investigate various model components to achieve the best performance. Finally, we explore whether, when, and how to use the image for better beam prediction. To answer this question, we split the dataset into three clusters -- (LOS, light NLOS, serious NLOS)-like -- based on the standard deviation of the beam sequence. With experiments we demonstrate that using the image indeed helps beam tracking especially when the user is in serious NLOS, and the solution relies on carefully-designed dataset for training a model. Generally speaking, including NLOS-like data for training a model does not benefit beam tracking of the user in LOS, but including light NLOS-like data for training a model benefits beam tracking of the user in serious NLOS.