Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis
This work addresses the challenge of efficient data processing at the network edge for mobile applications, though it is incremental as it builds on existing over-the-air aggregation and stochastic gradient descent methods.
The paper tackles the problem of accelerating distributed principal component analysis (PCA) over multi-access channels by exploiting channel noise to speed up convergence around saddle points, achieving a faster convergence rate than without power control.
Recently years, the attempts on distilling mobile data into useful knowledge has been led to the deployment of machine learning algorithms at the network edge. Principal component analysis (PCA) is a classic technique for extracting the linear structure of a dataset, which is useful for feature extraction and data compression. In this work, we propose the deployment of distributed PCA over a multi-access channel based on the algorithm of stochastic gradient descent to learn the dominant feature space of a distributed dataset at multiple devices. Over-the-air aggregation is adopted to reduce the multi-access latency, giving the name over-the-air PCA. The novelty of this design lies in exploiting channel noise to accelerate the descent in the region around each saddle point encountered by gradient descent, thereby increasing the convergence speed of over-the-air PCA. The idea is materialized by proposing a power-control scheme which detects the type of descent region and controlling the level of channel noise accordingly. The scheme is proved to achieve a faster convergence rate than in the case without power control.