Efficient Temporal Modeling for Mobile Sleep Staging via Lightweight Random Attention
For researchers and engineers developing mobile sleep monitoring systems, RA offers an efficient alternative to RNNs and Transformers, enabling real-time deployment on wearables.
The paper tackles the problem of computationally expensive sequential models for mobile sleep staging. It proposes Random Attention (RA), a lightweight temporal modeling module using fixed random projections, which improves epoch-wise baselines by 1-3% in accuracy and F1 score while being suitable for real-time wearable applications.
Mobile sleep staging serves as a foundational infrastructure for in-home sleep monitoring and closed-loop modulation. But existing sequential models such as RNNs and Transformers are computationally expensive for mobile deployment. In this paper, we propose Random Attention (RA), a lightweight temporal modeling module based on fixed random projections, which replaces learnable sequence modeling with similarity-based aggregation. RA introduces little additional parameters beyond the epoch encoder while enabling effective temporal smoothing. We further provide a theoretical interpretation via the Random Attention Prior Kernel (RAPK), which decomposes RA into a global smoothing term and a feature similarity term, offering an interpretable view of temporal sleep structure. Experiments on Sleep-EDF-20 and Sleep-EDF-78 show that RA consistently improves epoch-wise baselines by 1-3\% in accuracy and F1 score, while achieving competitive performance compared with LSTM, GRU, and Transformer models. RA also demonstrates strong generalization across different backbone encoders and improved robustness over conventional temporal smoothing methods. These results indicate that efficient sleep staging can be achieved through lightweight similarity-based temporal aggregation, making RA suitable for real-time wearable applications.