Attention-based Neural Bag-of-Features Learning for Sequence Data
This work addresses sequence data analysis for domains like finance and healthcare, offering an incremental improvement by adding attention to existing NBoF models.
The authors tackled the problem of enhancing sequence data learning by proposing 2D-Attention (2DA), a generic attention module integrated into Neural Bag of Feature (NBoF) models, which improved performance and resilience to noise in financial forecasting, audio analysis, and medical diagnosis tasks.
In this paper, we propose 2D-Attention (2DA), a generic attention formulation for sequence data, which acts as a complementary computation block that can detect and focus on relevant sources of information for the given learning objective. The proposed attention module is incorporated into the recently proposed Neural Bag of Feature (NBoF) model to enhance its learning capacity. Since 2DA acts as a plug-in layer, injecting it into different computation stages of the NBoF model results in different 2DA-NBoF architectures, each of which possesses a unique interpretation. We conducted extensive experiments in financial forecasting, audio analysis as well as medical diagnosis problems to benchmark the proposed formulations in comparison with existing methods, including the widely used Gated Recurrent Units. Our empirical analysis shows that the proposed attention formulations can not only improve performances of NBoF models but also make them resilient to noisy data.