Self-Reinforcement Attention Mechanism For Tabular Learning
This addresses the need for interpretable models in high-risk fields such as fraud detection and credit scoring, where linear models are often used despite performance trade-offs, though it appears incremental as it builds on existing attention mechanisms.
The paper tackles the problem of interpretability and performance in tabular data learning, particularly for imbalanced datasets like fraud detection, by introducing a Self-Reinforcement Attention (SRA) mechanism that provides feature relevance weights to learn intelligible representations, showing effectiveness in end-to-end combinations with baseline models on synthetic and real-world data.
Apart from the high accuracy of machine learning models, what interests many researchers in real-life problems (e.g., fraud detection, credit scoring) is to find hidden patterns in data; particularly when dealing with their challenging imbalanced characteristics. Interpretability is also a key requirement that needs to accompany the used machine learning model. In this concern, often, intrinsically interpretable models are preferred to complex ones, which are in most cases black-box models. Also, linear models are used in some high-risk fields to handle tabular data, even if performance must be sacrificed. In this paper, we introduce Self-Reinforcement Attention (SRA), a novel attention mechanism that provides a relevance of features as a weight vector which is used to learn an intelligible representation. This weight is then used to reinforce or reduce some components of the raw input through element-wise vector multiplication. Our results on synthetic and real-world imbalanced data show that our proposed SRA block is effective in end-to-end combination with baseline models.