IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms
This addresses feature selection challenges in machine learning, particularly for uncovering interactions, but appears incremental as it builds on Relief-based algorithms.
The paper tackles the problem of differentiating interaction terms from marginal effects in feature selection by proposing the IMMIGRATE algorithm, which includes and trains weights for interaction terms and achieves state-of-the-art results on several tasks.
Relief based algorithms have often been claimed to uncover feature interactions. However, it is still unclear whether and how interaction terms will be differentiated from marginal effects. In this paper, we propose IMMIGRATE algorithm by including and training weights for interaction terms. Besides applying the large margin principle, we focus on the robustness of the contributors of margin and consider local and global information simultaneously. Moreover, IMMIGRATE has been shown to enjoy attractive properties, such as robustness and combination with Boosting. We evaluate our proposed method on several tasks, which achieves state-of-the-art results significantly.