E2E-FS: An End-to-End Feature Selection Method for Neural Networks
This work addresses the problem of balancing accuracy and explainability in feature selection for machine learning practitioners, offering an incremental improvement over existing methods.
This paper introduces E2E-FS, a novel embedded feature selection algorithm that aims to balance accuracy and explainability. It uses gradient descent with non-convex regularization to select a maximum number of features for subsequent classification.
Classic embedded feature selection algorithms are often divided in two large groups: tree-based algorithms and lasso variants. Both approaches are focused in different aspects: while the tree-based algorithms provide a clear explanation about which variables are being used to trigger a certain output, lasso-like approaches sacrifice a detailed explanation in favor of increasing its accuracy. In this paper, we present a novel embedded feature selection algorithm, called End-to-End Feature Selection (E2E-FS), that aims to provide both accuracy and explainability in a clever way. Despite having non-convex regularization terms, our algorithm, similar to the lasso approach, is solved with gradient descent techniques, introducing some restrictions that force the model to specifically select a maximum number of features that are going to be used subsequently by the classifier. Although these are hard restrictions, the experimental results obtained show that this algorithm can be used with any learning model that is trained using a gradient descent algorithm.