Feature selection in linear SVMs via a hard cardinality constraint: a scalable SDP decomposition approach
This work addresses the need for interpretable classification models in machine learning, though it is incremental as it builds on existing SVM and optimization techniques.
The authors tackled the NP-hard problem of feature selection in linear SVMs with a cardinality constraint by proposing scalable semidefinite relaxations and decomposition methods, achieving efficient and effective results on classical benchmarking datasets.
In this paper, we study the embedded feature selection problem in linear Support Vector Machines (SVMs), in which a cardinality constraint is employed, leading to an interpretable classification model. The problem is NP-hard due to the presence of the cardinality constraint, even though the original linear SVM amounts to a problem solvable in polynomial time. To handle the hard problem, we first introduce two mixed-integer formulations for which novel semidefinite relaxations are proposed. Exploiting the sparsity pattern of the relaxations, we decompose the problems and obtain equivalent relaxations in a much smaller cone, making the conic approaches scalable. To make the best usage of the decomposed relaxations, we propose heuristics using the information of its optimal solution. Moreover, an exact procedure is proposed by solving a sequence of mixed-integer decomposed semidefinite optimization problems. Numerical results on classical benchmarking datasets are reported, showing the efficiency and effectiveness of our approach.