Piecewise-Linear Approximation for Feature Subset Selection in a Sequential Logit Model
This work improves feature selection for sequential logit models, which is incremental as it builds on prior optimization methods.
The paper tackles the problem of selecting a subset of features for a sequential logit model by addressing the gap between the logistic loss function and its quadratic approximation, resulting in a piecewise-linear approximation method that finds better feature subsets than the quadratic approach.
This paper concerns a method of selecting a subset of features for a sequential logit model. Tanaka and Nakagawa (2014) proposed a mixed integer quadratic optimization formulation for solving the problem based on a quadratic approximation of the logistic loss function. However, since there is a significant gap between the logistic loss function and its quadratic approximation, their formulation may fail to find a good subset of features. To overcome this drawback, we apply a piecewise-linear approximation to the logistic loss function. Accordingly, we frame the feature subset selection problem of minimizing an information criterion as a mixed integer linear optimization problem. The computational results demonstrate that our piecewise-linear approximation approach found a better subset of features than the quadratic approximation approach.