Multiclass classification by sparse multinomial logistic regression
This work addresses feature selection in multiclass classification for high-dimensional data, offering incremental improvements through computationally feasible methods like group Lasso and Slope.
The paper tackles high-dimensional multiclass classification by proposing sparse multinomial logistic regression methods, deriving nonasymptotic bounds for misclassification excess risk and showing they achieve minimax optimality, with results varying based on the number of classes and noise conditions.
In this paper we consider high-dimensional multiclass classification by sparse multinomial logistic regression. We propose first a feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the nonasymptotic bounds for misclassification excess risk of the resulting classifier. We establish also their tightness by deriving the corresponding minimax lower bounds. In particular, we show that there exist two regimes corresponding to small and large number of classes. The bounds can be reduced under the additional low noise condition. To find a penalized maximum likelihood solution with a complexity penalty requires, however, a combinatorial search over all possible models. To design a feature selection procedure computationally feasible for high-dimensional data, we propose multinomial logistic group Lasso and Slope classifiers and show that they also achieve the minimax order.