Automatic Response Category Combination in Multinomial Logistic Regression
This work provides a domain-specific solution for statistical modeling in categorical data analysis, but it appears incremental as it builds on existing penalized methods without introducing a new paradigm.
The authors tackled the problem of automatically combining response categories in multinomial logistic regression by proposing a penalized likelihood method that encourages pairwise equality of columns, resulting in a method that addresses prediction and model selection with an ADMM algorithm for computation.
We propose a penalized likelihood method that simultaneously fits the multinomial logistic regression model and combines subsets of the response categories. The penalty is non differentiable when pairs of columns in the optimization variable are equal. This encourages pairwise equality of these columns in the estimator, which corresponds to response category combination. We use an alternating direction method of multipliers algorithm to compute the estimator and we discuss the algorithm's convergence. Prediction and model selection are also addressed.