Regularization Strategies for Quantile Regression
This work provides improved regularization strategies for researchers and practitioners working with quantile regression, particularly for ensuring non-crossing quantiles and better calibration.
This paper explores regularization techniques for quantile regression, focusing on predicting subsets of quantiles or the full inverse CDF. It demonstrates that minimizing an expected pinball loss over a continuous distribution of quantiles acts as an effective regularizer, even for single-quantile predictions. For multiple quantiles, the paper proposes using deep lattice networks with monotonic input features to ensure non-crossing quantiles and regularize the predicted distribution to a location-scale family.
We investigate different methods for regularizing quantile regression when predicting either a subset of quantiles or the full inverse CDF. We show that minimizing an expected pinball loss over a continuous distribution of quantiles is a good regularizer even when only predicting a specific quantile. For predicting multiple quantiles, we propose achieving the classic goal of non-crossing quantiles by using deep lattice networks that treat the quantile as a monotonic input feature, and we discuss why monotonicity on other features is an apt regularizer for quantile regression. We show that lattice models enable regularizing the predicted distribution to a location-scale family. Lastly, we propose applying rate constraints to improve the calibration of the quantile predictions on specific subsets of interest and improve fairness metrics. We demonstrate our contributions on simulations, benchmark datasets, and real quantile regression problems.