A sparse PAC-Bayesian approach for high-dimensional quantile prediction
This addresses robust quantile estimation in high-dimensional settings for fields like econometrics and machine learning, representing an incremental improvement over existing penalized and Bayesian methods.
The paper tackles high-dimensional quantile prediction by proposing a sparse PAC-Bayesian method with a scaled Student-t prior and Langevin Monte Carlo, achieving minimax-optimal prediction error and competitive performance in simulations and real-world data.
Quantile regression, a robust method for estimating conditional quantiles, has advanced significantly in fields such as econometrics, statistics, and machine learning. In high-dimensional settings, where the number of covariates exceeds sample size, penalized methods like lasso have been developed to address sparsity challenges. Bayesian methods, initially connected to quantile regression via the asymmetric Laplace likelihood, have also evolved, though issues with posterior variance have led to new approaches, including pseudo/score likelihoods. This paper presents a novel probabilistic machine learning approach for high-dimensional quantile prediction. It uses a pseudo-Bayesian framework with a scaled Student-t prior and Langevin Monte Carlo for efficient computation. The method demonstrates strong theoretical guarantees, through PAC-Bayes bounds, that establish non-asymptotic oracle inequalities, showing minimax-optimal prediction error and adaptability to unknown sparsity. Its effectiveness is validated through simulations and real-world data, where it performs competitively against established frequentist and Bayesian techniques.