Out-of-Distribution generalization of quantile regression with heavy tailed inputs: an SVM approach
For statisticians and machine learning practitioners, this work provides a theoretically grounded method for extreme quantile regression in high-dimensional nonlinear settings, though it is an incremental extension of existing SVM and extreme value theory ideas.
The paper tackles quantile regression with heavy-tailed covariates in an extrapolation regime, proposing an SVM framework that achieves finite-sample learning guarantees. Empirical results on Danube river flow data demonstrate practical relevance.
We study quantile regression in an extrapolation regime where the covariate takes unusually large values. Under regular variation assumptions, extreme observations can be effectively characterized through their angular components, enabling learning strategies that focus on the angle of the most extreme observations. This approach is formalized through the minimization of an asymptotic conditional risk that localizes learning in the tail of the covariate distribution. We propose a novel Support Vector Machine (SVM) framework for extreme quantile regression, leveraging reproducing kernel Hilbert spaces to handle high-dimensional and nonlinear settings. Our method also accommodates unbounded response variables and avoids restrictive transformations. We establish finite-sample learning guarantees under mild regularity assumptions. The proposed framework unifies ideas from statistical learning and multivariate extremes, providing a tractable and theoretically grounded approach to extrapolation. We complement our theoretical findings with an empirical study on river flow data from the Danube, demonstrating the practical relevance of our methods.