Support Vector Regression: Risk Quadrangle Framework
It provides a theoretical unification for SVR methods, linking them to risk management and optimization frameworks, which is incremental for researchers in machine learning and statistics.
This paper investigates Support Vector Regression (SVR) within the Risk Quadrangle (RQ) theory, showing that ε-SVR and ν-SVR reduce to minimizing the Vapnik error and Conditional Value-at-Risk (CVaR) norm, respectively, and are asymptotically unbiased estimators of symmetric conditional quantiles.
This paper investigates Support Vector Regression (SVR) within the framework of the Risk Quadrangle (RQ) theory. Every RQ includes four stochastic functionals -- error, regret, risk, and \emph{deviation}, bound together by a so-called statistic. The RQ framework unifies stochastic optimization, risk management, and statistical estimation. Within this framework, both $\varepsilon$-SVR and $ν$-SVR are shown to reduce to the minimization of the \emph{Vapnik error} and the Conditional Value-at-Risk (CVaR) norm, respectively. The Vapnik error and CVaR norm define quadrangles with a statistic equal to the average of two symmetric quantiles. Therefore, RQ theory implies that $\varepsilon$-SVR and $ν$-SVR are asymptotically unbiased estimators of the average of two symmetric conditional quantiles. Moreover, the equivalence between $\varepsilon$-SVR and $ν$-SVR is demonstrated in a general stochastic setting. Additionally, SVR is formulated as a deviation minimization problem. Another implication of the RQ theory is the formulation of $ν$-SVR as a Distributionally Robust Regression (DRR) problem. Finally, an alternative dual formulation of SVR within the RQ framework is derived. Theoretical results are validated with a case study.