Efficient Strongly Polynomial Algorithms for Quantile Regression
This work addresses a computational bottleneck for statisticians and machine learning practitioners by providing faster algorithms for robust regression, though it is incremental as it builds on existing QR methods.
The paper tackles the lack of efficient strongly polynomial algorithms for Quantile Regression (QR) compared to Ordinary Least Squares, proposing new algorithms with deterministic worst-case time complexity of O(n^{4/3} polylog(n)) and expected complexities as low as O(n log^2(n)) for two-dimensional QR, and O(n^{d-1} log^2(n)) for higher dimensions.
Linear Regression is a seminal technique in statistics and machine learning, where the objective is to build linear predictive models between a response (i.e., dependent) variable and one or more predictor (i.e., independent) variables. In this paper, we revisit the classical technique of Quantile Regression (QR), which is statistically a more robust alternative to the other classical technique of Ordinary Least Square Regression (OLS). However, while there exist efficient algorithms for OLS, almost all of the known results for QR are only weakly polynomial. Towards filling this gap, this paper proposes several efficient strongly polynomial algorithms for QR for various settings. For two dimensional QR, making a connection to the geometric concept of $k$-set, we propose an algorithm with a deterministic worst-case time complexity of $\mathcal{O}(n^{4/3} polylog(n))$ and an expected time complexity of $\mathcal{O}(n^{4/3})$ for the randomized version. We also propose a randomized divide-and-conquer algorithm -- RandomizedQR with an expected time complexity of $\mathcal{O}(n\log^2{(n)})$ for two dimensional QR problem. For the general case with more than two dimensions, our RandomizedQR algorithm has an expected time complexity of $\mathcal{O}(n^{d-1}\log^2{(n)})$.