Sequential Gradient Descent and Quasi-Newton's Method for Change-Point Analysis
This work addresses a computational bottleneck for researchers and practitioners in statistics and data analysis by providing a faster method for change-point detection in models like generalized linear models and penalized regression.
The paper tackles the computational inefficiency in change-point detection by introducing a sequential method (SE) that integrates with gradient descent (SeGD) and quasi-Newton's method (SeN) to update cost values without re-optimization, resulting in orders of magnitude faster performance than the PELT method while maintaining accuracy.
One common approach to detecting change-points is minimizing a cost function over possible numbers and locations of change-points. The framework includes several well-established procedures, such as the penalized likelihood and minimum description length. Such an approach requires finding the cost value repeatedly over different segments of the data set, which can be time-consuming when (i) the data sequence is long and (ii) obtaining the cost value involves solving a non-trivial optimization problem. This paper introduces a new sequential method (SE) that can be coupled with gradient descent (SeGD) and quasi-Newton's method (SeN) to find the cost value effectively. The core idea is to update the cost value using the information from previous steps without re-optimizing the objective function. The new method is applied to change-point detection in generalized linear models and penalized regression. Numerical studies show that the new approach can be orders of magnitude faster than the Pruned Exact Linear Time (PELT) method without sacrificing estimation accuracy.