Quickest Change Detection for Unnormalized Statistical Models
This work addresses change detection for unnormalized models in machine learning, offering a practical solution for scenarios where explicit distributions are hard to compute, though it is incremental as it builds on classical CUSUM methods.
The paper tackled the problem of quickest change detection for unnormalized statistical models, where traditional methods are infeasible due to complex distribution computations, and developed the Score-based CUSUM (SCUSUM) algorithm based on Fisher divergence and Hyvärinen score, achieving asymptotic optimality with demonstrated performance in numerical results.
Classical quickest change detection algorithms require modeling pre-change and post-change distributions. Such an approach may not be feasible for various machine learning models because of the complexity of computing the explicit distributions. Additionally, these methods may suffer from a lack of robustness to model mismatch and noise. This paper develops a new variant of the classical Cumulative Sum (CUSUM) algorithm for the quickest change detection. This variant is based on Fisher divergence and the Hyvärinen score and is called the Score-based CUSUM (SCUSUM) algorithm. The SCUSUM algorithm allows the applications of change detection for unnormalized statistical models, i.e., models for which the probability density function contains an unknown normalization constant. The asymptotic optimality of the proposed algorithm is investigated by deriving expressions for average detection delay and the mean running time to a false alarm. Numerical results are provided to demonstrate the performance of the proposed algorithm.