Cost Trade-offs in Matrix Inversion Updates for Streaming Outlier Detection
This provides an incremental improvement for researchers and practitioners developing efficient online outlier detection techniques by optimizing computational methods for matrix inversion updates.
The paper tackled the problem of efficiently updating matrix inverses for online outlier detection using the Christoffel function, comparing three methods (Direct Inversion, Iterative Sherman-Morrison, and Woodbury Matrix Identity) through theoretical cost analysis and simulations, resulting in a rule that ISM is optimal for rank-1 updates, WMI for small updates relative to matrix size, and DI otherwise.
Outlier detection identifies data points that deviate significantly from expected patterns, revealing anomalies that may require special attention. Incorporating online learning further improves accuracy by continuously updating the model to reflect the most recent data. When employing the Christoffel function as an outlier score, online learning requires updating the inverse of a matrix following a rank-k update, given the initial inverse. Surprisingly, there is no consensus on the optimal method for this task. This technical note aims to compare three different updating methods: Direct Inversion (DI), Iterative Sherman-Morrison (ISM), and Woodbury Matrix Identity (WMI), to identify the most suitable approach for different scenarios. We first derive the theoretical computational costs of each method and then validate these findings through comprehensive Python simulations run on a CPU. These results allow us to propose a simple, quantitative, and easy-to-remember rule that can be stated qualitatively as follows: ISM is optimal for rank-1 updates, WMI excels for small updates relative to matrix size, and DI is preferable otherwise. This technical note produces a general result for any problem involving a matrix inversion update. In particular, it contributes to the ongoing development of efficient online outlier detection techniques.