Statistical process monitoring of artificial neural networks
This work addresses the need for real-time, low-cost monitoring of AI models during deployment, which is crucial for maintaining prediction accuracy in applications like machine learning systems, though it is incremental as it builds on existing statistical process control methods.
The paper tackles the problem of monitoring artificial neural networks for nonstationarity in data streams by using latent feature embeddings and multivariate control charts, achieving competitive performance compared to benchmark approaches across various architectures and data formats.
The rapid advancement of models based on artificial intelligence demands innovative monitoring techniques which can operate in real time with low computational costs. In machine learning, especially if we consider artificial neural networks (ANNs), the models are often trained in a supervised manner. Consequently, the learned relationship between the input and the output must remain valid during the model's deployment. If this stationarity assumption holds, we can conclude that the ANN provides accurate predictions. Otherwise, the retraining or rebuilding of the model is required. We propose considering the latent feature representation of the data (called "embedding") generated by the ANN to determine the time when the data stream starts being nonstationary. In particular, we monitor embeddings by applying multivariate control charts based on the data depth calculation and normalized ranks. The performance of the introduced method is compared with benchmark approaches for various ANN architectures and different underlying data formats.