Quality Monitoring and Assessment of Deployed Deep Learning Models for Network AIOps
This work addresses the need for maintaining and updating DL models in network operations, but it is incremental as it applies existing DevOps practices to AI in a specific domain.
The paper tackles the problem of monitoring and assessing the quality of deployed deep learning models in network AIOps, proposing techniques for individual inference assessment and overall model tracking, and applies them to network management and image recognition use cases.
Artificial Intelligence (AI) has recently attracted a lot of attention, transitioning from research labs to a wide range of successful deployments in many fields, which is particularly true for Deep Learning (DL) techniques. Ultimately, DL models being software artifacts, they need to be regularly maintained and updated: AIOps is the logical extension of the DevOps software development practices to AI-software applied to network operation and management. In the lifecycle of a DL model deployment, it is important to assess the quality of deployed models, to detect "stale" models and prioritize their update. In this article, we cover the issue in the context of network management, proposing simple yet effective techniques for (i) quality assessment of individual inference, and for (ii) overall model quality tracking over multiple inferences, that we apply to two use cases, representative of the network management and image recognition fields.