DiagNet: towards a generic, Internet-scale root cause analysis solution
This addresses the costly and difficult task of problem diagnosis for content providers and ISPs in decentralized Internet services, though it appears incremental by adapting existing image processing techniques to network metrics.
The paper tackles the problem of diagnosing issues in Internet-scale services by developing DiagNet, a machine learning-based root cause analysis solution that uses end-user measurements and achieves a recall of 73.9% for identifying causes, including those introduced only during inference.
Diagnosing problems in Internet-scale services remains particularly difficult and costly for both content providers and ISPs. Because the Internet is decentralized, the cause of such problems might lie anywhere between an end-user's device and the service datacenters. Further, the set of possible problems and causes is not known in advance, making it impossible in practice to train a classifier with all combinations of problems, causes and locations. In this paper, we explore how different machine learning techniques can be used for Internet-scale root cause analysis using measurements taken from end-user devices. We show how to build generic models that (i) are agnostic to the underlying network topology, (ii) do not require to define the full set of possible causes during training, and (iii) can be quickly adapted to diagnose new services. Our solution, DiagNet, adapts concepts from image processing research to handle network and system metrics. We evaluate DiagNet with a multi-cloud deployment of online services with injected faults and emulated clients with automated browsers. We demonstrate promising root cause analysis capabilities, with a recall of 73.9% including causes only being introduced at inference time.