Predicting Breakdowns in Cloud Services (with SPIKE)
This addresses the critical issue of service reliability for cloud service providers, where downtime leads to revenue and reputation loss, though it is incremental as it builds on existing machine learning techniques.
The paper tackles the problem of predicting service breakdowns in cloud services to prevent downtime, and presents SPIKE, a data mining tool that predicts breakdowns 30 minutes in advance with recalls and precision of 75% and above.
Maintaining web-services is a mission-critical task where any down-time means loss of revenue and reputation (of being a reliable service provider). In the current competitive web services market, such a loss of reputation causes extensive loss of future revenue. To address this issue, we developed SPIKE, a data mining tool which can predict upcoming service breakdowns, half an hour into the future. Such predictions let an organization alert and assemble the tiger team to address the problem (e.g. by reconfiguring cloud hardware in order to reduce the likelihood of that breakdown). SPIKE utilizes (a) regression tree learning (with CART); (b) synthetic minority over-sampling (to handle how rare spikes are in our data); (c) hyperparameter optimization (to learn best settings for our local data) and (d) a technique we called "topology sampling" where training vectors are built from extensive details of an individual node plus summary details on all their neighbors. In the experiments reported here, SPIKE predicted service spikes 30 minutes into future with recalls and precision of 75% and above. Also, SPIKE performed relatively better than other widely-used learning methods (neural nets, random forests, logistic regression).