Improving Predictability of User-Affecting Metrics to Support Anomaly Detection in Cloud Services
This work provides insights for system administrators and security engineers on configuring cloud services to balance performance and security, particularly for signature-based intrusion detection systems.
This paper explores the trade-off between anomaly detection using performance signatures and system scalability in cloud services. It finds that an optimal number of servers exists to minimize response time variance and maximize classification accuracy, which is often less than or equal to the number that minimizes mean response time.
Anomaly detection systems aim to detect and report attacks or unexpected behavior in networked systems. Previous work has shown that anomalies have an impact on system performance, and that performance signatures can be effectively used for implementing an IDS. In this paper, we present an analytical and an experimental study on the trade-off between anomaly detection based on performance signatures and system scalability. The proposed approach combines analytical modeling and load testing to find optimal configurations for the signature-based IDS. We apply a heavy-tail bi-modal modeling approach, where "long" jobs represent large resource consuming transactions, e.g., generated by DDoS attacks; the model was parametrized using results obtained from controlled experiments. For performance purposes, mean response time is the key metric to be minimized, whereas for security purposes, response time variance and classification accuracy must be taken into account. The key insights from our analysis are: (i) there is an optimal number of servers which minimizes the response time variance, (ii) the sweet-spot number of servers that minimizes response time variance and maximizes classification accuracy is typically smaller than or equal to the one that minimizes mean response time. Therefore, for security purposes, it may be worth slightly sacrificing performance to increase classification accuracy.