Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead
This survey identifies challenges and mitigation strategies for ensuring the security and reliability of ML systems, especially for developers working with resource-constrained edge ML devices.
This paper surveys vulnerabilities in machine learning (ML) systems, particularly in resource-constrained edge devices, and reviews existing defense mechanisms. It covers threats and mitigation techniques across hardware and software, from cloud training to edge inference, and discusses formal verification.
Machine Learning (ML) techniques have been rapidly adopted by smart Cyber-Physical Systems (CPS) and Internet-of-Things (IoT) due to their powerful decision-making capabilities. However, they are vulnerable to various security and reliability threats, at both hardware and software levels, that compromise their accuracy. These threats get aggravated in emerging edge ML devices that have stringent constraints in terms of resources (e.g., compute, memory, power/energy), and that therefore cannot employ costly security and reliability measures. Security, reliability, and vulnerability mitigation techniques span from network security measures to hardware protection, with an increased interest towards formal verification of trained ML models. This paper summarizes the prominent vulnerabilities of modern ML systems, highlights successful defenses and mitigation techniques against these vulnerabilities, both at the cloud (i.e., during the ML training phase) and edge (i.e., during the ML inference stage), discusses the implications of a resource-constrained design on the reliability and security of the system, identifies verification methodologies to ensure correct system behavior, and describes open research challenges for building secure and reliable ML systems at both the edge and the cloud.