Breaking SafetyCore: Exploring the Risks of On-Device AI Deployment
This highlights a critical security problem for developers and users of on-device AI systems, exposing vulnerabilities that adversaries can exploit.
The paper tackled the security risks of on-device AI deployment by analyzing SafetyCore, an Android system service, and demonstrated that its AI model can be extracted and manipulated to bypass detection, rendering the protection ineffective.
Due to hardware and software improvements, an increasing number of AI models are deployed on-device. This shift enhances privacy and reduces latency, but also introduces security risks distinct from traditional software. In this article, we examine these risks through the real-world case study of SafetyCore, an Android system service incorporating sensitive image content detection. We demonstrate how the on-device AI model can be extracted and manipulated to bypass detection, effectively rendering the protection ineffective. Our analysis exposes vulnerabilities of on-device AI models and provides a practical demonstration of how adversaries can exploit them.