Robustness Analysis of Deep Learning Frameworks on Mobile Platforms
This study addresses the robustness of deep learning frameworks for mobile devices, which is important for developers and users relying on on-device ML tasks, but it is incremental as it builds on existing adversarial attack research.
The paper empirically compares the robustness of two on-device deep learning frameworks against adversarial attacks on mobile platforms, finding that neither framework is generally more robust, but quantization improves robustness when moving from PC to mobile.
With the recent increase in the computational power of modern mobile devices, machine learning-based heavy tasks such as face detection and speech recognition are now integral parts of such devices. This requires frameworks to execute machine learning models (e.g., Deep Neural Networks) on mobile devices. Although there exist studies on the accuracy and performance of these frameworks, the quality of on-device deep learning frameworks, in terms of their robustness, has not been systematically studied yet. In this paper, we empirically compare two on-device deep learning frameworks with three adversarial attacks on three different model architectures. We also use both the quantized and unquantized variants for each architecture. The results show that, in general, neither of the deep learning frameworks is better than the other in terms of robustness, and there is not a significant difference between the PC and mobile frameworks either. However, in cases like Boundary attack, mobile version is more robust than PC. In addition, quantization improves robustness in all cases when moving from PC to mobile.