Jiliang Zhang

Yansong Gao, Bao Gia Doan, Zhi Zhang et al.

This work provides the community with a timely comprehensive review of backdoor attacks and countermeasures on deep learning. According to the attacker's capability and affected stage of the machine learning pipeline, the attack surfaces are recognized to be wide and then formalized into six categorizations: code poisoning, outsourcing, pretrained, data collection, collaborative learning and post-deployment. Accordingly, attacks under each categorization are combed. The countermeasures are categorized into four general classes: blind backdoor removal, offline backdoor inspection, online backdoor inspection, and post backdoor removal. Accordingly, we review countermeasures, and compare and analyze their advantages and disadvantages. We have also reviewed the flip side of backdoor attacks, which are explored for i) protecting intellectual property of deep learning models, ii) acting as a honeypot to catch adversarial example attacks, and iii) verifying data deletion requested by the data contributor.Overall, the research on defense is far behind the attack, and there is no single defense that can prevent all types of backdoor attacks. In some cases, an attacker can intelligently bypass existing defenses with an adaptive attack. Drawing the insights from the systematic review, we also present key areas for future research on the backdoor, such as empirical security evaluations from physical trigger attacks, and in particular, more efficient and practical countermeasures are solicited.

Xinbo Liu, Jiliang Zhang, Yaping Lin et al.

Since the threat of malicious software (malware) has become increasingly serious, automatic malware detection techniques have received increasing attention, where machine learning (ML)-based visualization detection methods become more and more popular. In this paper, we demonstrate that the state-of-the-art ML-based visualization detection methods are vulnerable to Adversarial Example (AE) attacks. We develop a novel Adversarial Texture Malware Perturbation Attack (ATMPA) method based on the gradient descent and L-norm optimization method, where attackers can introduce some tiny perturbations on the transformed dataset such that ML-based malware detection methods will completely fail. The experimental results on the MS BIG malware dataset show that a small interference can reduce the accuracy rate down to 0% for several ML-based detection methods, and the rate of transferability is 74.1% on average.

Jiliang Zhang, Haihan Su

It is a challenging task to deploy lightweight security protocols in resource-constrained IoT applications. A hardware-oriented lightweight authentication protocol based on device signature generated during voltage over-scaling (VOS) was recently proposed to address this issue. VOS-based authentication employs the computation unit such as adders to generate the process variation dependent error which is combined with secret keys to create a two-factor authentication protocol. In this paper, machine learning (ML)-based modeling attacks to break such authentication is presented. We also propose a dynamic obfuscation mechanism based on keys (DOMK) for the VOS-based authentication to resist ML attacks. Experimental results show that ANN, RNN and CMA-ES can clone the challenge-response behavior of VOS-based authentication with up to 99.65% predication accuracy, while the predication accuracy is less than 51.2% after deploying our proposed ML resilient technique.

Jiliang Zhang, Chaoqun Shen

Strong physical unclonable function (PUF) is a promising solution for device authentication in resourceconstrained applications but vulnerable to machine learning attacks. In order to resist such attack, many defenses have been proposed in recent years. However, these defenses incur high hardware overhead, degenerate reliability and are inefficient against advanced machine learning attacks such as approximation attacks. In order to address these issues, we propose a Random Set-based Obfuscation (RSO) for Strong PUFs to resist machine learning attacks. The basic idea is that several stable responses are derived from the PUF itself and pre-stored as the set for obfuscation in the testing phase, and then a true random number generator is used to select any two keys to obfuscate challenges and responses with XOR operations. When the number of challenge-response pairs (CRPs) collected by the attacker exceeds the given threshold, the set will be updated immediately. In this way, machine learning attacks can be prevented with extremely low hardware overhead. Experimental results show that for a 64x64 Arbiter PUF, when the size of set is 32 and even if 1 million CRPs are collected by attackers, the prediction accuracies of Logistic regression, support vector machines, artificial neural network, convolutional neural network and covariance matrix adaptive evolutionary strategy are about 50% which is equivalent to the random guessing.