Liehuang Zhu

Meng Shen, Zhehui Tan, Dusit Niyato et al.

Web 3.0 is the new generation of the Internet that is reconstructed with distributed technology, which focuses on data ownership and value expression. Also, it operates under the principle that data and digital assets should be owned and controlled by users rather than large corporations. In this survey, we explore the current development state of Web 3.0 and the application of AI Technology in Web 3.0. Through investigating the existing applications and components of Web 3.0, we propose an architectural framework for Web 3.0 from the perspective of ecological application scenarios. We outline and divide the ecology of Web 3.0 into four layers. The main functions of each layer are data management, value circulation, ecological governance, and application scenarios. Our investigation delves into the major challenges and issues present in each of these layers. In this context, AI has shown its strong potential to solve existing problems of Web 3.0. We illustrate the crucial role of AI in the foundation and growth of Web 3.0. We begin by providing an overview of AI, including machine learning algorithms and deep learning techniques. Then, we thoroughly analyze the current state of AI technology applications in the four layers of Web 3.0 and offer some insights into its potential future development direction.

Zhibo Xing, Zijian Zhang, Ziang Zhang et al.

Over recent decades, machine learning has significantly advanced network communication, enabling improved decision-making, user behavior analysis, and fault detection. Decentralized approaches, where participants exchange computation results instead of raw private data, mitigate these risks but introduce challenges related to trust and verifiability. A critical issue arises: How can one ensure the integrity and validity of computation results shared by other participants? Existing survey articles predominantly address security and privacy concerns in decentralized machine learning, whereas this survey uniquely highlights the emerging issue of verifiability. Recognizing the critical role of zero-knowledge proofs in ensuring verifiability, we present a comprehensive review of Zero-Knowledge Proof-based Verifiable Machine Learning (ZKP-VML). To clarify the research problem, we present a definition of ZKP-VML consisting of four algorithms, along with several corresponding key security properties. Besides, we provide an overview of the current research landscape by systematically organizing the research timeline and categorizing existing schemes based on their security properties. Furthermore, through an in-depth analysis of each existing scheme, we summarize their technical contributions and optimization strategies, aiming to uncover common design principles underlying ZKP-VML schemes. Building on the reviews and analysis presented, we identify current research challenges and suggest future research directions. To the best of our knowledge, this is the most comprehensive survey to date on verifiable decentralized machine learning and ZKP-VML.

Xiaoyan Yu, Shen Zhou, Huafeng Li et al.

The use of a single image restoration framework to achieve multi-task image restoration has garnered significant attention from researchers. However, several practical challenges remain, including meeting the specific and simultaneous demands of different tasks, balancing relationships between tasks, and effectively utilizing task correlations in model design. To address these challenges, this paper explores a multi-expert adaptive selection mechanism. We begin by designing a feature representation method that accounts for both the pixel channel level and the global level, encompassing low-frequency and high-frequency components of the image. Based on this method, we construct a multi-expert selection and ensemble scheme. This scheme adaptively selects the most suitable expert from the expert library according to the content of the input image and the prompts of the current task. It not only meets the individualized needs of different tasks but also achieves balance and optimization across tasks. By sharing experts, our design promotes interconnections between different tasks, thereby enhancing overall performance and resource utilization. Additionally, the multi-expert mechanism effectively eliminates irrelevant experts, reducing interference from them and further improving the effectiveness and accuracy of image restoration. Experimental results demonstrate that our proposed method is both effective and superior to existing approaches, highlighting its potential for practical applications in multi-task image restoration.

Jialing He, Jiamou Liu, Zijian Zhang et al.

Non-intrusive load monitoring (NILM) aims to decompose aggregated electrical usage signal into appliance-specific power consumption and it amounts to a classical example of blind source separation tasks. Leveraging recent progress on deep learning techniques, we design a new neural NILM model Multi-State Dual CNN (MSDC). Different from previous models, MSDC explicitly extracts information about the appliance's multiple states and state transitions, which in turn regulates the prediction of signals for appliances. More specifically, we employ a dual-CNN architecture: one CNN for outputting state distributions and the other for predicting the power of each state. A new technique is invented that utilizes conditional random fields (CRF) to capture state transitions. Experiments on two real-world datasets REDD and UK-DALE demonstrate that our model significantly outperform state-of-the-art models while having good generalization capacity, achieving 6%-10% MAE gain and 33%-51% SAE gain to unseen appliances.

Xiaoyan Yu, Yifan Wei, Pu Li et al.

Training social event detection models through federated learning (FedSED) aims to improve participants' performance on the task. However, existing federated learning paradigms are inadequate for achieving FedSED's objective and exhibit limitations in handling the inherent heterogeneity in social data. This paper proposes a personalized federated learning framework with a dual aggregation mechanism for social event detection, namely DAMe. We present a novel local aggregation strategy utilizing Bayesian optimization to incorporate global knowledge while retaining local characteristics. Moreover, we introduce a global aggregation strategy to provide clients with maximum external knowledge of their preferences. In addition, we incorporate a global-local event-centric constraint to prevent local overfitting and ``client-drift''. Experiments within a realistic simulation of a natural federated setting, utilizing six social event datasets spanning six languages and two social media platforms, along with an ablation study, have demonstrated the effectiveness of the proposed framework. Further robustness analyses have shown that DAMe is resistant to injection attacks.

Xiaoyan Yu, Tongxu Luo, Yifan Wei et al.

Large Language Models (LLMs) have revolutionized open-domain dialogue agents but encounter challenges in multi-character role-playing (MCRP) scenarios. To address the issue, we present Neeko, an innovative framework designed for efficient multiple characters imitation. Unlike existing methods, Neeko employs a dynamic low-rank adapter (LoRA) strategy, enabling it to adapt seamlessly to diverse characters. Our framework breaks down the role-playing process into agent pre-training, multiple characters playing, and character incremental learning, effectively handling both seen and unseen roles. This dynamic approach, coupled with distinct LoRA blocks for each character, enhances Neeko's adaptability to unique attributes, personalities, and speaking patterns. As a result, Neeko demonstrates superior performance in MCRP over most existing methods, offering more engaging and versatile user interaction experiences. Code and data are available at https://github.com/weiyifan1023/Neeko.

Xiaoyan Yu, Neng Dong, Liehuang Zhu et al.

Visible-infrared person re-identification (VIReID) primarily deals with matching identities across person images from different modalities. Due to the modality gap between visible and infrared images, cross-modality identity matching poses significant challenges. Recognizing that high-level semantics of pedestrian appearance, such as gender, shape, and clothing style, remain consistent across modalities, this paper intends to bridge the modality gap by infusing visual features with high-level semantics. Given the capability of CLIP to sense high-level semantic information corresponding to visual representations, we explore the application of CLIP within the domain of VIReID. Consequently, we propose a CLIP-Driven Semantic Discovery Network (CSDN) that consists of Modality-specific Prompt Learner, Semantic Information Integration (SII), and High-level Semantic Embedding (HSE). Specifically, considering the diversity stemming from modality discrepancies in language descriptions, we devise bimodal learnable text tokens to capture modality-private semantic information for visible and infrared images, respectively. Additionally, acknowledging the complementary nature of semantic details across different modalities, we integrate text features from the bimodal language descriptions to achieve comprehensive semantics. Finally, we establish a connection between the integrated text features and the visual features across modalities. This process embed rich high-level semantic information into visual representations, thereby promoting the modality invariance of visual representations. The effectiveness and superiority of our proposed CSDN over existing methods have been substantiated through experimental evaluations on multiple widely used benchmarks. The code will be released at \url{https://github.com/nengdong96/CSDN}.

Xiaoyan Yu, Yifan Wei, Shuaishuai Zhou et al.

The vast, complex, and dynamic nature of social message data has posed challenges to social event detection (SED). Despite considerable effort, these challenges persist, often resulting in inadequately expressive message representations (ineffective) and prolonged learning durations (inefficient). In response to the challenges, this work introduces an unsupervised framework, HyperSED (Hyperbolic SED). Specifically, the proposed framework first models social messages into semantic-based message anchors, and then leverages the structure of the anchor graph and the expressiveness of the hyperbolic space to acquire structure- and geometry-aware anchor representations. Finally, HyperSED builds the partitioning tree of the anchor message graph by incorporating differentiable structural information as the reflection of the detected events. Extensive experiments on public datasets demonstrate HyperSED's competitive performance, along with a substantial improvement in efficiency compared to the current state-of-the-art unsupervised paradigm. Statistically, HyperSED boosts incremental SED by an average of 2%, 2%, and 25% in NMI, AMI, and ARI, respectively; enhancing efficiency by up to 37.41 times and at least 12.10 times, illustrating the advancement of the proposed framework. Our code is publicly available at https://github.com/XiaoyanWork/HyperSED.

Shuo Wang, Keke Gai, Jing Yu et al.

Vertical Federated Learning (VFL) has garnered significant attention as a privacy-preserving machine learning framework for sample-aligned feature federation. However, traditional VFL approaches do not address the challenges of class and feature continual learning, resulting in catastrophic forgetting of knowledge from previous tasks. To address the above challenge, we propose a novel vertical federated continual learning method, named Vertical Federated Continual Learning via Evolving Prototype Knowledge (V-LETO), which primarily facilitates the transfer of knowledge from previous tasks through the evolution of prototypes. Specifically, we propose an evolving prototype knowledge method, enabling the global model to retain both previous and current task knowledge. Furthermore, we introduce a model optimization technique that mitigates the forgetting of previous task knowledge by restricting updates to specific parameters of the local model, thereby enhancing overall performance. Extensive experiments conducted in both CIL and FIL settings demonstrate that our method, V-LETO, outperforms the other state-of-the-art methods. For example, our method outperforms the state-of-the-art method by 10.39% and 35.15% for CIL and FIL tasks, respectively. Our code is available at https://anonymous.4open.science/r/V-LETO-0108/README.md.

Liangqi Lei, Keke Gai, Jing Yu et al.

Latent Diffusion Models (LDMs) enable a wide range of applications but raise ethical concerns regarding illegal utilization. Adding watermarks to generative model outputs is a vital technique employed for copyright tracking and mitigating potential risks associated with Artificial Intelligence (AI)-generated contents. However, post-processed watermarking methods are unable to withstand generative watermark attacks and there exists a trade-off between image fidelity and watermark strength. Therefore, we propose a novel technique called DiffuseTrace. DiffuseTrace does not rely on fine-tuning of the diffusion model components. The multi-bit watermark is a embedded into the image space semantically without compromising image quality. The watermark component can be utilized as a plug-in in arbitrary diffusion models. We validate through experiments the effectiveness and flexibility of DiffuseTrace. Under 8 types of image processing watermark attacks and 3 types of generative watermark attacks, DiffuseTrace maintains watermark detection rate of 99% and attribution accuracy of over 94%.

Xuhao Ren, Haotian Liang, Yajie Wang et al.

To boost the encoder stealing attack under the perturbation-based defense that hinders the attack performance, we propose a boosting encoder stealing attack with perturbation recovery named BESA. It aims to overcome perturbation-based defenses. The core of BESA consists of two modules: perturbation detection and perturbation recovery, which can be combined with canonical encoder stealing attacks. The perturbation detection module utilizes the feature vectors obtained from the target encoder to infer the defense mechanism employed by the service provider. Once the defense mechanism is detected, the perturbation recovery module leverages the well-designed generative model to restore a clean feature vector from the perturbed one. Through extensive evaluations based on various datasets, we demonstrate that BESA significantly enhances the surrogate encoder accuracy of existing encoder stealing attacks by up to 24.63\% when facing state-of-the-art defenses and combinations of multiple defenses.

Chuan Zhang, Xixi Zheng, Xiaolong Tao et al.

With the rapid advancement of artificial intelligence, generative artificial intelligence (GAI) has taken a leading role in transforming data processing methods. However, the high computational demands of GAI present challenges for devices with limited resources. As we move towards the sixth generation of mobile networks (6G), the higher data rates and improved energy efficiency of 6G create a need for more efficient data processing in GAI. Traditional GAI, however, shows its limitations in meeting these demands. To address these challenges, we introduce a multi-level collaborative inference system designed for next-generation networks and communication. Our proposed system features a deployment strategy that assigns models of varying sizes to devices at different network layers. Then, we design a task offloading strategy to optimise both efficiency and latency. Furthermore, a modified early exit mechanism is implemented to enhance the inference process for single models. Experimental results demonstrate that our system effectively reduces inference latency while maintaining high-quality output. Specifically, compared to existing work, our system can reduce inference time by up to 17% without sacrificing the inference accuracy.

Jialing He, Jiacheng Wang, Ning Wang et al.

Smart grid, through networked smart meters employing the non-intrusive load monitoring (NILM) technique, can considerably discern the usage patterns of residential appliances. However, this technique also incurs privacy leakage. To address this issue, we propose an innovative scheme based on adversarial attack in this paper. The scheme effectively prevents NILM models from violating appliance-level privacy, while also ensuring accurate billing calculation for users. To achieve this objective, we overcome two primary challenges. First, as NILM models fall under the category of time-series regression models, direct application of traditional adversarial attacks designed for classification tasks is not feasible. To tackle this issue, we formulate a novel adversarial attack problem tailored specifically for NILM and providing a theoretical foundation for utilizing the Jacobian of the NILM model to generate imperceptible perturbations. Leveraging the Jacobian, our scheme can produce perturbations, which effectively misleads the signal prediction of NILM models to safeguard users' appliance-level privacy. The second challenge pertains to fundamental utility requirements, where existing adversarial attack schemes struggle to achieve accurate billing calculation for users. To handle this problem, we introduce an additional constraint, mandating that the sum of added perturbations within a billing period must be precisely zero. Experimental validation on real-world power datasets REDD and UK-DALE demonstrates the efficacy of our proposed solutions, which can significantly amplify the discrepancy between the output of the targeted NILM model and the actual power signal of appliances, and enable accurate billing at the same time. Additionally, our solutions exhibit transferability, making the generated perturbation signal from one target model applicable to other diverse NILM models.

Liangqi Lei, Keke Gai, Jing Yu et al.

The personalization techniques of diffusion models succeed in generating images with specific concepts. This ability also poses great threats to copyright protection and network security since malicious users can generate unauthorized content and disinformation relevant to a target concept. Model watermarking is an effective solution to trace the malicious generated images and safeguard their copyright. However, existing model watermarking techniques merely achieve image-level tracing without concept traceability. When tracing infringing or harmful concepts, current approaches execute image concept detection and model tracing sequentially, where performance is critically constrained by concept detection accuracy. In this paper, we propose a lightweight concept watermarking framework that efficiently binds target concepts to model watermarks, supporting simultaneous concept identification and model tracing via single-stage watermark verification. To further enhance the robustness of concept watermarking, we propose an adversarial perturbation injection method collaboratively embedded with watermarks during image generation, avoiding watermark removal by model purification attacks. Experimental results demonstrate that ConceptWM significantly outperforms state-of-the-art watermarking methods, improving detection accuracy by 6.3%-19.3% across diverse datasets including COCO and StableDiffusionDB. Additionally, ConceptWM possesses a critical capability absent in other watermarking methods: it sustains a 21.7% FID/CLIP degradation under adversarial fine-tuning of Stable Diffusion models on WikiArt and CelebA-HQ, demonstrating its capability to mitigate model misuse.

Mingfei Zeng, Ming Xie, Xixi Zheng et al.

The rapid development of Internet of Things (IoT) technology has transformed people's way of life and has a profound impact on both production and daily activities. However, with the rapid advancement of IoT technology, the security of IoT devices has become an unavoidable issue in both research and applications. Although some efforts have been made to detect or mitigate IoT security vulnerabilities, they often struggle to adapt to the complexity of IoT environments, especially when dealing with dynamic security scenarios. How to automatically, efficiently, and accurately understand these vulnerabilities remains a challenge. To address this, we propose an IoT security assistant driven by Large Language Model (LLM), which enhances the LLM's understanding of IoT security vulnerabilities and related threats. The aim of the ICoT method we propose is to enable the LLM to understand security issues by breaking down the various dimensions of security vulnerabilities and generating responses tailored to the user's specific needs and expertise level. By incorporating ICoT, LLM can gradually analyze and reason through complex security scenarios, resulting in more accurate, in-depth, and personalized security recommendations and solutions. Experimental results show that, compared to methods relying solely on LLM, our proposed LLM-driven IoT security assistant significantly improves the understanding of IoT security issues through the ICoT approach and provides personalized solutions based on the user's identity, demonstrating higher accuracy and reliability.

Jianbo Gao, Keke Gai, Jing Yu et al.

Recent advancement in large-scale Artificial Intelligence (AI) models offering multimodal services have become foundational in AI systems, making them prime targets for model theft. Existing methods select Out-of-Distribution (OoD) data as backdoor watermarks and retrain the original model for copyright protection. However, existing methods are susceptible to malicious detection and forgery by adversaries, resulting in watermark evasion. In this work, we propose Model-\underline{ag}nostic Black-box Backdoor W\underline{ate}rmarking Framework (AGATE) to address stealthiness and robustness challenges in multimodal model copyright protection. Specifically, we propose an adversarial trigger generation method to generate stealthy adversarial triggers from ordinary dataset, providing visual fidelity while inducing semantic shifts. To alleviate the issue of anomaly detection among model outputs, we propose a post-transform module to correct the model output by narrowing the distance between adversarial trigger image embedding and text embedding. Subsequently, a two-phase watermark verification is proposed to judge whether the current model infringes by comparing the two results with and without the transform module. Consequently, we consistently outperform state-of-the-art methods across five datasets in the downstream tasks of multimodal image-text retrieval and image classification. Additionally, we validated the robustness of AGATE under two adversarial attack scenarios.

Yanli Yuan, Bingbing Wang, Chuan Zhang et al.

Segmentation of organs of interest in medical CT images is beneficial for diagnosis of diseases. Though recent methods based on Fully Convolutional Neural Networks (F-CNNs) have shown success in many segmentation tasks, fusing features from images with different scales is still a challenge: (1) Due to the lack of spatial awareness, F-CNNs share the same weights at different spatial locations. (2) F-CNNs can only obtain surrounding information through local receptive fields. To address the above challenge, we propose a new segmentation framework based on attention mechanisms, named MFA-Net (Multi-Scale Feature Fusion Attention Network). The proposed framework can learn more meaningful feature maps among multiple scales and result in more accurate automatic segmentation. We compare our proposed MFA-Net with SOTA methods on two 2D liver CT datasets. The experimental results show that our MFA-Net produces more precise segmentation on images with different scales.

Shulin Lan, Kanlin Liu, Yazhou Zhao et al.

Current passive deepfake face-swapping detection methods encounter significance bottlenecks in model generalization capabilities. Meanwhile, proactive detection methods often use fixed watermarks which lack a close relationship with the content they protect and are vulnerable to security risks. Dynamic watermarks based on facial features offer a promising solution, as these features provide unique identifiers. Therefore, this paper proposes a Facial Feature-based Proactive deepfake detection method (FaceProtect), which utilizes changes in facial characteristics during deepfake manipulation as a novel detection mechanism. We introduce a GAN-based One-way Dynamic Watermark Generating Mechanism (GODWGM) that uses 128-dimensional facial feature vectors as inputs. This method creates irreversible mappings from facial features to watermarks, enhancing protection against various reverse inference attacks. Additionally, we propose a Watermark-based Verification Strategy (WVS) that combines steganography with GODWGM, allowing simultaneous transmission of the benchmark watermark representing facial features within the image. Experimental results demonstrate that our proposed method maintains exceptional detection performance and exhibits high practicality on images altered by various deepfake techniques.

Keke Gai, Ziyue Shen, Jing Yu et al.

With the growing demand for protecting the intellectual property (IP) of text-to-image diffusion models, we propose PCDiff -- a proactive access control framework that redefines model authorization by regulating generation quality. At its core, PCDIFF integrates a trainable fuser module and hierarchical authentication layers into the decoder architecture, ensuring that only users with valid encrypted credentials can generate high-fidelity images. In the absence of valid keys, the system deliberately degrades output quality, effectively preventing unauthorized exploitation.Importantly, while the primary mechanism enforces active access control through architectural intervention, its decoupled design retains compatibility with existing watermarking techniques. This satisfies the need of model owners to actively control model ownership while preserving the traceability capabilities provided by traditional watermarking approaches.Extensive experimental evaluations confirm a strong dependency between credential verification and image quality across various attack scenarios. Moreover, when combined with typical post-processing operations, PCDIFF demonstrates powerful performance alongside conventional watermarking methods. This work shifts the paradigm from passive detection to proactive enforcement of authorization, laying the groundwork for IP management of diffusion models.

Liangqi Lei, Keke Gai, Jing Yu et al.

Latent diffusion models have exhibited considerable potential in generative tasks. Watermarking is considered to be an alternative to safeguard the copyright of generative models and prevent their misuse. However, in the context of model distribution scenarios, the accessibility of models to large scale of model users brings new challenges to the security, efficiency and robustness of existing watermark solutions. To address these issues, we propose a secure and efficient watermarking solution. A new security mechanism is designed to prevent watermark leakage and watermark escape, which considers watermark randomness and watermark-model association as two constraints for mandatory watermark injection. To reduce the time cost of training the security module, watermark injection and the security mechanism are decoupled, ensuring that fine-tuning VAE only accomplishes the security mechanism without the burden of learning watermark patterns. A watermark distribution-based verification strategy is proposed to enhance the robustness against diverse attacks in the model distribution scenarios. Experimental results prove that our watermarking consistently outperforms existing six baselines on effectiveness and robustness against ten image processing attacks and adversarial attacks, while enhancing security in the distribution scenarios.

Zhibo Xing, Zijian Zhang, Zi'ang Zhang et al.

Federated learning allows several clients to train one machine learning model jointly without sharing private data, providing privacy protection. However, traditional federated learning is vulnerable to poisoning attacks, which can not only decrease the model performance, but also implant malicious backdoors. In addition, direct submission of local model parameters can also lead to the privacy leakage of the training dataset. In this paper, we aim to build a privacy-preserving and Byzantine-robust federated learning scheme to provide an environment with no vandalism (NoV) against attacks from malicious participants. Specifically, we construct a model filter for poisoned local models, protecting the global model from data and model poisoning attacks. This model filter combines zero-knowledge proofs to provide further privacy protection. Then, we adopt secret sharing to provide verifiable secure aggregation, removing malicious clients that disrupting the aggregation process. Our formal analysis proves that NoV can protect data privacy and weed out Byzantine attackers. Our experiments illustrate that NoV can effectively address data and model poisoning attacks, including PGD, and outperforms other related schemes.

Tianxiu Xie, Keke Gai, Jing Yu et al.

Recent booming development of Generative Artificial Intelligence (GenAI) has facilitated an emerging model commercialization for the purpose of reinforcement on model performance, such as licensing or trading Deep Neural Network (DNN) models. However, DNN model trading may trigger concerns of the unauthorized replications or misuses over the model, so that the benefit of the model ownership will be violated. Model identity auditing is a challenging issue in protecting intellectual property of DNN models and verifying the integrity and ownership of models for guaranteeing trusts in transactions is one of the critical obstacles. In this paper, we focus on the above issue and propose a novel Accumulator-enabled Auditing for Distributed Identity of DNN Model (A2-DIDM) that utilizes blockchain and zero-knowledge techniques to protect data and function privacy while ensuring the lightweight on-chain ownership verification. The proposed model presents a scheme of identity records via configuring model weight checkpoints with corresponding zero-knowledge proofs, which incorporates predicates to capture incremental state changes in model weight checkpoints. Our scheme ensures both computational integrity of DNN training process and programmability, so that the uniqueness of the weight checkpoint sequence in a DNN model is preserved, ensuring the correctness of the model identity auditing. In addition, A2-DIDM also addresses privacy protections in distributed identity via a proposed method of accumulators. We systematically analyze the security and robustness of our proposed model and further evaluate the effectiveness and usability of auditing DNN model identities.

Tianxiu Xie, Keke Gai, Jing Yu et al.

Distributed machine learning enables parallel training of extensive datasets by delegating computing tasks across multiple workers. Despite the cost reduction benefits of distributed machine learning, the dissemination of final model weights often leads to potential conflicts over model ownership as workers struggle to substantiate their involvement in the training computation. To address the above ownership issues and prevent accidental failures and malicious attacks, verifying the computational integrity and effectiveness of workers becomes particularly crucial in distributed machine learning. In this paper, we proposed a novel binary linear tree commitment-based ownership protection model to ensure computational integrity with limited overhead and concise proof. Due to the frequent updates of parameters during training, our commitment scheme introduces a maintainable tree structure to reduce the costs of updating proofs. Distinguished from SNARK-based verifiable computation, our model achieves efficient proof aggregation by leveraging inner product arguments. Furthermore, proofs of model weights are watermarked by worker identity keys to prevent commitments from being forged or duplicated. The performance analysis and comparison with SNARK-based hash commitments validate the efficacy of our model in preserving computational integrity within distributed machine learning.

Meng Shen, Hao Yu, Liehuang Zhu et al.

Deep neural networks (DNNs) have been increasingly used in face recognition (FR) systems. Recent studies, however, show that DNNs are vulnerable to adversarial examples, which can potentially mislead the FR systems using DNNs in the physical world. Existing attacks on these systems either generate perturbations working merely in the digital world, or rely on customized equipments to generate perturbations and are not robust in varying physical environments. In this paper, we propose FaceAdv, a physical-world attack that crafts adversarial stickers to deceive FR systems. It mainly consists of a sticker generator and a transformer, where the former can craft several stickers with different shapes and the latter transformer aims to digitally attach stickers to human faces and provide feedbacks to the generator to improve the effectiveness of stickers. We conduct extensive experiments to evaluate the effectiveness of FaceAdv on attacking 3 typical FR systems (i.e., ArcFace, CosFace and FaceNet). The results show that compared with a state-of-the-art attack, FaceAdv can significantly improve success rate of both dodging and impersonating attacks. We also conduct comprehensive evaluations to demonstrate the robustness of FaceAdv.

Liehuang Zhu, Xiangyun Tang, Meng Shen et al.

To develop Smart City, the growing popularity of Machine Learning (ML) that appreciates high-quality training datasets generated from diverse IoT devices raises natural questions about the privacy guarantees that can be provided in such settings. Privacy-preserving ML training in an aggregation scenario enables a model demander to securely train ML models with the sensitive IoT data gathered from personal IoT devices. Existing solutions are generally server-aided, cannot deal with the collusion threat between the servers or between the servers and data owners, and do not match the delicate environments of IoT. We propose a privacy-preserving ML training framework named Heda that consists of a library of building blocks based on partial homomorphic encryption (PHE) enabling constructing multiple privacy-preserving ML training protocols for the aggregation scenario without the assistance of untrusted servers and defending the security under collusion situations. Rigorous security analysis demonstrates the proposed protocols can protect the privacy of each participant in the honest-but-curious model and defend the security under most collusion situations. Extensive experiments validate the efficiency of Heda which achieves the privacy-preserving ML training without losing the model accuracy.

Chuan Zhang, Liehuang Zhu, Chang Xu et al.

The explosive growth of vehicle amount has given rise to a series of traffic problems, such as traffic congestion, road safety, and fuel waste. Collecting vehicles' speed information is an effective way to monitor the traffic condition and avoid vehicles being congested, which however may bring threats to vehicles' location and trajectory privacy. Motivated by the fact that traffic monitoring does not need to know each individual vehicle's speed and the average speed would be sufficient, we propose a privacy-preserving traffic monitoring (PPTM) scheme to aggregate vehicles' speeds at different locations. In PPTM, the roadside unit (RSU) collects vehicles' speed information at multiple road segments, and further cooperates with a service provider to calculate the average speed information for every road segment. To preserve vehicles' privacy, both homomorphic Paillier cryptosystem and super-increasing sequence are adopted. A comprehensive security analysis indicates that the proposed PPTM can preserve vehicles' identities, speeds, locations, and trajectories privacy from being disclosed. In addition, extensive simulations are conducted to validate the effectiveness and efficiency of the proposed PPTM scheme.

Chuan Zhang, Liehuang Zhu, Chang Xu et al.

The recent proliferation of smart devices has given rise to ubiquitous computing, an emerging computing paradigm which allows anytime & anywhere computing possible. In such a ubiquitous computing environment, customers release different computing or sensing tasks, and people, also known as data processors, participate in these tasks and get paid for providing their idle computing and communication resources. Thus, how to select an appropriate and reliable customer while not disclosing processors' privacy has become an interesting problem. In this article, we present a trust-based and privacy-preserving customer selection scheme in ubiquitous computing, called TPCS, to enable potential processors select the customers with good reputation. The basic concept of TPCS is that each data processor holds a trust value, and the reputation score of the customer is calculated based on processors' trust values and feedbacks via a truth discovery process. To preserve processors' privacy, pseudonyms and Paillier cryptosystem are applied to conceal each processor's real identity. In addition, three authentication protocols are designed to ensure that only the valid data processors (i.e., the processors registering in the system, holding the truthful trust values, and joining the computing tasks) can pass the authentication. A comprehensive security analysis is conducted to prove that our proposed TPCS scheme is secure and can defend against several sophisticated attacks. Moreover, extensive simulations are conducted to demonstrate the correctness and effectiveness of the proposed scheme.

Rixin Xu, Qiang Zeng, Liehuang Zhu et al.

The combination of smart home platforms and automation apps introduces much convenience to smart home users. However, this also brings the potential for privacy leakage. If a smart home platform is permitted to collect all the events of a user day and night, then the platform will learn the behavior patterns of this user before long. In this paper, we investigate how IFTTT, one of the most popular smart home platforms, has the capability of monitoring the daily life of a user in a variety of ways that are hardly noticeable. Moreover, we propose multiple ideas for mitigating privacy leakages, which altogether forms a Filter-and-Fuzz (F&F) process: first, it filters out events unneeded by the IFTTT platform; then, it fuzzes the values and frequencies of the remaining events. We evaluate the F&F process, and the results show that the proposed solution makes IFTTT unable to recognize any of the user's behavior patterns.

Xiangyun Tang, Liehuang Zhu, Meng Shen et al.

Machine learning (ML) classifiers are invaluable building blocks that have been used in many fields. High quality training dataset collected from multiple data providers is essential to train accurate classifiers. However, it raises concern about data privacy due to potential leakage of sensitive information in training dataset. Existing studies have proposed many solutions to privacy-preserving training of ML classifiers, but it remains a challenging task to strike a balance among accuracy, computational efficiency, and security. In this paper, we propose Heda, an efficient privacypreserving scheme for training ML classifiers. By combining homomorphic cryptosystem (HC) with differential privacy (DP), Heda obtains the tradeoffs between efficiency and accuracy, and enables flexible switch among different tradeoffs by parameter tuning. In order to make such combination efficient and feasible, we present novel designs based on both HC and DP: A library of building blocks based on partially HC are proposed to construct complex training algorithms without introducing a trusted thirdparty or computational relaxation; A set of theoretical methods are proposed to determine appropriate privacy budget and to reduce sensitivity. Security analysis demonstrates that our solution can construct complex ML training algorithm securely. Extensive experimental results show the effectiveness and efficiency of the proposed scheme.

Liehuang Zhu, Baokun Zheng, Meng Shen et al.

With the more and more extensive application of blockchain, blockchain security has been widely concerned by the society and deeply studied by scholars. Moreover, the security of blockchain data directly affects the security of various applications of blockchain. In this survey, we perform a comprehensive classification and summary of the security of blockchain data. First, we present classification of blockchain data attacks. Subsequently, we present the attacks and defenses of blockchain data in terms of privacy, availability, integrity and controllability. Data privacy attacks present data leakage or data obtained by attackers through analysis. Data availability attacks present abnormal or incorrect access to blockchain data. Data integrity attacks present blockchain data being tampered. Data controllability attacks present blockchain data accidentally manipulated by smart contract vulnerability. Finally, we present several important open research directions to identify follow-up studies in this area.

Jingwei Liu, Fanghui Cai, Longfei Wu et al.

As a popular application, mobile crowd sensing systems aim at providing more convenient service via the swarm intelligence. With the popularity of sensor-embedded smart phones and intelligent wearable devices, mobile crowd sensing is becoming an efficient way to obtain various types of sensing data from individuals, which will make people's life more convenient. However, mobile crowd sensing systems today are facing a critical challenge, namely the privacy leakage of the sensitive information and valuable data, which can raise grave concerns among the participants. To address this issue, we propose an enhanced secure certificateless privacy-preserving verifiable data authentication scheme for mobile crowd sensing, named EPDA. The proposed scheme provides unconditional anonymous data authentication service for mobile crowd sensing, by deploying an improved certificateless ring signature as the cryptogram essential, in which the big sensing data should be signed by one of legitimate members in a specific group and could be verified without exposing the actual identity of the participant. The formal security proof demonstrates that EPDA is secure against existential forgery under adaptive chosen message and identity attacks in random oracle model. Finally, extensive simulations are conducted. The results show that the proposed EPDA efficiently decreases computational cost and time consumption in the sensing data authentication process.

Zhitao Guan, Jing Li, Liehuang Zhu et al.

In the Smart Grid with Renewable Energy Resources (RERs), the Residential Units (RUs) with Distributed Energy Resources (DERs) are considered to be both power consumers and suppliers. Specifically, RUs with excessive renewable generations can trade with the utility in deficit of power supplies for mutual benefits. It causes two challenging issues. First, the trading data of RUs is quite sensitive, which should be only accessed by authorized users with fine-grained policies. Second, the behaviors of the RUs to generate trading data are spontaneous and unpredictable, then the problem is how to guarantee system efficiency and delay tolerance simultaneously. In this paper, we propose a delay-tolerant flexible data access control scheme based on Key Policy Attribute Based Encryption (KP-ABE) for Smart Grid with Renewable Energy Resources (RERs). We adopt the secret sharing scheme (SSS) to realize a flexible access control with encryption delay tolerance. Furthermore, there is no central trusted server to perform the encryption/decryption. We reduce the computation cost on RUs and operators via a semi-trusted model. The analysis shows that the proposed scheme can meet the data security requirement of the Smart Grid with RERs, and it also has less cost compared with other popular models.

Meng Shen, Zelin Liao, Liehuang Zhu et al.

Face liveness detection has become a widely used technique with a growing importance in various authentication scenarios to withstand spoofing attacks. Existing methods that perform liveness detection generally focus on designing intelligent classifiers or customized hardware to differentiate between the image or video samples of a real legitimate user and the imitated ones. Although effective, they can be resource-consuming and detection results may be sensitive to environmental changes. In this paper, we take iris movement as a significant liveness sign and propose a simple and efficient liveness detection system named IriTrack. Users are required to move their eyes along with a randomly generated poly-line, and trajectories of irises are then used as evidences for liveness detection. IriTrack allows checking liveness by using data collected during user-device interactions. We implemented a prototype and conducted extensive experiments to evaluate the performance of the proposed system. The results show that IriTrack can fend against spoofing attacks with a moderate and adjustable time overhead.

Meng Shen, Guohua Cheng, Liehuang Zhu et al.

Content-based image retrieval (CBIR) is one of the fundamental image retrieval primitives. Its applications can be found in various areas, such as art collections and medical diagnoses. With an increasing prevalence of cloud computing paradigm, image owners desire to outsource their images to cloud servers. In order to deal with the risk of privacy leakage of images, images are typically encrypted before they are outsourced to the cloud, which makes CBIR an extremely challenging task. Existing studies focus on the scenario with only a single image owner, leaving the problem of CBIR with multiple image sources (i.e., owners) unaddressed. In this paper, we propose a secure CBIR scheme that supports Multiple Image owners with Privacy Protection (MIPP). We encrypt image features with a secure multi-party computation technique, which allows image owners to encrypt image features with their own keys. This enables efficient image retrieval over images gathered from multiple sources, while guaranteeing that image privacy of an individual image owner will not be leaked to other image owners. We also propose a new method for similarity measurement of images that can avoid revealing image similarity information to the cloud. Theoretical analysis and experimental results demonstrate that MIPP achieves retrieval accuracy and efficiency simultaneously, while preserving image privacy.

Meng Shen, Baoli Ma, Liehuang Zhu et al.

Phrase search allows retrieval of documents containing an exact phrase, which plays an important role in many machine learning applications for cloud-based IoT, such as intelligent medical data analytics. In order to protect sensitive information from being leaked by service providers, documents (e.g., clinic records) are usually encrypted by data owners before being outsourced to the cloud. This, however, makes the search operation an extremely challenging task. Existing searchable encryption schemes for multi-keyword search operations fail to perform phrase search, as they are unable to determine the location relationship of multiple keywords in a queried phrase over encrypted data on the cloud server side. In this paper, we propose P3, an efficient privacy-preserving phrase search scheme for intelligent encrypted data processing in cloud-based IoT. Our scheme exploits the homomorphic encryption and bilinear map to determine the location relationship of multiple queried keywords over encrypted data. It also utilizes a probabilistic trapdoor generation algorithm to protect users search patterns. Thorough security analysis demonstrates the security guarantees achieved by P3. We implement a prototype and conduct extensive experiments on real-world datasets. The evaluation results show that compared with existing multikeyword search schemes, P3 can greatly improve the search accuracy with moderate overheads.

Meng Shen, Baoli Ma, Liehuang Zhu et al.

Constrained shortest distance (CSD) querying is one of the fundamental graph query primitives, which finds the shortest distance from an origin to a destination in a graph with a constraint that the total cost does not exceed a given threshold. CSD querying has a wide range of applications, such as routing in telecommunications and transportation. With an increasing prevalence of cloud computing paradigm, graph owners desire to outsource their graphs to cloud servers. In order to protect sensitive information, these graphs are usually encrypted before being outsourced to the cloud. This, however, imposes a great challenge to CSD querying over encrypted graphs. Since performing constraint filtering is an intractable task, existing work mainly focuses on unconstrained shortest distance queries. CSD querying over encrypted graphs remains an open research problem. In this paper, we propose Connor, a novel graph encryption scheme that enables approximate CSD querying. Connor is built based on an efficient, tree-based ciphertext comparison protocol, and makes use of symmetric-key primitives and the somewhat homomorphic encryption, making it computationally efficient. Using Connor, a graph owner can first encrypt privacy-sensitive graphs and then outsource them to the cloud server, achieving the necessary privacy without losing the ability of querying. Extensive experiments with real-world datasets demonstrate the effectiveness and efficiency of the proposed graph encryption scheme.

Liehuang Zhu, Chuan Zhang, Chang Xu et al.

Recent advances in Socially Aware Networks (SANs) have allowed its use in many domains, out of which social Internet of vehicles (SIOV) is of prime importance. SANs can provide a promising routing and forwarding paradigm for SIOV by using interest-based communication. Though able to improve the forwarding performance, existing interest-based schemes fail to consider the important issue of protecting users' interest information. In this paper, we propose a PRivacy-preserving Interest-based Forwarding scheme (PRIF) for SIOV, which not only protects the interest information, but also improves the forwarding performance. We propose a privacy-preserving authentication protocol to recognize communities among mobile nodes. During data routing and forwarding, a node can know others' interests only if they are affiliated with the same community. Moreover, to improve forwarding performance, a new metric {\em community energy} is introduced to indicate vehicular social proximity. Community energy is generated when two nodes encounter one another and information is shared among them. PRIF considers this energy metric to select forwarders towards the destination node or the destination community. Security analysis indicates PRIF can protect nodes' interest information. In addition, extensive simulations have been conducted to demonstrate that PRIF outperforms the existing algorithms including the BEEINFO, Epidemic, and PRoPHET.

Chang Xu, Xuan Xie, Liehuang Zhu et al.

Recent advances in Socially Aware Networks (SANs) have allowed its use in many domains, out of which social Internet of vehicles (SIOV) is of prime importance. SANs can provide a promising routing and forwarding paradigm for SIOV by using interest-based communication. Though able to improve the forwarding performance, existing interest-based schemes fail to consider the important issue of protecting users' interest information. In this paper, we propose a PRivacy-preserving Interest-based Forwarding scheme (PRIF) for SIOV, which not only protects the interest information, but also improves the forwarding performance. We propose a privacy-preserving authentication protocol to recognize communities among mobile nodes. During data routing and forwarding, a node can know others' interests only if they are affiliated with the same community. Moreover, to improve forwarding performance, a new metric {\em community energy} is introduced to indicate vehicular social proximity. Community energy is generated when two nodes encounter one another and information is shared among them. PRIF considers this energy metric to select forwarders towards the destination node or the destination community. Security analysis indicates PRIF can protect nodes' interest information. In addition, extensive simulations have been conducted to demonstrate that PRIF outperforms the existing algorithms including the BEEINFO, Epidemic, and PRoPHET.

Chuan Zhang, Liehuang Zhu, Chang Xu et al.

In recent years, cognitive Internet of Things (CIoT) has received considerable attention because it can extract valuable information from various Internet of Things (IoT) devices. In CIoT, truth discovery plays an important role in identifying truthful values from large scale data to help CIoT provide deeper insights and value from collected information. However, the privacy concerns of IoT devices pose a major challenge in designing truth discovery approaches. Although existing schemes of truth discovery can be executed with strong privacy guarantees, they are not efficient or cannot be applied in real-life CIoT applications. This article proposes a novel framework for lightweight and privacy-preserving truth discovery called LPTD-I, which is implemented by incorporating fog and cloud platforms, and adopting the homomorphic Paillier encryption and one-way hash chain techniques. This scheme not only protects devices' privacy, but also achieves high efficiency. Moreover, we introduce a fault tolerant (LPTD-II) framework which can effectively overcome malfunctioning CIoT devices. Detailed security analysis indicates the proposed schemes are secure under a comprehensively designed threat model. Experimental simulations are also carried out to demonstrate the efficiency of the proposed schemes.

Zijian Zhang, Zhan Qin, Liehuang Zhu et al.

The technology of differential privacy, adding a noise drawn from the Laplace distribution, successfully overcomes a difficulty of keeping both the privacy of individual data and the utility of the statistical result simultaneously. Therefore, it is prevalent to use a rechargeable battery as the noise for achieving differential privacy in the application of smart grid. Unfortunately, to the best of our knowledge, we observe that the existing privacy protection mechanisms cannot satisfy differential privacy, when considering physical restrictions of the battery in practice. In this paper, we first classify two types of challenges caused by two physical restrictions, the maximum charging/discharging rate and the capacity of battery. We then propose a stateless privacy protection scheme by exploring a boundary-changeable distribution for noise, and prove this scheme satisfies differential privacy, with regard to the first type of challenge. We further explain the difficulty to achieve differential privacy under the second type of challenge, and formalize the definition of a relaxed differential privacy. Finally, we present a stateful privacy protection scheme that satisfies the relaxed differential privacy. Experimental analysis shows that the maximum privacy leakage of our privacy protection schemes at each time point stably outperforms that of the existing work.