Sen Zhang

Baokui Li, Sen Zhang, Wangshu Zhang et al.

Supplying data augmentation to conversational question answering (CQA) can effectively improve model performance. However, there is less improvement from single-turn datasets in CQA due to the distribution gap between single-turn and multi-turn datasets. On the other hand, while numerous single-turn datasets are available, we have not utilized them effectively. To solve this problem, we propose a novel method to convert single-turn datasets to multi-turn datasets. The proposed method consists of three parts, namely, a QA pair Generator, a QA pair Reassembler, and a question Rewriter. Given a sample consisting of context and single-turn QA pairs, the Generator obtains candidate QA pairs and a knowledge graph based on the context. The Reassembler utilizes the knowledge graph to get sequential QA pairs, and the Rewriter rewrites questions from a conversational perspective to obtain a multi-turn dataset S2M. Our experiments show that our method can synthesize effective training resources for CQA. Notably, S2M ranks 1st place on the QuAC leaderboard at the time of submission (Aug 24th, 2022).

Sen Zhang, Qingqing Ye, Haibo Hu et al.

The skip-gram model (SGM), which employs a neural network to generate node vectors, serves as the basis for numerous popular graph embedding techniques. However, since the training datasets contain sensitive linkage information, the parameters of a released SGM may encode private information and pose significant privacy risks. Differential privacy (DP) is a rigorous standard for protecting individual privacy in data analysis. Nevertheless, when applying differential privacy to skip-gram in graphs, it becomes highly challenging due to the complex link relationships, which potentially result in high sensitivity and necessitate substantial noise injection. To tackle this challenge, we present AdvSGM, a differentially private skip-gram for graphs via adversarial training. Our core idea is to leverage adversarial training to privatize skip-gram while improving its utility. Towards this end, we develop a novel adversarial training module by devising two optimizable noise terms that correspond to the parameters of a skip-gram. By fine-tuning the weights between modules within AdvSGM, we can achieve differentially private gradient updates without additional noise injection. Extensive experimental results on six real-world graph datasets show that AdvSGM preserves high data utility across different downstream tasks.

Sen Zhang, Qingqing Ye, Haibo Hu

Graph embedding generation techniques aim to learn low-dimensional vectors for each node in a graph and have recently gained increasing research attention. Publishing low-dimensional node vectors enables various graph analysis tasks, such as structural equivalence and link prediction. Yet, improper publication opens a backdoor to malicious attackers, who can infer sensitive information of individuals from the low-dimensional node vectors. Existing methods tackle this issue by developing deep graph learning models with differential privacy (DP). However, they often suffer from large noise injections and cannot provide structural preferences consistent with mining objectives. Recently, skip-gram based graph embedding generation techniques are widely used due to their ability to extract customizable structures. Based on skip-gram, we present SE-PrivGEmb, a structure-preference enabled graph embedding generation under DP. For arbitrary structure preferences, we design a unified noise tolerance mechanism via perturbing non-zero vectors. This mechanism mitigates utility degradation caused by high sensitivity. By carefully designing negative sampling probabilities in skip-gram, we theoretically demonstrate that skip-gram can preserve arbitrary proximities, which quantify structural features in graphs. Extensive experiments show that our method outperforms existing state-of-the-art methods under structural equivalence and link prediction tasks.

Sen Zhang, Weiwei Ni, Nan Fu

The goal of privacy-preserving social graph publishing is to protect individual privacy while preserving data utility. Community structure, which is an important global pattern of nodes, is a crucial data utility as it serves as fundamental operations for many graph analysis tasks. Yet, most existing methods with differential privacy (DP) commonly fall in edge-DP to sacrifice security in exchange for utility. Moreover, they reconstruct graphs from the local feature-extraction of nodes, resulting in poor community preservation. Motivated by this, we propose PrivCom, a strict node-DP graph publishing algorithm to maximize the utility on the community structure while maintaining a higher level of privacy. Specifically, to reduce the huge sensitivity, we devise a Katz index-based private graph feature extraction method, which can capture global graph structure features while greatly reducing the global sensitivity via a sensitivity regulation strategy. Yet, with a fixed sensitivity, the feature captured by Katz index, which is presented in matrix form, requires privacy budget splits. As a result, plenty of noise is injected, thereby mitigating global structural utility. To this end, we design a private Oja algorithm approximating eigen-decomposition, which yields the noisy Katz matrix via privately estimating eigenvectors and eigenvalues from extracted low-dimensional vectors. Experimental results confirm our theoretical findings and the efficacy of PrivCom.