Towards Differentially Private Text Representations
This addresses privacy concerns in distributed learning for applications where server trust is lacking, representing an incremental improvement over existing LDP protocols.
The paper tackles the problem of training deep learning models without a trusted server by proposing a new framework with a local differentially private protocol, achieving comparable or better performance than non-private and existing LDP methods.
Most deep learning frameworks require users to pool their local data or model updates to a trusted server to train or maintain a global model. The assumption of a trusted server who has access to user information is ill-suited in many applications. To tackle this problem, we develop a new deep learning framework under an untrusted server setting, which includes three modules: (1) embedding module, (2) randomization module, and (3) classifier module. For the randomization module, we propose a novel local differentially private (LDP) protocol to reduce the impact of privacy parameter $ε$ on accuracy, and provide enhanced flexibility in choosing randomization probabilities for LDP. Analysis and experiments show that our framework delivers comparable or even better performance than the non-private framework and existing LDP protocols, demonstrating the advantages of our LDP protocol.