On the privacy-utility trade-off in differentially private hierarchical text classification
It addresses privacy risks in hierarchical text classification for users deploying neural networks, but is incremental as it focuses on empirical comparisons of existing architectures.
This work investigates the privacy-utility trade-off in hierarchical text classification with differential privacy, showing that large differential privacy parameters can completely mitigate membership inference attacks with only a moderate decrease in model utility, such as Transformer-based models achieving favorable trade-offs for large datasets with long texts.
Hierarchical text classification consists in classifying text documents into a hierarchy of classes and sub-classes. Although artificial neural networks have proved useful to perform this task, unfortunately they can leak training data information to adversaries due to training data memorization. Using differential privacy during model training can mitigate leakage attacks against trained models, enabling the models to be shared safely at the cost of reduced model accuracy. This work investigates the privacy-utility trade-off in hierarchical text classification with differential privacy guarantees, and identifies neural network architectures that offer superior trade-offs. To this end, we use a white-box membership inference attack to empirically assess the information leakage of three widely used neural network architectures. We show that large differential privacy parameters already suffice to completely mitigate membership inference attacks, thus resulting only in a moderate decrease in model utility. More specifically, for large datasets with long texts we observed Transformer-based models to achieve an overall favorable privacy-utility trade-off, while for smaller datasets with shorter texts convolutional neural networks are preferable.