Feature Selection from Differentially Private Correlations
This addresses the need for efficient and private feature selection for data scientists handling sensitive high-dimensional data, representing an incremental improvement over existing methods.
The paper tackled the problem of feature selection in high-dimensional datasets while ensuring differential privacy, showing that an established baseline method is unstable under sparsity and performs poorly on real-world data. The authors proposed a correlations-based order statistic approach, which significantly outperforms the baseline on many datasets.
Data scientists often seek to identify the most important features in high-dimensional datasets. This can be done through $L_1$-regularized regression, but this can become inefficient for very high-dimensional datasets. Additionally, high-dimensional regression can leak information about individual datapoints in a dataset. In this paper, we empirically evaluate the established baseline method for feature selection with differential privacy, the two-stage selection technique, and show that it is not stable under sparsity. This makes it perform poorly on real-world datasets, so we consider a different approach to private feature selection. We employ a correlations-based order statistic to choose important features from a dataset and privatize them to ensure that the results do not leak information about individual datapoints. We find that our method significantly outperforms the established baseline for private feature selection on many datasets.