WaKA: Data Attribution using K-Nearest Neighbors and Membership Privacy Principles
This work addresses data attribution and privacy assessment for machine learning practitioners, offering a unified framework but is incremental as it builds on existing methods like LiRA and k-NN.
The paper tackles the problem of attributing individual data points' contributions to model performance and privacy risks by introducing WaKA, a method that uses Wasserstein distance and k-nearest neighbors to measure loss distribution without subset sampling, showing performance close to LiRA with greater efficiency and robustness in tasks like data minimization.
In this paper, we introduce WaKA (Wasserstein K-nearest-neighbors Attribution), a novel attribution method that leverages principles from the LiRA (Likelihood Ratio Attack) framework and k-nearest neighbors classifiers (k-NN). WaKA efficiently measures the contribution of individual data points to the model's loss distribution, analyzing every possible k-NN that can be constructed using the training set, without requiring to sample subsets of the training set. WaKA is versatile and can be used a posteriori as a membership inference attack (MIA) to assess privacy risks or a priori for privacy influence measurement and data valuation. Thus, WaKA can be seen as bridging the gap between data attribution and membership inference attack (MIA) by providing a unified framework to distinguish between a data point's value and its privacy risk. For instance, we have shown that self-attribution values are more strongly correlated with the attack success rate than the contribution of a point to the model generalization. WaKA's different usage were also evaluated across diverse real-world datasets, demonstrating performance very close to LiRA when used as an MIA on k-NN classifiers, but with greater computational efficiency. Additionally, WaKA shows greater robustness than Shapley Values for data minimization tasks (removal or addition) on imbalanced datasets.