On the Impact of the Utility in Semivalue-based Data Valuation
This work addresses a critical issue for practitioners using semivalue-based data valuation, particularly when selecting among multiple equally valid utilities, by providing a practical methodology to assess the robustness of their results.
The authors tackled the problem of robustness in semivalue-based data valuation to changes in utility, and found that their proposed methodology can inform practitioners about the potential shift in data valuation results, with strong agreement across diverse datasets and semivalues. Their approach demonstrated robustness in data valuation, with specific results showing strong correlation with rank-correlation analyses.
Semivalue-based data valuation uses cooperative-game theory intuitions to assign each data point a value reflecting its contribution to a downstream task. Still, those values depend on the practitioner's choice of utility, raising the question: How robust is semivalue-based data valuation to changes in the utility? This issue is critical when the utility is set as a trade-off between several criteria and when practitioners must select among multiple equally valid utilities. We address it by introducing the notion of a dataset's spatial signature: given a semivalue, we embed each data point into a lower-dimensional space where any utility becomes a linear functional, making the data valuation framework amenable to a simpler geometric picture. Building on this, we propose a practical methodology centered on an explicit robustness metric that informs practitioners whether and by how much their data valuation results will shift as the utility changes. We validate this approach across diverse datasets and semivalues, demonstrating strong agreement with rank-correlation analyses and offering analytical insight into how choosing a semivalue can amplify or diminish robustness.