Dataset Usage Inference without Shadow Models or Held-out Data
For data owners and model auditors, this provides a practical tool to determine dataset usage in real-world scenarios without impractical assumptions.
Dataset Usage Inference (DUI) estimates the fraction of a dataset used to train a model. The proposed method removes the need for shadow models and held-out data by using synthetic non-members and mixture proportion estimation, achieving reliable quantification on large image generative models.
How much of my data was used to train a machine learning model? Dataset Usage Inference (DUI) aims to answer this by estimating what fraction of a dataset contributed to a model's training. However, existing DUI methods rely on assumptions that rarely hold in practice: they require training expensive shadow models to imitate the target model, and they assume access to both known training samples and an in-distribution held-out set confirmed to be absent from training. These conditions make current approaches impractical for modern large models and real data ownership disputes. We introduce a practical DUI framework that removes these constraints. Our method requires neither shadow models nor real held-out data. Instead, it generates synthetic non-member samples, extracts diverse membership signals, and casts DUI as a mixture proportion estimation problem to estimate what share of the candidate dataset was used during training. Experiments on large image generative models show that our method reliably quantifies dataset usage, providing a practical tool for data owners to determine how much of their data was used to train a model.