Fairshare Data Pricing via Data Valuation for Large Language Models
This addresses fairness and sustainability in data markets for LLM builders and human annotators, offering a concrete solution to a domain-specific problem.
The paper tackles exploitative pricing in data markets for large language models (LLMs) by introducing fairshare, a pricing mechanism based on data valuation, which theoretically maximizes long-term utility and profit while empirically boosting seller earnings and improving buyers' performance-per-dollar on tasks like math and medical diagnosis.
Training data is the backbone of large language models (LLMs), yet today's data markets often operate under exploitative pricing -- sourcing data from marginalized groups with little pay or recognition. This paper introduces a theoretical framework for LLM data markets, modeling the strategic interactions between buyers (LLM builders) and sellers (human annotators). We begin with theoretical and empirical analysis showing how exploitative pricing drives high-quality sellers out of the market, degrading data quality and long-term model performance. Then we introduce fairshare, a pricing mechanism grounded in data valuation that quantifies each data's contribution. It aligns incentives by sustaining seller participation and optimizing utility for both buyers and sellers. Theoretically, we show that fairshare yields mutually optimal outcomes: maximizing long-term buyer utility and seller profit while sustaining market participation. Empirically when training open-source LLMs on complex NLP tasks, including math problems, medical diagnosis, and physical reasoning, fairshare boosts seller earnings and ensures a stable supply of high-quality data, while improving buyers' performance-per-dollar and long-term welfare. Our findings offer a concrete path toward fair, transparent, and economically sustainable data markets for LLM.