Ballpark Learning: Estimating Labels from Rough Group Comparisons
This addresses a practical data labeling challenge for scenarios where only rough group comparisons are available, offering an incremental improvement over prior methods.
The paper tackles the problem of estimating individual labels when only aggregated constraints on label proportions are available, relaxing the unrealistic assumption of known exact proportions. It demonstrates surprisingly high accuracy across domains like income prediction and sentiment analysis using only proportion bounds and no labeled examples.
We are interested in estimating individual labels given only coarse, aggregated signal over the data points. In our setting, we receive sets ("bags") of unlabeled instances with constraints on label proportions. We relax the unrealistic assumption of known label proportions, made in previous work; instead, we assume only to have upper and lower bounds, and constraints on bag differences. We motivate the problem, propose an intuitive formulation and algorithm, and apply our methods to real-world scenarios. Across several domains, we show how using only proportion constraints and no labeled examples, we can achieve surprisingly high accuracy. In particular, we demonstrate how to predict income level using rough stereotypes and how to perform sentiment analysis using very little information. We also apply our method to guide exploratory analysis, recovering geographical differences in twitter dialect.