Feature Multi-Selection among Subjective Features
This addresses feature selection challenges in noisy, subjective data for crowdsourced learning applications, but it appears incremental as it builds on existing wisdom-of-crowds concepts.
The paper tackles the problem of selecting and determining the number of judgments for subjective features in crowdsourced tasks, demonstrating effectiveness through linear regression on predicting height and weight from photos with features like 'gender' and 'attractive'.
When dealing with subjective, noisy, or otherwise nebulous features, the "wisdom of crowds" suggests that one may benefit from multiple judgments of the same feature on the same object. We give theoretically-motivated `feature multi-selection' algorithms that choose, among a large set of candidate features, not only which features to judge but how many times to judge each one. We demonstrate the effectiveness of this approach for linear regression on a crowdsourced learning task of predicting people's height and weight from photos, using features such as 'gender' and 'estimated weight' as well as culturally fraught ones such as 'attractive'.