Crowdsourcing Feature Discovery via Adaptively Chosen Comparisons
This work addresses the challenge of efficient feature discovery for data analysis via crowdsourcing, representing an incremental improvement over nonadaptive algorithms.
The paper tackles the problem of discovering underlying features in a dataset through crowdsourcing, using adaptively chosen triples and binary labels, and shows that this approach recovers all features with less labor than nonadaptive methods in hierarchical and independent feature models.
We introduce an unsupervised approach to efficiently discover the underlying features in a data set via crowdsourcing. Our queries ask crowd members to articulate a feature common to two out of three displayed examples. In addition we also ask the crowd to provide binary labels to the remaining examples based on the discovered features. The triples are chosen adaptively based on the labels of the previously discovered features on the data set. In two natural models of features, hierarchical and independent, we show that a simple adaptive algorithm, using "two-out-of-three" similarity queries, recovers all features with less labor than any nonadaptive algorithm. Experimental results validate the theoretical findings.