Approximating Wisdom of Crowds using K-RBMs
This addresses the challenge of improving data quality for tasks like web search evaluation and product ratings, but it appears incremental as it builds on existing clustering and RBM techniques.
The paper tackles the problem of aggregating noisy labels from crowds of non-experts to create large training sets, proposing a method that uses K-RBMs for clustering and shows empirical evaluations on real datasets.
An important way to make large training sets is to gather noisy labels from crowds of non experts. We propose a method to aggregate noisy labels collected from a crowd of workers or annotators. Eliciting labels is important in tasks such as judging web search quality and rating products. Our method assumes that labels are generated by a probability distribution over items and labels. We formulate the method by drawing parallels between Gaussian Mixture Models (GMMs) and Restricted Boltzmann Machines (RBMs) and show that the problem of vote aggregation can be viewed as one of clustering. We use K-RBMs to perform clustering. We finally show some empirical evaluations over real datasets.