Expertise-based Weighting for Regression Models with Noisy Labels
This provides a flexible solution for scenarios where accurate labels are unavailable, relying on diverse expert opinions, but it is incremental as it builds on existing noisy label approaches.
The paper tackles the problem of training regression models with noisy labels from multiple experts by proposing a two-step method that estimates labeler expertise and combines opinions with learned weights, demonstrating empirical outperformance over existing techniques on simulated and real data.
Regression methods assume that accurate labels are available for training. However, in certain scenarios, obtaining accurate labels may not be feasible, and relying on multiple specialists with differing opinions becomes necessary. Existing approaches addressing noisy labels often impose restrictive assumptions on the regression function. In contrast, this paper presents a novel, more flexible approach. Our method consists of two steps: estimating each labeler's expertise and combining their opinions using learned weights. We then regress the weighted average against the input features to build the prediction model. The proposed method is formally justified and empirically demonstrated to outperform existing techniques on simulated and real data. Furthermore, its flexibility enables the utilization of any machine learning technique in both steps. In summary, this method offers a simple, fast, and effective solution for training regression models with noisy labels derived from diverse expert opinions.