Constrained Labeling for Weakly Supervised Learning
This addresses the problem of dataset curation bottlenecks for machine learning practitioners, offering an incremental improvement in weak supervision techniques.
The paper tackles the challenge of combining noisy weak supervision signals by proposing a data-free method that trains with random labels within a constrained space, proving theoretical error bounds and showing experimental outperformance on text- and image-classification tasks.
Curation of large fully supervised datasets has become one of the major roadblocks for machine learning. Weak supervision provides an alternative to supervised learning by training with cheap, noisy, and possibly correlated labeling functions from varying sources. The key challenge in weakly supervised learning is combining the different weak supervision signals while navigating misleading correlations in their errors. In this paper, we propose a simple data-free approach for combining weak supervision signals by defining a constrained space for the possible labels of the weak signals and training with a random labeling within this constrained space. Our method is efficient and stable, converging after a few iterations of gradient descent. We prove theoretical conditions under which the worst-case error of the randomized label decreases with the rank of the linear constraints. We show experimentally that our method outperforms other weak supervision methods on various text- and image-classification tasks.