Label Augmentation with Reinforced Labeling for Weak Supervision
This addresses the need for more efficient weak supervision in domains like text and sensor data, though it is an incremental improvement on existing data programming techniques.
The paper tackles the problem of low coverage in weak supervision by proposing reinforced labeling (RL), which augments labeling functions using sample similarities to increase labeling coverage, resulting in up to +21 points in accuracy and +61 points in F1 score improvements over state-of-the-art methods.
Weak supervision (WS) is an alternative to the traditional supervised learning to address the need for ground truth. Data programming is a practical WS approach that allows programmatic labeling data samples using labeling functions (LFs) instead of hand-labeling each data point. However, the existing approach fails to fully exploit the domain knowledge encoded into LFs, especially when the LFs' coverage is low. This is due to the common data programming pipeline that neglects to utilize data features during the generative process. This paper proposes a new approach called reinforced labeling (RL). Given an unlabeled dataset and a set of LFs, RL augments the LFs' outputs to cases not covered by LFs based on similarities among samples. Thus, RL can lead to higher labeling coverage for training an end classifier. The experiments on several domains (classification of YouTube comments, wine quality, and weather prediction) result in considerable gains. The new approach produces significant performance improvement, leading up to +21 points in accuracy and +61 points in F1 scores compared to the state-of-the-art data programming approach.