Limitations of weak labels for embedding and tagging
This work addresses the problem of data annotation costs for researchers and practitioners in sound analysis, but it is incremental as it focuses on isolating the effect of weak labels from other common challenges.
The paper investigates the performance impact of using weakly labeled data versus strongly labeled data for training embedding models and end-to-end classifiers in ambient sound analysis, finding that weak labels lead to significant performance degradation in certain applications.
Many datasets and approaches in ambient sound analysis use weakly labeled data.Weak labels are employed because annotating every data sample with a strong label is too expensive.Yet, their impact on the performance in comparison to strong labels remains unclear.Indeed, weak labels must often be dealt with at the same time as other challenges, namely multiple labels per sample, unbalanced classes and/or overlapping events.In this paper, we formulate a supervised learning problem which involves weak labels.We create a dataset that focuses on the difference between strong and weak labels as opposed to other challenges. We investigate the impact of weak labels when training an embedding or an end-to-end classifier.Different experimental scenarios are discussed to provide insights into which applications are most sensitive to weakly labeled data.