Towards Unbiased Exploration in Partial Label Learning
This addresses a specific bias issue in partial label learning for machine learning practitioners, offering an incremental improvement over existing methods.
The paper tackles the problem of biased exploration in partial label learning due to softmax layers, and introduces a novel loss function that enables unbiased exploration, achieving improved performance on synthetic data, standard benchmarks, and a new contributed benchmark.
We consider learning a probabilistic classifier from partially-labelled supervision (inputs denoted with multiple possibilities) using standard neural architectures with a softmax as the final layer. We identify a bias phenomenon that can arise from the softmax layer in even simple architectures that prevents proper exploration of alternative options, making the dynamics of gradient descent overly sensitive to initialisation. We introduce a novel loss function that allows for unbiased exploration within the space of alternative outputs. We give a theoretical justification for our loss function, and provide an extensive evaluation of its impact on synthetic data, on standard partially labelled benchmarks and on a contributed novel benchmark related to an existing rule learning challenge.