Task Agnostic Robust Learning on Corrupt Outputs by Correlation-Guided Mixture Density Networks
This addresses the problem of robust learning with noisy outputs for machine learning practitioners, offering an incremental improvement over existing methods.
The paper tackles weakly supervised learning with noisy training data by assuming outputs come from a mixture of target and correlated noise distributions, proposing a method that simultaneously estimates the target distribution and data quality via correlation. The result shows it achieves comparable or superior performance to baselines in handling noisy data across classification and regression tasks.
In this paper, we focus on weakly supervised learning with noisy training data for both classification and regression problems.We assume that the training outputs are collected from a mixture of a target and correlated noise distributions.Our proposed method simultaneously estimates the target distribution and the quality of each data which is defined as the correlation between the target and data generating distributions.The cornerstone of the proposed method is a Cholesky Block that enables modeling dependencies among mixture distributions in a differentiable manner where we maintain the distribution over the network weights.We first provide illustrative examples in both regression and classification tasks to show the effectiveness of the proposed method.Then, the proposed method is extensively evaluated in a number of experiments where we show that it constantly shows comparable or superior performances compared to existing baseline methods in the handling of noisy data.