A Modulation Layer to Increase Neural Network Robustness Against Data Quality Issues
This addresses data quality issues for high-stakes applications like healthcare, enabling deployment in real-time environments, though it is an incremental improvement over existing methods.
The paper tackles the problem of data missingness and quality in machine learning by proposing a novel neural network modification that replaces fixed weights with a function of an additional input, inspired by neuromodulation. In testing, models with modulating layers showed increased robustness against data quality degradation, saving training time compared to imputation.
Data missingness and quality are common problems in machine learning, especially for high-stakes applications such as healthcare. Developers often train machine learning models on carefully curated datasets using only high quality data; however, this reduces the utility of such models in production environments. We propose a novel neural network modification to mitigate the impacts of low quality and missing data which involves replacing the fixed weights of a fully-connected layer with a function of an additional input. This is inspired from neuromodulation in biological neural networks where the cortex can up- and down-regulate inputs based on their reliability and the presence of other data. In testing, with reliability scores as a modulating signal, models with modulating layers were found to be more robust against degradation of data quality, including additional missingness. These models are superior to imputation as they save on training time by completely skipping the imputation process and further allow the introduction of other data quality measures that imputation cannot handle. Our results suggest that explicitly accounting for reduced information quality with a modulating fully connected layer can enable the deployment of artificial intelligence systems in real-time applications.