PROMISSING: Pruning Missing Values in Neural Networks
This addresses the issue of handling missing data in real-world machine learning applications, offering a novel approach that avoids preprocessing steps and encourages models to express uncertainty, though it is incremental in its performance gains.
The paper tackles the problem of missing values in neural networks by proposing PROMISSING, a method that prunes missing values during learning and inference without imputation, resulting in similar prediction performance to imputation techniques and enabling models to become less decisive when facing incomplete data.
While data are the primary fuel for machine learning models, they often suffer from missing values, especially when collected in real-world scenarios. However, many off-the-shelf machine learning models, including artificial neural network models, are unable to handle these missing values directly. Therefore, extra data preprocessing and curation steps, such as data imputation, are inevitable before learning and prediction processes. In this study, we propose a simple and intuitive yet effective method for pruning missing values (PROMISSING) during learning and inference steps in neural networks. In this method, there is no need to remove or impute the missing values; instead, the missing values are treated as a new source of information (representing what we do not know). Our experiments on simulated data, several classification and regression benchmarks, and a multi-modal clinical dataset show that PROMISSING results in similar prediction performance compared to various imputation techniques. In addition, our experiments show models trained using PROMISSING techniques are becoming less decisive in their predictions when facing incomplete samples with many unknowns. This finding hopefully advances machine learning models from being pure predicting machines to more realistic thinkers that can also say "I do not know" when facing incomplete sources of information.