A Quantitative Evaluation Framework for Missing Value Imputation Algorithms
This provides a principled way to rank imputation algorithms for researchers and practitioners, though it appears incremental as it builds on existing distribution comparison methods.
The paper tackles the problem of quantitatively evaluating missing value imputation algorithms by developing a framework that treats evaluation as comparing two distributions, resulting in metrics like the proposed Neighborhood-based Dissimilarity Score that is fast and effective across datasets.
We consider the problem of quantitatively evaluating missing value imputation algorithms. Given a dataset with missing values and a choice of several imputation algorithms to fill them in, there is currently no principled way to rank the algorithms using a quantitative metric. We develop a framework based on treating imputation evaluation as a problem of comparing two distributions and show how it can be used to compute quantitative metrics. We present an efficient procedure for applying this framework to practical datasets, demonstrate several metrics derived from the existing literature on comparing distributions, and propose a new metric called Neighborhood-based Dissimilarity Score which is fast to compute and provides similar results. Results are shown on several datasets, metrics, and imputations algorithms.