Data Quality Evaluation using Probability Models
This addresses data quality assessment for data scientists, but it is incremental as it applies an existing method to a new problem.
The paper tackles the problem of evaluating data quality by using machine-learning probability models with a decision tree algorithm to predict data quality without domain knowledge, showing accurate prediction for the examined data but noting it may be insufficient for production use.
This paper discusses an approach with machine-learning probability models to evaluate the difference between good and bad data quality in a dataset. A decision tree algorithm is used to predict data quality based on no domain knowledge of the datasets under examination. It is shown that for the data examined, the ability to predict the quality of data based on simple good/bad pre-labelled learning examples is accurate, however in general it may not be sufficient for useful production data quality assessment.