Visual Model Validation via Inline Replication
This addresses the issue for consumers of data visualizations who rely on them for decision-making, though it is incremental as it adapts existing validation methods to visualization contexts.
The paper tackles the problem of visualizations being used as predictive models without validation, leading to invalid inferences, by introducing Inline Replication (IR) as a technique for visual model assessment and repeatability, similar to cross-validation in machine learning.
Data visualizations typically show retrospective views of an existing dataset with little or no focus on repeatability. However, consumers of these tools often use insights gleaned from retrospective visualizations as the basis for decisions about future events. In this way, visualizations often serve as visual predictive models despite the fact that they are typically designed to present historical views of the data. This "visual predictive model" approach, however, can lead to invalid inferences. In this paper, we describe an approach to visual model validation called Inline Replication (IR) which, similar to the cross-validation technique used widely in machine learning, provides a nonparametric and broadly applicable technique for visual model assessment and repeatability. This paper describes the overall IR process and outlines how it can be integrated into both traditional and emerging "big data" visualization pipelines. Examples are provided showing IR integrated within common visualization techniques (such as bar charts and linear regression lines) as well as a more fully-featured visualization system designed for complex exploratory analysis tasks.