Seeing Eye to AI? Applying Deep-Feature-Based Similarity Metrics to Information Visualization
This work addresses the need for better similarity assessments in visualization, which could improve visual analysis tools and techniques, but it is incremental as it extends existing methods to a new domain.
The paper tackled the problem of judging visualization similarity by applying deep-feature-based similarity metrics, and found that their metric using pre-trained ImageNet weights outperformed gradient-descent tuned MS-SSIM in replicating perceptual judgments from crowd-sourced studies.
Judging the similarity of visualizations is crucial to various applications, such as visualization-based search and visualization recommendation systems. Recent studies show deep-feature-based similarity metrics correlate well with perceptual judgments of image similarity and serve as effective loss functions for tasks like image super-resolution and style transfer. We explore the application of such metrics to judgments of visualization similarity. We extend a similarity metric using five ML architectures and three pre-trained weight sets. We replicate results from previous crowd-sourced studies on scatterplot and visual channel similarity perception. Notably, our metric using pre-trained ImageNet weights outperformed gradient-descent tuned MS-SSIM, a multi-scale similarity metric based on luminance, contrast, and structure. Our work contributes to understanding how deep-feature-based metrics can enhance similarity assessments in visualization, potentially improving visual analysis tools and techniques. Supplementary materials are available at https://osf.io/dj2ms.