Seeing Eye to AI? Applying Deep-Feature-Based Similarity Metrics to Information Visualization
ACM Human Factors in Computing Systems (CHI) 2025
An overview of how prior studies from Veras & Collins (2019) and Demiralp et al. (2016) were conceptually replicated and compared using deep-feature-based similarity metrics in this work.
Abstract
Judging the similarity of visualizations is crucial to various applications, such as visualization-based search and visualization recommendation systems. Recent studies show deep-feature-based similarity metrics correlate well with perceptual judgments of image similarity and serve as effective loss functions for tasks like image super-resolution and style transfer. We explore the application of such metrics to judgments of visualization similarity. We extend a similarity metric using five ML architectures and three pre-trained weight sets. We replicate results from previous crowd-sourced studies on scatterplot and visual channel similarity perception. Notably, our metric using pre-trained ImageNet weights outperformed gradient-descent tuned MS-SSIM, a multi-scale similarity metric based on luminance, contrast, and structure. Our work contributes to understanding how deep-feature-based metrics can enhance similarity assessments in visualization, potentially improving visual analysis tools and techniques.
Citation
BibTeX
@inproceedings{long2025seeing,
title={Seeing Eye to AI? Applying Deep-Feature-Based Similarity Metrics to Information Visualization},
author={Long, Sheng and Chatzimparmpas, Angelos and Alexander, Emma and Kay, Matthew and Hullman, Jessica},
booktitle={Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems},
pages={1--20},
year={2025}
}