DeepVA: Bridging Cognition and Computation through Semantic Interaction and Deep Learning
This work addresses the challenge of modeling user cognitive reasoning in visual analytics, but it appears incremental as it extends existing semantic interaction pipelines with deep learning features.
The paper tackled the problem of bridging cognition and computation in visual analytics by proposing DeepVA, which uses high-level deep learning representations for semantic interaction instead of low-level hand-crafted features, and found that it effectively hastened interactive convergence, especially in high abstraction tasks.
This paper examines how deep learning (DL) representations, in contrast to traditional engineered features, can support semantic interaction (SI) in visual analytics. SI attempts to model user's cognitive reasoning via their interaction with data items, based on the data features. We hypothesize that DL representations contain meaningful high-level abstractions that can better capture users' high-level cognitive intent. To bridge the gap between cognition and computation in visual analytics, we propose DeepVA (Deep Visual Analytics), which uses high-level deep learning representations for semantic interaction instead of low-level hand-crafted data features. To evaluate DeepVA and compare to SI models with lower-level features, we design and implement a system that extends a traditional SI pipeline with features at three different levels of abstraction. To test the relationship between task abstraction and feature abstraction in SI, we perform visual concept learning tasks at three different task abstraction levels, using semantic interaction with three different feature abstraction levels. DeepVA effectively hastened interactive convergence between cognitive understanding and computational modeling of the data, especially in high abstraction tasks.