Fengling Zheng

Dekun Qian, Ruiqi Yu, Fengling Zheng et al.

Composition in Traditional Chinese Paintings (TCPs) carries spatial, narrative, and cultural-aesthetic meaning. Systematic compositional analysis is therefore important for understanding their visual language and artistic meaning. Traditional compositional analysis is mainly qualitative and interpretation-driven. It supports close reading of individual paintings, but it is difficult to discover, compare, and verify compositional patterns across large painting collections. To better understand these challenges, we conducted a literature review and in-depth interviews with two art historians. Based on these findings, we introduce the Composition Graph, a scene-graph-based representation for TCP composition. It models a painting through four layers: entities, relations, void space, and context. Based on this representation, we develop CompoVista, a canvas-based visual analytics system for composition-oriented exploration of TCPs. CompoVista allows art historians to construct and revise format-aware painting cohorts through visual queries and context queries. It also supports cohort-level inspection of entity distributions and relations, comparison of compositional differences across cohorts, and tracing aggregate patterns back to painting-level evidence.We evaluated CompoVista through a task-based user study with 12 domain participants, two case studies, and expert interviews. The results show that CompoVista supports composition-oriented cohort construction, pattern discovery, iterative refinement, and evidence inspection. The evaluation also reveals future needs, including clearer result explanations, fuzzier composition queries, and stronger exploration history management. Our work contributes a composition-specific structured representation and an integrated visual analytics workflow for studying TCP composition at collection scale.

Zhiguang Zhou, Fengling Zheng, Miaoxin Hu et al.

Structured representation can characterize semantic objects and relationships in images. It provides a possible effective way for the semantic understanding of Traditional Chinese Paintings (TCPs) to better support archaeology and art history research. However, most image-oriented structured representation methods perform poorly on TCPs, due to two major challenges: 1) the objects and events of TCPs exhibit substantial differences from modern natural images, which results in semantic misunderstandings of TCPs; and 2) it is difficult to achieve accurate identification of ancient objects and events in TCPs, even for domain experts.In this paper, we propose VisTCP, a visualization framework that combines a TCP-oriented intelligent model and expert knowledge, which enables art historians to achieve trustworthy structured representations of TCPs in a human-in-the-loop manner. Firstly, we conduct a pilot study with three domain experts to build a semantic taxonomy of TCPs. Then, expert-annotated data are used to train a TCP-oriented structured representation model, which can automatically extract meaningful objects and their relationships in TCPs. To inform users of the model uncertainty, we design a joint embedding visualization view to show the differences between expert annotations and model predictions. This allows users to refine the structured representation based on their domain knowledge, enabling iterative optimization of the model. Finally, we conduct a case study, a usage scenario, and expert interviews on a real dataset to demonstrate the effectiveness of VisTCP in supporting the structured representation and semantic understanding of TCPs.

Zhiguang Zhou, Haoxuan Wang, Zhengqing Zhao et al.

Chart images, such as bar charts, pie charts, and line charts, are explosively produced due to the wide usage of data visualizations. Accordingly, knowledge mining from chart images is becoming increasingly important, which can benefit downstream tasks like chart retrieval and knowledge graph completion. However, existing methods for chart knowledge mining mainly focus on converting chart images into raw data and often ignore their visual encodings and semantic meanings, which can result in information loss for many downstream tasks. In this paper, we propose ChartKG, a novel knowledge graph (KG) based representation for chart images, which can model the visual elements in a chart image and semantic relations among them including visual encodings and visual insights in a unified manner. Further, we develop a general framework to convert chart images to the proposed KG-based representation. It integrates a series of image processing techniques to identify visual elements and relations, e.g., CNNs to classify charts, yolov5 and optical character recognition to parse charts, and rule-based methods to construct graphs. We present four cases to illustrate how our knowledge-graph-based representation can model the detailed visual elements and semantic relations in charts, and further demonstrate how our approach can benefit downstream applications such as semantic-aware chart retrieval and chart question answering. We also conduct quantitative evaluations to assess the two fundamental building blocks of our chart-to-KG framework, i.e., object recognition and optical character recognition. The results provide support for the usefulness and effectiveness of ChartKG.