Natural Language-Driven Viewpoint Navigation for Volume Exploration via Semantic Block Representation
This work addresses the problem of volumetric data exploration for users without domain expertise, offering an incremental improvement through automation and semantic guidance.
The paper tackles the challenge of selecting optimal viewpoints for exploring volumetric data by proposing a framework that uses natural language interaction and semantic block representation to guide navigation via reinforcement learning, resulting in improved efficiency and interpretability.
Exploring volumetric data is crucial for interpreting scientific datasets. However, selecting optimal viewpoints for effective navigation can be challenging, particularly for users without extensive domain expertise or familiarity with 3D navigation. In this paper, we propose a novel framework that leverages natural language interaction to enhance volumetric data exploration. Our approach encodes volumetric blocks to capture and differentiate underlying structures. It further incorporates a CLIP Score mechanism, which provides semantic information to the blocks to guide navigation. The navigation is empowered by a reinforcement learning framework that leverage these semantic cues to efficiently search for and identify desired viewpoints that align with the user's intent. The selected viewpoints are evaluated using CLIP Score to ensure that they best reflect the user queries. By automating viewpoint selection, our method improves the efficiency of volumetric data navigation and enhances the interpretability of complex scientific phenomena.