Xizhuo Zhang

Yannian Gu, Zhongzhen Huang, Linjie Mu et al.

Multimodal large language models (MLLMs) demonstrate considerable potential in clinical diagnostics, a domain that inherently requires synthesizing complex visual and textual data alongside consulting authoritative medical literature. However, existing benchmarks primarily evaluate MLLMs in end-to-end answering scenarios. This limits the ability to disentangle a model's foundational multimodal reasoning from its proficiency in evidence retrieval and application. We introduce the Clinical Understanding and Retrieval Evaluation (CURE) benchmark. Comprising $500$ multimodal clinical cases mapped to physician-cited reference literature, CURE evaluates reasoning and retrieval under controlled evidence settings to disentangle their respective contributions. We evaluate state-of-the-art MLLMs across distinct evidence-gathering paradigms in both closed-ended and open-ended diagnosis tasks. Evaluations reveal a stark dichotomy: while advanced models demonstrate clinical reasoning proficiency when supplied with physician reference evidence (achieving up to $73.4\%$ accuracy on differential diagnosis), their performance substantially declines (as low as $25.4\%$) when reliant on independent retrieval mechanisms. This disparity highlights the dual challenges of effectively integrating multimodal clinical evidence and retrieving precise supporting literature. CURE is publicly available at https://github.com/yanniangu/CURE.

Xizhuo Zhang, Bing Yao

Rapid developments in advanced sensing and imaging have significantly enhanced information visibility, opening opportunities for predictive modeling of complex dynamic systems. However, sensing signals acquired from such complex systems are often distributed across 3D geometries and rapidly evolving over time, posing significant challenges in spatiotemporal predictive modeling. This paper proposes a geometry-aware active learning framework for modeling spatiotemporal dynamic systems. Specifically, we propose a geometry-aware spatiotemporal Gaussian Process (G-ST-GP) to effectively integrate the temporal correlations and geometric manifold features for reliable prediction of high-dimensional dynamic behaviors. In addition, we develop an adaptive active learning strategy to strategically identify informative spatial locations for data collection and further maximize the prediction accuracy. This strategy achieves the adaptive trade-off between the prediction uncertainty in the G-ST-GP model and the space-filling design guided by the geodesic distance across the 3D geometry. We implement the proposed framework to model the spatiotemporal electrodynamics in a 3D heart geometry. Numerical experiments show that our framework outperforms traditional methods lacking the mechanism of geometric information incorporation or effective data collection.

Xizhuo Zhang, Bing Yao

Recent advances in sensing and imaging technologies have enabled the collection of high-dimensional spatiotemporal data across complex geometric domains. However, effective modeling of such data remains challenging due to irregular spatial structures, rapid temporal dynamics, and the need to jointly predict multiple interrelated physical variables. This paper presents a physics-augmented multi-task Gaussian Process (P-M-GP) framework tailored for spatiotemporal dynamic systems. Specifically, we develop a geometry-aware, multi-task Gaussian Process (M-GP) model to effectively capture intrinsic spatiotemporal structure and inter-task dependencies. To further enhance the model fidelity and robustness, we incorporate governing physical laws through a physics-based regularization scheme, thereby constraining predictions to be consistent with governing dynamical principles. We validate the proposed P-M-GP framework on a 3D cardiac electrodynamics modeling task. Numerical experiments demonstrate that our method significantly improves prediction accuracy over existing methods by effectively incorporating domain-specific physical constraints and geometric prior.