Hailiang Huang

Yutao Hou, Zeguan Xiao, Fei Yu et al.

Large language models (LLMs) have achieved distinguished performance on various reasoning-intensive tasks. However, LLMs might still face the challenges of robustness issues and fail unexpectedly in some simple reasoning tasks. Previous works evaluate the LLM robustness with hand-crafted templates or a limited set of perturbation rules, indicating potential data contamination in pre-training or fine-tuning datasets. In this work, inspired by stress testing in software engineering, we propose a novel framework, Automatic Robustness Checker (AR-Checker), to generate mathematical problem variants that maintain the semantic meanings of the original one but might fail the LLMs. The AR-Checker framework generates mathematical problem variants through multi-round parallel streams of LLM-based rewriting and verification. Our framework can generate benchmark variants dynamically for each LLM, thus minimizing the risk of data contamination. Experiments on GSM8K and MATH-500 demonstrate the strong performance of AR-Checker on mathematical tasks. We also evaluate AR-Checker on benchmarks beyond mathematics, including MMLU, MMLU-Pro, and CommonsenseQA, where it also achieves strong performance, further proving the effectiveness of AR-Checker.

Qingwen Li, Xiaohang Zhao, Xiao Han et al.

Intensive Care Unit (ICU) mortality prediction, which estimates a patient's mortality status at discharge using EHRs collected early in an ICU admission, is vital in critical care. For this task, predictive accuracy alone is insufficient; interpretability is equally essential for building clinical trust and meeting regulatory standards, a topic that has attracted significant attention in information system research. Accordingly, an ideal solution should enable intrinsic interpretability and align its reasoning with three key elements of the ICU decision-making practices: clinical course identification, demographic heterogeneity, and prognostication awareness. However, conventional approaches largely focus on demographic heterogeneity, overlooking clinical course identification and prognostication awareness. Recent prototype learning methods address clinical course identification, yet the integration of the other elements into such frameworks remains underexplored. To address these gaps, we propose ProtoDoctor, a novel ICU mortality prediction framework that delivers intrinsic interpretability while integrating all three elements of the ICU decision-making practices into its reasoning process. Methodologically, ProtoDoctor features two key innovations: the Prognostic Clinical Course Identification module and the Demographic Heterogeneity Recognition module. The former enables the identification of clinical courses via prototype learning and achieves prognostication awareness using a novel regularization mechanism. The latter models demographic heterogeneity through cohort-specific prototypes and risk adjustments. Extensive empirical evaluations demonstrate that ProtoDoctor outperforms state-of-the-art baselines in predictive accuracy. Human evaluations further confirm that its interpretations are more clinically meaningful, trustworthy, and applicable in ICU practice.