D Hu

h-index1
4papers
4citations

4 Papers

14.1CVJun 16Code
Enhancing Pathological VLMs with Cross-scale Reasoning

Chi Phan, Tianyi Zhang, Qiaochu Xue et al.

Pathological images are inherently multi-scale, requiring pathologists to integrate evidence from global tissue architecture at low magnification to cellular morphology at higher magnification for accurate diagnosis. While existing pathological datasets for vision-language model (VLM) include various scales, they often lack an explicit cross-scale reasoning objective. This limitation prevents VLMs from capturing essential cross-scale representations and learning evidence-based reasoning. To bridge this gap, we introduce the first cross-scale training and evaluation paradigm that formulates pathology interpretation as multi-magnification reasoning. However, creating such a task reveals a critical challenge: multi-image visual question answering (VQA) is prone to text-only shortcuts, which allow models to guess answers using magnification-dependent artifacts rather than visual evidence. To address this, we propose a leakage-aware curation pipeline that combines adversarial text-only screening with constraint-guided question design. Using this pipeline, we construct Scale-VQA, a high-quality benchmark with 4,685 multiple-choice questions grounded in 2,537 pathology images across multiple magnification levels. Finally, we present ScaleReasoner-R1, a model trained via reinforcement learning to optimize performance on the cross-scale VQA task. ScaleReasoner-R1 achieves state-of-the-art performance on our cross-scale reasoning benchmark and generalizes to SOTA performance on established single-scale benchmarks. Findings suggest that even the limited cross-scale supervision can significantly improve pathological understanding. The code and demos will be open-sourced.

CVJun 17Code
GeoISF: Instance Semantic Forest Inspired Large-Scale Cross-View Geo-Localization via Ground LiDAR-to-Satellite Image

Di Hu, Xia Yuan, Chunxia Zhao

The problem of localization on a large-scale satellite image given a frame of query ground view point clouds remains challenging. Existing LiDAR-to-image cross-view localization methods struggle in large-scale scenarios due to limited semantic alignment and the modality gap between point clouds and satellite images. This paper introduces the large-scale LiDAR-to-image geo-localization pipeline called GeoISF. GeoISF introduces an instance semantic forest constructed using WordNet, which enhances temporal semantic representation and discriminative power by integrating semantic trees from multiple frames. By leveraging environmental semantic representation as a shared medium, GeoISF effectively bridges the modality gap and improves semantic matching accuracy. Extensive experiments demonstrate the superior performance of GeoISF in large-scale cross-view localization, 13.22 times better than the parallel LiDAR-to-image method in the R@10 metric on the KITTI dataset. The proposed method addresses the existing gap in large-scale LiDAR-to-image cross-view localization, offering a robust solution to the computational and accuracy challenges inherent in such scenarios. We will release the code as an open-source resource available online for the broader research community.

14.3CVJul 7
Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning

Yake Wei, Yuan Wang, Fengyun Rao et al.

Recent advancements in Multimodal Large Language Models (MLLMs) have evolved from static perception to interleaved visual-language reasoning, often referred to as ``thinking with images''. A basic operation in this reasoning process is to zoom in on regions of interest (often represented with bounding boxes) to acquire finer visual details. In this paper, we propose \textbf{Seg}mentation before \textbf{Answer}ing (SegAnswer), which shifts the unit of zoom-in from the popular bounding box to pixel-level segmentation mask. By employing fine-grained masks to isolate the target area from cluttered environments, segmented visual input yields a more precise region of interest, effectively filtering out redundant background and interfering objects. Furthermore, the discrete patches of segmented visual input align more seamlessly with how MLLMs structure visual tokens via positional embeddings. In experiments, we evaluate SegAnswer across diverse benchmarks, including high-resolution perception, general perception, and hallucination. It achieves consistent improvements and also exhibits considerable performance on segmentation tasks, validating its capability for reliable pixel grounding.

9.2OCJun 15
Robust Koopman MPC with Sets Updates for Time Delayed Systems

Xinglong Zhang, Xinxin Yao, Xin Xu et al.

Koopman operators have shown significant potential in designing linear model predictive control (MPC) schemes for nonlinear systems on a lifted observable space. Recent advances have tackled the robust Koopman MPC design issue in the presence of modeling errors, relying on the prior estimation of the modeling uncertainty set. However, deriving a robust positively invariant set using a precalculated uncertainty set can be conservative because the uncertainty set bound is time-varying and dependent on the state and control. Additionally, no existing Koopman MPC design has addressed the closed-loop robustness challenge for nonlinear time delayed systems. Thereby, this article presents a robust adaptive Koopman MPC approach with online updates of uncertainty sets for a class of nonlinear time delayed systems. The unknown nonlinear time delayed system is first modeled in a data-driven manner to derive a lifted time delayed Koopman model in the feature space. By analyzing fundamental properties such as controllability and observability, a robust tube-based MPC algorithm is designed for the time delayed Koopman model. The robust adaptive Koopman MPC algorithm with online updates of the uncertainty sets is then presented to reduce conservatism. Closed-loop robustness under exogenous disturbances and asymptotic convergence in the nominal scenario are proven. Finally, numerical examples verify the effectiveness of the proposed approach.