CVSep 16, 2025
MSGFusion: Multimodal Scene Graph-Guided Infrared and Visible Image FusionGuihui Li, Bowei Dong, Kaizhi Dong et al.
Infrared and visible image fusion has garnered considerable attention owing to the strong complementarity of these two modalities in complex, harsh environments. While deep learning-based fusion methods have made remarkable advances in feature extraction, alignment, fusion, and reconstruction, they still depend largely on low-level visual cues, such as texture and contrast, and struggle to capture the high-level semantic information embedded in images. Recent attempts to incorporate text as a source of semantic guidance have relied on unstructured descriptions that neither explicitly model entities, attributes, and relationships nor provide spatial localization, thereby limiting fine-grained fusion performance. To overcome these challenges, we introduce MSGFusion, a multimodal scene graph-guided fusion framework for infrared and visible imagery. By deeply coupling structured scene graphs derived from text and vision, MSGFusion explicitly represents entities, attributes, and spatial relations, and then synchronously refines high-level semantics and low-level details through successive modules for scene graph representation, hierarchical aggregation, and graph-driven fusion. Extensive experiments on multiple public benchmarks show that MSGFusion significantly outperforms state-of-the-art approaches, particularly in detail preservation and structural clarity, and delivers superior semantic consistency and generalizability in downstream tasks such as low-light object detection, semantic segmentation, and medical image fusion.
OPTICSMay 7, 2025
All-optical temporal integration mediated by subwavelength heat antennasYi Zhang, Nikolaos Farmakidis, Ioannis Roumpos et al.
Optical computing systems deliver unrivalled processing speeds for scalar operations. Yet, integrated implementations have been constrained to low-dimensional tensor operations that fall short of the vector dimensions required for modern artificial intelligence. We demonstrate an all-optical neuromorphic computing system based on time division multiplexing, capable of processing input vectors exceeding 250,000 elements within a unified framework. The platform harnesses optically driven thermo-optic modulation in standing wave optical fields, with titanium nano-antennas functioning as wavelength-selective absorbers. Counterintuitively, the thermal time dynamics of the system enable simultaneous time integration of ultra-fast (50GHz) signals and the application of programmable, non-linear activation functions, entirely within the optical domain. This unified framework constitutes a leap towards large-scale photonic computing that satisfies the dimensional requirements of AI workloads.
AIFeb 26, 2021
Multi-Agent Path Planning based on MPC and DDPGJunxiao Xue, Xiangyan Kong, Bowei Dong et al.
The problem of mixed static and dynamic obstacle avoidance is essential for path planning in highly dynamic environment. However, the paths formed by grid edges can be longer than the true shortest paths in the terrain since their headings are artificially constrained. Existing methods can hardly deal with dynamic obstacles. To address this problem, we propose a new algorithm combining Model Predictive Control (MPC) with Deep Deterministic Policy Gradient (DDPG). Firstly, we apply the MPC algorithm to predict the trajectory of dynamic obstacles. Secondly, the DDPG with continuous action space is designed to provide learning and autonomous decision-making capability for robots. Finally, we introduce the idea of the Artificial Potential Field to set the reward function to improve convergence speed and accuracy. We employ Unity 3D to perform simulation experiments in highly uncertain environment such as aircraft carrier decks and squares. The results show that our method has made great improvement on accuracy by 7%-30% compared with the other methods, and on the length of the path and turning angle by reducing 100 units and 400-450 degrees compared with DQN (Deep Q Network), respectively.