Xiujin Liu

Xuewen Luo, Ding Fan, Ruiqi Chen et al.

As autonomous vehicles progress toward fully driverless mobility, a critical question emerges: who understands and responds to passengers when the human driver is absent? Existing autonomous driving systems primarily optimize predefined navigation and control objectives from external scene observations, but they remain limited in perceiving and reasoning about in-cabin human intent. In this paper, we propose Intent2Drive, a unified framework for holistic human intent understanding and human-aligned planning. Instead of treating passenger intent as explicit commands alone, Intent2Drive models intent as a latent cognitive state shaped by language, personal attributes, emotional and physical conditions, behavioral signals, and situational context. To support this formulation, we construct a Holistic Intent Dataset (HID) that provides structured supervision over both explicit and implicit intent cues. Built upon HID, our Theory-of-Mind-inspired Human Intent Reasoner (HIR) infers a Latent Human State (LHS) and further translates it into a planner-compatible Human Intent Objective (HIO). We then introduce a Hierarchical Intent-Conditioned Planner (HICP) that incorporates HIO into route-level and trajectory-level planning, enabling driving behaviors to remain aligned with passenger needs across different planning horizons. Extensive experiments show that Intent2Drive improves structured human intent inference and HIO construction while preserving competitive closed-loop planning performance. These results demonstrate a promising step toward passenger-responsive autonomous driving systems that can reason about, interpret, and act upon human intent in driverless mobility.

Pan Wang, Yihao Hu, Xiujin Liu et al.

Vision-language model (VLM) agents increasingly rely on memory-augmented reinforcement learning to reuse experience across long-horizon tasks, yet most existing frameworks store memory as text and depend on proprietary teacher models to summarize or refine it. This design is poorly matched to spatial decision making: geometric priors are compressed into lossy language, and sparse interaction is often supervised through delayed textual feedback rather than dense visually grounded signals. We argue that reusable experience for VLM agents should remain visually grounded. Based on this insight, we propose \textbf{AtlasVA}, a teacher-free visual skill memory framework that organizes memory into three complementary layers: spatial heatmaps, visual exemplars, and symbolic text skills. AtlasVA further evolves danger and affinity atlases directly from trajectory statistics and lightweight grid heuristics, and reuses these self-evolving atlases as potential-based shaping rewards for reinforcement learning. This unifies perception, memory, and optimization without external LLM supervision. Experiments on \textsc{Sokoban}, \textsc{FrozenLake}, 3D embodied navigation, and 3D robotic manipulation benchmarks show that AtlasVA consistently outperforms text-centric memory baselines and competitive VLM agents, with especially strong gains on spatially intensive tasks. Homepage: https://wangpan-ustc.github.io/AtlasvaWeb

Yihao Hu, Zhihao Wen, Xiujin Liu et al.

Large Language Model (LLM) agents are increasingly improved through interaction, yet most self-evolution methods adapt either the policy or the learning environment in isolation. We identify this structural gap as \emph{Agent-Environment Misalignment}: the agent's capability frontier changes during training, while the environment that provides supervision remains static or only weakly coupled to the agent's revealed failures. We propose SEAL, a closed-loop co-evolution framework for interactive tool-use agents. SEAL collects on-policy trajectories under executable verification, diagnoses failed rollouts into turn-level failure labels, and uses these diagnoses as a shared signal for both environment-side adaptation and model-side policy optimization. The environment evolves its training-time learning interface by exposing clearer tool affordance cues, constraint information, and recovery-oriented feedback, while the policy is updated with diagnosis-guided advantage reweighting. Extensive experiments across in-distribution and out-of-distribution multi-turn tool-use evaluations show that SEAL improves low-resource agent learning: with only 400 training samples, it yields +8.25 to +26.25 average-point gains across three backbones and exhibits positive out-of-distribution transfer. These results demonstrate the value of jointly adapting the learner and its training-time learning substrate for robust self-improving LLM agents.