Yan Liu

Ao Sun, Hongtao Zhang, Heng Zhou et al.

Linear Attention Large Language Models (LLMs) offer a compelling recurrent formulation that compresses context into a fixed-size state matrix, enabling constant-time inference. However, the internal dynamics of this compressed state remain largely opaque. In this work, we present a comprehensive study on the runtime state dynamics of state-of-the-art Linear Attention models. We uncover a fundamental phenomenon termed State Rank Stratification, characterized by a distinct spectral bifurcation among linear attention heads: while one group maintains an effective rank oscillating near zero, the other exhibits rapid growth that converges to an upper bound. Extensive experiments across diverse inference contexts reveal that these dynamics remain strikingly consistent, indicating that the identity of a head,whether low-rank or high-rank,is an intrinsic structural property acquired during pre-training, rather than a transient state dependent on the input data. Furthermore, our diagnostic probes reveal a surprising functional divergence: low-rank heads are indispensable for model reasoning, whereas high-rank heads exhibit significant redundancy. Leveraging this insight, we propose Joint Rank-Norm Pruning, a zero-shot strategy that achieves a 38.9\% reduction in KV-cache overhead while largely maintaining model accuracy.

Zhi Zhang, Yan Liu, Zhejing Hu et al.

Cross-subject EEG generalization is challenging due to substantial heterogeneity across subjects. Existing methods typically learn either a shared subject-invariant model or multiple subject-specialized experts, but these two paradigms fail in complementary ways: the former may over-reduce subject-specific discriminative signals, while the latter may under-reduce transferable structure. We show that their suitability depends on the reducibility cost of branch-specific functions to branch-invariant ones, and we further provide a theory-to-method mapping that instantiates alignment principles in cross-subject EEG learning. Based on this insight, we propose Shared-Routed Expert Alignment (SREA), a collaborative framework that couples a shared expert for reducible invariant functions with routed experts for irreducible subject-specific functions. SREA trains the shared branch with joint embedding over augmented temporal neighbors, the routed branch with prototype-based sparse routing and expert specialization, and both branches with numerically stable mutual-guided reweighting based on cross-branch learnability gaps. Experiments on seven public EEG benchmarks across different tasks show that SREA consistently outperforms state-of-the-art methods and EEG foundation models.

Jiate Li, Defu Cao, Li Li et al.

Large language models (LLMs) have been serving as effective backbones for retrieval systems, including Retrieval-Augmentation-Generation (RAG), Dense Information Retriever (IR), and Agent Memory Retrieval. Recent studies have demonstrated that such LLM-based Retrieval (LLMR) is vulnerable to adversarial attacks, which manipulates documents by token-level injections and enables adversaries to either boost or diminish these documents in retrieval tasks. However, existing attack studies mainly (1) presume a known query is given to the attacker, and (2) highly rely on access to the victim model's parameters or interactions, which are hardly accessible in real-world scenarios, leading to limited validity. To further explore the secure risks of LLMR, we propose a practical black-box attack method that generates transferable injection tokens based on zero-shot surrogate LLMs without need of victim queries or victim models knowledge. The effectiveness of our attack raises such a robustness issue that similar effects may arise from benign or unintended document edits in the real world. To achieve our attack, we first establish a theoretical framework of LLMR and empirically verify it. Under the framework, we simulate the transferable attack as a min-max problem, and propose an adversarial learning mechanism that finds optimal adversarial tokens with learnable query samples. Our attack is validated to be effective on benchmark datasets across popular LLM retrievers.