Yan Ming

Wenbo Qiao, Shuaixian Wang, Peng Zhang et al.

Data re-uploading quantum circuits (DRQC) are a key approach to implementing quantum neural networks and have been shown to outperform classical neural networks in fitting high-frequency functions. However, their practical application is limited by the scalability of current quantum hardware. In this paper, we introduce the mathematical paradigm of DRQC into classical models by proposing a quantum-inspired data re-uploading network (Q-RUN), which retains the Fourier-expressive advantages of quantum models without any quantum hardware. Experimental results demonstrate that Q-RUN delivers superior performance across both data modeling and predictive modeling tasks. Compared to the fully connected layers and the state-of-the-art neural network layers, Q-RUN reduces model parameters while decreasing error by approximately one to three orders of magnitude on certain tasks. Notably, Q-RUN can serve as a drop-in replacement for standard fully connected layers, improving the performance of a wide range of neural architectures. This work illustrates how principles from quantum machine learning can guide the design of more expressive artificial intelligence.

Samuel Soo, Chen Guang, Wesley Teng et al.

Effective and reliable control over large language model (LLM) behavior is a significant challenge. While activation steering methods, which add steering vectors to a model's hidden states, are a promising approach, existing techniques often lack precision and interpretability in how they influence model outputs. We introduce Feature Guided Activation Additions (FGAA), a novel activation steering method that leverages insights from Contrastive Activation Addition (CAA) and Sparse Autoencoder-Targeted Steering (SAE-TS). By operating in the latent space of a Sparse Autoencoder (SAE) and employing optimization techniques to select desired SAE features, FGAA constructs precise steering vectors that provide better steering effects while maintaining coherence of steered model outputs. In this regard, evaluations on Gemma-2-2B and Gemma-2-9B models across various steering tasks demonstrate that FGAA outperforms existing steering methods of CAA, SAE decoder steering, and SAE-TS. Our results also highlight important trade-offs between steering scale and general model capabilities that are consistent across all tested steering methods.