The Blessing and Curse of Dimensionality in Safety Alignment
This work addresses safety vulnerabilities in LLMs, which is critical for their safe deployment across domains, but it is incremental as it builds on existing concepts like activation engineering.
The paper investigates how high-dimensional representations in large language models (LLMs) can be exploited via linear structures to circumvent safety alignment, and shows that projecting representations onto lower-dimensional subspaces reduces susceptibility to jailbreaking, with empirical results confirming significant reductions in vulnerability.
The focus on safety alignment in large language models (LLMs) has increased significantly due to their widespread adoption across different domains. The scale of LLMs play a contributing role in their success, and the growth in parameter count follows larger hidden dimensions. In this paper, we hypothesize that while the increase in dimensions has been a key advantage, it may lead to emergent problems as well. These problems emerge as the linear structures in the activation space can be exploited, in the form of activation engineering, to circumvent its safety alignment. Through detailed visualizations of linear subspaces associated with different concepts, such as safety, across various model scales, we show that the curse of high-dimensional representations uniquely impacts LLMs. Further substantiating our claim, we demonstrate that projecting the representations of the model onto a lower dimensional subspace can preserve sufficient information for alignment while avoiding those linear structures. Empirical results confirm that such dimensional reduction significantly reduces susceptibility to jailbreaking through representation engineering. Building on our empirical validations, we provide theoretical insights into these linear jailbreaking methods relative to a model's hidden dimensions. Broadly speaking, our work posits that the high dimensions of a model's internal representations can be both a blessing and a curse in safety alignment.