Can LLM Safety Be Ensured by Constraining Parameter Regions?
This addresses the problem of ensuring AI safety for developers and users, but the findings are incremental as they highlight limitations in existing approaches.
The paper investigated whether safety in large language models can be ensured by identifying specific parameter regions, finding that current methods yield low to moderate overlap and fail to reliably identify stable, dataset-agnostic safety regions.
Large language models (LLMs) are often assumed to contain ``safety regions'' -- parameter subsets whose modification directly influences safety behaviors. We conduct a systematic evaluation of four safety region identification methods spanning different parameter granularities, from individual weights to entire Transformer layers, across four families of backbone LLMs with varying sizes. Using ten safety identification datasets, we find that the identified safety regions exhibit only low to moderate overlap, as measured by IoU. The overlap drops significantly when the safety regions are further refined using utility datasets (\ie non-harmful queries). These results suggest that current techniques fail to reliably identify a stable, dataset-agnostic safety region.