SELF: A Robust Singular Value and Eigenvalue Approach for LLM Fingerprinting
This addresses the critical challenge of IP protection for LLM developers, offering a more secure method against attacks, though it appears incremental as an improvement over existing fingerprinting techniques.
The paper tackled the problem of protecting intellectual property in large language models by proposing SELF, a robust fingerprinting scheme that uses singular value and eigenvalue decomposition of attention weights, achieving high accuracy in detecting IP infringement while resisting modifications like quantization and fine-tuning.
The protection of Intellectual Property (IP) in Large Language Models (LLMs) represents a critical challenge in contemporary AI research. While fingerprinting techniques have emerged as a fundamental mechanism for detecting unauthorized model usage, existing methods -- whether behavior-based or structural -- suffer from vulnerabilities such as false claim attacks or susceptible to weight manipulations. To overcome these limitations, we propose SELF, a novel intrinsic weight-based fingerprinting scheme that eliminates dependency on input and inherently resists false claims. SELF achieves robust IP protection through two key innovations: 1) unique, scalable and transformation-invariant fingerprint extraction via singular value and eigenvalue decomposition of LLM attention weights, and 2) effective neural network-based fingerprint similarity comparison based on few-shot learning and data augmentation. Experimental results demonstrate SELF maintains high IP infringement detection accuracy while showing strong robustness against various downstream modifications, including quantization, pruning, and fine-tuning attacks. Our code is available at https://github.com/HanxiuZhang/SELF_v2.