The Challenge of Identifying the Origin of Black-Box Large Language Models
This addresses the issue of unauthorized LLM use for stakeholders like regulators and developers, though it is incremental as it builds on existing proactive methods.
The paper tackles the problem of identifying unauthorized fine-tuned derivatives of large language models (LLMs) when only black-box API access is available, proposing PlugAE, a proactive technique that optimizes adversarial token embeddings to achieve substantial improvement in identification accuracy.
The tremendous commercial potential of large language models (LLMs) has heightened concerns about their unauthorized use. Third parties can customize LLMs through fine-tuning and offer only black-box API access, effectively concealing unauthorized usage and complicating external auditing processes. This practice not only exacerbates unfair competition, but also violates licensing agreements. In response, identifying the origin of black-box LLMs is an intrinsic solution to this issue. In this paper, we first reveal the limitations of state-of-the-art passive and proactive identification methods with experiments on 30 LLMs and two real-world black-box APIs. Then, we propose the proactive technique, PlugAE, which optimizes adversarial token embeddings in a continuous space and proactively plugs them into the LLM for tracing and identification. The experiments show that PlugAE can achieve substantial improvement in identifying fine-tuned derivatives. We further advocate for legal frameworks and regulations to better address the challenges posed by the unauthorized use of LLMs.