Stylometric Watermarks for Large Language Models
This addresses accountability and societal harm for proprietary large language models, though it is incremental as it builds on existing watermarking techniques.
The paper tackles the problem of distinguishing human-written from machine-generated text by proposing a watermarking method that alters token probabilities using linguistic features like stylometry, achieving false positive and false negative rates of 0.02 for three or more sentences.
The rapid advancement of large language models (LLMs) has made it increasingly difficult to distinguish between text written by humans and machines. Addressing this, we propose a novel method for generating watermarks that strategically alters token probabilities during generation. Unlike previous works, this method uniquely employs linguistic features such as stylometry. Concretely, we introduce acrostica and sensorimotor norms to LLMs. Further, these features are parameterized by a key, which is updated every sentence. To compute this key, we use semantic zero shot classification, which enhances resilience. In our evaluation, we find that for three or more sentences, our method achieves a false positive and false negative rate of 0.02. For the case of a cyclic translation attack, we observe similar results for seven or more sentences. This research is of particular of interest for proprietary LLMs to facilitate accountability and prevent societal harm.