Short-PHD: Detecting Short LLM-generated Text with Topological Data Analysis After Off-topic Content Insertion
This addresses the problem of malicious LLM usage for short texts, offering an incremental improvement over prior topological data analysis methods.
The paper tackles the challenge of detecting short LLM-generated texts by proposing Short-PHD, a zero-shot method that stabilizes persistent homology dimension estimation through off-topic content insertion, and it outperforms existing zero-shot methods in experiments.
The malicious usage of large language models (LLMs) has motivated the detection of LLM-generated texts. Previous work in topological data analysis shows that the persistent homology dimension (PHD) of text embeddings can serve as a more robust and promising score than other zero-shot methods. However, effectively detecting short LLM-generated texts remains a challenge. This paper presents Short-PHD, a zero-shot LLM-generated text detection method tailored for short texts. Short-PHD stabilizes the estimation of the previous PHD method for short texts by inserting off-topic content before the given input text and identifies LLM-generated text based on an established detection threshold. Experimental results on both public and generated datasets demonstrate that Short-PHD outperforms existing zero-shot methods in short LLM-generated text detection. Implementation codes are available online.