Generation, Evaluation, and Explanation of Novelists' Styles with Single-Token Prompts
This work addresses challenges in stylometry for researchers and practitioners by providing a framework for generating and evaluating stylistic text without paired data or reliance on human judgment, though it is incremental as it builds on existing language model techniques.
The authors tackled the problem of generating and evaluating sentences in the style of 19th-century novelists using large language models fine-tuned with single-token prompts, and found that the generated text reflects authors' distinctive patterns with AI-based evaluation offering a reliable alternative to human assessment.
Recent advances in large language models have created new opportunities for stylometry, the study of writing styles and authorship. Two challenges, however, remain central: training generative models when no paired data exist, and evaluating stylistic text without relying only on human judgment. In this work, we present a framework for both generating and evaluating sentences in the style of 19th-century novelists. Large language models are fine-tuned with minimal, single-token prompts to produce text in the voices of authors such as Dickens, Austen, Twain, Alcott, and Melville. To assess these generative models, we employ a transformer-based detector trained on authentic sentences, using it both as a classifier and as a tool for stylistic explanation. We complement this with syntactic comparisons and explainable AI methods, including attention-based and gradient-based analyses, to identify the linguistic cues that drive stylistic imitation. Our findings show that the generated text reflects the authors' distinctive patterns and that AI-based evaluation offers a reliable alternative to human assessment. All artifacts of this work are published online.