Detection and Measurement of Syntactic Templates in Generated Text
This provides a tool for analyzing style memorization and model differentiation, but it is incremental as it extends existing diversity evaluation from word-level to syntactic features.
The paper tackles the problem of evaluating syntactic repetition in LLM-generated text, finding that models produce templated text at a higher rate than human references, with 76% of templates traceable to pre-training data.
Recent work on evaluating the diversity of text generated by LLMs has focused on word-level features. Here we offer an analysis of syntactic features to characterize general repetition in models, beyond frequent n-grams. Specifically, we define syntactic templates and show that models tend to produce templated text in downstream tasks at a higher rate than what is found in human-reference texts. We find that most (76%) templates in model-generated text can be found in pre-training data (compared to only 35% of human-authored text), and are not overwritten during fine-tuning processes such as RLHF. This connection to the pre-training data allows us to analyze syntactic templates in models where we do not have the pre-training data. We also find that templates as features are able to differentiate between models, tasks, and domains, and are useful for qualitatively evaluating common model constructions. Finally, we demonstrate the use of templates as a useful tool for analyzing style memorization of training data in LLMs.