Assessing the Stylistic Properties of Neurally Generated Text in Authorship Attribution
This addresses the need for systematic evaluation of neural text generation for researchers in NLP and authorship attribution, though it is incremental as it builds on existing models.
The paper tackled the problem of evaluating the stylistic quality of neurally generated text, specifically its ability to reproduce authorial writing styles, and found that neural models produce fuzzier text that is harder to attribute correctly but offers valuable perspectives for data augmentation.
Recent applications of neural language models have led to an increased interest in the automatic generation of natural language. However impressive, the evaluation of neurally generated text has so far remained rather informal and anecdotal. Here, we present an attempt at the systematic assessment of one aspect of the quality of neurally generated text. We focus on a specific aspect of neural language generation: its ability to reproduce authorial writing styles. Using established models for authorship attribution, we empirically assess the stylistic qualities of neurally generated text. In comparison to conventional language models, neural models generate fuzzier text that is relatively harder to attribute correctly. Nevertheless, our results also suggest that neurally generated text offers more valuable perspectives for the augmentation of training data.