The Topic Confusion Task: A Novel Scenario for Authorship Attribution
This work addresses a specific issue in authorship attribution for researchers, but it is incremental as it builds on existing scenarios without introducing a new paradigm.
The authors tackled the problem of distinguishing errors in authorship attribution caused by topic shifts versus inability to capture writing styles by proposing a topic confusion task, showing that stylometric features with part-of-speech tags are least susceptible to topic variations and combining them improves accuracy, while pretrained models like BERT perform poorly.
Authorship attribution is the problem of identifying the most plausible author of an anonymous text from a set of candidate authors. Researchers have investigated same-topic and cross-topic scenarios of authorship attribution, which differ according to whether new, unseen topics are used in the testing phase. However, neither scenario allows us to explain whether errors are caused by a failure to capture authorship writing style or by a topic shift. Motivated by this, we propose the \emph{topic confusion} task where we switch the author-topic configuration between the training and testing sets. This setup allows us to distinguish two types of errors: those caused by the topic shift and those caused by the features' inability to capture the writing styles. We show that stylometric features with part-of-speech tags are the least susceptible to topic variations. We further show that combining them with other features leads to significantly lower topic confusion and higher attribution accuracy. Finally, we show that pretrained language models such as BERT and RoBERTa perform poorly on this task and are surpassed by simple features such as word-level $n$-grams.