Self-Calibrating Neural-Probabilistic Model for Authorship Verification Under Covariate Shift
This work addresses critical reliability issues in authorship verification systems, which is important for forensic and legal applications, but it appears incremental as it builds on an existing framework.
The authors tackled the problems of topic variability and miscalibration in authorship verification by expanding a prior framework with Bayes factor scoring and an uncertainty adaptation layer, resulting in significant reductions in sensitivity to topical variations and improvements in calibration.
We are addressing two fundamental problems in authorship verification (AV): Topic variability and miscalibration. Variations in the topic of two disputed texts are a major cause of error for most AV systems. In addition, it is observed that the underlying probability estimates produced by deep learning AV mechanisms oftentimes do not match the actual case counts in the respective training data. As such, probability estimates are poorly calibrated. We are expanding our framework from PAN 2020 to include Bayes factor scoring (BFS) and an uncertainty adaptation layer (UAL) to address both problems. Experiments with the 2020/21 PAN AV shared task data show that the proposed method significantly reduces sensitivities to topical variations and significantly improves the system's calibration.