WR-ONE2SET: Towards Well-Calibrated Keyphrase Generation
This work addresses calibration issues in keyphrase generation for NLP applications, but it is incremental as it builds directly on the existing ONE2SET paradigm.
The paper tackled the problem of calibration errors in keyphrase generation, specifically over-estimation of the 'no keyphrase' token in the ONE2SET paradigm, by proposing WR-ONE2SET with adaptive weighting and target re-assignment, which improved calibration without specifying concrete numbers.
Keyphrase generation aims to automatically generate short phrases summarizing an input document. The recently emerged ONE2SET paradigm (Ye et al., 2021) generates keyphrases as a set and has achieved competitive performance. Nevertheless, we observe serious calibration errors outputted by ONE2SET, especially in the over-estimation of $\varnothing$ token (means "no corresponding keyphrase"). In this paper, we deeply analyze this limitation and identify two main reasons behind: 1) the parallel generation has to introduce excessive $\varnothing$ as padding tokens into training instances; and 2) the training mechanism assigning target to each slot is unstable and further aggravates the $\varnothing$ token over-estimation. To make the model well-calibrated, we propose WR-ONE2SET which extends ONE2SET with an adaptive instance-level cost Weighting strategy and a target Re-assignment mechanism. The former dynamically penalizes the over-estimated slots for different instances thus smoothing the uneven training distribution. The latter refines the original inappropriate assignment and reduces the supervisory signals of over-estimated slots. Experimental results on commonly-used datasets demonstrate the effectiveness and generality of our proposed paradigm.