APRIL: Interactively Learning to Summarise by Combining Active Preference Learning and Reinforcement Learning
This work addresses the challenge of high sample complexity in preference-based interactive summarization, making it more practical for users who find it easier to provide preferences than reference summaries.
The paper tackles the problem of automatic document summarization without reference summaries by interactively learning from user preferences, and it achieves a significant reduction in sample complexity, advancing the state of the art in both simulation and real-user experiments.
We propose a method to perform automatic document summarisation without using reference summaries. Instead, our method interactively learns from users' preferences. The merit of preference-based interactive summarisation is that preferences are easier for users to provide than reference summaries. Existing preference-based interactive learning methods suffer from high sample complexity, i.e. they need to interact with the oracle for many rounds in order to converge. In this work, we propose a new objective function, which enables us to leverage active learning, preference learning and reinforcement learning techniques in order to reduce the sample complexity. Both simulation and real-user experiments suggest that our method significantly advances the state of the art. Our source code is freely available at https://github.com/UKPLab/emnlp2018-april.