Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature
This addresses the need for the scientific community to keep up with rapidly flowing COVID-19 literature, though it is incremental as it adapts existing methods to a specific domain.
The authors tackled the problem of summarizing the overwhelming volume of daily COVID-19 research by proposing a novel BERT architecture for continual learning, which provides brief and original summaries while minimizing catastrophic forgetting, with benchmark and manual examination showing it delivers sound summaries.
The scientific community continues to publish an overwhelming amount of new research related to COVID-19 on a daily basis, leading to much literature without little to no attention. To aid the community in understanding the rapidly flowing array of COVID-19 literature, we propose a novel BERT architecture that provides a brief yet original summarization of lengthy papers. The model continually learns on new data in online fashion while minimizing catastrophic forgetting, thus fitting to the need of the community. Benchmark and manual examination of its performance show that the model provide a sound summary of new scientific literature.