Plug-and-Play Adaptation for Continuously-updated QA
This addresses the need for efficient knowledge updates in language models for practical applications, though it is incremental as it builds on existing methods for knowledge retention.
The paper tackles the problem of updating knowledge in language models used as knowledge bases by proposing a new task, Continuously-updated QA (CuQA), to assess performance after multiple large-scale updates, and introduces plug-in modules that achieve a 4x improvement in updates/forgets ratio compared to fine-tuning baselines.
Language models (LMs) have shown great potential as implicit knowledge bases (KBs). And for their practical use, knowledge in LMs need to be updated periodically. However, existing tasks to assess LMs' efficacy as KBs do not adequately consider multiple large-scale updates. To this end, we first propose a novel task--Continuously-updated QA (CuQA)--in which multiple large-scale updates are made to LMs, and the performance is measured with respect to the success in adding and updating knowledge while retaining existing knowledge. We then present LMs with plug-in modules that effectively handle the updates. Experiments conducted on zsRE QA and NQ datasets show that our method outperforms existing approaches. We find that our method is 4x more effective in terms of updates/forgets ratio, compared to a fine-tuning baseline.