Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU
This addresses the challenge of integrating state-of-the-art NLP models into interactive systems that require real-time processing, though it is incremental as it adapts existing methods rather than introducing new paradigms.
The paper tackled the problem of using bidirectional language encoders, which typically require full input sequences, in incremental processing scenarios where partial output must be provided based on partial input, and found that these models can retain most of their non-incremental quality under incremental interfaces, with BERT being more impacted but mitigable through adaptations like truncated training or delayed output.
While humans process language incrementally, the best language encoders currently used in NLP do not. Both bidirectional LSTMs and Transformers assume that the sequence that is to be encoded is available in full, to be processed either forwards and backwards (BiLSTMs) or as a whole (Transformers). We investigate how they behave under incremental interfaces, when partial output must be provided based on partial input seen up to a certain time step, which may happen in interactive systems. We test five models on various NLU datasets and compare their performance using three incremental evaluation metrics. The results support the possibility of using bidirectional encoders in incremental mode while retaining most of their non-incremental quality. The "omni-directional" BERT model, which achieves better non-incremental performance, is impacted more by the incremental access. This can be alleviated by adapting the training regime (truncated training), or the testing procedure, by delaying the output until some right context is available or by incorporating hypothetical right contexts generated by a language model like GPT-2.