Neural Semantic Encoders
This addresses natural language processing challenges for researchers and practitioners, offering a flexible model for multiple tasks, though it appears incremental as it builds on existing memory-augmented architectures.
The authors tackled the problem of natural language understanding by introducing Neural Semantic Encoders (NSE), a memory-augmented neural network with a novel memory update rule and variable-sized encoding memory, achieving state-of-the-art performance on tasks like natural language inference and machine translation, with an improvement of about 1.0 BLEU over a baseline.
We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time and maintains the understanding of input sequences through read}, compose and write operations. NSE can also access multiple and shared memories. In this paper, we demonstrated the effectiveness and the flexibility of NSE on five different natural language tasks: natural language inference, question answering, sentence classification, document sentiment analysis and machine translation where NSE achieved state-of-the-art performance when evaluated on publically available benchmarks. For example, our shared-memory model showed an encouraging result on neural machine translation, improving an attention-based baseline by approximately 1.0 BLEU.