Online Segment to Segment Neural Transduction
This addresses the problem of slow or inefficient sequence generation in neural machine translation and related tasks for AI researchers, representing a novel method rather than an incremental improvement.
The paper tackles the bottleneck of vanilla encoder-decoders that must read the entire input before output by introducing an online neural sequence-to-sequence model that alternates between encoding and decoding segments, allowing exact polynomial marginalization of latent segmentation during training and beam search for alignment during decoding. Experiments on abstractive sentence summarization and morphological inflection show significant performance gains over baseline encoder-decoders.
We introduce an online neural sequence to sequence model that learns to alternate between encoding and decoding segments of the input as it is read. By independently tracking the encoding and decoding representations our algorithm permits exact polynomial marginalization of the latent segmentation during training, and during decoding beam search is employed to find the best alignment path together with the predicted output sequence. Our model tackles the bottleneck of vanilla encoder-decoders that have to read and memorize the entire input sequence in their fixed-length hidden states before producing any output. It is different from previous attentive models in that, instead of treating the attention weights as output of a deterministic function, our model assigns attention weights to a sequential latent variable which can be marginalized out and permits online generation. Experiments on abstractive sentence summarization and morphological inflection show significant performance gains over the baseline encoder-decoders.