Learning to Decode for Future Success
This addresses the problem of controlling specific output properties in sequence generation for NLP researchers and practitioners, representing an incremental advancement over existing decoder methods.
The paper tackles the problem of neural decoders lacking control over output properties like sequence length, introducing a simple actor-critic method that interpolates between token generation and a value function for future property estimation. The result shows consistent improvements in abstractive summarization and machine translation when optimizing for BLEU or ROUGE scores, while handling previously unmanageable properties.
We introduce a simple, general strategy to manipulate the behavior of a neural decoder that enables it to generate outputs that have specific properties of interest (e.g., sequences of a pre-specified length). The model can be thought of as a simple version of the actor-critic model that uses an interpolation of the actor (the MLE-based token generation policy) and the critic (a value function that estimates the future values of the desired property) for decision making. We demonstrate that the approach is able to incorporate a variety of properties that cannot be handled by standard neural sequence decoders, such as sequence length and backward probability (probability of sources given targets), in addition to yielding consistent improvements in abstractive summarization and machine translation when the property to be optimized is BLEU or ROUGE scores.