A Simple and Effective Approach to the Story Cloze Test
This work addresses the Story Cloze Test for natural language understanding, but it is incremental as it builds on prior findings about dataset issues.
The paper tackled the Story Cloze Test by proposing a fully-neural approach using skip-thought embeddings in a feed-forward network, achieving close to state-of-the-art performance without feature engineering, and found that using only the last sentence of the prompt improved accuracy.
In the Story Cloze Test, a system is presented with a 4-sentence prompt to a story, and must determine which one of two potential endings is the 'right' ending to the story. Previous work has shown that ignoring the training set and training a model on the validation set can achieve high accuracy on this task due to stylistic differences between the story endings in the training set and validation and test sets. Following this approach, we present a simpler fully-neural approach to the Story Cloze Test using skip-thought embeddings of the stories in a feed-forward network that achieves close to state-of-the-art performance on this task without any feature engineering. We also find that considering just the last sentence of the prompt instead of the whole prompt yields higher accuracy with our approach.