Story Cloze Ending Selection Baselines and Data Examination
This work provides incremental baselines for a specific NLP task, aiding researchers in evaluating story understanding systems.
The paper tackled the Story Cloze Test by developing supervised baselines, including a classifier with word embedding and similarity features and neural LSTM models, finding that the feature-based model outperformed neural approaches with an accuracy of 72.42%.
This paper describes two supervised baseline systems for the Story Cloze Test Shared Task (Mostafazadeh et al., 2016a). We first build a classifier using features based on word embeddings and semantic similarity computation. We further implement a neural LSTM system with different encoding strategies that try to model the relation between the story and the provided endings. Our experiments show that a model using representation features based on average word embedding vectors over the given story words and the candidate ending sentences words, joint with similarity features between the story and candidate ending representations performed better than the neural models. Our best model achieves an accuracy of 72.42, ranking 3rd in the official evaluation.