No Training Required: Exploring Random Encoders for Sentence Classification
This work addresses the need for more rigorous evaluation in sentence classification, offering incremental improvements in baseline methods for researchers in natural language processing.
The paper tackled the problem of evaluating sentence embeddings by comparing modern methods to random encoders without training, finding that random methods perform surprisingly well, with gains often minimal, and provided stronger baselines and experimental protocols for future research.
We explore various methods for computing sentence representations from pre-trained word embeddings without any training, i.e., using nothing but random parameterizations. Our aim is to put sentence embeddings on more solid footing by 1) looking at how much modern sentence embeddings gain over random methods---as it turns out, surprisingly little; and by 2) providing the field with more appropriate baselines going forward---which are, as it turns out, quite strong. We also make important observations about proper experimental protocol for sentence classification evaluation, together with recommendations for future research.