TTTTTackling WinoGrande Schemas
This work addresses the WinoGrande schema problem for AI and NLP researchers, but it is incremental as it applies an existing method to a specific dataset.
The authors tackled the AI2 WinoGrande Challenge by applying the T5 sequence-to-sequence model to decompose examples into hypotheses and using entailment token probabilities for scoring, achieving a best-known result of 0.7673 AUC, beating the previous state of the art by over five points.
We applied the T5 sequence-to-sequence model to tackle the AI2 WinoGrande Challenge by decomposing each example into two input text strings, each containing a hypothesis, and using the probabilities assigned to the "entailment" token as a score of the hypothesis. Our first (and only) submission to the official leaderboard yielded 0.7673 AUC on March 13, 2020, which is the best known result at this time and beats the previous state of the art by over five points.