Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5
This work addresses the challenge of cross-task transfer learning for explainable natural language inference, offering an effective method for researchers and practitioners in NLP, though it is incremental as it builds on existing text-to-text models.
The paper tackled the problem of boosting performance on two interdependent tasks—predicting language inference labels on figurative language and generating textual explanations—by comparing sequential fine-tuning with multi-task learning using T5 models. The result showed that sequential fine-tuning achieved the tied highest score on the FigLang2022 shared task, outperforming multi-task learning on the second task and avoiding overfitting issues.
We compare sequential fine-tuning with a model for multi-task learning in the context where we are interested in boosting performance on two tasks, one of which depends on the other. We test these models on the FigLang2022 shared task which requires participants to predict language inference labels on figurative language along with corresponding textual explanations of the inference predictions. Our results show that while sequential multi-task learning can be tuned to be good at the first of two target tasks, it performs less well on the second and additionally struggles with overfitting. Our findings show that simple sequential fine-tuning of text-to-text models is an extraordinarily powerful method for cross-task knowledge transfer while simultaneously predicting multiple interdependent targets. So much so, that our best model achieved the (tied) highest score on the task.