Toward Zero-Shot Instruction Following
This addresses the challenge of cross-task generalization in NLP, enabling models to follow instructions without prior examples, though it is incremental as it builds on existing zero-shot settings.
The paper tackles the problem of zero-shot instruction following, where models must perform tasks based solely on paragraph-style definitions without demonstrations, achieving state-of-the-art performance on the Super-NaturalInstructions benchmark.
This work proposes a challenging yet more realistic setting for zero-shot cross-task generalization: zero-shot instruction following, presuming the existence of a paragraph-style task definition while no demonstrations exist. To better learn the task supervision from the definition, we propose two strategies: first, to automatically find out the critical sentences in the definition; second, a ranking objective to force the model to generate the gold outputs with higher probabilities when those critical parts are highlighted in the definition. The joint efforts of the two strategies yield state-of-the-art performance on the Super-NaturalInstructions. Our code is available on GitHub.