jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
This toolkit addresses the problem of fragmented experimentation for researchers in natural language understanding, though it is incremental as it builds on existing models and benchmarks.
The authors tackled the need for a unified toolkit for multitask and transfer learning experiments in English NLU by introducing jiant, an open-source software toolkit that reproduces published performance on over 50 tasks, including GLUE and SuperGLUE benchmarks, with models like BERT and RoBERTa.
We introduce jiant, an open source toolkit for conducting multitask and transfer learning experiments on English NLU tasks. jiant enables modular and configuration-driven experimentation with state-of-the-art models and implements a broad set of tasks for probing, transfer learning, and multitask training experiments. jiant implements over 50 NLU tasks, including all GLUE and SuperGLUE benchmark tasks. We demonstrate that jiant reproduces published performance on a variety of tasks and models, including BERT and RoBERTa. jiant is available at https://jiant.info.