EXPATS: A Toolkit for Explainable Automated Text Scoring
This toolkit addresses the need for accessible and interpretable tools for practitioners in educational applications of natural language processing, though it is incremental as it builds on existing frameworks and tools.
The authors tackled the challenge of developing and experimenting with automated text scoring models by introducing EXPATS, an open-source toolkit that offers flexible components, an easy-to-use configuration system, and command-line interface, enabling quick model development with minimal engineering efforts.
Automated text scoring (ATS) tasks, such as automated essay scoring and readability assessment, are important educational applications of natural language processing. Due to their interpretability of models and predictions, traditional machine learning (ML) algorithms based on handcrafted features are still in wide use for ATS tasks. Practitioners often need to experiment with a variety of models (including deep and traditional ML ones), features, and training objectives (regression and classification), although modern deep learning frameworks such as PyTorch require deep ML expertise to fully utilize. In this paper, we present EXPATS, an open-source framework to allow its users to develop and experiment with different ATS models quickly by offering flexible components, an easy-to-use configuration system, and the command-line interface. The toolkit also provides seamless integration with the Language Interpretability Tool (LIT) so that one can interpret and visualize models and their predictions. We also describe two case studies where we build ATS models quickly with minimal engineering efforts. The toolkit is available at \url{https://github.com/octanove/expats}.