The Cost of Training NLP Models: A Concise Overview
arXiv:2004.08900v1235 citations
Originality Synthesis-oriented
AI Analysis
It addresses budgeting and economic understanding for those involved in NLP model training, but it is incremental as it reviews existing information without new findings.
The paper reviews the costs and drivers of training large-scale language models, targeting engineers, scientists, and non-practitioners to help with budgeting and understanding NLP economics.
We review the cost of training large-scale language models, and the drivers of these costs. The intended audience includes engineers and scientists budgeting their model-training experiments, as well as non-practitioners trying to make sense of the economics of modern-day Natural Language Processing (NLP).