Analyze the Effects of Weighting Functions on Cost Function in the Glove Model
This addresses computational efficiency issues for researchers or practitioners using GloVe models on limited hardware, though it appears incremental.
The authors tackled the problem of long training times for GloVe models with large vocabularies and corpora, which can take dozens of hours for 500MB data, by deriving a weighting function that reduces parameter selection time and achieves nearly similar accuracy without extensive experimentation.
When dealing with the large vocabulary size and corpus size, the run-time for training Glove model is long, it can even be up to several dozen hours for data, which is approximately 500MB in size. As a result, finding and selecting the optimal parameters for the weighting function create many difficulties for weak hardware. Of course, to get the best results, we need to test benchmarks many times. In order to solve this problem, we derive a weighting function, which can save time for choosing parameters and making benchmarks. It also allows one to obtain nearly similar accuracy at the same given time without concern for experimentation.