Efficiency Evaluation of Character-level RNN Training Schedules
This work addresses efficiency optimization for researchers and practitioners using RNNs, but it is incremental as it compares existing schedules without introducing new methods.
The paper investigated how different training and prediction schedules affect the efficiency of character-level recurrent neural networks, finding that schedule choice can significantly impact prediction effectiveness relative to training time and data usage.
We present four training and prediction schedules from the same character-level recurrent neural network. The efficiency of these schedules is tested in terms of model effectiveness as a function of training time and amount of training data seen. We show that the choice of training and prediction schedule potentially has a considerable impact on the prediction effectiveness for a given training budget.