Distributed Deep Learning for Question Answering
This work addresses efficiency in training for question answering, but it is incremental as it applies existing distributed methods to this domain.
The paper tackles the problem of accelerating deep learning for question answering tasks by evaluating multiple distributed optimization algorithms, achieving a 24x speedup with 48 workers that reduces training time from 138.2 hours to 5.81 hours.
This paper is an empirical study of the distributed deep learning for question answering subtasks: answer selection and question classification. Comparison studies of SGD, MSGD, ADADELTA, ADAGRAD, ADAM/ADAMAX, RMSPROP, DOWNPOUR and EASGD/EAMSGD algorithms have been presented. Experimental results show that the distributed framework based on the message passing interface can accelerate the convergence speed at a sublinear scale. This paper demonstrates the importance of distributed training. For example, with 48 workers, a 24x speedup is achievable for the answer selection task and running time is decreased from 138.2 hours to 5.81 hours, which will increase the productivity significantly.