A Nesterov's Accelerated quasi-Newton method for Global Routing using Deep Reinforcement Learning
This is an incremental improvement for global routing in electronic design automation, potentially speeding up training for practitioners.
The paper tackled slow training of deep Q-networks in reinforcement learning by introducing a second-order Nesterov's accelerated quasi-Newton method, showing it obtains better routing solutions than first-order methods like Adam and RMSprop.
Deep Q-learning method is one of the most popularly used deep reinforcement learning algorithms which uses deep neural networks to approximate the estimation of the action-value function. Training of the deep Q-network (DQN) is usually restricted to first order gradient based methods. This paper attempts to accelerate the training of deep Q-networks by introducing a second order Nesterov's accelerated quasi-Newton method. We evaluate the performance of the proposed method on deep reinforcement learning using double DQNs for global routing. The results show that the proposed method can obtain better routing solutions compared to the DQNs trained with first order Adam and RMSprop methods.