Wei Zhang

Songyan Wang, Non-member, Jilai Yu et al.

Analyzing system trajectory from the perspective of individual machines provides a distinctive angle to analyze the transient stability of power systems. This two-paper series propose a direct-time-domain method that is based on the individual-machine equal area criterion. In the first paper, by examining the mapping between the trajectory and power-vs-angle curve of an individual machine, the stability property to characterize a critical machine is clarified. The mapping between the system trajectory and individual-machine equal area criterion is established. Furthermore, a unity principle between the individual-machine stability and system stability is proposed. It is proved that the instability of the system can be confirmed by finding any one unstable critical machine, thence, the transient stability of a multimachine system can be monitored in an individual-machine way in transient stability assessment.

Bowen Weng, Huaqing Xiong, Lin Zhao et al.

Existing studies indicate that momentum ideas in conventional optimization can be used to improve the performance of Q-learning algorithms. However, the finite-sample analysis for momentum-based Q-learning algorithms is only available for the tabular case without function approximations. This paper analyzes a class of momentum-based Q-learning algorithms with finite-sample guarantee. Specifically, we propose the MomentumQ algorithm, which integrates the Nesterov's and Polyak's momentum schemes, and generalizes the existing momentum-based Q-learning algorithms. For the infinite state-action space case, we establish the convergence guarantee for MomentumQ with linear function approximations and Markovian sampling. In particular, we characterize the finite-sample convergence rate which is provably faster than the vanilla Q-learning. This is the first finite-sample analysis for momentum-based Q-learning algorithms with function approximations. For the tabular case under synchronous sampling, we also obtain a finite-sample convergence rate that is slightly better than the SpeedyQ \citep{azar2011speedy} when choosing a special family of step sizes. Finally, we demonstrate through various experiments that the proposed MomentumQ outperforms other momentum-based Q-learning algorithms.

Wei Zhang, Yang Yu, Bowen Zhou

Neural Turing Machines (NTM) contain memory component that simulates "working memory" in the brain to store and retrieve information to ease simple algorithms learning. So far, only linearly organized memory is proposed, and during experiments, we observed that the model does not always converge, and overfits easily when handling certain tasks. We think memory component is key to some faulty behaviors of NTM, and better organization of memory component could help fight those problems. In this paper, we propose several different structures of memory for NTM, and we proved in experiments that two of our proposed structured-memory NTMs could lead to better convergence, in term of speed and prediction accuracy on copy task and associative recall task as in (Graves et al. 2014).