An Analytic End-to-End Deep Learning Algorithm based on Collaborative Learning
This work addresses the problem of ensuring stability and reliability in control applications for researchers and practitioners by offering a theoretically grounded deep learning method, though it is incremental as it builds on prior convergence analyses.
The paper tackled the lack of theoretical convergence analysis in deep learning for control tasks by providing a convergence analysis for end-to-end deep learning with smooth activation functions, avoiding chattering and gradient vanishing issues, and demonstrated performance with a classification case on MNIST and an online control task for a UR5e robot arm.
In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. Recently, some results have been obtained for convergence analysis of end-to end deep learning based on non-smooth ReLU activation functions, which may result in chattering for control tasks. This paper presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method therefore avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning can be used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset was done to demonstrate the performance of the proposed approach. Then an online kinematics control task of a UR5e robot arm was performed to illustrate the regression approximation and online updating ability of our algorithm.