A Letter on Convergence of In-Parameter-Linear Nonlinear Neural Architectures with Gradient Learnings
This work addresses stability and convergence issues in neural network training for real-time applications, but it appears incremental as it extends existing stability concepts to a specific architecture family.
The paper tackles the problem of ensuring weight convergence for a broad family of in-parameter-linear nonlinear neural architectures using incremental gradient learning algorithms, and it proves a bounded-input bounded-state (BIBS) stability condition that applies to individual learning points or batches for real-time applications.
This letter summarizes and proves the concept of bounded-input bounded-state (BIBS) stability for weight convergence of a broad family of in-parameter-linear nonlinear neural architectures as it generally applies to a broad family of incremental gradient learning algorithms. A practical BIBS convergence condition results from the derived proofs for every individual learning point or batches for real-time applications.