On the Equivalence between Implicit and Explicit Neural Networks: A High-dimensional Viewpoint
This provides foundational theoretical insights for researchers in machine learning, though it is incremental as it builds on existing kernel theories.
The paper tackles the lack of theoretical analysis connecting implicit and explicit neural networks by studying high-dimensional equivalents for conjugate and neural tangent kernels, establishing their equivalence in high dimensions.
Implicit neural networks have demonstrated remarkable success in various tasks. However, there is a lack of theoretical analysis of the connections and differences between implicit and explicit networks. In this paper, we study high-dimensional implicit neural networks and provide the high dimensional equivalents for the corresponding conjugate kernels and neural tangent kernels. Built upon this, we establish the equivalence between implicit and explicit networks in high dimensions.