Implicit Deep Learning
This foundational approach could impact all of ML/AI by offering a more flexible and interpretable framework for deep learning.
The paper introduces implicit deep learning, a framework that generalizes feedforward neural networks by using fixed-point equations to define hidden features, simplifying notation and enabling new architectures, algorithms, and analyses.
Implicit deep learning prediction rules generalize the recursive rules of feedforward neural networks. Such rules are based on the solution of a fixed-point equation involving a single vector of hidden features, which is thus only implicitly defined. The implicit framework greatly simplifies the notation of deep learning, and opens up many new possibilities, in terms of novel architectures and algorithms, robustness analysis and design, interpretability, sparsity, and network architecture optimization.