Analysis of function approximation and stability of general DNNs in directed acyclic graphs using un-rectifying analysis
This work addresses the lack of mathematical tools for analyzing diverse DNN architectures, which is an incremental theoretical contribution for researchers in machine learning theory.
The paper tackles the problem of analyzing deep neural networks (DNNs) by developing a method using directed acyclic graphs (DAGs) and un-rectifying analysis to derive properties of general DNNs, advancing theoretical understanding without specific numerical results.
A general lack of understanding pertaining to deep feedforward neural networks (DNNs) can be attributed partly to a lack of tools with which to analyze the composition of non-linear functions, and partly to a lack of mathematical models applicable to the diversity of DNN architectures. In this paper, we made a number of basic assumptions pertaining to activation functions, non-linear transformations, and DNN architectures in order to use the un-rectifying method to analyze DNNs via directed acyclic graphs (DAGs). DNNs that satisfy these assumptions are referred to as general DNNs. Our construction of an analytic graph was based on an axiomatic method in which DAGs are built from the bottom-up through the application of atomic operations to basic elements in accordance with regulatory rules. This approach allows us to derive the properties of general DNNs via mathematical induction. We show that using the proposed approach, some properties hold true for general DNNs can be derived. This analysis advances our understanding of network functions and could promote further theoretical insights if the host of analytical tools for graphs can be leveraged.