Some Geometrical and Topological Properties of DNNs' Decision Boundaries
This work provides principled guidance for designing and regularizing DNNs to improve classification performance and robustness, though it is incremental in applying existing mathematical tools.
The paper theoretically explores the geometrical and topological properties of decision boundaries in deep neural networks using differential geometry, deriving conditions for flat boundaries and a method to compute Euler characteristics, with experimental verification.
Geometry and topology of decision regions are closely related with classification performance and robustness against adversarial attacks. In this paper, we use differential geometry to theoretically explore the geometrical and topological properties of decision regions produced by deep neural networks (DNNs). The goal is to obtain some geometrical and topological properties of decision boundaries for given DNN models, and provide some principled guidance to design and regularization of DNNs. First, we present the curvatures of decision boundaries in terms of network parameters, and give sufficient conditions on network parameters for producing flat or developable decision boundaries. Based on the Gauss-Bonnet-Chern theorem in differential geometry, we then propose a method to compute the Euler characteristics of compact decision boundaries, and verify it with experiments.