Nearest Neighbor Representations of Neural Circuits
This work addresses a theoretical gap in understanding the computational equivalence between NN representations and neural networks, which is incremental as it extends known results from single neurons to small-depth networks.
The paper tackles the problem of representing neural networks using Nearest Neighbor (NN) representations, specifically focusing on depth-2 threshold circuits, and provides explicit constructions with bounds on the number of bits required for functions like convex polytopes and decision lists.
Neural networks successfully capture the computational power of the human brain for many tasks. Similarly inspired by the brain architecture, Nearest Neighbor (NN) representations is a novel approach of computation. We establish a firmer correspondence between NN representations and neural networks. Although it was known how to represent a single neuron using NN representations, there were no results even for small depth neural networks. Specifically, for depth-2 threshold circuits, we provide explicit constructions for their NN representation with an explicit bound on the number of bits to represent it. Example functions include NN representations of convex polytopes (AND of threshold gates), IP2, OR of threshold gates, and linear or exact decision lists.