Linear-Size Neural Network Representation of Piecewise Affine Functions in $\mathbb{R}^2$
This provides a theoretical foundation for neural network expressiveness in approximating complex functions, though it is incremental by extending to non-convex pieces.
The paper tackles the problem of representing continuous piecewise affine functions in 2D with ReLU neural networks, showing that such functions with p pieces can be represented using two hidden layers and O(p) neurons, without requiring convex pieces as in prior work.
It is shown that any continuous piecewise affine (CPA) function $\mathbb{R}^2\to\mathbb{R}$ with $p$ pieces can be represented by a ReLU neural network with two hidden layers and $O(p)$ neurons. Unlike prior work, which focused on convex pieces, this analysis considers CPA functions with connected but potentially non-convex pieces.