Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects
It resolves a conjecture about the structure of minimal architectures in polynomial neural networks, which is a theoretical problem for researchers in neural network architecture design.
The paper disproves the minimal unimodal conjecture for polynomial neural networks by providing a counterexample found via frontier search, verified with recursive dimension bounds and symbolic computation, and shows that some subarchitectures have large defect, unlike prior examples.
We provide a counterexample to the minimal unimodal conjecture for polynomial neural networks (PNNs) with power activation functions. Fixing the input and output widths, the conjecture states that any minimal filling architecture has unimodal widths for the hidden layers. We found a counterexample via a frontier search and certified it using recursive dimension bounds and symbolic computation. Notably, several subarchitectures of this example exhibit large defect, in contrast with the predominantly small-defect behavior observed in prior examples.