Statistical model-based evaluation of neural networks
This work provides a benchmark for evaluating neural networks against theoretical limits, which is useful for researchers designing and applying NNs.
The paper proposes a statistical model-based data generation setup to evaluate neural networks against minimum-mean-square-error (MMSE) performance bounds. This setup was used to test the effects of various data conditions on NN performance.
Using a statistical model-based data generation, we develop an experimental setup for the evaluation of neural networks (NNs). The setup helps to benchmark a set of NNs vis-a-vis minimum-mean-square-error (MMSE) performance bounds. This allows us to test the effects of training data size, data dimension, data geometry, noise, and mismatch between training and testing conditions. In the proposed setup, we use a Gaussian mixture distribution to generate data for training and testing a set of competing NNs. Our experiments show the importance of understanding the type and statistical conditions of data for appropriate application and design of NNs