Collaboration between parallel connected neural networks -- A possible criterion for distinguishing artificial neural networks from natural organs
This provides a simple criterion for researchers to assess the biological realism of artificial neural networks, though it is incremental as it builds on existing network architectures.
The study tackled the problem of distinguishing artificial neural networks from natural biological organs by experimentally identifying three properties of parallel-connected neural networks (PNNs) that are unlikely in natural systems, such as sub-networks not being optimized individually and the PNN outputting correct results even when all sub-networks are incorrect. The result proposed a criterion for measuring bionic levels, showing that ReLU activation functions make networks more bionic than sigmoid or Tanh.
We find experimentally that when artificial neural networks are connected in parallel and trained together, they display the following properties. (i) When the parallel-connected neural network (PNN) is optimized, each sub-network in the connection is not optimized. (ii) The contribution of an inferior sub-network to the whole PNN can be on par with that of the superior sub-network. (iii) The PNN can output the correct result even when all sub-networks give incorrect results. These properties are unlikely for natural biological sense organs. Therefore, they could serve as a simple yet effective criterion for measuring the bionic level of neural networks. With this criterion, we further show that when serving as the activation function, the ReLU function can make an artificial neural network more bionic than the sigmoid and Tanh functions do.