Some Theoretical Properties of a Network of Discretely Firing Neurons
This addresses theoretical properties in neural network optimization, but appears incremental as it builds on existing concepts without broad practical application.
The paper tackled the optimization of discretely firing neuron networks by introducing an objective function that measures the average bits needed for state encoding, and minimizing it led to results like topographic mappings and factorial encoder networks.
The problem of optimising a network of discretely firing neurons is addressed. An objective function is introduced which measures the average number of bits that are needed for the network to encode its state. When this is minimised, it is shown that this leads to a number of results, such as topographic mappings, piecewise linear dependence on the input of the probability of a neuron firing, and factorial encoder networks.