Neural Networks Built from Unreliable Components
This addresses the challenge of building reliable neural networks from unreliable components, which is incremental as it extends existing associative memory models by incorporating internal noise.
The paper tackles the problem of associative memory systems with noisy internal components, showing that recall error probability can be made very small despite noise, and characterizes a threshold phenomenon while optimizing inference parameters based on noise statistics.
Recent advances in associative memory design through strutured pattern sets and graph-based inference algorithms have allowed the reliable learning and retrieval of an exponential number of patterns. Both these and classical associative memories, however, have assumed internally noiseless computational nodes. This paper considers the setting when internal computations are also noisy. Even if all components are noisy, the final error probability in recall can often be made exceedingly small, as we characterize. There is a threshold phenomenon. We also show how to optimize inference algorithm parameters when knowing statistical properties of internal noise.