Express Wavenet -- a low parameter optical neural network with random shift wavelet pattern
This work addresses the challenge of parameter efficiency in optical neural networks, which is incremental as it builds on existing optical diffractive networks with specific improvements.
The authors tackled the problem of high parameter counts in optical diffractive neural networks by introducing Express Wavenet, which reduces parameters from O(n^2) to O(n) using wavelet-like patterns and an expressway structure, achieving 92% accuracy on MNIST with only 1,229 parameters compared to 125,440 in standard networks.
Express Wavenet is an improved optical diffractive neural network. At each layer, it uses wavelet-like pattern to modulate the phase of optical waves. For input image with n2 pixels, express wavenet reduce parameter number from O(n2) to O(n). Only need one percent of the parameters, and the accuracy is still very high. In the MNIST dataset, it only needs 1229 parameters to get accuracy of 92%, while the standard optical network needs 125440 parameters. The random shift wavelets show the characteristics of optical network more vividly. Especially the vanishing gradient phenomenon in the training process. We present a modified expressway structure for this problem. Experiments verified the effect of random shift wavelet and expressway structure. Our work shows optical diffractive network would use much fewer parameters than other neural networks. The source codes are available at https://github.com/closest-git/ONNet.