A foundation for exact binarized morphological neural networks
This work addresses energy-efficient AI for hardware-constrained applications, but it is incremental as it builds on existing binarization and morphological methods.
The paper tackles the problem of high computational and energy costs in deep neural networks by proposing a binarized morphological neural network model, achieving performance comparable to standard ConvNets under certain conditions with new approximation methods and regularization losses.
Training and running deep neural networks (NNs) often demands a lot of computation and energy-intensive specialized hardware (e.g. GPU, TPU...). One way to reduce the computation and power cost is to use binary weight NNs, but these are hard to train because the sign function has a non-smooth gradient. We present a model based on Mathematical Morphology (MM), which can binarize ConvNets without losing performance under certain conditions, but these conditions may not be easy to satisfy in real-world scenarios. To solve this, we propose two new approximation methods and develop a robust theoretical framework for ConvNets binarization using MM. We propose as well regularization losses to improve the optimization. We empirically show that our model can learn a complex morphological network, and explore its performance on a classification task.