Training Deep Morphological Neural Networks as Universal Approximators
This work addresses the problem of efficient training and generalization for morphological networks, which is incremental as it builds on existing concepts with specific constraints and improvements.
The authors tackled the challenge of training deep morphological neural networks (DMNNs) by proposing constrained architectures with 'linear' activations, residual connections, and weight dropout, resulting in successfully trained networks that are more prunable than linear networks and accelerate convergence in hybrid setups.
We investigate deep morphological neural networks (DMNNs). We demonstrate that despite their inherent non-linearity, "linear" activations are essential for DMNNs. To preserve their inherent sparsity, we propose architectures that constraint the parameters of the "linear" activations: For the first (resp. second) architecture, we work under the constraint that the majority of parameters (resp. learnable parameters) should be part of morphological operations. We improve the generalization ability of our networks via residual connections and weight dropout. Our proposed networks can be successfully trained, and are more prunable than linear networks. To the best of our knowledge, we are the first to successfully train DMNNs under such constraints. Finally, we propose a hybrid network architecture combining linear and morphological layers, showing empirically that the inclusion of morphological layers significantly accelerates the convergence of gradient descent with large batches.