Optimal Density Functions for Weighted Convolution in Learning Models
This method addresses the problem of enhancing approximation accuracy in convolutional neural networks for image processing tasks, representing an incremental improvement over standard convolution.
The paper tackles the problem of improving convolution in neural networks by introducing a weighted convolution with an optimal density function that scales pixel contributions based on distance, rather than treating them uniformly. Experimental results on an image denoising task show up to a 53% reduction in loss and increased test accuracy, though with an 11% increase in execution time.
The paper introduces the weighted convolution, a novel approach to the convolution for signals defined on regular grids (e.g., 2D images) through the application of an optimal density function to scale the contribution of neighbouring pixels based on their distance from the central pixel. This choice differs from the traditional uniform convolution, which treats all neighbouring pixels equally. Our weighted convolution can be applied to convolutional neural network problems to improve the approximation accuracy. Given a convolutional network, we define a framework to compute the optimal density function through a minimisation model. The framework separates the optimisation of the convolutional kernel weights (using stochastic gradient descent) from the optimisation of the density function (using DIRECT-L). Experimental results on a learning model for an image-to-image task (e.g., image denoising) show that the weighted convolution significantly reduces the loss (up to 53% improvement) and increases the test accuracy compared to standard convolution. While this method increases execution time by 11%, it is robust across several hyperparameters of the learning model. Future work will apply the weighted convolution to real-case 2D and 3D image convolutional learning problems.