Autoselection of the Ensemble of Convolutional Neural Networks with Second-Order Cone Programming
This work addresses the problem of computational efficiency for practitioners using deep learning ensembles, though it is incremental as it builds on existing ensemble pruning techniques.
The study tackled the computational complexity of ensemble pruning in deep learning by proposing a mathematical model that prunes ensembles of CNNs with varying depths and layers to maximize accuracy and diversity using sparse second-order conic optimization, achieving promising results on CIFAR-10, CIFAR-100, and MNIST datasets while significantly reducing model complexity.
Ensemble techniques are frequently encountered in machine learning and engineering problems since the method combines different models and produces an optimal predictive solution. The ensemble concept can be adapted to deep learning models to provide robustness and reliability. Due to the growth of the models in deep learning, using ensemble pruning is highly important to deal with computational complexity. Hence, this study proposes a mathematical model which prunes the ensemble of Convolutional Neural Networks (CNN) consisting of different depths and layers that maximizes accuracy and diversity simultaneously with a sparse second order conic optimization model. The proposed model is tested on CIFAR-10, CIFAR-100 and MNIST data sets which gives promising results while reducing the complexity of models, significantly.