Evolving Deep Neural Networks by Multi-objective Particle Swarm Optimization for Image Classification
This addresses the difficulty of deploying state-of-the-art deep CNNs in industrial applications by automating hyperparameter optimization, though it is incremental as it builds on existing methods like particle swarm optimization.
The paper tackles the problem of manually fine-tuning hyperparameters and balancing accuracy with computational cost in deep CNNs for image classification by proposing a multi-objective particle swarm optimization method, which evolves non-dominant solutions to form a clear Pareto front and reduces running time almost linearly through a new infrastructure.
In recent years, convolutional neural networks (CNNs) have become deeper in order to achieve better classification accuracy in image classification. However, it is difficult to deploy the state-of-the-art deep CNNs for industrial use due to the difficulty of manually fine-tuning the hyperparameters and the trade-off between classification accuracy and computational cost. This paper proposes a novel multi-objective optimization method for evolving state-of-the-art deep CNNs in real-life applications, which automatically evolves the non-dominant solutions at the Pareto front. Three major contributions are made: Firstly, a new encoding strategy is designed to encode one of the best state-of-the-art CNNs; With the classification accuracy and the number of floating point operations as the two objectives, a multi-objective particle swarm optimization method is developed to evolve the non-dominant solutions; Last but not least, a new infrastructure is designed to boost the experiments by concurrently running the experiments on multiple GPUs across multiple machines, and a Python library is developed and released to manage the infrastructure. The experimental results demonstrate that the non-dominant solutions found by the proposed algorithm form a clear Pareto front, and the proposed infrastructure is able to almost linearly reduce the running time.