Designing Neural Network Architectures using Reinforcement Learning
This addresses the labor-intensive process of CNN design for researchers and practitioners, offering an automated alternative that is competitive but incremental in improving performance.
The authors tackled the problem of manually designing convolutional neural network architectures by introducing MetaQNN, a reinforcement learning-based algorithm that automatically generates high-performing CNN architectures. On image classification benchmarks, the agent-designed networks outperformed existing networks with the same layer types and were competitive against state-of-the-art methods using more complex layers.
At present, designing convolutional neural network (CNN) architectures requires both human expertise and labor. New architectures are handcrafted by careful experimentation or modified from a handful of existing networks. We introduce MetaQNN, a meta-modeling algorithm based on reinforcement learning to automatically generate high-performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using $Q$-learning with an $ε$-greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task. On image classification benchmarks, the agent-designed networks (consisting of only standard convolution, pooling, and fully-connected layers) beat existing networks designed with the same layer types and are competitive against the state-of-the-art methods that use more complex layer types. We also outperform existing meta-modeling approaches for network design on image classification tasks.