Auto-MAP: A DQN Framework for Exploring Distributed Execution Plans for DNN Workloads
This work addresses the problem of inefficient and framework-specific manual parallelization for DNN workloads, offering an automated solution that is incremental in improving exploration efficiency.
The paper tackles the challenge of manually designing distributed execution plans for deep neural network training by proposing Auto-MAP, a framework that uses reinforcement learning to automatically discover fast parallelization strategies, achieving better throughput on NLP and convolution models and finding optimal solutions in two hours.
The last decade has witnessed growth in the computational requirements for training deep neural networks. Current approaches (e.g., data/model parallelism, pipeline parallelism) parallelize training tasks onto multiple devices. However, these approaches always rely on specific deep learning frameworks and requires elaborate manual design, which make it difficult to maintain and share between different type of models. In this paper, we propose Auto-MAP, a framework for exploring distributed execution plans for DNN workloads, which can automatically discovering fast parallelization strategies through reinforcement learning on IR level of deep learning models. Efficient exploration remains a major challenge for reinforcement learning. We leverage DQN with task-specific pruning strategies to help efficiently explore the search space including optimized strategies. Our evaluation shows that Auto-MAP can find the optimal solution in two hours, while achieving better throughput on several NLP and convolution models.