MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
This work addresses the need for scalable and efficient machine learning tools for researchers and practitioners working with diverse hardware setups, representing a significant but incremental advancement in library design.
The paper tackles the challenge of developing efficient machine learning algorithms for heterogeneous distributed systems by introducing MXNet, a flexible library that blends declarative symbolic expression with imperative tensor computation, achieving promising results on large-scale deep neural network applications using multiple GPU machines.
MXNet is a multi-language machine learning (ML) library to ease the development of ML algorithms, especially for deep neural networks. Embedded in the host language, it blends declarative symbolic expression with imperative tensor computation. It offers auto differentiation to derive gradients. MXNet is computation and memory efficient and runs on various heterogeneous systems, ranging from mobile devices to distributed GPU clusters. This paper describes both the API design and the system implementation of MXNet, and explains how embedding of both symbolic expression and tensor operation is handled in a unified fashion. Our preliminary experiments reveal promising results on large scale deep neural network applications using multiple GPU machines.