Benchmark Assessment for DeepSpeed Optimization Library
This work provides a benchmark assessment for practitioners using DeepSpeed, but it is incremental as it extends existing evaluations to new architectures.
The paper evaluated the Microsoft DeepSpeed optimization library on modern neural network architectures like CNNs and Vision Transformers, finding that it improves performance in some cases but has no or negative impact in others.
Deep Learning (DL) models are widely used in machine learning due to their performance and ability to deal with large datasets while producing high accuracy and performance metrics. The size of such datasets and the complexity of DL models cause such models to be complex, consuming large amount of resources and time to train. Many recent libraries and applications are introduced to deal with DL complexity and efficiency issues. In this paper, we evaluated one example, Microsoft DeepSpeed library through classification tasks. DeepSpeed public sources reported classification performance metrics on the LeNet architecture. We extended this through evaluating the library on several modern neural network architectures, including convolutional neural networks (CNNs) and Vision Transformer (ViT). Results indicated that DeepSpeed, while can make improvements in some of those cases, it has no or negative impact on others.