Benchmark Analysis of Representative Deep Neural Network Architectures
This work provides a comparative analysis for researchers and practitioners to select appropriate DNN architectures based on resource constraints, but it is incremental as it benchmarks existing methods without introducing new techniques.
This paper conducted a comprehensive benchmark analysis of deep neural network architectures for image recognition, evaluating performance indices like accuracy, complexity, and inference time across two computer architectures, and found that the study aids in comparing DNNs under different computational constraints.
This work presents an in-depth analysis of the majority of the deep neural networks (DNNs) proposed in the state of the art for image recognition. For each DNN multiple performance indices are observed, such as recognition accuracy, model complexity, computational complexity, memory usage, and inference time. The behavior of such performance indices and some combinations of them are analyzed and discussed. To measure the indices we experiment the use of DNNs on two different computer architectures, a workstation equipped with a NVIDIA Titan X Pascal and an embedded system based on a NVIDIA Jetson TX1 board. This experimentation allows a direct comparison between DNNs running on machines with very different computational capacity. This study is useful for researchers to have a complete view of what solutions have been explored so far and in which research directions are worth exploring in the future; and for practitioners to select the DNN architecture(s) that better fit the resource constraints of practical deployments and applications. To complete this work, all the DNNs, as well as the software used for the analysis, are available online.