AutoML for Multilayer Perceptron and FPGA Co-design
This work addresses the need for automated hardware-aware design of MLPs, which are crucial for data center workloads, though it is incremental as it extends existing AutoML methods to a specific network type.
The authors tackled the challenge of automatically designing and implementing efficient neural network architectures for general-purpose deep neural networks, which are underrepresented in current AutoML research focused on convolutional networks. They developed a co-design flow for multilayer perceptrons (MLPs) that achieved higher accuracy than published MLP results and demonstrated competitive efficiency and throughput on FPGAs compared to GPUs across six benchmarks.
State-of-the-art Neural Network Architectures (NNAs) are challenging to design and implement efficiently in hardware. In the past couple of years, this has led to an explosion in research and development of automatic Neural Architecture Search (NAS) tools. AutomML tools are now used to achieve state of the art NNA designs and attempt to optimize for hardware usage and design. Much of the recent research in the auto-design of NNAs has focused on convolution networks and image recognition, ignoring the fact that a significant part of the workload in data centers is general-purpose deep neural networks. In this work, we develop and test a general multilayer perceptron (MLP) flow that can take arbitrary datasets as input and automatically produce optimized NNAs and hardware designs. We test the flow on six benchmarks. Our results show we exceed the performance of currently published MLP accuracy results and are competitive with non-MLP based results. We compare general and common GPU architectures with our scalable FPGA design and show we can achieve higher efficiency and higher throughput (outputs per second) for the majority of datasets. Further insights into the design space for both accurate networks and high performing hardware shows the power of co-design by correlating accuracy versus throughput, network size versus accuracy, and scaling to high-performance devices.