QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration
This work addresses the practical deployment challenges of QDNNs for researchers and practitioners in machine learning, though it appears incremental as it builds on existing QDNN concepts.
The authors tackled the problem of quadratic deep neural networks (QDNNs) having drawbacks in neuron design by proposing a new QDNN neuron architecture and developing QuadraLib, a library for architecture optimization and design exploration, resulting in good performance in prediction accuracy and computation consumption across multiple learning tasks.
The significant success of Deep Neural Networks (DNNs) is highly promoted by the multiple sophisticated DNN libraries. On the contrary, although some work have proved that Quadratic Deep Neuron Networks (QDNNs) show better non-linearity and learning capability than the first-order DNNs, their neuron design suffers certain drawbacks from theoretical performance to practical deployment. In this paper, we first proposed a new QDNN neuron architecture design, and further developed QuadraLib, a QDNN library to provide architecture optimization and design exploration for QDNNs. Extensive experiments show that our design has good performance regarding prediction accuracy and computation consumption on multiple learning tasks.