The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design
It addresses the need for new computational devices to support deep learning, targeting researchers and engineers in computer architecture and chip design, but is incremental as it builds on existing trends.
This paper discusses the implications of deep learning advances on computer architecture and chip design, particularly in the post-Moore's Law era, and explores how machine learning can aid circuit design while proposing directions for larger-scale, sparsely activated multi-task models.
The past decade has seen a remarkable series of advances in machine learning, and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas, including computer vision, speech recognition, language translation, and natural language understanding tasks. This paper is a companion paper to a keynote talk at the 2020 International Solid-State Circuits Conference (ISSCC) discussing some of the advances in machine learning, and their implications on the kinds of computational devices we need to build, especially in the post-Moore's Law-era. It also discusses some of the ways that machine learning may also be able to help with some aspects of the circuit design process. Finally, it provides a sketch of at least one interesting direction towards much larger-scale multi-task models that are sparsely activated and employ much more dynamic, example- and task-based routing than the machine learning models of today.