NetBooster: Empowering Tiny Deep Learning By Standing on the Shoulders of Deep Giants
It addresses the challenge of deploying deep learning on intelligent IoT devices, which is an incremental improvement in domain-specific applications.
The paper tackles the under-fitting problem in tiny deep learning on large-scale datasets and downstream tasks by proposing NetBooster, a framework that augments tiny neural network architectures using an expansion-then-contraction strategy, resulting in consistent outperformance over state-of-the-art solutions.
Tiny deep learning has attracted increasing attention driven by the substantial demand for deploying deep learning on numerous intelligent Internet-of-Things devices. However, it is still challenging to unleash tiny deep learning's full potential on both large-scale datasets and downstream tasks due to the under-fitting issues caused by the limited model capacity of tiny neural networks (TNNs). To this end, we propose a framework called NetBooster to empower tiny deep learning by augmenting the architectures of TNNs via an expansion-then-contraction strategy. Extensive experiments show that NetBooster consistently outperforms state-of-the-art tiny deep learning solutions.