Adaptative Inference Cost With Convolutional Neural Mixture Models
This addresses the issue of limited computational resources in vision tasks by providing an adaptive inference method, though it is incremental as it builds on existing CNN frameworks.
The paper tackles the problem of high computational cost during inference in convolutional neural networks (CNNs) by proposing Convolutional Neural Mixture Models (CNMMs), which embed multiple CNNs and allow pruning to adapt inference cost, achieving excellent accuracy-compute trade-offs in image classification and semantic segmentation without re-training.
Despite the outstanding performance of convolutional neural networks (CNNs) for many vision tasks, the required computational cost during inference is problematic when resources are limited. In this context, we propose Convolutional Neural Mixture Models (CNMMs), a probabilistic model embedding a large number of CNNs that can be jointly trained and evaluated in an efficient manner. Within the proposed framework, we present different mechanisms to prune subsets of CNNs from the mixture, allowing to easily adapt the computational cost required for inference. Image classification and semantic segmentation experiments show that our method achieve excellent accuracy-compute trade-offs. Moreover, unlike most of previous approaches, a single CNMM provides a large range of operating points along this trade-off, without any re-training.