Context-Aware Multipath Networks
This addresses the need for cost-effective generalized intelligence in machine learning by enabling a single model to handle variations within and across datasets, though it appears incremental as it builds on multi-path network concepts.
The paper tackles the problem of making a single network effectively address diverse contexts by introducing Context-Aware Multipath Network (CAMNet), which uses data-dependent routing to allocate resources and regulate information flow, resulting in performance surpassing equivalent single-path, multi-path, and deeper networks in classification and pixel-labeling tasks across individual, sequential, and combined datasets.
Making a single network effectively address diverse contexts---learning the variations within a dataset or multiple datasets---is an intriguing step towards achieving generalized intelligence. Existing approaches of deepening, widening, and assembling networks are not cost effective in general. In view of this, networks which can allocate resources according to the context of the input and regulate flow of information across the network are effective. In this paper, we present Context-Aware Multipath Network (CAMNet), a multi-path neural network with data-dependant routing between parallel tensors. We show that our model performs as a generalized model capturing variations in individual datasets and multiple different datasets, both simultaneously and sequentially. CAMNet surpasses the performance of classification and pixel-labeling tasks in comparison with the equivalent single-path, multi-path, and deeper single-path networks, considering datasets individually, sequentially, and in combination. The data-dependent routing between tensors in CAMNet enables the model to control the flow of information end-to-end, deciding which resources to be common or domain-specific.