Modular Blended Attention Network for Video Question Answering
This addresses the problem of complex network design in multimodal machine learning for researchers and practitioners, though it appears incremental as it builds on existing modular approaches.
The paper tackles the cumbersome and expensive problem of constructing ad hoc subnetworks for processing divergent modalities in multimodal machine learning tasks by introducing a reusable and composable neural unit that facilitates straightforward network construction and reduces space complexity through parameter sharing. The method achieves impressive performance compared to several video QA baselines on three commonly used datasets.
In multimodal machine learning tasks, it is due to the complexity of the assignments that the network structure, in most cases, is assembled in a sophisticated way. The holistic architecture can be separated into several logical parts according to the respective ends that the modules are devised to achieve. As the number of modalities of information representation increases, constructing ad hoc subnetworks for processing the data from divergent modalities while mediating the fusion of different information types has become a cumbersome and expensive problem. In this paper, we present an approach to facilitate the question with a reusable and composable neural unit; by connecting the units in series or parallel, the arduous network constructing of multimodal machine learning tasks will be accomplished in a much straightforward way. Additionally, through parameter sharing (weights replication) among the units, the space complexity will be significantly reduced. We have conducted experiments on three commonly used datasets; our method achieves impressive performance compared to several video QA baselines.