FastFusionNet: New State-of-the-Art for DAWNBench SQuAD
This work addresses computational bottlenecks for researchers and practitioners using reading comprehension models, but it is incremental as it builds on existing FusionNet architecture.
The authors tackled the problem of improving computational efficiency in reading comprehension models by introducing FastFusionNet, an efficient variant of FusionNet, which achieved state-of-the-art results on DAWNBench and the lowest training and inference time on SQuAD to-date.
In this technical report, we introduce FastFusionNet, an efficient variant of FusionNet [12]. FusionNet is a high performing reading comprehension architecture, which was designed primarily for maximum retrieval accuracy with less regard towards computational requirements. For FastFusionNets we remove the expensive CoVe layers [21] and substitute the BiLSTMs with far more efficient SRU layers [19]. The resulting architecture obtains state-of-the-art results on DAWNBench [5] while achieving the lowest training and inference time on SQuAD [25] to-date. The code is available at https://github.com/felixgwu/FastFusionNet.