Table-based Fact Verification with Self-adaptive Mixture of Experts
This work addresses the challenging problem of verifying facts against tables for applications in data validation and AI systems, presenting an incremental improvement over existing methods.
The paper tackles table-based fact verification by proposing a Self-adaptive Mixture-of-Experts Network (SaMoE) to handle diverse reasoning types like numerical and logical operations, achieving 85.1% accuracy on the TabFact benchmark, comparable to previous state-of-the-art models.
The table-based fact verification task has recently gained widespread attention and yet remains to be a very challenging problem. It inherently requires informative reasoning over natural language together with different numerical and logical reasoning on tables (e.g., count, superlative, comparative). Considering that, we exploit mixture-of-experts and present in this paper a new method: Self-adaptive Mixture-of-Experts Network (SaMoE). Specifically, we have developed a mixture-of-experts neural network to recognize and execute different types of reasoning -- the network is composed of multiple experts, each handling a specific part of the semantics for reasoning, whereas a management module is applied to decide the contribution of each expert network to the verification result. A self-adaptive method is developed to teach the management module combining results of different experts more efficiently without external knowledge. The experimental results illustrate that our framework achieves 85.1% accuracy on the benchmark dataset TabFact, comparable with the previous state-of-the-art models. We hope our framework can serve as a new baseline for table-based verification. Our code is available at https://github.com/THUMLP/SaMoE.