MTPareto: A MultiModal Targeted Pareto Framework for Fake News Detection
This work addresses fake news detection for internet multimedia, offering an incremental improvement in multimodal fusion methods.
The paper tackles the problem of multimodal fake news detection by proposing the MTPareto framework, which uses a Targeted Pareto optimization algorithm to optimize fusion across different levels, resulting in accuracy improvements of 2.40% and 1.89% on two datasets compared to baselines.
Multimodal fake news detection is essential for maintaining the authenticity of Internet multimedia information. Significant differences in form and content of multimodal information lead to intensified optimization conflicts, hindering effective model training as well as reducing the effectiveness of existing fusion methods for bimodal. To address this problem, we propose the MTPareto framework to optimize multimodal fusion, using a Targeted Pareto(TPareto) optimization algorithm for fusion-level-specific objective learning with a certain focus. Based on the designed hierarchical fusion network, the algorithm defines three fusion levels with corresponding losses and implements all-modal-oriented Pareto gradient integration for each. This approach accomplishes superior multimodal fusion by utilizing the information obtained from intermediate fusion to provide positive effects to the entire process. Experiment results on FakeSV and FVC datasets show that the proposed framework outperforms baselines and the TPareto optimization algorithm achieves 2.40% and 1.89% accuracy improvement respectively.