Out-of-Context Misinformation Detection via Variational Domain-Invariant Learning with Test-Time Training
This work addresses the challenge of domain adaptation for misinformation detection, which is incremental as it builds on existing methods by enhancing robustness to novel news domains.
The paper tackles the problem of detecting out-of-context misinformation in news reports by addressing poor performance when models encounter novel domains, proposing a method that learns domain-invariant features and uses test-time training to outperform state-of-the-art baselines on the NewsCLIPpings dataset under most domain adaptation settings.
Out-of-context misinformation (OOC) is a low-cost form of misinformation in news reports, which refers to place authentic images into out-of-context or fabricated image-text pairings. This problem has attracted significant attention from researchers in recent years. Current methods focus on assessing image-text consistency or generating explanations. However, these approaches assume that the training and test data are drawn from the same distribution. When encountering novel news domains, models tend to perform poorly due to the lack of prior knowledge. To address this challenge, we propose \textbf{VDT} to enhance the domain adaptation capability for OOC misinformation detection by learning domain-invariant features and test-time training mechanisms. Domain-Invariant Variational Align module is employed to jointly encodes source and target domain data to learn a separable distributional space domain-invariant features. For preserving semantic integrity, we utilize domain consistency constraint module to reconstruct the source and target domain latent distribution. During testing phase, we adopt the test-time training strategy and confidence-variance filtering module to dynamically updating the VAE encoder and classifier, facilitating the model's adaptation to the target domain distribution. Extensive experiments conducted on the benchmark dataset NewsCLIPpings demonstrate that our method outperforms state-of-the-art baselines under most domain adaptation settings.