Virtual University Journals
Search
Search
Cross-Modal Inconsistency Detection with Pooled Fusion: A Controlled Study for Multimodal Fake News
Waqas Gulzar Gondal, Sajid Ullah Khan, Atlas Gondal
Abstract:
Fake news often pairs misleading text with authentic images, so cross-modal consistency can be a useful signal for multimodal detection. In this paper, we test whether a lightweight pooled attention fusion layer—operating on pooled robustly optimised bidirectional encoder representations from transformers model and vision transformer model embeddings—provides consistent improvements over strong fusion baselines on binary Fakeddit classification. Using stratified subsets from the official splits (training set/validation set/testing set = 50,000/10,000/10,000) and three random seeds (mean ± standard deviation), multimodal fusion improves over unimodal models, but pooled attention does not consistently outperform late fusion or gated fusion. On the primary setting, late fusion achieves the best mean accuracy (88.78%), gated fusion the best mean area under the receiver operating characteristic curve (0.9516), and pooled attention remains competitive (88.50% accuracy; 0.9460 area under the receiver operating characteristic curve). Overall, pooled attention is a useful coarse interaction term, but it is not sufficient evidence of fine-grained inconsistency reasoning.
Keywords:
Keywords: Multimodal Fake News Detection, Cross-Modal Consistency, Multimodal Fusion, Vision Transformer (ViT), RoBERTa, Misinformation Detection
Full Text:
PDF