DAISY LabSharif University of Technology
Publications

FNR: A Similarity and Transformer-Based Approach to Detect Multi-Modal Fake News in Social Media

Faeze Ghorbanpour, Maryam Ramezani, Mohammad Amin Fazli, Hamid R. Rabiee

Social Network Analysis and Mining, 28 March 2023

DOIarXiv

Abstract

The availability and interactive nature of social media have made them the primary source of news around the globe. The popularity of social media tempts criminals to pursue their immoral intentions by producing and disseminating fake news using seductive text and misleading images. This work analyses multi-modal features from texts and images in social media for detecting fake news. We propose a Fake News Revealer (FNR) method that uses transfer learning to extract contextual and semantic features and contrastive loss to determine the similarity between image and text. Applied to two real social media datasets, FNR achieves higher accuracy in detecting fake news compared to previous work.

In short

A fake news post is rarely fake in one modality alone. FNR looks at the relationship between what a post says and what its image shows: transformer encoders extract contextual and semantic features from each modality, and a contrastive objective measures how well the two agree.

That agreement signal is what separates FNR from classifiers that treat text and image as two independent feature bags.

Cite this paper

@article{ghorbanpour2023similarity,
  title = {{FNR: A Similarity and Transformer-Based Approach to Detect Multi-Modal Fake News in Social Media}},
  author = {Faeze Ghorbanpour and Maryam Ramezani and Mohammad Amin Fazli and Hamid R. Rabiee},
  journal = {Social Network Analysis and Mining},
  year = {2023},
  doi = {10.1007/s13278-023-01065-0},
  eprint = {2112.01131},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2112.01131}
}

Related work