An image and text-based fake news detection with transfer learning.
| Title: | An image and text-based fake news detection with transfer learning. |
|---|---|
| Authors: | Setiawan EI; Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia.; Sutanto P; Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia.; Purwanto CN; Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia.; Santoso J; Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia.; Ferdinandus FX; Information Technology Department, Institut Sains dan Teknologi Terpadu Surabaya, Surabaya, Jawa Timur, Indonesia.; Pah ND; Electrical Engineering Department, University of Surabaya, Surabaya, Jawa Timur, Indonesia.; School of Engineering, RMIT University, Melbourne, Victoria, Australia.; Purnomo MH; Electrical Engineering Department, Institut Teknologi Sepuluh Nopember, Surabaya, Jawa Timur, Indonesia. |
| Source: | PloS one [PLoS One] 2025 Jun 17; Vol. 20 (6), pp. e0324394. Date of Electronic Publication: 2025 Jun 17 (Print Publication: 2025). |
| Publication Type: | Journal Article |
| Language: | English |
| Journal Info: | Publisher: Public Library of Science Country of Publication: United States NLM ID: 101285081 Publication Model: eCollection Cited Medium: Internet ISSN: 1932-6203 (Electronic) Linking ISSN: 19326203 NLM ISO Abbreviation: PLoS One Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: San Francisco, CA : Public Library of Science |
| MeSH Terms: | Image Processing, Computer-Assisted*/methods ; Deception*; Humans ; Deep Learning ; Algorithms |
| Abstract: | Fake news has emerged as a significant problem in today's information age, threatening the reliability of information sources. Detecting fake news is crucial for maintaining trust and ensuring access to factual information. While deep learning offers solutions, most approaches focus on the text, neglecting the potential of visual information, which may contradict or misrepresent the accompanying text. This research proposes a multimodal classification approach that combines text and images to improve fake news detection, particularly in low-resource settings where labeled data is scarce. We leverage CLIP, a model that understands relationships between images and text, to extract features from both modalities. These features are concatenated and fed into a simple one-layer multi-layer perceptron (MLP) for classification. To enhance data efficiency, we apply LoRA (Low-Rank Adaptation), a parameter-efficient fine-tuning technique, to the CLIP model. We also explore the effects of integrating features from other models. The model achieves an 83% accuracy when using LoRA in classifying whether an image and its accompanying text constitute fake or factual news. These results highlight the potential of multimodal learning and efficient fine-tuning techniques for robust fake news detection, even with limited data.; (Copyright: © 2025 Setiawan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.) |
| Competing Interests: | The authors have declared that no competing interests exist. |
| References: | Multimed Tools Appl. 2021;80(8):11765-11788. (PMID: 33432264); Sci Rep. 2021 Dec 8;11(1):23705. (PMID: 34880354) |
| Entry Date(s): | Date Created: 20250617 Date Completed: 20250617 Latest Revision: 20250620 |
| Update Code: | 20260130 |
| PubMed Central ID: | PMC12173408 |
| DOI: | 10.1371/journal.pone.0324394 |
| PMID: | 40526614 |
| Database: | MEDLINE |
Journal Article