MEENAKSHI SUNDARARAJAN ENGINEERING COLLEGE

Fake News Detection Using Deep Learning Models

Team No: 17

INTERNAL GUIDE: MRS.C. JERIN MAHIBA

Kunguma Akshatra M 311520104026

Madhumitha M 311520104026

Tejashri J 311520104026
 OVERVIEW
1. Introduction 8. Module Description
2. Abstract 9. System Requirements
3. Problem Statement 10. Use cases
4. Literature Survey 11. Implementation
5. Existing System 12. Conclusion
6. Proposed System 13. Future Enhancements
7. System Architecture 14. References
Abstract
❖ Transformer Models: Utilizing mBERT, ALBERT, and XML RoBERTa for Malayalam fake news
detection.

❖ Performance: mBERT and XML RoBERTa achieved outstanding macro F1 scores of 0.84 in binary
classification, while ALBERT showed robust capability with a score of 0.56.

❖ Deeper Analysis: XML RoBERTa led Task 2 with a macro F1 score of 0.21, followed by mBERT
and BERT with scores of 0.16 and 0.11.

❖ Significance: Research highlights the importance of developing robust systems to distinguish

authentic information from deceptive content in social media.

❖ Implications: Findings offer insights for constructing tools to safeguard information credibility,
crucial for combating the dissemination of fake news.
Introduction
❖ Detection methods scrutinize social media text meticulously,
aiming to distinguish between original and misleading content,
combating the spread of fake news.

❖ The task focuses on developing models to classify fake news,

particularly in Malayalam-language articles, aiming to address the
challenge of misleading information dissemination.

❖ Enhancing NLP promotes informed, reliable digital space, upholds

integrity, and encourages responsible communication.
Problem statement
❖ Existing fake news detection methods face challenges in early
detection and limited labeled data.

❖ A novel framework utilizing advanced deep learning, like the

Transformer model, is crucial.

❖ It aims to integrate temporal dynamics and contextual signals,

enhancing accuracy and scalability, and enabling early detection to
combat misinformation spread effectively.
Literature survey
TITLE
Fake news detection based on news
content and social contexts: a
transformer-based approach by
Shaina Raza and Chen Ding.
Covid-19 fake news detection using
bidirectional encoder
representations from transformers
based models by Yuxiang Wang,
Yongheng Zhang, Xuebo Li, and
Xinyao Yu.
Automatic fake news detection
model based on bidirectional
encoder representations from
transformers (bert) by Heejung
Jwa, Dongsuk Oh, Kinam Park, Jang
Mook Kang, and Heuiseok Lim.
Year of Publication Salient Features
2022 Transformer-based approach enables
efficient processing, focusing on
early detection by integrating news
content and social contexts.
Encoder-decoder architecture and
effective labeling enhance model
performance for fake news
detection.
2021 Utilizes BERT for fake news
detection, incorporates BiLSTM and
CNN layers, achieves state-of-the-art
performance, explores keyword
evaluation, and offers adaptable
parameter freezing methods for
optimization.
2019 Leveraging BERT, the model detects
fake news automatically, prioritizing
data-driven approaches, pre-training
with extra data, and surpassing
previous model performance.
Limitations
Limitations: Real-world data quality
affects performance, scalability
concerns for large datasets, limited
generalization, interpretability
challenges, resource-intensive
training.
Limitations include increased model
complexity, varying performance
across fake news types, reliance on
keywords, interpretability
challenges, and dependence on
training data quality and quantity for
generalizability.
Challenges include interpreting
results, demanding training process,
dependence on data quality, and
potential limitations in contextual
understanding.
Literature survey
TITLE
Detection of Fake News using
Transformer Model iCoMET by
Momina Qazi, Muhammad US
Khan, and Mazhar Ali.
BERT: Pre-training of Deep
Bidirectional Transformers for
Language Understanding. CoRR
by Jacob Devlin, Ming-Wei Chang,
Kenton Lee, and Kristina Toutanova.
Fake News Detection based on
News Content and Social Contexts:
a Transformer-based approach by
Shaina Raza and Chen Ding.
Year of Publication Salient Features
2020 Utilizes attention-based transformer
model for fake news detection on
publicly available dataset. Aims to
compare with state-of-the-art
algorithms, showing 15% accuracy
improvement over Hybrid CNN.
2018 Introduces BERT, a Bidirectional
Encoder Representations from
Transformers, pre-trained on
unlabeled text with deep
bidirectional context conditioning in
all layers. Achieves state-of-the-art
results on eleven NLP tasks.
2022 Novel fake news detection
framework, Transformer
architecture, multiple features
integration, effective labeling, and
higher accuracy in early detection.
Limitations
Potential reliance on dataset quality,
scope limitations in contextual
understanding, and interpretability
challenges with transformer model
predictions.
May require substantial
computational resources for
pre-training and fine-tuning.
Evaluation primarily focuses on task
performance without thorough
analysis of failure cases.
Generalization challenges, limited
interpretability, resource intensity,
labeling technique effectiveness
variation, and potential validation
necessity across diverse datasets.
Existing System
❖ Current fake news detection systems employ preprocessing and feature engineering but struggle with
nuanced characteristics and evolving tactics due to imbalanced datasets and temporal dynamics.
❖ Advanced frameworks integrating deep learning architectures are vital for enhanced accuracy, scalability,
and early detection capabilities.
❖ These solutions must adeptly utilize contextual cues and temporal dynamics to combat misinformation
effectively in the digital landscape.
❖ Addressing the pressing need for sophisticated detection methods, these frameworks offer promising
avenues for mitigating the impact of fake news and safeguarding information integrity in the evolving
online environment.
Proposed System
❖ Incorporation of Advanced Transformer Models:- Implement state-of-the-art transformer models like
mBERT, ALBERT, and XMLRoBERTa to enhance the system's language coverage and understanding,
particularly focusing on Dravidian languages like Malayalam.

❖ Context-Aware Approaches:- Develop algorithms that go beyond keyword matching, incorporating

context-aware techniques to capture the subtle nuances and variations in the presentation of misinformation on
social media.

❖ Dynamic Feature Adaptation:- Introduce dynamic feature sets that adapt to evolving misinformation
patterns, ensuring the system remains effective in identifying and categorizing fake news in the rapidly
changing landscape of social media.

❖ Scalable Architecture:- Design a scalable and efficient architecture capable of handling large volumes of
social media data, ensuring real-time or near-real-time processing for timely and effective fake news detection.

❖ Utilization of Transfer Learning: - Implement transfer learning techniques to leverage pre-trained models
and enhance the system's contextual understanding, enabling more accurate classification of fake news by
considering the broader linguistic and cultural context.
Project Architecture
System Architecture
Module Description

Data Gathering and Model Training with

Reporting and Visualization
Collection Transformer Architectures

Data Preprocessing Model Testing and Validation

Hardware requirements
S.No REQUIREMENTS RECOMMENDED MINIMUM REQUIREMENTS

1 Operating System Windows 10 Windows 7

2 RAM 4 GB 4 GB

3 GPU NVIDIA Geforce RTX Intel(R) UHD Graphics 630

4 Processor Intel Core i7 Intel Core i5

5 SDD 1 TB 500 GB

6 NIC Cards Gigabit Ethernet Gigabit Ethernet

Software requirements
REQUIREMENTS SPECIFICATIONS

Operating System Windows 10

Deep Learning Frameworks Hugging Face Transformers

NLP Libraries NLTK, Transformers

Development Environment Google Colab, Jupyter Notebook

UML Diagrams
Use case diagram
UML Diagrams
Class diagram
UML Diagrams
Sequence diagram
UML Diagrams
Collaboration diagram
UML Diagrams
Activity diagram
UML Diagrams
State diagram
UML Diagrams
Component diagram
UML Diagrams
Package diagram
UML Diagrams
Deployment diagram
Implementation
Implementation
Implementation
Implementation
Conclusion
❖ DravidianLangTech@EACL 2024 initiative employs transformer
models to detect fake news in Malayalam social media.

❖ mBERT and XMLRoBERTa excel in binary classification, while

XMLRoBERTa demonstrates superior performance in
categorizing types of misinformation.
❖ The study underscores the importance of accurate fake news
detection in combating misinformation on social media.
❖ It sets a standard for future research and promotes the
development of trustworthy tools to counter disinformation in
the digital era.
Future Scope
Data Balance and Augmentation
● Address data imbalance
with sophisticated methods
like oversampling or
synthetic data synthesis to
improve classification
accuracy and reduce
biases.
Multimodal Analysis Expansion
● Enhance detection performance
by integrating non-textual
variables such as audio and visual
data, providing a more
comprehensive context for fake
news identification.
Cross-lingual Generalization
and Efficiency
● Ensure consistent performance
across languages, optimize
transformer topologies for
scalability and efficiency, and
enhance interpretability
through Explainable AI
techniques.
References
1. Maryam Heidari, Samira Zad, Parisa Hajibabaee, Masoud Malekzadeh, SeyyedPooya HekmatiAthar, Ozlem Uzuner, and
James H Jones. 2021. BERT Model for Fake News Detection Based on Social Bot Activities in the COVID-19 Pandemic. In
2021 IEEE 12th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), pages
0103–0109. IEEE.

2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional
Transformers for Language Understanding. CoRR, abs/1810.04805.

3. Quanliang Jing, Di Yao, Xinxin Fan, Baoli Wang, Haining Tan, Xiangpeng Bu, and Jingping Bi. 2021. TRANSFAKE:
Multi-Task Transformer for Multimodal Enhanced Fake News Detection. In 2021 International Joint Conference on Neural
Networks (IJCNN), pages 1–8. IEEE.

4. Heejung Jwa, Dongsuk Oh, Kinam Park, Jang Mook Kang, and Heuiseok Lim. 2019. exBAKE: Automatic Fake News
Detection model based on Bidirectional Encoder Representations from Transformers (BERT). Applied Sciences, 9(19):4062.

5. Sebastian Kula, Rafał Kozik, Michał Choras, and ´ Michał Wo´zniak. 2021. Transformer Based Models in Fake News
Detection. In the International Conference on Computational Science, pages 28–38. Springer.

6. Ashok Kumar, Tina Esther Trueman, and Erik Cambria. 2021. Fake news detection using XLNet fine-tuning model. In 2021
International Conference on Computational Intelligence and Computing Applications (ICCICA), pages 1–4. IEEE.
Publication
❖ TechWhiz@DravidianLangTech 2024: Fake News Detection Using Deep Learning Models, Madhumitha M,
Kunguma Akshatra M, Tejashri, Jerin Mahibha C published in Proceedings of the Fourth Workshop on
Speech, Vision, and Language Technologies for Dravidian Languages, Association for Computational
Linguistics.
Thank-you