Hybrid Deep Learning-Based System for Real-Time

Misinformation and Crime Detection on Social Networks

MR.L.Sankar Rao Assistant Professor
A.Divyesh Phani Giridhar, CH. Durga Prasad, J. Uma Surya Teja, T.
Dileep Sai
Raghu Engineering College, Department of Computer Science with
Artificial Intelligence and Machine Learning, Visakhapatnam,

ABSTRACT
The rapid growth of social networks has led to an unprecedented dissemination of both useful information and
harmful content, such as misinformation and criminal activities. Addressing these challenges in real time
requires advanced technological solutions. This project presents a Hybrid Deep Learning-Based System
designed to detect misinformation and identify crime detection on social networks in real time.
The system integrates multiple deep learning models to analyze diverse content types, such as text and images,
making it versatile in its detection capabilities. For misinformation detection, the system utilizes transformer-
based models like BERT and LSTM to classify text as real or fake news, capturing both contextual and
sequential patterns in the data. For crime detection, a pre-trained Convolutional Neural Network (CNN) like
YOLO (You Only Look Once) is employed to analyze visual content and identify harmful behavior in images.
Data preprocessing techniques, including tokenization, stop-word removal, and lemmatization, are applied to
text data, while image data undergoes resizing and normalization. The system leverages a hybrid architecture,
combining the outputs of BERT, LSTM, and CNN models using a fully connected neural network, resulting in
an ensemble capable of capturing nuanced patterns of misinformation and crime detection across different
media types.
To enable real-time detection, the system is designed for scalability and efficiency using streaming data analysis
and parallel processing frameworks. Evaluation metrics such as accuracy, precision, recall, and F1-score are
used to assess the system's performance.
Potential applications of this system include enhancing social media platforms' ability to flag harmful content
proactively, aiding law enforcement in crime detection, and providing media outlets with tools to counter
disinformation during critical events. The project also addresses challenges related to data privacy, scalability,
and adversarial attacks.
Keywords: Deep learning, BERT, LSTM, CNN, YOLO, real-time detection, misinformation, social networks,
crime detection, data privacy

INTRODUCTION
Social networks have become vital channels for information sharing, but their rapid growth has also accelerated
the spread of misinformation and criminal activities online. Effectively addressing these issues requires
sophisticated systems capable of real-time detection and mitigation. Hybrid deep learning-based systems have
emerged as a robust solution, harnessing diverse neural network architectures to enhance detection accuracy and
operational efficiency.

METHODOLOGY
This research proposes a Hybrid Deep Learning-Based System for real-time social network detection of
misinformation and crime. The system leverages advanced neural network architectures to analyze diverse
content types, ensuring high accuracy and efficiency in identifying harmful content. The methodology involves
two primary components: text-based analysis using transformer and sequential models and image-based analysis
using convolutional networks.
BERT/LSTM
For misinformation detection, this system combines Bidirectional Encoder Representations from Transformers
(BERT) and Long Short-Term Memory (LSTM) networks. BERT captures contextual relationships between
words in a text, while LSTM identifies sequential dependencies, enabling the system to classify text as real or
fake news with enhanced accuracy. Preprocessing steps include tokenization, stop-word removal, and
lemmatization to prepare the text data for analysis.
CNN YOLO
The system employs a pre-trained Convolutional Neural Network (CNN) model for crime detection, specifically
YOLO (You Only Look Once). YOLO is renowned for its speed and accuracy in object detection tasks, making
it suitable for the real-time identification of harmful behavior in images. Image data undergoes resizing and
normalization as part of the preprocessing pipeline to ensure consistency and model efficiency.

MODELING AND ANALYSIS

The hybrid deep learning system leverages multiple neural network architectures, including Convolutional
Neural Networks (CNN) with the YOLO (You Only Look Once) algorithm for crime detection and
BERT/LSTM models for misinformation detection. CNN YOLO provides fast and accurate object detection
capabilities, making it well-suited for identifying visual indicators of criminal activity in social media content.
BERT (Bidirectional Encoder Representations from Transformers) combined with LSTM (Long Short-Term
Memory) captures contextual understanding and sequential dependencies, improving the accuracy of
misinformation detection.
BERT’s bidirectional nature allows it to understand the context of words in a sentence more effectively, while
LSTM handles sequential dependencies over long text spans. This combination ensures robust performance in
detecting misinformation by capturing nuanced meaning and temporal relationships in social media content.
Together, they enhance the system's ability to distinguish between verified and unverified information.

Figure 1: BERT Architecture

CNN YOLO is renowned for its speed and precision in real-time object detection. It processes entire images in a
single forward pass, enabling the system to identify potential criminal activities swiftly and efficiently. This
architecture balances accuracy and performance, making it ideal for analyzing visual data on social networks
where immediate response is crucial.
 Figure 2: CNN YOLO Architecture

RESULTS AND DISCUSSION

The Hybrid Deep Learning-Based System for Real-Time Misinformation and Crime Detection on Social
Networks demonstrates significant effectiveness and efficiency in detecting harmful content across text and
image data. The system’s integration of multiple deep learning models enhances its versatility and accuracy.
BERT/LSTM
The evaluation of the proposed BERT/LSTM model's performance was conducted by analyzing key metrics
such as accuracy, precision, and recall. The presented confusion matrix offers insights into the model's
classification capabilities, illustrating the distribution of true positive, true negative, false positive, and false
negative predictions. The substantial number of correctly predicted instances in both classes indicates the
model’s strong discriminative power. Misclassification rates remain minimal, further reinforcing the model's
effectiveness in distinguishing between positive and negative samples.

Figure 3: Confusion Matrix for BERT/LSTM

The training and testing loss curves provide an overview of the model's convergence behavior. The rapid
decrease in training loss during initial epochs signifies efficient learning, while the test loss stabilizes at a
relatively low value, confirming minimal overfitting. This convergence pattern demonstrates that the model
generalizes well to unseen data.

Figure 3: Model Loss

Similarly, the accuracy curve (Figure 3) reflects the model's performance throughout training. The sharp rise in
training accuracy in the early epochs, followed by stable behavior, indicates effective learning. The marginal
difference between training and test accuracy highlights the model's capacity to generalize without significant
overfitting.

Figure 3: Model Accuracy

CNN YOLO
The evaluation of the proposed CNN YOLO model's performance was conducted using multiple metrics, such
as precision, recall, and accuracy, to comprehensively assess its effectiveness in object detection and
classification tasks. The confusion matrix provides insights into the model's classification performance by
illustrating the distribution of true positive, true negative, false positive, and false negative predictions. This
matrix highlights the model's capability to accurately classify both positive and negative classes while also
identifying potential misclassification patterns.

Figure 3: Confusion Matrix for CNN YOLO

Additionally, the precision-recall curve was examined to analyze the trade-off between precision and recall. A
larger area under this curve signifies improved performance, particularly in scenarios with imbalanced data
distributions. Precision reflects the proportion of accurately identified positive cases among all predicted
positive cases, while recall measures the proportion of correctly identified positive instances relative to the total
number of actual positives.

Figure 4: Precision-Recall Curve

The model's performance consistency during training is further demonstrated through the results in the
recall confidence curve. The batch performance visualizationreveals stable learning patterns, affirming
the model's ability to converge effectively. The steady decline in the loss function alongside consistent
accuracy throughout training supports the robustness of the proposed methodology.

Figure 5: Recall-Confidence Curve

Figure 6: Batch Performance

CONCLUSION
The Hybrid Deep Learning-Based System proves to be a powerful and efficient solution for real-time detection
of misinformation and criminal activities on social networks. By combining advanced models like BERT/LSTM
and CNN YOLO, the system achieves high accuracy and responsiveness across both text and image data. Its
hybrid architecture enables comprehensive content analysis, making it well-suited for identifying harmful
content swiftly and effectively. Future improvements could focus on enhancing model robustness against
adversarial attacks and expanding the system’s capabilities to handle more diverse media types. Overall, this
system has strong potential to support digital safety initiatives and contribute to the fight against online
misinformation and crime.

REFERENCES
1. Hu L, Wei S, Zhao Z, Wu B. Deep learning for fake news detection: A comprehensive survey. AI Open.
2022;. doi: 10.1016/j.aiopen.2022.09.001 [DOI] [Google Scholar]
2. Greenwood S, Perrin A, Duggan M. Social media update 2016. Pew Research Center. 2016;11(2):1–18.
[Google Scholar]
3. Bzai J, Alam F, Dhafer A, Bojović M, Altowaijri SM, Niazi IK, et al. Machine Learning-Enabled Internet of
Things (IoT): Data, Applications, and Industry Perspective. Electronics. 2022;11(17):2676. doi:
10.3390/electronics11172676 [DOI] [Google Scholar]
4. Zhou X, Zafarani R. A survey of fake news: Fundamental theories, detection methods, and opportunities.
ACM Computing Surveys (CSUR). 2020;53(5):1–40. doi: 10.1145/3395046 [DOI] [Google Scholar]
5. Shu K, Sliva A, Wang S, Tang J, Liu H. Fake news detection on social media: A data mining perspective.
ACM SIGKDD explorations newsletter. 2017;19(1):22–36. doi: 10.1145/3137597.3137600 [DOI] [Google
Scholar]
6. Wang WY. “liar, liar pants on fire”: A new benchmark dataset for fake news detection. arXiv preprint
arXiv:170500648. 2017;.
7. Chauhan T, Palivela H. Optimization and improvement of fake news detection using deep learning
approaches for societal benefit. International Journal of Information Management Data Insights.
2021;1(2):100051. doi: 10.1016/j.jjimei.2021.100051 [DOI] [Google Scholar]
8. Monti F, Frasca F, Eynard D, Mannion D, Bronstein MM. Fake news detection on social media using
geometric deep learning. arXiv preprint arXiv:190206673. 2019;.
9. Nasir JA, Khan OS, Varlamis I. Fake news detection: A hybrid CNN-RNN based deep learning approach.
International Journal of Information Management Data Insights. 2021;1(1):100007. doi:
10.1016/j.jjimei.2020.100007 [DOI] [Google Scholar]
10.Oshikawa R, Qian J, Wang WY. A survey on natural language processing for fake news detection. arXiv
preprint arXiv:181100770. 2018;.
11. Keya AJ, Afridi S, Maria AS, Pinki SS, Ghosh J, Mridha M. Fake news detection based on deep learning. In:
2021 International Conference on Science & Contemporary Technologies (ICSCT). IEEE; 2021. p. 1–6.
12. Zhang Y, Wallace B. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for
sentence classification. arXiv preprint arXiv:151003820. 2015;.
13. Shu K, Mahudeswaran D, Wang S, Lee D, Liu H. Fakenewsnet: A data repository with news content, social
context, and spatiotemporal information for studying fake news on social media. Big data. 2020;8(3):171–188.
doi: 10.1089/big.2020.0062 [DOI] [PubMed] [Google Scholar]
14. Liu Y, Wu YF. Early detection of fake news on social media through propagation path classification with
recurrent and convolutional networks. In: Proceedings of the AAAI conference on artificial intelligence. vol. 32;
2018.
15. Hochreiter S, Schmidhuber J. Long short-term memory. Neural computation. 1997;9(8):1735–1780. doi:
10.1162/neco.1997.9.8.1735 [DOI] [PubMed] [Google Scholar]
16. Roy SS, Roy A, Samui P, Gandomi M, Gandomi AH. Hateful Sentiment Detection in Real-Time Tweets: An
LSTM-Based Comparative Approach. IEEE Transactions on Computational Social Systems. 2023;. doi:
10.1109/TCSS.2023.3260217 [DOI] [Google Scholar]
17. Roy SS, Awad AI, Amare LA, Erkihun MT, Anas M. Multimodel phishing URL detection using LSTM,
bidirectional LSTM, and GRU models. Future Internet. 2022;14(11):340. doi: 10.3390/fi14110340 [DOI]
[Google Scholar]
18. Umer M, Imtiaz Z, Ullah S, Mehmood A, Choi GS, On BW. Fake news stance detection using deep learning
architecture (CNN-LSTM). IEEE Access. 2020;8:156695–156706. doi: 10.1109/ACCESS.2020.3019735 [DOI]
[Google Scholar]
19. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE
conference on computer vision and pattern recognition; 2016. p. 770–778.
20.Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In:
Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 4700–4708.
21. Pennington J, Socher R, Manning CD. Glove: Global vectors for word representation. In: Proceedings of the
2014 conference on empirical methods in natural language processing (EMNLP); 2014. p. 1532–1543.
22. Devlin J, Chang MW, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for
language understanding. arXiv preprint arXiv:181004805. 2018;.
23. Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, et al. Deep contextualized word
representations. CoRR abs/1802.05365 (2018). arXiv preprint arXiv:180205365. 1802;.
24. Kaliyar RK, Goswami A, Narang P. FakeBERT: Fake news detection in social media with a BERT-based
deep learning approach. Multimedia tools and applications. 2021;80(8):11765–11788. doi: 10.1007/s11042-020-
10183-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Fawaid J, Awalina A, Krisnabayu RY, Yudistira N. Indonesia’s Fake News Detection using Transformer
Network. In: 6th International Conference on Sustainable Information Engineering and Technology 2021; 2021.
p. 247–251.
26. Keya AJ, Wadud MAH, Mridha M, Alatiyyah M, Hamid MA. AugFake-BERT: Handling Imbalance through
Augmentation of Fake News Using BERT to Enhance the Performance of Fake News Classification. Applied
Sciences. 2022;12(17):8398. doi: 10.3390/app12178398 [DOI] [Google Scholar]
27. Yang Y, Zheng L, Zhang J, Cui Q, Li Z, Yu PS. TI-CNN: Convolutional neural networks for fake news
detection. arXiv preprint arXiv:180600749. 2018;.
28. Kaliyar RK, Goswami A, Narang P, Sinha S. FNDNet–a deep convolutional neural network for fake news
detection. Cognitive Systems Research. 2020;61:32–44. doi: 10.1016/j.cogsys.2019.12.005 [DOI] [Google
Scholar]
29. Shan G, Zhao B, Clavin JR, Zhang H, Duan S. Poligraph: Intrusion-tolerant and distributed fake news
detection system. IEEE Transactions on Information Forensics and Security. 2021;17:28–41. doi:
10.1109/TIFS.2021.3131026 [DOI] [Google Scholar]
30. Sedik A, Abohany AA, Sallam KM, Munasinghe K, Medhat T. Deep fake news detection system based on
concatenated and recurrent modalities. Expert Systems with Applications. 2022;208:117953. doi:
10.1016/j.eswa.2022.117953 [DOI] [Google Scholar]