Deepfake Image and Video Detection Using Deep Learning Algorithms
Deepfake Image and Video Detection Using Deep Learning Algorithms
Mohana
Computer Science & Engineering
(Cyber Security)
RV College of Engineering®
Bengaluru, India
[email protected]
Abstract— Deep learning (DL) algorithms are swiftly finding deepfakes raises fearfulness on spreading false information.
applications in computer vision and natural language Various approaches, including feature-based, temporal-
processing. Nonetheless, they can also be employed for creating based, and deep feature-based detection methods, have been
convincing deepfakes, which are challenging to distinguish from explored to analyze and classify images and video recordings.
reality. The advancements in image and video technology and
tools, especially on social media platforms, potentially lead to
Detecting deepfakes is essential to mitigate potential harm
misuse for malicious purposes like blackmail or defamation. To caused by their misuse and protect individuals from malicious
tackle this issue, several group of researchers tried upon activities.
spreading or creating awareness on real or fake data. The
proposed approach involves combining Deepfake generation II. LITERATURE SURVEY
using GANs and Autoencoders with a Deepfake detection Hafsa Ilyas et al [1] Proposed DL model called
method. The aim of this initiative is exclusively to combat InceptionResNet-BiLSTM to detect Deepfake in video
disinformation and online fraud for the welfare of the general sequence. The model combines InceptionResNetV2 for
population. Deepfakes, products of AI, have become
feature extraction and a bidirectional LSTM for classification.
increasingly realistic, rendering it nearly difficult to distinguish
the content. Auto-encoders with sufficient time can achieve It achieves high accuracy in detecting deepfakes on diverse
about 92% accuracy. As the generator improves, the datasets and outperforms existing methods. The model is Face
discriminator performance worsens as it struggles to Forensics++ and FakeAVCeleb datasets, showing promising
differentiate real or fake data. A perfect generator results in results in both cross-set and close-set evaluations. Lalitha S et
50% accuracy. With advancements in computational capacity al [2] explains about a model that combines CNN and GAN
and data availability, the proposed DDM (Deepfake Detection to recognize Deepfake videos. The model uses CNN to
Model) has achieved greater accuracy rate of up to 92.3%. extract video properties and a GAN to pinpoint deepfakes. A
GAN comprises a generator and discriminator, and the
Keywords—Deep-fake, Deep learning, GAN (Generative
aforementioned generator produces false images from video
Adversarial Nets), Autoencoders, CNN (Convolutional Neural
Network), DNN (Deep Neural Networks), Computer Vision(CV). frames. The paper focuses on training the GAN with different
setups and techniques and achieves 97.2% accuracy in
detecting Deepfake videos. Yushaa Shafqat Malik et al[3]
explores concepts of image animation using Deepfakes and
I. INTRODUCTION
spotting Deepfake generated animations. The authors suggest
Deepfake technology has been growing additional prevalent, a framework for animating objects onto driving videos
making it difficult to differentiate real and manipulated without prior evidence on object. They achieve realistic
videos. This paper strives to portray a complete analysis of results by training their dataset on images and video
DL models and techniques for detecting deepfakes using recordings of objects category. The paper also discusses
GANs. The proposed work examines and discusses the execution of a Deepfake detector technology using
potential for further advancements in DL knowledge to XceptionNet, which achieves high accuracy. However, the
enhance the detection of deepfakes, such as exploring authors find that Deepfake generation can be perturbed to fool
generator-discriminator models with multiple neural network the detector, reducing its accuracy significantly. Jixin Zhang
structures with increased complexity. Development of more et al [4] presents a Deepfake detection method that utilizes
effective Deepfake detection methods and contribute to the techniques of ensemble learning and heterogeneous features.
protection against misuse of Deepfake technology. As AI Extracted features and characteristics are integrated into an
technology continues to advance, potential commercial ensemble feature vector and used to carve a Deepfake
applications of deepfakes are expanding, but the rise of detector using a back-propagation neural network.
Shan Jia et al [5] talks about the problem of model attribution models, including CNN, RNN, and hybrid models, for
for Deepfake videos and proposes a spatial and temporal detecting Deepfake videos. The results reiterate that CNN
attention-based method called DMA-STA. This method models with SVM show superior performance to hybrid
achieves over 70% accuracy in identifying Deepfakes in a models. This paper also portrays various approaches and
dataset with videos from different models. It outperforms architectures used in Deepfake detection and displays a
existing methods and future projects includes developing and satisfactory overview of the datasets used for evaluation. Ali
improving its robustness to video compression. Manoj Kumar Berjawi et al [15] presents an optimization approach for
Das et al [6] discusses the use of DL techniques in deepfakes, Deepfake rooted video detection by introducing a pre-
emphasizing the importance of Deepfake detection in the processing phase that involves dataset cleansing and image
context of social media platforms. Authors propose a enhancement using CLAHE. Experiments demonstrate that
methodology for detecting fake faces in photographs using a this optimization improves accuracy. The proposed
deep neural network. The paper also reviews existing optimization can be valuable in detecting and mitigating the
literature on Deepfake detection and discusses efforts of effect of Deepfake video recordings, which are a growing
major tech companies in this field. This method is more concern in the area of cybersecurity.
efficient in categorizing images are real or fake, and the paper
III. DESIGN AND IMPLEMENTATION
concludes by highlighting potential threats posed by
manipulated images and videos to privacy and world peace.
Pu Sun et al [7] presents a proactive defence method called
FakeTracer by adding sustainable and erasable traces to
training data available. This method is effective in exposing
Deepfake faces and is robust against various perturbations.
Also highlighted other techniques and approaches for
Deepfake detection, including geometric features, frequency-
aware clues, and artificial fingerprinting. This article
discusses the impact of Deepfake technology, which creates
fake videos that closely resemble real ones, on various
industries and the potential risks and challenges it poses.
Samer Shorman et al [8] It emphasizes use of Deepfake
detection methods and need to address the cognitive impact
and uncertainty caused by deepfakes. Surendra singh chouhan
et al [9] discusses use of DL models and datasets for
Deepfake detection in videos and images. It explores GANs
and provides a comparative examination of various methods.
The paper also discusses limitations of existing datasets and Fig.1. Block diagram of real or fake image classification using GANs
models and aims to develop Deepfake detection technology
to prevent misuse of deepfakes. Anis Trabelsi et al [10]
discusses the problem of Deepfake detection and analyses the
top five solutions from DFDC. It explores the
complementarity between solutions and experiments with
ensemble methods to improve detection accuracy. The
authors also highlight the importance of interpretability and
explainability in Deepfake detection. Additionally, this paper
provides a list of references to related research papers and
articles on Deepfake detection and manipulation. Sio Jurnalis
Pipin et al [11] proposes a method to spot and identify
Deepfake in videos. PRNU analysis helps to overcome
performance evaluation inconsistencies and improves the
detection process. Norah M Alnaim et al [12] presents a
Deepfake face mask dataset (DFFMD) and proposes
Deepfake detection algorithms to approach hardships thrown
by the staunch increasing realism of Deepfake video
recordings. Evaluated efficiency of different DL models in
detecting Deepfake videos and discusses various approaches
and techniques for Deepfake detection. It also provides a
comprehensive survey of existing literature on Deepfake
detection and suggests future research directions. Jerry John
et al [13] explores different methods for detecting Deepfake
GANS, autoencoder.kes. This article discusses the utilization
of neural network architecture and references various
techniques and datasets used in Deepfake detection research.
Athirashree Das et al [14] presents an important survey on
Deepfake video using DL. It compares precision of different Fig.2.Flow chart of GAN Model for real or fake image classification.
Figure 1 and 2 shows the block diagram and flowchart of is trained, it can possibly be utilized to generate deepfake
GAN Model for real or fake image classification. By training images. For creation of deepfake, the autoencoder is fed with
GAN model on dataset, it learns to differentiate real and fake a sample input image of person whose face will be swapped.
faces based on attributes extracted from the images. This The encoder compresses input images into the latent space
network successfully is able to identify patterns and representation. Then, instead of using the original latent
discrepancies that differentiate real faces from deepfake representation, a modified latent representation is created by
faces. The advantage is its efficiency in classifying fake or replacing original face features with the desired face features.
real images. It leverages the power of DNN and availability Finally, the decoder takes this modified latent representation
of surplus amounts of data to develop an effective network and reconstructs the image, resulting in a deepfake image with
structure. Overall, this methodology combines the a swapped face. Detailed flowchart for the proposed
capabilities of DNN with a carefully curated dataset to implementation as shown in figure 4. This process enables the
precisely detect and classify fake faces in photographs. generation of realistic-looking deepfake images by leveraging
the power of autoencoders to encode and decode facial
features. Nonetheless, it is important to acknowledge that
numerous other methods and architectures exist for the
creation of deepfakes, and autoencoders are just one
approach.
Software and Hardware Details
Name of software: DeepFaceLive
Language Used: Anaconda Python
Requirements:
● Any DirectX12 compatible graphics card
● 4GB RAM
● 32GB+ paging file
● Windows 10
Fig. 3. Encoder-Decoder Architecture (Auto encoder) ● Modern CPU with AVX instructions
V. APPLICATIONS
Deepfake technology has numerous applications in various
sectors, including entertainment, advertising, education, art,
forensics, privacy, and political commentary. It can create
realistic visual effects, create engaging content, and help
Fig.7.Deepfake Result 1-Joker data set. identify manipulated evidence in forensic investigations. It
can also uplift awareness regarding privacy and security risks
and be utilized for political satire or social commentary.
However, ethical and legal implications must be addressed to
ensure responsible and beneficial applications. Deepfake
detection can help prevent fraudulent activities, authenticate
video authenticity, and prevent the spread of misinformation.
It also finds its effect in law enforcement to identify cases
involving digital evidence manipulation. Ultimately,
Deepfake detection contributes to the protection of
individuals, organizations, and society [16].
Fig.8. Deepfake Result 2-Mr Bean data set.
VI. RESEARCH AND IMPLEMENTATION
CHALLENGES
The field of Deepfake detection faces several open issues,
including the development of robust and reliable detection
methods, creation of larger and diverse datasets for training
and evaluating models, and exploring the potential of
combining multiple DL techniques, leveraging ensemble
methods. As Deepfake technology advances, it becomes
increasingly challenging to detect in deployed videos,
making it essential for ongoing research and development to
maintain a lead. the evolving capabilities of Deepfake
Fig.9. Deepfake Result 3-Jackie Chan data set. creation. Ethical and legal implications of Deepfake concepts
are also a concern, as inclusion of Deepfake principles for
malicious purposes raises concerns about privacy, consent,
and potential harm. Clear guidelines and regulations are
required to address these challenges and ensure responsible
use of Deepfake technology. Future directives in Deepfake
detection include upcoming of better advanced algorithms,
collaborations between researchers, industry experts, and
policymakers, and creating awareness among public
regarding deepfakes [17] [18]. In conclusion, addressing
open issues and future directives in the domain of deepfakes
requires a multidisciplinary approach involving researchers,
industry professionals, policymakers, and the public. By
directing their attention to these domains, researchers have
Fig.10. Deepfake Result 4-Meghan Markle data set.
the potential to provide valuable insights into contributing to
the development of more effective and reliable deep fake capabilities of Deepfake creation and safeguard against its
detection systems. detrimental consequences.