Criminal Recognition and Tracking System
Criminal Recognition and Tracking System
I. INTRODUCTION
We live in an age of information, where anything and everything is connected through the internet. Connecting and
interfacing different devices has become easier than ever. Information is available at the tip of a finger. Yet,
everything is not well. The crime rate is ever growing. The increase of fraudulent behavior in society has caused
many professional institutions to turn to biometric measures for positive identification of their clients. There is
thus a renewed interest in the collection of biometric measures of people to strengthen existing identity checks.
Face recognition is one identification technique that employs biometric measures. Before dwelling further, we
need to understand how face recognition works. In human beings, it is the temporal lobe of the brain which is
responsible for recognition of faces. The neurons of the temporal lobe respond to certain features of the face and
stores them eventually leading to face identification. In machine learning system, the machines are often fed a huge
bank of images which the system absorbs and stores. When aface is to be matched, it tries matching it with the
stored images using face recognition algorithm. A Grid OF CCTVs installed in an area can give information about
human activities happening in the area. The existing network of Government and public sector-installed Closed
Circuit Television Cameras can be virtually networked to create a system that will monitor key public arenas,
financial institutions, tourism sites and key infrastructure. However, installing CCTV cameras and forgetting them
until something happens, gives us a false sense of security.
___________________________________________________________________________________________________________________________________
IRJCS: Mendeley (Elsevier Indexed) CiteFactor Journal Citations Impact Factor 1.81 –SJIF: Innospace,
Morocco (2016): 4.281 Indexcopernicus: (ICV 2016): 88.80
© 2014-19, IRJCS- All Rights Reserved Page-118
International Research Journal of Computer Science (IRJCS) ISSN: 2393-9842
Issue 06, Volume 6 (June 2019) [Link]
SPECIAL ISSUE - 5th International Conference - “ACCE – 2019”
The truth is most CCTVs cameras go unmonitored and unmaintained. Some even stop functioning and we find out
this only when there is a manual intervention to try to access some footage. No Number of Human Being will be
able to monitor the terabytes of information being generated on a second-to-second basis. Eyeballing petabytes of
video is simply impossible for the eyeballs we employ in police. It also requires lots of patience. Manual video
tracking (CCTV video analysis) mostly relies on teams of specially trained officers watching thousands of hours of
footage, waiting for that one crucial second of evidence. But this is the problem we are trying to solve. Now,
Advanced Computer Vision technology can monitor the footage (if not disapproved by law) for movement. We just
need to bring these cameras online, and link them up in an encrypted grid using advanced Artificial Intelligence
and Machine Learning technologies. The data from these live streams can be processed and analyzed in near-real
time to produce a wealth of information, which will be a significant input when it comes to the topic of
surveillance. The main concept of our project is to experiment with use of deep learning neural networks to detect
and quickly respond to crimes in progress with effective criminal tracking system to reduce the crime rate. The
idea behind the project is to being able to sort through a massive volume of data to find patterns that are useful for
law enforcement. This kind of data analysis was technologically impossible a few decades ago, but the hope is that
recent developments in deep learning neural networks are up to the task. The project uses computer vision and
other techniques to spot potentially criminal anomalies on real-time surveillance video with facial recognition and
gait analysis technology help us to use advanced intelligence concepts to find and track individuals.
II. LITERATURE SURVEY
Face Recognition and Detection using Neural Networks [1] is one of the latest technologies being studied area
in biometric as it has wide area of applications. But Face detection is one of the challenging problems in Image
processing. The basic aim of face detection is to determine if there is any face in an image & then locate position of
a face in an image. Evidently face detection is the first step towards creating an automated system which may
involve other face processing. The neural network is created & trained with training set of faces & non-faces.
Neural network has a feature of adaptive learning i.e. an ability to learn how to do tasks. Also it can create its own
organization. It has a remarkable ability to derive meaning from complicated or imprecise [Link]: The
number of subjects has to be increased in the training set.
Face Recognition Using Neural Networks [2] is an approach to the problem of face recognition using a discrete
cosine transform (DCT) and neural networks is presented. The DCT is used to extract features from the high
dimensional facial data. Two different neural networks are used. The standard backpropagation neural network is
used in a "network per person" implementation, while the counter propagation network is used in a database
implementation. Two types of networks were used for the classification for the data a standard backpropagation
(for person identity) with momentum and a counter propagation network (for determining database
membership). The database comprised 83 distinct subjects, with ten views per subject (830 images). Using the
standard backpropagation algorithm with momentum (for determining identity), 83 networks were trained
separately. A single network was trained using the counter propagation algorithm for feed-forward networks (for
determining database membership). No attempt was made to control the light that filtered in through the blinds in
the laboratory. The fluorescent lights were switched on and off at [Link]: The system is not feasible
with 500 or more subjects.
Real Time Face Authentication Using Convolutional Neural Network [3] the face analysis is the recent and
challenging technology for authentication and recognition. Researchers have been developing new techniques for
face analysis for the past decades. This paper deals with the real time face authentication using Deep learning
based Convolutional Neural Network (CNN) for high True Acceptance Rate (TAR). Face analysis in computer vision
technology has two modes of operation Face identification (Recognition) and Face Authentication (verification),
where face identification is the one to many mapping, compares the unknown query image to the group of known
images in the database whereas face authentication is the one to one mapping, compares the query image to the
person’s image. The process involved in the face verification using Convolutional Neural Network (CNN) and
Constrained Local Model (CLM) for alignment of the faces. The class of 10 images for each user is trained by deep
neural network; features are matched against the query image and provide the confidence score for that
[Link]: The system fails to recognize the person in different poses and at different lighting levels.
Real Time Face Detection Using Neural Networks [4] is an active interdisciplinary area of research that uses
techniques from computer vision, image processing and pattern recognition. On the other hand, neural networks
have been widely used to address problems in feature extraction, pattern recognition, and in general, the same
kind of problems. The proposed system here uses neural networks in the development of a face detection system
capable of operating in real time. The system performs a guided face search on interest regions exhibiting human
skin color properties. These properties are detected in a pixel by pixel basis. The proposed system can be used as a
module of face recognition systems, video surveillance systems, access control systems.
___________________________________________________________________________________________________________________________________
IRJCS: Mendeley (Elsevier Indexed) CiteFactor Journal Citations Impact Factor 1.81 –SJIF: Innospace,
Morocco (2016): 4.281 Indexcopernicus: (ICV 2016): 88.80
© 2014-19, IRJCS- All Rights Reserved Page-119
International Research Journal of Computer Science (IRJCS) ISSN: 2393-9842
Issue 06, Volume 6 (June 2019) [Link]
SPECIAL ISSUE - 5th International Conference - “ACCE – 2019”
The system combines two methods, one based in features and another based on image to detect more accurately
faces and to provide a faster response in order to be able to perform in real [Link]: The image that have
the same color properties than the skin color model difficult to identify and the robustness of the system can be
improved by using larger training sets.
FaceNet: A Unified Embedding for Face Recognition and Clustering [5] the face analysis is the recent and
challenging technology for authentication and recognition. The proposed system uses a system called FaceNet,
which directly learns a mapping from face images to a compact Euclidean space where distances directly
correspond to a measure of face similarity. Once this space has been produced, tasks such as face recognition,
verification and clustering can be easily implemented using standard techniques with FaceNet embedding’s as
feature vectors. The system uses a deep convolutional network trained to directly optimize the embedding itself,
rather than an intermediate bottleneck layer as in previous deep learning approaches. To train, it uses triplets of
roughly aligned matching / non-matching face patches generated using a novel online triplet mining method. The
benefit of this approach is much greater representational efficiency. On the widely used Labeled Faces in the Wild
(LFW) dataset, the system achieves a new record accuracy of 99.63%. On YouTube Faces DB it achieves 95.12%.
This method is based on learning a Euclidean embedding per image using a deep convolutional
[Link]: The system requires huge model and high end CPU, also requires long training time.
III. METHODOLOGY
In the proposed system the first feature is to detect the Face in real time. At the time the face is detected the
system will try to recognize the face, if the recognized face is a wanted criminal or person missing, the system will
automatically notify the concerned officials. The data of the victim’s face will be initially stored in the server where
each camera with the system will receive this data. And other primary focus is given to the tracking down the
victim by getting the previous records and successive switching between the cameras to find the last known
location of the victim.
A. Advantages of Proposed System
• Reduced human effort.
• Tracking criminals at faster rate.
• Completely autonomous system.
• Footage is available at all times.
• Provides efficient surveillance.
Using the manual way of finding out the criminal and tracking is highly time consuming. Hence better to make an
automated system that hunts or searches for the criminal all time. The criminal recognition system will search for
the criminal all the time, provided the picture of the criminal is uploaded to the server, and when criminal turns up
in any one of the cameras the concerned official will be notified. Every year 80,000 children go missing in India.
Many notorious criminals are found absconding. The proposed work is to alert the officials on finding the missing
people or criminals, providing better surveillance and data access and reducing human effort and time.
IV. DESIGN
A. System Architecture
The Figure1 describes the System Architecture where there three components- User, Server and CCTVs. The CCTV
provides continuous video stream to the server. The server monitors each CCTV video stream hunting for the
required person. The User provides information about the person to be tracked and gets a notification from
server when person is identified.
___________________________________________________________________________________________________________________________________
IRJCS: Mendeley (Elsevier Indexed) CiteFactor Journal Citations Impact Factor 1.81 –SJIF: Innospace,
Morocco (2016): 4.281 Indexcopernicus: (ICV 2016): 88.80
© 2014-19, IRJCS- All Rights Reserved Page-120
International Research Journal of Computer Science (IRJCS) ISSN: 2393-9842
Issue 06, Volume 6 (June 2019) [Link]
SPECIAL ISSUE - 5th International Conference - “ACCE – 2019”
A use case diagram is a dynamic or behavior diagram in UML. Use case diagrams model the functionality of a system
using actors and use cases. The Figure 2. describes the use case of the system where there are two actors- Admin
(Server), Camera (CCTVs). Admin performs all the modules to in order for each frame from the CCTV camera. The
Camera only inputs the video to the system.
• Image Acquisition acquires the frame from camera streams.
• Face Detection detects the face in acquired images, resizes and records it.
• Face Recognition module compares selected facial features from image and compare with the faces it already
knows.
• Person Tracking module notifies the User with an email containing the snapshot of the victim.
C. Data Flow Diagram
A data-flow diagram (DFD) is a way of representing a flow of a data of a process or a system (usually an
information system).
The DFD also provides information about the outputs and inputs of each entity and the process itself. A data-flow
diagram has no control flow, there are no decision rules and no loops. When the system is started, the first
process to be carried out is the image capture, where a live camera feed is used as input. The camera feed is
processed, and passed to the Face detection and recognition module, here as and when a face is detected it is
matched against the database to see if there is match. If and when a match is found with confidence more than the
set threshold value a notification is sent to the admin. The Figure 3. describes the data flow of the system.
V. IMPLEMENTATION
A. The Pipeline
The Criminal Recognition and Tracking System Pipeline as shown in Figure 4, consists the Image Acquisition, Face
Detection, Face Recognition and Person Tracking Modules. Pipelining is the process of accumulating instruction
from the processor through a pipeline. It allows storing and executing instructions in an orderly process. It is also
known as pipeline processing. Pipelining is a technique where multiple instructions are overlapped during
execution. Pipeline is divided into stages and these stages are connected with one another to form a pipe like
structure. Instructions enter from one end and exit from another end. Pipelining increases the overall instruction
throughput.
___________________________________________________________________________________________________________________________________
IRJCS: Mendeley (Elsevier Indexed) CiteFactor Journal Citations Impact Factor 1.81 –SJIF: Innospace,
Morocco (2016): 4.281 Indexcopernicus: (ICV 2016): 88.80
© 2014-19, IRJCS- All Rights Reserved Page-122
International Research Journal of Computer Science (IRJCS) ISSN: 2393-9842
Issue 06, Volume 6 (June 2019) [Link]
SPECIAL ISSUE - 5th International Conference - “ACCE – 2019”
The Figure 9 shows the Criminal Recognition and Tracking dashboard with real time Criminal Recognition and
Tracking pipeline implementation, where the streams of the multiple cameras are fed and the image acquisition,
face detection, face recognition and person tracking modules are executed simultaneously without any user
intervention, thereby making it a most secure application. The User can make the selection and give the input to
the system only through the respective sections that are defined above.
___________________________________________________________________________________________________________________________________
IRJCS: Mendeley (Elsevier Indexed) CiteFactor Journal Citations Impact Factor 1.81 –SJIF: Innospace,
Morocco (2016): 4.281 Indexcopernicus: (ICV 2016): 88.80
© 2014-19, IRJCS- All Rights Reserved Page-124
International Research Journal of Computer Science (IRJCS) ISSN: 2393-9842
Issue 06, Volume 6 (June 2019) [Link]
SPECIAL ISSUE - 5th International Conference - “ACCE – 2019”
VII. CONCLUSION
The computational models, which were implemented in this project, were chosen after extensive research, and the
successful testing results confirm that the choices made by the researcher were reliable. This system was tested
under very robust conditions in this experimental study and it is envisaged that real-world performance will be far
more accurate. The fully automated frontal view face detection system displayed virtually perfect accuracy. The
most suitable real-world applications for face detection and recognition systems are for surveillance. There are
better techniques such as iris or retina recognition and face recognition using the thermal spectrum for user access
and user verification applications since these need a very high degree of accuracy. The real-time automated pose
invariant face detection and recognition system proposed would be ideal for crowd surveillance applications. If
such a system were widely implemented its potential for locating and tracking suspects for law enforcement
agencies is immense. The implemented fully automated face detection and recognition system could be used for
simple surveillance applications such as ATM user security, while the implemented manual face detection and
automated recognition system is ideal of criminal recognition and tracking.
A. Applications
• Quick notifications about criminal location.
• Better approach to find missing and suspicious people.
• Better scan to run facial recognition to search for potential criminals or missing people.
• Reduced human effort.
• Tracking criminals at faster rate.
• Footage is available at all times.
• Provides efficient surveillance.
REFRENCES
1. Face Recognition and Detection using Neural Networks Vinita Bhandiwad and Bhanu Tekwani, Assistant
Professors, Department of Information technology, Vidyalankar Institute of technology, Mumbai, 2017.
2. Face Recognition Using Neural Networks by S. C. Debiyersild arid A. D Broadhurst Department of Electroriic
Engineering University of Natal, Durban, South Africa, 1997.
3. Real Time Face Authentication Using Convolutional Neural Network by Sharma, Sameera Shaik School of
Electronics, Department of Computer Science 1,2 Vignan’s University, Guntur ,Andhra Pradesh, India, 2016.
4. Real Time Face Detection Using Neural Networks by Angel Noe Martinez-Gonzalez and Victor Ayala-Ramirez
Electronics Engineering Department Universidad de Guanajuato DICIS Salamanca, Mexico, 2011.
5. FaceNet: A Unified Embedding for Face Recognition and Clustering by Florian Schroff, Dmitry Kalenichenko,
James Philbin Google Inc, 2015.
6. S. Z. Li, Daniel Gatica-Perez, Dong Zhang, "Real-Time Face Detection Using Boosting in Hierarchical Feature
Spaces".
7. C. G. García, D. M. Llorian, B.C.P. G-Bustelo, J.M.C. Lovelle, “A review about smart objects, sensors, and
actuators,” 2017.
8. Das, M. Pukhrambam, A. Saha, "Real-time robust face detection and tracking using extended haar functions
and improved boosting algorithm", 2015.
9. B. Martinez and M. Valstar. Advances, challenges, and opportunities in automatic facial expression
recognition. In B. S. M. Kawulok, E. Celebi, editor, Advances in Face Detection and Facial Image Analysis.
Springer, 2015. In press.
10. [Link],[Link],[Link],[Link],andJ.F. [Link]: Aspontaneousfacialactionintensitydatabase.
IEEE T-AFFC, 2013.
11. E. Sariyanidi, H. Gunes, and A. Cavallaro. Automatic analysis of facial affect: A survey of registration,
representation and recognition. IEEE TPAMI, 2014.
___________________________________________________________________________________________________________________________________
IRJCS: Mendeley (Elsevier Indexed) CiteFactor Journal Citations Impact Factor 1.81 –SJIF: Innospace,
Morocco (2016): 4.281 Indexcopernicus: (ICV 2016): 88.80
© 2014-19, IRJCS- All Rights Reserved Page-125