Proceedings of the Third International Conference on Trends in Electronics and Informatics (ICOEI 2019)
IEEE Xplore Part Number: CFP19J32-ART; ISBN: 978-1-5386-9439-8
An Efficient Helmet Detection for MVD
using Deep learning
Rohith C A, Shilpa A Nair, Parvathi Sanil Nair, Nithin Prince John
Sneha Alphonsa, UG Students Assistant Professor
Department of Computer Science and Engineering Department of Computer Science and Engineering,
Saintgits College of Engineering Saintgits College of Engineering
Kottayam, Kerala, India Kottayam, Kerala, India
[email protected],
[email protected],
[email protected] [email protected],
[email protected] Abstract— Bike riders have been rapidly increasing numerous drivers don’t fully utilize it. A road safety manual
amid time in various countries. Motorbikes are favoured by [3] was created by United Nations road safety collaboration
citizens belonging to different classes of the society due to which explains the importance of wearing helmets. The
many reasons such as its economic value. Wearing helmets is manual [3] also gives the reader a better understanding of the
compulsory according to the standard however the vast actual purpose of the helmet. The fundamental objective of a
majority avoid it. A principal goal of the helmet is to helmet is to ensure driver's safety. A study conducted by the
guarantee the safety of the riders. Our research intends to United Nations in 2015 estimated that the chances of surviving
make an automated system to distinguish whether a biker is an accident rose by 42% on wearing a helmet.
wearing a helmet or not and to impose fine to defaulters as a
part of law enforcement. According to our research, such a Even though helmets are for the safety of the riders, most
system is not currently used by the police or by any other
of them avoid it due to reasons like “it spoils my hairstyle”, “it
feels uncomfortable”,” good helmets are costly” or “it
authority. Executing the proposed framework can convey
obstructs my peripheral vision”. These reasons are not
more mindfulness and need to wear a helmet at any rate with
comparable to losing a life. We realised that creating new and
the goal that they don't get captured on camera and avoid fine.
advanced methodologies to find the defaulters will help reduce
This project intends to make deep learning based automated
the number of accidents, increase safety and create a
detection system for helmet identification using trained models
realisation in the minds of the people about the need for
and datasets that would be useful for the police department to wearing a helmet. The existing mechanism to check whether a
enforce the law for the betterment of the society. rider is wearing a helmet or not is a checkpoint by police or
other personnel to manually check each rider. In this system,
Keywords—Image classification; deep learning; helmet there is a possibility of riders evading checkpoints.
detection; Motor Vehicle Department; Caffe model; InceptionV3
The main aim of our research is to apply the concepts of
I. INTRODUCTION machine learning algorithms for creating an automated system
for helmet detection. Machine Learning is the subset of
Almost everyone uses two-wheelers for transportation
Artificial Intelligence that deals with the extraction of patterns
purposes, and due to increased use of these vehicles, more
from datasets. Machine learning is the act of utilizing
people are prone to accidents. Since the number of accidents
algorithms to parse information, gain from it, and afterward
in our country is increasing day by day, the need for safety
make a prediction or expectation about something in the
measures is a must, and usage of helmets is an important
world. In our new system, we apply the techniques of deep
factor in terms of safety which most of the people avoid.
learning which is a subset of machine learning algorithm that
According to a report by TOI [1], about 28 motor bike riders
utilizes complex artificial neural networks. One of the main
died daily on Indian roads in 2016 and 98 riders daily in 2017
areas of research for our proposed innovation includes the
for not wearing a helmet. From the survey, almost 20% of the
usage of pre-trained real time object detection models and
people who died in an accident were not wearing a helmet
image classification models using convolutional neural
with a total of 10,135 in 2016 which rose to 36,000 in 2017
networks (CNNs). Image classification using CNNs aims to
[2]. Helmet is the primary security measure of riders; however
classify an input image based on visual content. Many models
such as CaffeNet, GoogLeNet, VGGnet, InceptionV3 etc.
978-1-5386-9439-8/19/$31.00 ©2019 IEEE 282
Proceedings of the Third International Conference on Trends in Electronics and Informatics (ICOEI 2019)
IEEE Xplore Part Number: CFP19J32-ART; ISBN: 978-1-5386-9439-8
have been developed over the years and some of them are
quite popular due to its high performance. InceptionV3 is a
very popular model which is known to show almost 78%
accuracy on the ImageNet dataset. The model is a combination
of ideas developed by many researchers. In our system, we use
the InceptionV3 model for transfer learning. By using such
machine learning models, we can automate the process of
helmet detection with the use of a real time surveillance
system.
Executing this automated system can convey more
awareness among riders to wear a helmet at any rate with the
goal that they don’t want to get captured and avoid fine. Such
a system will help to impose fine on the defaulters and this
continuous generation of a fine will compel citizens to wear
helmet. The earlier mentioned reasons for not wearing a
helmet will slowly be no more. Since automatic detection is
involved, it will reduce the impact of accidents to an extent.
Incorporating an automatic model will help the police
No-Helmet
department to find the violators and save lives. With the new
innovative system, the number of accidental deaths caused by
not wearing a helmet can be reduced. SAVE
IMAGES
II OBJECTIVES AND PROPOSED INNOVATION
Fig. 1. Working model of the proposed system.
A. Objective
The main objective is to use the concepts of machine
learning and deep learning to create real time classifier. The Fig.1 illustrates the flow diagram of the proposed model.
studies and research done on real-time object detection are First, a camera is used to obtain a live video stream. This
inherited to gain knowledge in creating a new specific model. video is up-linked to a real-time object detection model. This
We focus on using pre-trained models to perform transfer model will detect the frames with motorbikes and capture
learning to create a new model specific to our task. We reuse them. The captured frame is then uploaded to the image
Caffe model and InceptionV3 model to fulfil our needs. classifier which will classify the frames into a helmet and non-
Explanation about the usage of the pre-trained models is helmet class. The frames in helmet class will be discarded and
mentioned below. Create real-time classifier for bike frames in non-helmet class are saved into the directory.
detection. Firstly, we plan to create a real-time classifier for
bike detection. After detecting a motorbike, we capture the
frames containing the motorbike from the video file. Next step III. RELATED WORKS
is to perform object segmentation on the converted frames to
identify biker class and rejected class. Feature extraction is to In the paper by Mistry et al [4] CNN and YOLOv2 models
be performed on the segmented image so that we'll get an are used. First YOLOv2 model guarantees the person
image containing any of the two class. It will be classified as detection which is trained on COCO dataset. This step
helmet and non-helmet. After a violator is identified the decreases the quantity of helmets being undetected. In the
segmented image of violators is saved to the directory. proposed methodology, detection of person class is utilized
rather than a motorcycle so as to build the accuracy of helmet
detection in the input image. The cropped images of the
B. Proposed Innovation
recognized person are utilized as input to the second YOLOv2
We apply the concepts of deep learning algorithm. we use model which was trained on helmeted image dataset. The non-
transfer learning and fine-tuning techniques to build a model helmeted pictures are processed further to remove license
to identify two-wheeler vehicles based on video footage from plate. In the proposed methodology, two diverse datasets
the camera and capture frames to verify whether a person is COCO and helmet datasets were utilized. Test on various
wearing a helmet or not. The image below shows a helmeted and non-helmeted images demonstrated that the
diagrammatic flow representation of the current plan for proposed technique performs better when contrasted with
helmet detection and task. other existing methodologies with 94.70% helmet detection
accuracy.
978-1-5386-9439-8/19/$31.00 ©2019 IEEE 283
Proceedings of the Third International Conference on Trends in Electronics and Informatics (ICOEI 2019)
IEEE Xplore Part Number: CFP19J32-ART; ISBN: 978-1-5386-9439-8
In the paper by Bhaskar, P. K., & Yong, S.-P. [12] the
In the paper by Vishnu et al [5] they introduced a system vehicle detection is based on image processing methods from
for the automatic detection of motorcyclists driving without the video frames, they incorporated algorithms for Vehicle
helmets in surveillance videos. They utilized versatile recognition and their tracking was done by combining
background subtraction on video edges to get moving objects. Gaussian mixture models and blob detection method. The
Later convolutional neural system (CNN) was utilized to mechanism followed was to differentiate the background and
choose motorcyclists among the moving objects. They foreground of the selected frame, detection is done by the
effectively identified 92.87% violators with a low false alarm foreground detector and binary computation to create
rate of 0.5% on an average and along these lines demonstrated rectangular areas around the detected object. Their detection
the proficiency of their proposed methodology. accuracy was found to be 91%.
The authors Kunal et al [6] present an approach for IV .DESIGN AND METHODOLOGY
automatic detection of riders not wearing helmet using
surveillance camera in real time. They use background We use Traffic cameras to shoot the videos from the
subtraction and object segmentation for detecting bike riders. desired locations. We then use a real-time object detection
By using a binary classifier, they detect whether the rider system (Caffe model) to detect the presence of people riding
wears a helmet or not [7] and also provided an approach for two-wheelers. If the model detects the presence of a two-
reporting the violation. They use different features to evaluate wheeler, a bounding box is created around it. This area is then
the results obtained. Based on the comparison of various cropped from the current frame and passed to the image
results, they got an accuracy of 93.08% on real-time data. classifier for further processing in the later stages. If the video
does not detect the presence of any motorbike riders the frame
In the paper by Rattapoom et al [8] they used a moving is simply discarded. The cropped frames are passed onto an
object detection mechanism and classified heads using the image classifier which classifies an image to detect if a rider
techniques comprising of mainly the head extraction and has a helmet or not. The image classifier will classify the test
classification. The extraction method is based on profiling image from the captured frame into one of the two destination
methods, while the classification method is based on features classes. If the predicted class is no helmet class the cropped
derived from head regions. The test results precisely frames are saved to the directory. An officer can then check
distinguished helmet wearers at the rate of 74% for both lanes. the vehicle number and calculate the fine for not wearing a
They further suggested the use of an automatic number plate helmet.
extraction system to generate an automatic helmet wearing To extract the frames from the video file, we use a caffe
monitoring system to reduce the hard work of police. model and perform object detection on the live video. When a
motorbike and a person is detected, and their bounding boxes
The authors Dharma et al [9] present the development of a are overlapped then those boxes are cropped from the current
system using image processing and deep convolutional neural frame and provided to the image classifier. The image
networks (CNNs) for finding motorcyclists who are violating classifier then classifies the image in helmet on no-helmet
helmet laws. Their system comprises motorcycle detection, class.
helmet vs. no-helmet classification, and motorcycle license
plate recognition. Developed CNN models for classifiers.
Main errors were when a motorcyclist is wearing a hat when
the front rider is wearing a helmet and the second person is
not. This issue can be solved by increasing the training data
belonging to these classes. i.e., more data containing hats, and
more data where the first rider is wearing a helmet and the
second is not.
In the paper by Rasli et al [10] present a mechanism for
accident prevention using a smart helmet with sensors. As it is
developed for the safety of people, they will be alarmed if the
speed limit is exceeded. A LED bulb is used to show the
excess speed. It will turn ON if speed is over 100 km/hr. They
use BLDC fan and Force Sensing Resistor (FSR) to detect
riders head and motorbike speed in [11]. They have Fig. 2. Real-time working of the proposed model
implemented the mechanism in such a way that the bike will
only start if the rider has a helmet on his head. In the image classification task, the cropped image is
passed through an InceptionV3 model. This then passes
through two more layers having 1024 nodes each and then
978-1-5386-9439-8/19/$31.00 ©2019 IEEE 284
Proceedings of the Third International Conference on Trends in Electronics and Informatics (ICOEI 2019)
IEEE Xplore Part Number: CFP19J32-ART; ISBN: 978-1-5386-9439-8
through a final SoftMax layer that gives out the binary We used Caffe model for the real-time motorbike
classification score for the two classifications. These scores recognition, where bounding boxes are created around the
are attained using transfer learning techniques. The pre-trained detected objects, we find the height and the width of the
InceptionV3 model is modified to suit our needs and is trained original bounding boxes as shown in Fig.3. After finding the
on the new dataset to give more accurate results on the task at height and the width we use the equation (1) to find an origin
hand. If the image is classified as no- helmet then it is saved (X, Y) for the new crop coordinates. The given equation (2) is
into the directory for future references. These details can be used for cropping the bounding boxes. The frames containing
used by MVD to impose fine to the violators. Fig. 2 illustrates the motorbikes are cropped and saved to a folder
the real-time working of the proposed model. At the initial
stage, the camera provides real-time video to the object crop_img = frame [y: y + height + 150, x:x + width + 150] (2)
detection model. The object detection model detects the
presence of motorbike and person along with other things. The In the image classification task, the segment of the image is
motorbike-person overlapping area at a specific point of time passed through an InceptionV3 model. Then it passes through
is detected and a set of coordinates are dynamically set two more layers having 1024 nodes each and at last through a
depending on the position of the bounding boxes created final SoftMax layer that gives out the binary classification
around the motorbike and person detected. This new set of score for the two classifications. The two classifications are
coordinates is used to crop the current frame. This cropped helmet and no-helmet. These final classifications scores are
framed with the motorbike person combination is then passed attained using transfer learning techniques. We had
onto the image classification model. The image classification approximately 6940 training images. Each experiment had
model then classifies the cropped Image into a helmet or no different images in training and testing sets.
helmet class as mentioned in the description of Fig. 1. If the
classifier identifies an image as no- helmet class, then the TABLE I SPLITTING OF DATASETS
image is saved to the directory else it is simply discarded.
V. EXPERIMENTS AND RESULTS
A. Experiments
We conducted the experiments in our campus. A phone
camera with aspect ratio 16:9, 3.68 mm focal length was used
to shoot the video, the shot video is of the top view. The
camera captures video with the resolution of 1920x1088
pixels. The system was implemented in python on MS
Windows 10 operating system that ran on an NVIDIA B. Results
GEFORCE 940 MX. We tested our system using these On analysing the conducted experiments, we obtain the
components for motorcycle recognition, frame extraction and following results:
head-helmet classification. These tests were carried out Using Caffe model, we got an accuracy of ≈ 76% for
independently, with the goal that the errors were minimum motorbike recognition and detected image are cropped and
and errors from the previous algorithms were not propagated. saved into a folder.
Fig. 4: Output Snapshot of Real Time Detection
Using InceptionV3 model, we got an accuracy of ≈ 81%
Fig 3: Creation of Crop coordinates
for Head-helmet classification and images containing no-
helmet were saved to a folder and others are discarded.
X = xRight_top – Width
(1)
Y = yLeft_bottom – Height
978-1-5386-9439-8/19/$31.00 ©2019 IEEE 285
Proceedings of the Third International Conference on Trends in Electronics and Informatics (ICOEI 2019)
IEEE Xplore Part Number: CFP19J32-ART; ISBN: 978-1-5386-9439-8
standards, safety can be ensured. The system can further be
automated by using an image to text classifier to identify
number plates and generate a fine automatically.
REFERENCES
[1] D. Dash., ‘In 2016, not wearing helmets, seat belts killed 43 a day ',The
Times of India, https://timesofindia.indiatimes.com/city/delhi/28-bikers-
without-helmets-died-daily-in 2016/articleshow/60050999.cms ,2017
[Accessed: Jan.23, 2019].
Fig. 5: Output Snapshot of Image Classifier
[2] A. Singh,”98 two-wheeler users without helmet died per day in 2017&
79 drivers died of not wearing seatbelts”,Financial Express,
https://www.financialexpress.com/auto/bike-news/98-two-wheeler-
VI. CONCLUSION AND FUTURE SCOPE users-without-helmet-died-per-day-in-2017-79-drivers-died-of-not-
wearing-seatbelts/1342771/[Accessed:Jan 23,2019]
A. Conclusion [3] WHO,” Helmets: a road safety manual for decision-makers and
practitioners”,United Nations ”
The condition selected for classifying the captured frames https://www.who.int/roadsafety/projects/manuals/helmet_manual/en/
is a person wearing a helmet and non-helmet while driving a [4] Jimit Mistry, Aashish K. Misraa, Meenu Agarwal, Ayushi Vyas, Vishal
two-wheeler. We are planning to extract images that contains M. Chudasama, Kishor P” An automatic detection of a helmeted and
a person wearing a helmet and non-helmet while driving a non-helmeted motorcyclist with license plate extraction using
two-wheeler. Firstly, we classify images that contain a person Convolutional Neural Network “ Upla Electronics Engineering
Department, Sardar Vallabhbhai National Institute of Technology
travelling in two-wheeler or not. Then, we analyze the (SVNIT), Surat, India.
classified images to obtain the dataset based on the condition. [5] C. Vishnu, Dinesh Singh, C. Krishna Mohan and Sobhan Babu
The captured frames that contain images other than the above- “Detection of motorcyclists without a helmet in Videos using
mentioned condition will be discarded or not considered. Convolutional Neural Network" Department of Computer Science and
Engineering Indian Institute of Technology Hyderabad, Kandi,
Sangareddy-502285, India.
We consider a video file as input to the model and create a
[6] K. Dahiya, D. Singh and C. K. Mohan, "Automatic detection of bike-
bounding box around frames containing person and riders without helmet using surveillance videos in real-time," 2016
motorbike. The frames where motorbike and person are International Joint Conference on Neural Networks (IJCNN),
overlapped are cropped and saved into a directory. The model Vancouver, BC, 2016
used for the detection and extraction is Caffe model. The [7] D. Selvathi, P. Pavithra, T. Preethi. "Intelligent transportation system for
accident prevention and detection", 2017 International Conference on
proposed model shows an accuracy score of ≈ 86%. The Intelligent Computing and Control Systems (ICICCS), 2017
dataset considered for training this new proposed model [8] Rattapoom Waranusast, Nannaphat Bundon, Vasan Timtong and
consists of images of people wearing helmets and people not Chainarong Tangnoi” Machine Vision Techniques for Motorcycle
wearing helmets. The image classifier was pre-trained for this Safety Helmet Detection” 2013 28th International Conference on Image
particular binary classification task. The model used for and Vision Computing New Zealand.
[9] Dharma Raj KC, Aphinya Chairat, Vasan Timtong, Matthew N. Dailey,
classification is Inception V3 model. The proposed model Mongkol Ekpanyapong” Helmet violation processing Using Deep
showed a validation accuracy score of ≈ 74%. Even though learning” Asian Institute of Technology Khlong Luang, Pathum Thani
this model gives a good result, we are still trying to optimize 12120, Thailand.
the model so that we have a better score. We will use different [10] M. K. A. Mohd Rasli, N. K. Madzhi and J. Johari, "Smart helmet with
models to analyze the accuracy and choose the best model. sensors for accident prevention," 2013 International Conference on
Electrical, Electronics and System Engineering (ICEESE), Kuala
Lumpur, 2013.
B. Future Scope [11] Sayan Tapadar, Shinjini Ray, Himadri Nath Saha,Arnab Kumar Saha,
Robin Karlose. "Accident and alcohol detection in bluetooth enabled
The model can further be improved with better training for smart helmets for motorbikes", 2018 IEEE 8th Annual Computing and
Communication Workshop and Conference (CCWC), 2018
the image classifier. Due to lack of resources the accuracy
[12] Bhaskar, P. K., & Yong, S.-P.,” Image Processing Based Vehicle
obtained is limited. This model can also be used in such a way Detection and Tracking Method” 2014 International Conference on
that a motorbike does not start unless the rider wears a helmet. Computer and Information Sciences (ICCOINS),2014.
By incorporating a portable model to the ignition switch the
power supply can be killed until a helmet is detected. This is
one of the future developments that can be made to this model.
Even though the vehicle cost will increase a bit than its
978-1-5386-9439-8/19/$31.00 ©2019 IEEE 286
