Ashim 

Saha
Nirmalya Kar
Suman Deb (Eds.)

Communications in Computer and Information Science 1192

Advances in Computational
Intelligence, Security and
Internet of Things
Second International Conference, ICCISIoT 2019
Agartala, India, December 13–14, 2019
Proceedings
Communications
in Computer and Information Science 1192
Commenced Publication in 2007
Founding and Former Series Editors:
Phoebe Chen, Alfredo Cuzzocrea, Xiaoyong Du, Orhun Kara, Ting Liu,
Krishna M. Sivalingam, Dominik Ślęzak, Takashi Washio, Xiaokang Yang,
and Junsong Yuan

Editorial Board Members

Simone Diniz Junqueira Barbosa
Pontifical Catholic University of Rio de Janeiro (PUC-Rio),
Rio de Janeiro, Brazil
Joaquim Filipe
Polytechnic Institute of Setúbal, Setúbal, Portugal
Ashish Ghosh
Indian Statistical Institute, Kolkata, India
Igor Kotenko
St. Petersburg Institute for Informatics and Automation of the Russian
Academy of Sciences, St. Petersburg, Russia
Lizhu Zhou
Tsinghua University, Beijing, China
More information about this series at http://www.springer.com/series/7899
Ashim Saha Nirmalya Kar
• •

Suman Deb (Eds.)

Advances in Computational
Intelligence, Security and
Internet of Things
Second International Conference, ICCISIoT 2019
Agartala, India, December 13–14, 2019
Proceedings

123
Editors
Ashim Saha Nirmalya Kar
National Institute of Technology Agartala National Institute of Technology Agartala
Agartala, India Agartala, India
Suman Deb
National Institute of Technology Agartala
Agartala, India

ISSN 1865-0929 ISSN 1865-0937 (electronic)

Communications in Computer and Information Science
ISBN 978-981-15-3665-6 ISBN 978-981-15-3666-3 (eBook)
https://doi.org/10.1007/978-981-15-3666-3
© Springer Nature Singapore Pte Ltd. 2020
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are
believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors
give a warranty, expressed or implied, with respect to the material contained herein or for any errors or
omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
 Preface

It is a matter of great privilege to have been tasked with the writing of this preface for
the proceedings of Second International Conference on Computational Intelligence,
Security & IoT (ICCISIoT 2019) organized by the Department of Computer Science
and Engineering of the National Institute of Technology Agartala, India, during
December 13–14, 2019. This mega event covered the major aspects of computational
intelligence, security, and the Internet of Things (IoT) where the scope was not limited
only to various engineering disciplines such as computer science and electronics
engineering researchers but also included researchers from allied communities working
in bioinformatics, computational intelligence for smart power and energy management,
artificial intelligence and cognitive science, modeling and simulation, smart factories
and Industry 4.0, etc. This conference series was initiated with a focus of bringing
together researchers, academics, scientists, and industry experts from around the globe
to share thoughts and promote research activities to derive benefits from the advances
of next-generation computing technologies in the listed areas. It was our wish and hope
that the people participating in the conference would find common ground on which
they could learn from each other, and that the conference would engender future fruitful
scientific activities. Such collaborative endeavors are necessary for the development
of the cutting-edge solutions embodied within our society and its fabrics.
The Program Committee of ICCISIoT 2019 is extremely grateful to the authors who
showed an overwhelming response to the call for papers, submitting over 153 papers in
four tracks. The entire review team (Technical Program Committee members along
with external experts as reviewers) expended tremendous effort to ensure fairness and
consistency during the selection process resulting in the best quality papers being
selected for presentation and publication. It was ensured that every paper received at
least three, and in most cases four, reviews. Checking of similarities and overlaps was
also done based on the international norms and standards. We are very thankful to our
reviewers for their efforts in finalizing the high-quality papers and finally, based on the
recommendation, 37 papers were accepted for publication. The proceedings of the
conference are published as one volume in the Communications in Computer and
Information Science (CCIS) series by Springer, and are also indexed by ISI Proceed-
ings, DBLP, Ulrich’s, EI-Compendex, SCOPUS, Zentralblatt Math, MetaPress, and
Springerlink. We, in our capacity as volume editors, convey our sincere gratitude to
Springer for providing the opportunity to publish the proceedings of ICCISIoT 2019 in
their CCIS series.
The conference program was enriched by a series of keynote presentations and the
keynote speakers included: Prof. Anu A. Gokhale (Illinois State University, USA);
Dr. Ahcene Bounceur (University of Brest, France); Prof. Manoj Sachan (Sant
Longowal Institute of Engineering and Technology, Punjab, India); etc. We are grateful
to them for sharing their insights on their latest research with us.
vi Preface

The Organizing Committee of ICCISIoT 2019 is indebted to Prof. H. K. Sharma,

Director of the National Institute of Technology Agartala and honorary chair, for the
confidence that he invested in us in organizing this international conference. We would
also like to take this opportunity to extend our heartfelt thanks to the honorary chair of
this conference, Prof. Tokunbo Ogunfunmi (Santa Clara University, USA), for his
active involvement from the very beginning until the end of the conference; without his
support, this conference could never have assumed such a successful shape. We
gratefully acknowledge the financial support received from TEQIP-III for organizing
the conference. Special thanks to EasyChair, whose paper submission platform was
used to organize reviews and collate the files for these proceedings.
Special words of appreciation are due to note the enthusiasm of all the faculty, staff,
and students of the Department of Computer Science and Engineering of the National
Institute of Technology Agartala, who organized the conference in a professional
manner. The involvement of faculty coordinators and student volunteers is particularly
praiseworthy in this regard. It is needless to mention the role of the contributors. The
editors would like to take this opportunity to thank the authors of all submitted papers
for not only their hard work but also for considering the conference a viable platform to
showcase some of their latest findings, not to mention their adherence to the deadlines
and patience with the tedious review process. The quality of a refereed volume pri-
marily depends on the expertise and dedication of the reviewers who volunteer with a
smiling face. The editors are further indebted to the Technical Program Committee
members and external reviewers who not only produced excellent reviews but also did
so in a short time frame, in spite of their very busy schedules. Because of their quality
work it was possible to maintain the high academic standard of the proceedings. We
also wish to express our thanks to Kamiya Khatter, Associate Editor, Springer Nature
India, New Delhi, for her help and cooperation.
Last but not least, the editors profusely thank all who directly or indirectly helped us
in making ICCISIoT 2019 a grand success and allowed for the conference to achieve its
goal, academic or otherwise. Happy Reading!

December 2019 Ashim Saha

Nirmalya Kar
Suman Deb
 Organization

Honorary Chairs
Harish Kumar Sharma NIT Agartala, India
Tokunbo Ogunfunmi Santa Clara University, USA

Organizing Chairs
Ashim Saha NIT Agartala, India
Nirmalya Kar NIT Agartala, India
Suman Deb NIT Agartala, India

Technical Program Committee

Alok Ranjan Prusty MSDE, India
Anish Kr. Saha NIT Silchar, India
Anupam Jamatia NIT Agartala, India
Ardhendu Saha NIT Agartala, India
B. Subramaniam NIT Warangal, India
Bidyut Bhattacharya NIT Agartala, India
Dhiman Saha IIT Bhilai, India
Dipankar Debnath IIT Kharagpur, India
Diptendu Bhattacharya NIT Agartala, India
Dwijen Rudrapal NIT Agartala, India
Farid Farahmand Sonama State University, USA
Kaoru Sakatani Nihon Univesity, Japan
Kunal Chakma NIT Agartala, India
Lov Kumar BITS Pilani, India
Mahendra Pratap Singh NIT Karnataka, India
Mridu Sahu NIT Raipur, India
Mrinal Kanti Debbarma NIT Agartala, India
Nikhil Debbarma NIT Agartala, India
P. M. Caledonian Muscat, Oman
Partha Pakrey NIT Silchar, India
R. P. Sharma NIT Agartala, India
Rupa G. Meheta SVNIT Surat, India
S. N. Pradhan NIT Agartala, India
Sadhan Gope Mizoram University, India
Samir S. Sahasrabudhe IIT Mumbai, India
Sergi Trilles Oliver University Jaume I, Spain
Sharmistha Roy Usha Martin University, India
Shital Raut VNIT Nagpur, India
viii Organization

Shivakumar Mathapathi California Polytecnic State University, USA

Shovan Bhoumik IIT Patna, India
Smita Das NIT Agartala, India
Sujoy Deb IIIT Delhi, India
Sutanu Chakraborti IIT Madras, India
Swapan Debbarma NIT Agartala, India
Tanmoy Roy Choudhury KIIT Bhubaneswar, India
Tribid Debbarma NIT Agartala, India

Additional Reviewers

Abhay Narayan Arjun Mane

Abhijit Debbarma Arnab Kumar Maji
Abhik Majumder Arnab Mishra
Abhishek Goyal Arpita Biswas
Abhishek Kumar Gaur Arti Vedula
Abhrajyoti Das Arundhati Das
Abirami S. Ashish Kumar Mourya
Abu Mitul Ashok Kumar Yadav
Abu Wasif Ashutosh Dhar Dwivedi
Ahnaf Hassan Aspin Pabi
Alak Roy Atowar Ul Islam
Alok Ranjan Prusty Ayatullah Faruk Mollah
Amal Htait Aylin Sevik
Amanpreet Kaur Baby Bhattacharya
Amit Agarwal Bao Thai Duong
Amit Doegar Bappaditya Jana
Amit Kumar Roy Bharat Tidke
Amit Roy Bhaskar Biswas
Ananya Roy Bhaskar Mondal
Anil Dubey Bidipto Chakraborty
Anil Mokhade Bidisha Choudhury
Animesh Debnath Biman Kumar Saha Roy
Anindita Jamatia Bishanka Brata Bhowmik
Aniruddha Bhattacharya C. Vyjayanthi
Ankur Gogoi Chandan Kumar
Ankuran Saha Chirag Modi
Anupam Biswas C. K. Bhensdadia
Anupam Das Claudio Moraga
Anurag Aeron D. K. Chaturvedi
Anurag De Deepak Garg
Apurbalal Senapati Dhrubajyoti Bhowmik
Arindam Debnath Diana Teresa Parra
Arindam Sanyal Diwakar Tripathi
 Organization ix

Dolendro Singh Laiphrakpam Naghma Khatoon

Eric Gamess Nitin Shrivastava
Fabio Leccese P. Srinivas Rao
G. U. Vasanthakumar Pakize Erdoğmuş
Gaurav Dhiman Papri Ghosh
Gautam Pal Parthasarathi De
Gautam Srivastava Pinaki Chakraborty
Gopa Bhaumik Pinky Roy
Gouranga Mandal Pradip Kumar Sahu
Goutam Majumder Prakash Mondal
Govind Gupta Prashant Bhardwaj
Gul Shahzad Prashant Pranav
Habila Basumatary Pritam Das
Hussein Abbass Priyadarshini Lamabam
Isha Agarwal Priyanka Mishra
Jabez Christopher Priyanka Sarkar
Jayanta Pal Punit Gupta
Jhunu Debbarma Purbani Kar
John Wang Rajagopal Eswari
Joyeeta Goswami Rajesh Panda
Kebira Azbeg Rajib Chowdhuri
Kiran Ravulakollu Rajiv Kumar Nath
Kishore Kumar Dhar Rakesh Roy
Kishorjit Nongmeikapam Ramesh Kumarasamy
Kusuma S. Ramjee Repaka
Kusworo Adi Ranjita Das
Lalita Kumari Ritushree Narayan
Luciano De Aguiar Monteiro Robert Matilo
Madhu Bala Myneni Rohit Chakraborty
Madhulika Bhatia Rupa Mehata
Mahendra Pratap Singh S. Raghavan
Mamia Saha S. Malliga
Manjeet Singh S. Prasanna
Manju Khari Sadhan Gope
Marimuthu C. Safial Islam Ayon
Marko Bošković Sajal Kanta Das
Md. Milon Islam Sambit Satpathy
Meryem Elmoulat Sanchali Das
Mili Dhar Sangeeta Lal
Miltos Alamaniotis Sanjib Debnath
Mitali Sinha Sanjoy Debbarma
Mohamed I. Ahmed Sanjoy Mitra
Mohammad Alibakhshikenari Santanu Koley
Moumita Majumder Satheesh Patchikolla
Mridu Sahu Saurabh Mehta
Muhammad Masroor Ali Sergio Trilles
x Organization

Sharmistha Majumder Sujata Sinha

Sharmistha Roy Sujit Das
Shirsha Ghosh Supratim Bhattacharya
Shivani Vora Sutanu Chakraborti
Shrabony Adhikary T. P. Singh
Shriram Raut Tanmoy Roy Choudhury
Shubhanshu Kumar Singh Tannistha Pal
Sibansu Mukhopadhyay Tanu Satija
Sigamani Annamary Amutha Tapajyoti Deb
Jeevakumari Tapodhir Acharjee
Sikdar Md. Sultan Askari Tejesh B. S. S.
Sonali Goel Thiago Lopes Trugillo da Silveira
Srinivas Rao Thinagaran Perumal
Subhadeep Bhattacharjee Tokunbo Ogunfuni
Subhankar Shome Tomislav Šekara
Subhojit Dawn Tutan Nama
Subhrajyoti Deb Usha Divakarla
Subrata Tikadar Yady Tatiana Solano-Correa
Sudhansu S. Sahoo Zhengjia Liu
 Contents

Computational Intelligence

An Integrated Image Processing and Deep Learning Approach

for Diabetic Retinopathy Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Anwayee Roy Choudhury, Debayan Bhattacharya, Abir Debnath,
and Ankur Biswas

Early Prediction of Brain Tumor Classification Using Convolution

Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
J. Arunnehru, Ajay Kumar, and J. P. Verma

Routine Statistical Framework to Speculate Kannada Lip Reading. . . . . . . . . 26

M. S. Nandini, Nagappa U. Bhajantri, and Trisiladevi C. Nagavi

Incepting on Language Structures with Phonological and Corpus

Analysis Using Multilingual Computing. . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Bidisha Choudhury and Anupam Das

Morphotactics of Manipuri Verbs: A Finite State Approach . . . . . . . . . . . . . 53

H. Kenny Devi, Gaurav Raghav, S. Bobita, and L. Sarbajit Singh

Proportional Analysis of Feature Extraction for Tamil Handwritten

Characters Using Centroid Based Method . . . . . . . . . . . . . . . . . . . . . . . . . 64
D. Shashi kala and M. P. Indra Gandhi

Mobile Supported Interaction Modeling to Find Engagement

of Acolyte in Live Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Barnita Debnath, Suman Deb, and Pooja Kumari

Contribution of Application of Deep Learning Approaches

on Biomedical Data in the Diagnosis of Neurological Disorders:
A Review on Recent Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Waseem Ahmad Mir, Izharuddin, and Iqra Nissar

Integration of BYOD Technology in Traditional Classroom:

A Statistical Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Pooja Kumari, Suman Deb, and Barnita Debnath

Statistical Analysis of Road Accident Data of UK Using Operational

Intelligence Tool - Splunk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Tapajyoti Deb, Niti Vishwas, and Ashim Saha
xii Contents

Security

Detection of Routing Infrastructure Attack in TCP Connection . . . . . . . . . . . 123

Asem Debala Chanu and Bobby Sharma

Optimizing e-KYC Process Using Distributed Ledger Technology

and Smart Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Hrushikesh Hanbar, Varan Shukla, Chirag Modi, and C. Vyjayanthi

Detecting Fake Image: A Review for Stopping Image Manipulation . . . . . . . 146

Jahanara Islam Lubna and S. M. Abrar Kabir Chowdhury

A View on the Blockchain as a Solution to the Healthcare Industry:

Challenges and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Sharmila, Krista Chaudhary, Umang Kant, and Pramod Kumar

An Approach Towards Encrypting Paired Digits Using Dynamic

Programming and Diffie-Hellman Key Exchange . . . . . . . . . . . . . . . . . . . . 170
Abhilash Kumar Das, Abhrajyoti Das, and Nirmalya Kar

Detecting Malicious Twitter Bots Using Machine Learning . . . . . . . . . . . . . 182

Tanu Satija and Nirmalya Kar

Content Based Routing Algorithm to Improve QoS in IoMT Networks . . . . . 195

Shailja Agnihotri and K. R. Ramkumar

An Improved Chaos Based Medical Image Encryption

Using DNA Encoding Techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Abhrajyoti Das, Abhilash Kumar Das, and Nirmalya Kar

Internet of Things (IoT)

Real Time Adaptive Street Lighting System . . . . . . . . . . . . . . . . . . . . . . . . 223

Sahil Garg, Sahil Ahuja, and Sukhchandan Randhawa

A Review on Coverage and Connectivity Through Random

Deployment of Nodes in Wireless Sensor Network . . . . . . . . . . . . . . . . . . . 240
Rajib Chowdhuri and Mrinal Kanti Deb Barma

Role of Open Hardware and IoT in Home Automation: A Case Study . . . . . . 251
Anurag De, Ashim Saha, and Praveen Kumar

Design and Simulation of Load Balancing Algorithms

in Wireless Mesh Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Priyanka Sarkar

6LoWSD: A Protocol Platform for Interoperability Between SDN and IoT. . . 273
Wanbanker Khongbuh and Goutam Saha
 Contents xiii

Machine Learning Based Efficient Multi-copy Routing

for OppIoT Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
N. N. Srinidhi, C. S. Sagar, S. Deepak Chethan, J. Shreyas,
and S. M. Dilip Kumar

An Open-Hardware Approach for IoT Enabled Smart Meter Monitoring

and Controlling System Using MQTT Protocol. . . . . . . . . . . . . . . . . . . . . . 303
G. M. Madhu, C. Vyjayanthi, and Chirag Modi

Performance of NOMA-MUSA System for 5G Using m-ZCZ Sequences . . . . 318

M. Jayasheela, E. Gomathi, M. Vijila, and Hepzibah A. Christinal

IoT Based Multi-sensor Data Acquisition System for the Application

of Smart Agriculture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
Saurabh Kumar and Nagesh Ch

Context-Oriented User-Centric Search System for the IoT Based

on Fuzzy Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
Santosh Pattar, Sandhya CR, Darshil Vala, Dharamendra Chouhan,
Rajkumar Buyya, Venugopal KR, S. S. Iyengar, and L. M. Patnaik

Implementation of a Smart Embedded System for Passenger Vessel Safety. . . 357

F. M. Javed Mehedi Shamrat, Naimul Islam Nobel, Zarrin Tasnim,
and Razu Ahmed

Extended Track

Empirical Analysis on the Effect of Image Compression and Denoising

Using Different Wavelets on Iris Recognition . . . . . . . . . . . . . . . . . . . . . . . 373
Pranita Baro, Malaya Dutta Borah, and Sushanta Mukhopadhyay

Segmentation of Chart Images Using H-Means Algorithm . . . . . . . . . . . . . . 387

Prerna Mishra, Santosh Kumar, and Mithilesh Kumar Chaube

Hand Vein Biometric Recognition Using Local Binary Pattern . . . . . . . . . . . 398

Koushlendra Kumar Singh, Arun Mishra, Anirudh Deep, Ashav Kumar,
and Bindesh Kumar

Distance Calculation of an Object in a Stereo Vision System . . . . . . . . . . . . 407

Ningthoujam Johny Singh, Wanghengbam Kanan Kumar,
and Kishorjit Nongmeikapam

Comparative Assessment of Consumer Perception on Sports Brands

Using Data Mined from Online Communities . . . . . . . . . . . . . . . . . . . . . . . 417
Vishu Singhvi and Prateek Srivastava
xiv Contents

Analytical Modelling of Internet of Vehicles (IoV) by IoT Cloud

for Dipping Traffic Congestion and Safety Alert: A Review . . . . . . . . . . . . . 430
Sanjib Debnath and Ashim Saha

Object Detection Using Point Feature Matching Based

on SURF Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
Monika Dey, Ashim Saha, and Anurag De

Diabetes Mellitus Prediction Using Ensemble Machine

Learning Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Md. Tanvir Islam, M. Raihan, Sheikh Raihan Islam Akash,
Fahmida Farzana, and Nasrin Aktar

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469

Computational Intelligence
 An Integrated Image Processing
and Deep Learning Approach for Diabetic
Retinopathy Classification

Anwayee Roy Choudhury, Debayan Bhattacharya, Abir Debnath,

and Ankur Biswas(B)

Department of Computer Science and Engineering, Tripura Institute of Technology,

Narsingarh, Agartala 799009, India
[email protected], [email protected],
[email protected], [email protected]

Abstract. Diabetic retinopathy is referred to as eye sight damage and

permanent blindness because of diabetic condition in humans. Diabetic
patients are growing up in numbers every year around the globe. Because
of modern day life style with elevated stress and tension the risk is even
higher. Hence it is crucial for diabetic patients to look for its effect on
other body parts as diabetic condition hit eye more likely and if left
unchecked, may lead to serious eye related issues. But it can be easily
monitored by regular checkups and proper health care and prevented
from further degradation, while an automatic screening system to iden-
tify whether an individual need follow up or referral for supplementary
action to avoid blindness can ease the task of detection at early stages to
a great extent. In this paper a model for automatic detection of diabetic
retinopathy is proposed using low complexity image processing technique
and modified Convolutional Neural Network (CNN) with better accuracy
and precision to help an ophthalmologist through detection of change
in retina features. The proposed model is used to classify the fundus
images into two categories, viz. healthy and infected and tested on Eye-
PACS dataset which obtained classification accuracy of 82% shows the
robustness of the system.

Keywords: Diabetic retinopathy · Image processing · Deep learning ·

Convolutional Neural Network · Fundus image

1 Introduction
Human body is a very complex system and brain control it by secreting various
types of hormone, pancreas is one which secret insulin and its very important
for the body as it restrains the sugar levels in the body. In diabetic condition
body produces less insulin then the required quantity which trigger the damage
to various body parts and the first to get hit is eye. According to a study number
TIT.
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 3–15, 2020.
https://doi.org/10.1007/978-981-15-3666-3_1
4 A. R. Choudhury et al.

of people across the globe suffering from diabetic will rise to double in coming
years and India is already on the top of the list having the largest population of
diabetic patients in the world [1]. Also, the dominance of this ailment has already
grown two times in the precedent 30 years [2] and is only expected to enhance
in the Asian region [3]. Among the diabetic population, about 33% are likely to
be spotted with diabetic retinopathy (DR), an ailment of eye which is chronic
and can step forward to irreversible loss in vision [4]. After duration of 15 years
10% of diabetic patients face visual impairment including diabetic retinopathy
[5]. According to UN report, India was the 140th country in happiness index
[6] and 15% of the population is going through any sort of mental illness [7].
India is one the top 3 country where corporate going people are not happy
about their working hours and do not play much sport. All this makes up a
perfect recipe for diabetic boom. Diabetic retinopathy is a condition in which
a person’s eyes sight gets affected because of internal damage of blood vessels
inside retina. In diabetic retinopathy the main reason for the damage of eye
sight is leaking of blood and other waxy substances inside retina which obstruct
the light from entering. Another phenomenon is fat concentration around fovea,
all these lead to obstruction of light and refection of light from optical disk to
macula. When a person is being a diabetic patient should go for regular eye
checkups since late detection can lead to even complete blindness. According to
the international monitoring agencies the problem of eye sight deterioration is
very common among the population of both developing and developed world.
The main problem is inadequate health care infrastructure in country like India
where a large number of people are not even entitled to basic healthcare. In the
young population the risk is high because of modern life style and negligence.
To ensure good quality of life and ensured economic growth the population
must be provided with good health care facilities, so it is mandatory to develop
systems and put up adequate infrastructure to deal with this situation in future.
As a developing nation India must achieve all these with limited resources and
frugal innovation. Diabetic retinopathy detection system proposed in this paper
is one of such initiative that will be backed by image processing and deep learn-
ing using existing CNN architecture with crucial modifications. The pro-posed
system will be trained up to a stage where it will be capable of identifying good
and affected retina. The concept that has been proposed for the detection of
diabetic retinopathy detection can also be replicated in other eye related prob-
lems. The proposed methodology is applied on EyePACS [8] LLC dataset and
obtained 83.32% on training set and 74% classification accuracy on testing set in
least possible time that shows the robustness of the methodology. The remain-
ing paper is organized as follows: Sect. 2 demonstrates the previous work, while
Sect. 3 provides necessary details of data used and methodology. Experimen-
tal details with evaluations are shown in Sect. 4 and finally Sect. 5 presents the
concluding remarks.
 An Integrated IP and DL Approach for DRC 5

2 Literature Survey
There are presently many developed systems to detect and diagnose diabetic
retinopathy and most of these existing systems which are based on supervised
learning classifier are discussed in this section:
Previous works on image processing techniques have achieved good results in
identification of one specific feature used in the detection of this ailment [9–11].
For other features, algorithms involving KNN, [12,13] support vector machines
(SVM) [14], and ensemble-based system [15] successfully achieved sensitivity and
specificity of 90% range, but involves huge preprocessing.
Previous CNN studies also achieved promising results in terms of sensitiv-
ity and specificity for binary classification on private datasets [16,17]. Rahim
et al. [18] utilized binary decision tree, K nearest algorithm, Support vector
and random forest, all these are self teaching system and they had done feature
extraction using MATLAB and SPSS. These systems are incapable of process-
ing big size and complex data thus bringing down the accuracy. The proposed
system does not require these steps as CNN classifier is used which follows rein-
forcement learning process and robust enough to handle big size and complex
data. Verma et al. [19] implemented technique of hybrid classification which is
designed using neural network and fuzzy logical system. In the proposed tech-
nique, texture, shape and statistical feature extraction was carried out which is
time consuming.

3 Materials and Methods

Data Collection and Segrigation

The EyePACs image datasets which is a free source data repository for web-based
diabetic retinopathy screening system is utilized for executing the methodology.
After the initial collection of data they are segregated into two classes namely
symptom and no-symptom as shown in Fig. 1. The dataset consist of a large set
of high resolution retina images which are taken under a variety of conditions.
Every single image were marked with a label either right or left with patient id
which are rated on a scale of 0 to 4 on the basis of severity level of the diabetic
retinopathy. The scale is as follows:
Level 0 - No Diabetic Retinopathy (DR), 1 – Mild DR, 2 – Moderate DR,
3 – Severe DR, 4 - Proliferative DR
Methodology
This section describes the stages involved in the proposed methodology. The
following stages are involved as represented in Fig. 2.
6 A. R. Choudhury et al.

Fig. 1. Number of image used for classification of retinal images with symptoms of
diabetic retinopathy and healthy

Fig. 2. Overall flowchart of the proposed system

3.1 Load Images

The data set containing 956 images is loaded in the system for classification. In
this system, an image will be either classified as normal or infected. The dataset
on load is shown in Fig. 3.
 An Integrated IP and DL Approach for DRC 7

Fig. 3. Original fundus image of retina (a), (b) with no symptoms of diabetic retinopa-
thy damage, (c), (d) with symptoms of diabetic retinopathy damage

3.2 Gray Scale Conversion

All the images used for the training and test cases were of higher resolutions and
because of RGB configuration the complexity was worse so all got converted to
grey scale format as shown in Fig. 4 for better efficiency. These images were also
put to normalization for keeping the range of the images from 0.0–1.0. using the
formulae,
X = X/255.0 (1)

3.3 Image Resizing

As the images in the EyePACS data set are of higher dimensions, it is resized and
downsampled to 244 × 244 pixel that significantly reduced the required memory
size preserving the intricate features of the image. The downsampled image is
represented in Fig. 5.

3.4 Classification Through CNN

The existing CNN architecture with certain modification as a part of deep learn-
ing method has been utilized and there are ten layers in proposed modified CNN
architecture and shown in Fig. 6.

Architecture of CNN

The model is a simple Sequential model and the window size is 3 × 3. The differ-
ent layers such as Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D
are used. At the end of all the layers a fully connected Dense layer is used. Flat-
ten layer is used before feeding data through the final Dense layer because the
Dense layer accepts one dimensional data. The Conv2D layer used in the model
8 A. R. Choudhury et al.

Fig. 4. Images after grey scale conversion and normalization

Fig. 5. Image after resizing

 An Integrated IP and DL Approach for DRC 9

is of 64 units. The Activation layer used is rectified linear (relu). Pool size of the
MaxPooling layer used is 2 × 2. Batch size is determined based on the number
of inputs as the numbers of images are 956 so, a batch size of 32 is suitable. The
detailed architecture of the model is shown in Fig. 7. Over fitting is one of the
most common issues in neural networks and it can be solved by reducing the
complexity of the neural network, i.e., by changing the values of weights and the
number of weights.

Epochs

Epoch in neural network expressions is one forward pass and one backward pass
of all the training set. Total epochs describe the number of times the algorithm
will work through the complete dataset of training. In every epoch the training
samples has a prospect to revise the parameters of model.

Training

Initially, the CNN was trained on 60% of total dataset to facilitate the train-
ing to reach a significant level. The training is essential for achieving a fast
classification outcome with no wastage of training time. The entire dataset was
repeatedly trained using 2, 5, 10, 15, 20, 25 epochs. The learning rate was kept
low for 2 and 5 epochs, later it was increased after the weights were stabilized.
Epochs while training the model were quite fast as the images were converted
to grayscale and resized to 224 × 224. One epoch took approximately 30 s with a
time of 51 ms/step which is very fast when compared to a RGB image. The total
training time for 15 epochs was roughly 10 min. In the beginning, 413 images
were taken which gave accuracy less than 60%. So the number of images were
increased which took the accuracy over 70%.

Testing

For testing 40% of the total number of images are taken in the validation set
which has given pretty good results. The images can be tested by loading this
saved model (model.h2). Saving the model is necessary as the weights and the
topology of the model need to be saved for future testing. The model is not ‘json
serializable’ it can be made ‘json serializable’ according to the desire of the user.
A single image can also be tested by the help of a prediction function which
takes the file path as an argument and returns a Boolean value i.e., TRUE or
FALSE. It also prints the category of the image. It can also print the index value
(0/1). It is completely based on the choice of the user.
10 A. R. Choudhury et al.

Fig. 6. Layers of the CNN model

 An Integrated IP and DL Approach for DRC 11

Fig. 7. Summary of the model

3.5 Classification Output

The output will be represented as “Infected” OR Normal as “No Symptom”

which is demonstrated in Result section.

4 Results

This section shows the illustration of the proposed Diabetic Retinopathy Detec-
tion system that will examine the images and give prognosis of the eye. And
the whole prediction process will be based on the diabetic retinopathy detection
module which is the most important module that comes after data resizing of
various fundus images. The system will predict severity of the eye with at least
1000 to 2000 or more fundus images. For the prediction process the image data
sets were taken from EyePACS where the data sets are available with the severity
level numbers. We have segregated the image data set into two sub image data
set namely healthy and infected. Convolution Neural Network classifier tech-
nique has been used for classification of the image data. After analyzing of the
images it has been observed that the system gives different accuracy for different
epoch. The highest accuracy given by our proposed system is all most 83.32%
for 25 epoch during training phase and during testing phase of the machine it
has shown accuracy rate near to 74% for 25 epoch. The number of epoch here
means how many times the system is trained or tested with a train dataset and
validation dataset respectively. The details of Epoch are shown in Fig. 8.
12 A. R. Choudhury et al.

Fig. 8. Epochs while training the model

Various comparison of Epochs against accuracy and loss are represented in

Fig. 9. The Epoch- Accuracy graph while training with various number of epochs
i.e., 2, 5, 10, 15, 25 is shown in Fig. 10.

4.1 Prediction

The model is to be loaded before testing it. Testing can be performed on a single
or multiple images. In this figure testing of a single image is shown. An image
can be tested by calling the function which requires the file path of the image
directory (for multiple images) and for any single image it takes the name of the
image as an argument. It prints the output as integer value or categorical value
as shown in the code below:

prediction = model.predict([(’10 left.jpeg’)])

print(CATEGORIES[int(prediction[0][0])])
OUTPUT: No Symptom

4.2 Time Complexity Analysis

The proposed system was implemented on Windows GPU environment with intel
CPU (i5, 2.2 GHz) using Python, Keras and Tensorflow. The average training
time for 956 images for each epoch is 30 s.
 An Integrated IP and DL Approach for DRC 13

Fig. 9. (a) Epoch vs accuracy (b) Epoch vs loss of final model (model.h2)

Fig. 10. Epoch vs accuracy graph (epochs- 2, 5, 10, 15, 25)

14 A. R. Choudhury et al.

5 Conclusions
Development in medical sciences is very important for tackling the problems of
human health issues and for this Information technology had played an impor-
tant role in last few decades for providing better procedures and innovative
ideas for mitigation of these problems. Currently most of the advancements in
the field of medical science because of advancements in the technologies of image
processing, machine learning and artificial intelligence. If we look into our past
the situation in the medieval period and the 20th century was tough in predicting
damages of internal organs. Prediction of diseases used to be done manually with
very less use of Holistic data but with advancement in data collection electronics
and more better computer programs which help us in gathering more and more
brut information about human internal organs, that helps in predicting more
accurately about any future health condition and also help us in segregating
and finding out the main cause which help in dealing with it more effectively.
While doing this work many project papers are explored and observed that a lot
of research is already going on across the globe on medical image analysis with
the help of artificial intelligence and machine learning and advanced electronic
devices. Doctors are using 3D modelling for orthopaedic surgeries and finding
out tumours and other anomalies in human body.

5.1 Future Scope

In future the existing system can be used to classify the input fundus images
in four classes with different stage of damage and healthy with better accuracy.
There can be a web based portal where any person who have fundus images of
retina can check for signs of diabetic retinopathy no matter from which corner
of world. This system can be used as an application on any device it can be a
pc or a smart phone with the help of a fundus image capture device can work
as any other bulky system.
In this paper, full concentration is on a very small topic diabetic retinopathy,
whereas there are many more eye related issues other than DR which may happen
because of diabetic condition. The goal is to automate the process with a slick
system in order to make it more cost effective and less complex to integrate.
The system would assist ophthalmologists to perform suitable classification by
monitoring the requisite result for early diagnosis.

References
1. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3920109
2. WHO. Global report on diabetes (2016)
3. Chan, J.C., et al.: Diabetes in Asia: epidemiology, risk factors, and pathophysiology.
JAMA 301(20), 2129–2140 (2009)
4. Congdon, N.G., Friedman, D.S., Lietman, T.: Important causes of visual impair-
ment in the world today. JAMA 290(15), 2057–2060 (2003)
 An Integrated IP and DL Approach for DRC 15

5. Verma, K., Deep, P., Ramakrishnan, A.G.: Detection and classification of diabetic
retinopathy using retinal images. In: 2011 Annual IEEE India Conference, Hyder-
abad, India. https://doi.org/10.1109/INDCON.2011.6139346
6. https://m.hindustantimes.com/india-news/indian-ranked-140th-among-lowest-
20nations-on-un-world-happiness-report/
7. National Mental Health Survey of India, 2015–2016, Supported by Ministry of
Health and Family Welfare Government of India. http://www.nimhans.ac.in
8. Cuadros, J., Bresnick, G.: EyePACS: an adaptable telemedicine system for diabetic
reti-nopathy screening. J. Diab. Sci. Technol. 3(3), 509–516 (2009)
9. Mookiah, M., Acharya, U., Chua, C., Lim, C., Ng, E., Laude, A.: Computer-aided
diagnosis of diabetic retinopathy: a review. Comput. Biolo. Med. 43(12), 2136–
2155 (2013)
10. Niemeijer, M., et al.: Retinopathy online challenge: auto-matic detection of microa-
neurysms in digital color fundus photographs. IEEE Trans. Med. Imaging 29(1),
185–195 (2010)
11. Wang, S., Tang, H.L., Hu, Y., Sanei, S., Saleh, G.M., Peto, T., et al.: Localizing
microaneurysms in fundus images through singular spectrum analysis. IEEE Trans.
Biomed. Eng. 64(5), 990–1002 (2017)
12. Abramoff, M.D., et al.: Automated early detection of diabetic retinopathy. Oph-
thalmology 117(6), 1147–1154 (2010)
13. Quellec, G., Lamard, M., Josselin, P.M., Cazuguel, G., Cochener, B., Roux, C.:
Optimal wavelet transform for the detection of microaneurysms in retina pho-
tographs. IEEE Trans. Med. Imaging 27(9), 1230–1241 (2008)
14. Noronha, K., Acharya, U.R.: Decision support system for diabetic retinopathy
using discrete wavelet transform. Proc. Inst. Mech. Eng. Part H J. Eng. Med.
227(3), 251–261 (2013)
15. Antal, B., Hajdu, A.: An ensemble-based system for microaneurysm detection and
dia-betic retinopathy grading. IEEE Trans. Biomed. Eng. 59(6), 1720–1726 (2012)
16. Gulshan, V., et al.: Development and validation of a deep learning algorithm for
detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22),
2402–2410 (2016)
17. Gargeya, R., Leng, T.: Automated identification of diabetic retinopathy using deep
learning. Ophthalmology 124(7), 962–969 (2017)
18. Rahim, S.S., Palade, V., Shuttleworth, J., Jayne, C.: Automatic screening and
classification of diabetic retinopathy fundus images. In: Mladenov, V., Jayne, C.,
Iliadis, L. (eds.) EANN 2014. CCIS, vol. 459, pp. 113–122. Springer, Cham (2014).
https://doi.org/10.1007/978-3-319-11071-4 11
19. Verma, K., Deep, P., Ramakrishnan, A.G.: Detection and Classification of Diabetic
Retinopathy Using Retinal Images
 Early Prediction of Brain Tumor
Classification Using Convolution
Neural Networks

J. Arunnehru1(B) , Ajay Kumar1 , and J. P. Verma2

1
Department of Computer Science and Engineering,
SRM Institute of Science and Technology, Chennai, India
[email protected], [email protected]
2
Department of Electronics and Communication Engineering,
SRM Institute of Science and Technology, Modinagar, India
[email protected]

Abstract. Automatic brain tumor classification of tissue types plays

a significant task in computer-aided medical diagnosis. In recent years,
classification of brain tumors types like meningioma (T1), glioma (T2),
and pituitary tumor (T3). Convolution Neural Networks (CNN), which
trains the image into increasingly sub-dividing as filter blocks for the
fine-tune of feature extraction from each sub-region, exhibits excellent
results and successfully used for object detection and classification. In
this paper, we present an approach to improve tumor detection and clas-
sification performance. Initially, the tumor area is clustered with the
fuzzy c-means algorithm for discovering the surrounded tumor tissues
and also gives important clues for tumor types. Second, the Canny edge
detection applied for the tumor region. Third, the spectral residual for
saliency map from the tumor region. Finally, we combine all three areas
into one representation for CNN training and testing on a large dataset,
which gives accuracies of 91.40%. These experimental results show that
the proposed method is realistic and useful for the classification of brain
tumors types.

Keywords: Convolution Neural Networks · MRI · Classification ·

Brain tumor

1 Introduction

The report released by the World Health Organization (WHO) classifies the
tumor grade by four types where grade I to grade IV and tumor types are benign
and malignant. The class III and IV are high-level grade tumors which grow very
fast, and it spreads to other parts of the brain and spinal cords while grade I and
II are low-level grade tumors. Detecting the tumors and classifying at an early
stage is the challenging task in medical science, which helps to give treatments
the for patients at the beginning stage and decides for suitable therapy or surgery
can be determined.
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 16–25, 2020.
https://doi.org/10.1007/978-981-15-3666-3_2
 Early Prediction of Brain Tumor 17

Nowadays, in the therapeutic universe, the rapid growing medicinal imag-

ing applications are assuming a computer-aided clinical tool for human groups.
Uniquely Magnetic Resonance Angiography, Position discharge tomography,
electronic tomography visualization are imagined by the method for the pro-
jection of greatest strength. Medicinal image representation preparing is viable
for its computational clarity and capacity. Since it is three-dimensional (3-D)
in cleverness, it shows precisely the vascular information of intrigue. The first
strides of therapeutic image handling are used to diminish the differentiation to-
commotion proportion (CNR), high recurrence parts of the images, and create
a weak representation of little vessels.
Tumor detection and classification from the human brain is the most challeng-
ing task and time-consuming process in medical image processing. MRI (Mag-
netic Resonance Imaging) is a medical imagining technique primarily used by
radiologist doctors for visualization of internal organs of the human body with-
out any surgery. Various approaches and methods have been developed for Brain
tumor detection and classification. A tumor is a small tissue that grows abnor-
mal in the brain or central spine that can obstruct regular brain function. A
primary brain tumor initializes from increasing cells in the brain. Another part
of the body mainly produces secondary tumors. MRI is the most appreciated
modality of brain tumor imaging for the proper diagnosis of patients who have
a brain tumor. Therefore, the main focus of this work is to develop an efficient
learning algorithm and accurate brain tumor detection system using deep neural
networks. Proper identification of the boundary line in tumor image is the cen-
tral issue because this is used to identify the medical treatment model and play
a vital role in analyzing the tissue state surrounding the tumor area to check up
the arising of unwanted process.
This research proposes a methodology to detect the brain tumor types (T1,
T2, and T3) from MRI with the help of deep neural networks (CNN) with a min-
imum number of configurable parameters. Initially, the fuzzy c-means algorithm
is applied for tumor region clustering for finding the tumor surrounding tissues,
which offer significant cues of the tumor types. Second, the Canny edge detection
used for the tumor region. Third, the spectral residual for saliency map from the
tumor region. Finally, we fuse all three output regions into one representation
for CNN training and testing on a large dataset called T1-weighted CE-MRI.

2 Related Work
The MRI image is initially pre-processed to remove signal noise and balance for
variations in intensity brain tissues [1]. The pre-processed image is then separated
into different regions of interest (ROI) - areas with a likely tumor. The segmented
image extracts various types of features. The number of extracted features is
large, which in the following stage increases processing time and storage needs.
Therefore, it retains only the most useful features [2]. The characteristics will
then be sent to the classification process to determine if they contain a tumor
or not. Classification performance is then evaluated for accuracy and processing
time over all these images.
18 J. Arunnehru et al.

The segmentation techniques like Fuzzy C Means (FCM) and Active Contour
Model are adopted as there are no image sets with ground truths [3]. The spatial
FCM algorithm [4] is used due to its sensitivity. Among the other available
segmentation methods, the mathematical morphological process is the mainly
efficient method for extracting tumor area more accurately with less processing
time [5]. Different studies use Gabor wavelet features, but these features are
partial to capturing only local MRI image structures such as regularity, location,
and direction. Similarly, many authors suggested using statistical features of
first-order, co-occurrence gray level matrix features, and matrix features of the
gray level run length [6].
In recent deep learning methods, CNNs are actively used in object clas-
sification challenges, which increase their popularity among computer vision
researchers [7]. In connection to conventional classification approaches, where
manually extracted features are fed into the classifier for training and testing,
CNNs learn low, mid, and high-level features automatically from the previous
layers. In our work, CNNs architecture design mainly focus on brain tumor
identification rather than conventional feature extraction using image process-
ing approaches. In [8], 3D CNN architecture proposed for classifying the multi-
model MRI glioma region detection. Initially, the input image is divided into 3D
patches and are called voxels cuboid, obtained from different MRI brain sam-
ples, where 3D sections consist of spatial intensity information. The methods
like multi-model properties (T1, T2, and Flair) working on pixel classification
[9] by semi-supervised, supervised, and unsupervised.
In general, the unsupervised approach applies to cluster for finding the simi-
larity distance measure between features and voxels. However, these approaches
do not consider the spatial correlation. In addition to that, semi-supervised
approaches like Markov Random Fields (MRF) give province to this problem
by diminishing the voxel noises [10,11]. Gering et al. [12] present a framework to
detect the brain tumor using multi-layer MRF, whereas, in this framework, each
layer targets the intensity information on inter-structure properties and relation-
ships for justifying precinct coherence. In [13], authors present Convolutional
Neural Networks (CNN) architecture for tumor segmentation using supervise
pixel classification [14] that can extract more non-linear features without para-
metric distribution hypothesis. In addition to that, the authors [7] uses voxel
patch extracted from the image with multi-channel intensity to CNN for classi-
fication. Havaei et al. [15] designed a double stream layered networks, to obtain
more contextual features from the input patches, which output feature maps
connected to the second network simultaneously.

3 Proposed Methods

In the initial step for the tumor classification, the input MRI image preprocessed
with a median filter of 3 × 3. After preprocessing the fuzzy c-means clustering,
edge detection, and spectral residual applied for tumor region segmentation.
Finally, we fuse all three regions with three channels of the RGB image into
 Early Prediction of Brain Tumor 19

one representation for CNN training and testing on a large dataset called T1-
weighted CE-MRI. The following subsections explain about fuzzy c-means clus-
tering algorithm, edge detection, and spectral residual for saliency mapping and
CNN for classification. Figure 1 gives an overview of the proposed approach.

Fig. 1. Overview of the proposed approach

3.1 Fuzzy C-Means Clustering Algorithm

The main idea of the algorithm is to obtain an iterating clustering that induces
the optimal n partition by reducing the weighted within-group sum of squared
error objective function JF :
m 
t p

JF = (uik ) q 2 (xk vi ) (1)
k=1 i=1

Where x = x1 , x2 , ...xn < B is the data in the q-dimensional vector repre-

sentation is the number of data present, t is the number of clusters is 3, Uik
is the degree of each membership in xk in the initial group is the exponent of
weighting of each fuzzy group, vi is the prototype of the middle cluster is the
measure within the objects being present in the xk and the cluster middle vi .
The iterative process is as follows.

1. Set values of t, p and α.

2. Initialize the fuzzy partition matrix M = Uik .
3. Set the loop counter g = 0.
(g)
4. Calculate the t cluster centers {vi } with M (g) :
p

m
(g)
(uik ) xk
k=1
vig = m
(g) p
(uik )
k=1
20 J. Arunnehru et al.

5. Calculate the membership U (g+1) . For k = 1 to m, calculate the following:

Ik = {i |1 ≤ i ≤ t, dik = xk − vi  = 0|},

(g+1) 1
– IF: IK = p, Then Uik = p 
 aik
 2
p−1
ajk
j=1
(g+1)  g+1
– ELSE: Uik = 0 for i ∈
/ I and Uik = 1; next k.
iαIK
 (g) 
6. IF: U − U (g+1)  ≺ α, stop. Otherwise set g = g + 1 and go to step 4.

3.2 Canny Edge Detection

Canny edge detection is a multistage edge detection algorithm. There are various
steps involved in the detection process. The steps are preprocessing, calculating
gradients, non-maximum suppression, and threshold with hysteresis. The key
parameters used are the upper threshold and the lower threshold. The edge
detectors are susceptible to noise. The first step is to remove the noise that takes
place with the help of Gaussian filters. The next stage of the edge detector is
to obtain the image gradient intensity, which represents the horizontal direction
and vertical direction, respectively is given in Eqs. 2 and 3.

EdgeGradient(Eg ) = Ep2 + Eq2 (2)
 
Eq
Angle = tan−1 (3)
Er
The unwanted pixels are removed by tracing the complete image. The
unwanted pixels eliminated by considering the pixels that do not constitute the
edge. For obtaining the Hysteresis thresholding, we require the max Threshold
and min Threshold. The edge which has an intensity gradient greater then max
Threshold and lesser than min Threshold are discarded because they are sure to
be non-edges.

3.3 Saliency Mapping

Bottom-up saliency maps are calculated to model the saliency using natural
statistics (SUN). SUN provides the bottom is saliency as P (E)−1 , which E
denotes the independent component analysis. The features of E has been more
interdependent by Independent component Analysis, SUN methods T (E) as a
result of similar dimensional distributors: where ei is the ith variable in vector e.
The similar dimensional distributional is modeled using the generalized Gaussian
distribution (GGD). The sparse visual feature chooses Laplace and Gaussian
distribution to make it the right choice of model [16].
 iσ
σi −e i
P (ei ) = −1 exp
(4)
2pi Γ (σi ) ρi
 Early Prediction of Brain Tumor 21

The distribution methodology is the purpose of obtaining the varied pro-

cessed models that are being inputted for the functional relativity among every
corresponding attribute in the function. The exponential term helps in obtaining
the segmented value that is stated in the functional measures. The standard up-
gradation of the system takes place in every segment of the processed model. The
sequential process which aids in obtaining the properly segmented mapping for
every numerical data. Where σi > o is the shape dimension, ρi > o is the mea-
suring parameter. For which the ICA constraint, similar dimensional Gaussian
distribution placed with the help of segments. The GGD data calculated with
the help of the algorithm defined. It is mainly used to structure with varying
ability of Independent component Analysis constrains by Gaussian distribution
place to their ability. The features are improved using e :
−αi −1
α(eαi
i pi αi )
e = (5)
Γ (σi−1 )
The overall manipulation of the data takes place with the help of the α. The
improved information standardized into the particular standards for obtaining
the simplified structure in securing the variables. The number of inputs varies
the complete iteration, which takes while building the system. The magnitude
of the structure obtained is with corresponds to the correlation equation and
simplified data collected with the renowned input data.

3.4 Convolutional Neural Network (CNN)

This algorithm is most awaiting in the field of machine learning. It is used for
feature extraction. It takes the input data directly from the images, video, text,
or sound. The features are being learned to recognize the object, faces, and
scenarios. This learning process takes place with the help of the patterns instead
of manual feature extraction. Many applications like face recognition and auto-
driven cars are deeply dependent on the CNN algorithm. The significant benefits
of CNN are the manual feature extraction can be eliminated, and features are
learned directly using CNN [17,18]. The new recognition tasks are retained using
CNN’s, and it allows you to overlap on the existing networks. It provides an
optimized architecture for image recognition and feature extraction. CNN has
evolved so much that it used as the key technology for automated cars and the
facial recognition process. The multi-layered process in which each layer is used
for learning and detecting different patterns from the input. The output data
obtained from one layer imported as the input to the next layer.
Just like the other neural networks CNN also contains the input layer, hidden
layer, and output layer. These layers alter the data that are inputted in such a
way the features can be learned and trained. The most commonly used features
are convolution, ReLU, and pooling. The convolution used as a set of filters
and each filter activates the features from the images. The Rectified Linear Unit
(ReLU) used for mapping the negative values to zero. It keeps the favorable
costs, and this layer used for the faster and most effective training process.
22 J. Arunnehru et al.

The simplification of the output takes place in the pooling layer, thereby the
number of parameters can be reduced. After the operation of the feature learning
process, the classification takes place. After the output layer, the next layer
obtains the output of the number of classes that are being present.
The CNN architecture contains the final layer as the classification layer to
provide the classification output. The CNN training from scratch or pre-trained
model can be performed using the Matlab. The process of training the system
from scratch requires the number of layers and filters, along with the necessary
parameters. The training to be accurate it should contain a large dataset, and
it consumes more time.
CNN derived from the visual cortex, which contains the arrangement of cells,
both the simple and complex cells. The working of the cell depends on the sub-
regions(receptive field). Similarly, in the case of CNN, the neurons are connected
with the sub-regions of the layer. The neurons become unresponsive once it is
out of the sub-regions in the image. Only CNN can produce spatially correlated
output. The neurons arranged in 3D manner for every cell. For the 3D model
as the input, the Convolution layer produces the 3D output, whereas, in the
hidden layer, it learns the non-linear combinations of the original data. This
process is known as feature extraction. Figure 2 gives a detailed overview, and
Table 1 contains the CNN model parameters for training the CNN.

Fig. 2. Proposed CNN architecture

4 Experimental Results

The proposed model was trained and tested on a system Intel i7 processor with
8GB RAM and NVIDIA 1050 Ti GPU with 4 GB of memory. MRI dataset con-
sists of T1 - 1416, T2 - 1426, and T3 - 1483 images were collected from internet
sources. The CNN feature extraction is the fastest way to use the representa-
tional power of trained deep networks. In our proposed CNN network has four
convolutional layers and one fully connected layer, as shown in Fig. 2 and Table 1
contains the CNN model parameters for training the CNN. In our experiment,
the input MRI image dataset divided into 70% training and 30% for test data.
 Early Prediction of Brain Tumor 23

Table 1. CNN parameters: 19 × 1 layer array with layers description

# Layer label Layer type Description

1 ‘imageinput’ Image input 128 × 128 × 3 images with ‘zerocenter’ normalization
2 ‘conv 1’ Convolution 8 – 3 × 3 × 3 convolutions with stride [1 1] and padding ‘same’
3 ‘batchnorm 1’ Batch normalization Batch normalization with 8 channels
4 ‘relu 1’ ReLU ReLU
5 ‘maxpool 1’ Max pooling 2 × 2 max pooling with stride [2 2] and padding [0 0 0 0]
6 ‘conv 2’ Convolution 16 – 3 × 3 × 8 convolutions with stride [1 1] and padding ‘same’
7 ‘batchnorm 2’ Batch normalization Batch normalization with 16 channels
8 ‘relu 2’ ReLU ReLU
9 ‘maxpool 2’ Max Pooling 2 × 2 max pooling with stride [2 2] and padding [0 0 0 0]
10 ‘conv 3’ Convolution 32 – 3 × 3 × 16 convolutions with stride [1 1] and padding ‘same’
11 ‘batchnorm 3’ Batch normalization Batch normalization with 32 channels
12 ‘relu 3’ ReLU ReLU
13 ‘maxpool 3’ Max pooling 2 × 2 max pooling with stride [2 2] and padding [0 0 0 0]
14 ‘conv 4’ Convolution 64 – 3 × 3 × 32 convolutions with stride [1 1] and padding ‘same’
15 ‘batchnorm 4’ Batch normalization Batch normalization with 64 channels
16 ‘relu 4’ ReLU ReLU
17 ‘fc’ Fully Connected 3 – fully connected layer
18 ‘softmax’ Softmax softmax
19 ‘classoutput’ Classification output crossentropyex with ‘T1’ and 2 other classes

Table 2. Training and validation results obtained from CNN

Epoch Iteration Time elapsed Mini-batch Validation Mini-batch Validation Base learning
(hh:mm:ss) accuracy accuracy loss loss rate
1 1 0:00:14 35.16% 39.85% 1.2186 4.0171 0.01
2 30 0:01:57 68.75% 74.94% 1.2068 0.9769 0.01
3 50 0:03:08 83.59% 77.28% 0.4608 0.6521 0.01
4 70 0:04:18 87.50% 81.51% 0.3302 0.4417 0.01
5 110 0:06:38 91.41% 86.57% 0.2069 0.3172 0.01
6 130 0:07:47 88.28% 86.19% 0.2568 0.3256 0.01
7 150 0:08:54 86.72% 85.74% 0.2995 0.3401 0.01
8 170 0:10:00 89.84% 89.13% 0.2387 0.2665 0.01
9 190 0:11:08 95.31% 87.92% 0.1976 0.3092 0.01
10 210 0:12:17 94.53% 89.58% 0.1793 0.2569 0.01
11 240 0:13:57 92.97% 90.04% 0.1505 0.2539 0.01
12 260 0:15:05 96.09% 90.79% 0.1359 0.2324 0.01
13 280 0:16:11 92.97% 90.72% 0.1945 0.2247 0.01
14 300 0:17:18 97.66% 90.34% 0.1051 0.2291 0.01
15 330 0:18:59 97.66% 91.85% 0.1259 0.2137 0.01
16 350 0:20:06 99.22% 91.92% 0.0672 0.2102 0.01
17 370 0:21:13 96.09% 91.09% 0.1456 0.2619 0.01
18 400 0:22:55 97.66% 91.85% 0.0828 0.2044 0.01
19 420 0:24:01 98.44% 92.53% 0.0827 0.1905 0.01
20 440 0:25:08 96.88% 91.40% 0.0951 0.2179 0.01
24 J. Arunnehru et al.

Table 3. Confusion matrix (%) obtained from CNN classifier

Class T1 T2 T3
T1 94.65 4.94 0.4
T2 5.91 93.79 0.3
T3 0.58 0.1 99.33

The CNN learns image features from MRI dataset, and use those features to
train an image classifier. The CNN requires input images of size 128 × 128 × 3.
Images are normalized. Table 2 shows the performance values obtained at each
epoch. The total time taken by the training phase of the MRI tumor classifica-
tion system is 25 min 8 s. We had a total of 20 epochs with 22 iterations per
epoch. The total numbers of iterations were 440 iterations with a learning rate
of 0.01, which was constant. This experiment resulted in a validation accuracy
of 91.40% and the corresponding confusion matrix of the CNN shown in Table 3
where correct predictions defined in the diagonal and the tumor classes like T3 is
recognized well and reached an accuracy of 99%. Also, T2 and T1 was a misclas-
sification rate of 4% to 5%. The experimental results show that CNN performs
well in tumor classification.

5 Conclusion
In this research work, we have used brain MR images, classified into brain tumors
types like meningioma (T1), glioma (T2), and pituitary tumor (T3). Next, we
have used the fuzzy c-means algorithm for finding cancer surrounding tissues
followed by canny edge detection and spectral residual for saliency map for
tumor region identification. Convolution Neural Networks (CNNs) used for the
classification of tumors from brain MRI images. From the experimental results,
it clearly expressed that the classification accuracy reported as 91.40% of brain
tumor types and also shows that our proposed approach is fast and accurate
when compared to the manual detection carried out by clinical experts.

References
1. Aslam, A., Khan, E., Beg, M.S.: Improved edge detection algorithm for brain tumor
segmentation. Procedia Comput. Sci. 58, 430–437 (2015)
2. Shanthakumar, P., Ganesh Kumar, P.: Computer aided brain tumor detection
system using watershed segmentation techniques. Int. J. Imaging Syst. Technol.
25(4), 297–301 (2015)
3. Natteshan, N.V.S., Angel Arul Jothi, J.: Automatic classification of brain MRI
images using SVM and neural network classifiers. In: El-Alfy, E.-S.M., Thampi,
S.M., Takagi, H., Piramuthu, S., Hanne, T. (eds.) Advances in Intelligent Infor-
matics. AISC, vol. 320, pp. 19–30. Springer, Cham (2015). https://doi.org/10.
1007/978-3-319-11218-3 3
 Early Prediction of Brain Tumor 25

4. Dandıl, E., Çakıroğlu, M., Ekşi, Z.: Computer-aided diagnosis of malign and benign
brain tumors on MR images. In: Bogdanova, A.M., Gjorgjevikj, D. (eds.) ICT
Innovations 2014. AISC, vol. 311, pp. 157–166. Springer, Cham (2015). https://
doi.org/10.1007/978-3-319-09879-1 16
5. Yazdani, S., Yusof, R., Karimian, A., Riazi, A.H., Bennamoun, M.: A unified frame-
work for brain segmentation in MR images. Comput. Math. Methods Med. 2015,
829893 (2015)
6. Nabizadeh, N., Kubat, M.: Brain tumors detection and segmentation in MR
images: Gabor wavelet vs. statistical features. Comput. Electr. Eng. 45, 286–301
(2015)
7. Zikic, D., Ioannou, Y., Brown, M. and Criminisi, A.: Segmentation of brain tumor
tissues with convolutional neural networks. In: Proceedings MICCAI-BRATS, pp.
36–39 (2014)
8. Urban, G., Bendszus, M., Hamprecht, F., Kleesiek, J.: Multi-modal brain tumor
segmentation using deep convolutional neural networks. MICCAI BraTS (Brain
Tumor Segmentation) Challenge. Proceedings, winning contribution, pp. 31–35
(2014)
9. Zhang, L., Han, Y., Yang, Y., Song, M., Yan, S., Tian, Q.: Discovering discrimina-
tive graphlets for aerial image categories recognition. IEEE Trans. Image Process.
22(12), 5071–5084 (2013)
10. Han, J., Zhang, D., Cheng, G., Liu, N., Xu, D.: Advanced deep-learning techniques
for salient and category-specific object detection: a survey. IEEE Signal Process.
Mag. 35(1), 84–100 (2018)
11. Menze, B.H., van Leemput, K., Lashkari, D., Weber, M.-A., Ayache, N., Golland,
P.: A generative model for brain tumor segmentation in multi-modal images. In:
Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part
II. LNCS, vol. 6362, pp. 151–159. Springer, Heidelberg (2010). https://doi.org/10.
1007/978-3-642-15745-5 19
12. Gering, D.T., Grimson, W.E.L., Kikinis, R.: Recognizing deviations from normalcy
for brain tumor segmentation. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002, Part
I. LNCS, vol. 2488, pp. 388–395. Springer, Heidelberg (2002). https://doi.org/10.
1007/3-540-45786-0 48
13. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to
document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
14. Bengio, Y.: Practical recommendations for gradient-based training of deep archi-
tectures. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks
of the Trade. LNCS, vol. 7700, pp. 437–478. Springer, Heidelberg (2012). https://
doi.org/10.1007/978-3-642-35289-8 26
15. Havaei, M., et al.: Brain tumor segmentation with deep neural networks. Med.
Image Anal. 35, 18–31 (2017)
16. Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: SUN: a Bayesian
framework for saliency using natural statistics. J. Vis. 8(7), 32–32 (2008)
17. Rajkumar, R., Arunnehru, J.: A study on convolutional neural networks with active
video tubelets for object detection and classification. In: Wang, J., Reddy, G.R.M.,
Prasad, V.K., Reddy, V.S. (eds.) Soft Computing and Signal Processing. AISC, vol.
898, pp. 107–115. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-
3393-4 12
18. Arunnehru, J., Chamundeeswari, G., Bharathi, S.P.: Human action recognition
using 3D convolutional neural networks with 3D motion cuboids in surveillance
videos. Procedia Comput. Sci. 133, 471–477 (2018)
 Routine Statistical Framework
to Speculate Kannada Lip Reading

M. S. Nandini1(B) , Nagappa U. Bhajantri2 , and Trisiladevi C. Nagavi3

1
Department of IS and Engineering, NIE Institute of Technology,
Mysore, Karnataka, India
[email protected]
2
Department of CS and Engineering, Government Engineering College,
Chamarajanagara, Karnataka, India
[email protected]
3
Department of Engineering, Jayachamaraja College of Engineering,
JSS Science and Technology University, Mysore, Mysore, Karnataka, India
[email protected]

Abstract. This paper envisage the system provides a statistical based

effort to predict lip movements of speaker. The words spoken by a person
is identified by analyzing the shape of lip movement in every instance
of time. The approach learns the process of prediction of shapes of lips
based on recognition of movement. The lip shapes are predicted by anno-
tating and tracking the movement of lips and synchronization of shape
recognition with respect to time is achieved by extracting the shape of
lips with different statistical information extracted from every frames of a
video. Hence, grooved statistical data lends the system with more appro-
priate shape based in terms of mean, variance, standard deviation and
various other statistical features. The proposed system based on statis-
tical features extraction leads to lip movement recognition and mapping
of various Kannada words into different classes based on recognition of
shape leads the system perform good initiation towards achieving the
Lip Reading. The effort has provided overall accuracy of 40.21% with
distinct statistical pattern of features extraction and classification.

Keywords: Statistical feature extraction · Shape classification · Lip

tracking · Lip shape recognition · Mapping · Kannada lip reading

1 Introduction
Artificial Intelligence plays a very important role in many core areas like robotics,
machine learning, as it involves significant aspects of imparting the information
to systems. Autonomous systems is another core area that includes driving of
vehicles from source destination with the help of speech recognition without any
assistance of drivers [2,3,5,12]. These core areas defines the need of artificially
intelligent system that does the task of reading of lips. The spoken language
may be English or Spanish or Dutch or any Indian language. These languages
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 26–38, 2020.
https://doi.org/10.1007/978-981-15-3666-3_3
 Routine Statistical Framework to Speculate Kannada Lip Reading 27

needs to be recognized based on pre-trained information from videos. The frames

extracted from videos needs to be annotated to determine the language used by a
person. In all these, the system intelligence is needed to determine the movement
of lips from videos. The system is to be intelligent enough to determine the
movement of lips. There are various statistical methods used to determine the
languages by annotations during the phase of training and testing of videos.
Some of those statistical methodology employed in erected method to obtain the
outcomes of language spoken based on the shapes of lip movement.
Lip reading [1,4,6,11] is a very effort oriented task in machine learning, as it
involves identifying the every small change in shape of lip needs to be identified
more accurately while testing every frames of a video. In addition to this, the
annotation at every instance of shape is required, so that the system maps the
recognized lip shapes into various words of specific language. Particularly, when
performing the lip reading of Kannada Language, the system needs to be trained
as such the system gets enough information and makes use of those trained
information while testing of videos of dataset. Irrespective of person, speaking
the words, the system needs to determine the movement of lips and recognize the
language and sentence spoken by a person. Thus, we need a robust method [7–10]
that determines the shape of lip movement at every instance of time and predicts
the sentence spoken by a person. Further, annotate the system accurately to get
enough information, while training and testing in terms of lip shapes as shown
in Fig. 1.

Fig. 1. Overall architecture of statistical approach

28 M. S. Nandini et al.

2 Related Work
Typically, there are some challenges with automating lip-reading process [20],
including the co-articulation effect, visual features diversity and speaker-
dependency of features. The history of these issues along with some proposed
solutions for them could be found in the literature. Regarding to the co-
articulation effect and visual feature diversity, the following studies have reached
to noticeable solutions. In [14,16,21] have demonstrated the effects of inter-
speaker variability of features could be reduced by applying the feature improve-
ment of per speaker z-score normalization and the hierarchical LDA (HiLDA)
techniques.Using HMM recognizer [18,19,22], the best viseme accuracies of 45.
In [1,15] the best viewing angle for an automated lip-reading system is stud-
ied, using a proposed built audio-visual speech database LiLiR. The database
contains multi-camera, multi-angle recordings of a speaker reciting 200 sentences
from the Resource Management Corpus, with the vocabulary size of 1000 words.
Visual features extracted in this work are based on the Active Appearance
Models, which includes the shape, appearance, a simple concatenation of the
shape and appearance parameters, and an application of PCA [13,17] over con-
catenation and HiLDA variations.

3 Datasets
The videos of a dataset is considered for features extraction by exercising various
simple statistical parameters resulting in appropriate accuracy. The statistical
methods have been used to essentially obtain the features like analysis of variance
and other strategies that are essentially required for calculating the features from
various tilted and frontal faces of small deaf and dumb child.
The dataset has been chosen with all challenging issues to address in frames
of a dataset. The frames of a video is extracted and analyzed to detect the facial
features like lip, mouth and teeth as a way of reference to extract statistical
featured information from every frames of a video as shown in Fig. 2.
The challenging issues like frontal face, tilted face are used to check the effi-
cacy of the erected method with various statistical features that are required for
features extraction and features classification. So that the assessment of effort
with different statistical models shall be checked. Some of the statistical cri-
teria explored are Analysis of variance, chi-square test, factor analysis, Mann
Whitney-U, mean square weighted deviation, and Pearson product moment cor-
relation coefficient, regression analysis. The spearman rank correlation coeffi-
cients are some of the statistics used to measure the efficacy of the present effort
with respect to features useful for achieving the desired task and calculating the
features classification.

4 Proposed Methods
The effortless statistical feature based representation learning towards recog-
nition of lip shapes has led system determine the words spoken by a person.
 Routine Statistical Framework to Speculate Kannada Lip Reading 29

Fig. 2. Data set used with statistical approach after detecting the mouth

Irrespective of language used, it needs to recognize its movement is to be tracked.

The elementary feature extraction based representation learning extraction of
information from every frames of a video by employing the straight forward
features extraction. Some of the normal feature extraction based on statistics
are analysis of variance, chi-square test, factor analysis, Mann Whitney U,
mean square weighted deviation, Pearson product moment correlation coeffi-
cient, regression analysis, Spearman rank correlation coefficient.

Analysis of Variance. The statistical models are collected together for analysis of
variance is based on certain principles like finding the mean of n different values
and determining the variance. We have noticed that the determined variance is
partitioned into different sources of variation. Further, it provides a statistical
test of whether two models and its population means are equal and generalizes
the test those two models are beyond two means. It employs data decomposition.
In other words, its sum of squares indicate the variance of each component of
the decomposition. It allows testing of nested sequence of models to achieve
comparison of mean squares.

Chi-Square Test. The approach to evaluate independent variables, when using

a cross tabulation as bivariate table. This bivariate table allows to represent the
independent variables into one table simultaneously. The test of independent
variables of features used for extraction makes use of existence of association of
variables across two variances. By comparing the observed pattern of responses
30 M. S. Nandini et al.

with the pattern that would be expected. The predicted values of variables
against the existing one together forms the system predict the characteristic
features of independent variables.
 (f0 − fc )2
x2 = (1)
fe
In Eq. 1 shows chi-square statistic based on predicted versus existence or ground
truth values. The independent variables together constitutes that the proposed
system is relying on completely independent variable also shows the similarity
in values of one variable with respect to the other.

Factor Analysis. The process of expressing the observed against the existing val-
ues and explored to reduce the large number of independent variables into fewer
number of factors. It extracts maximum variance among common independent
variables and keeps them into a form of score. A part of general linear model,
predicts several observations like linear relationships and multi co-linearity, and
includes relevant variables into analysis. If there is a true co-relation between
variables then, several factors are available but principal component analysis is
used as a reference to determine the relations among independent variables.

cov(x − μ) = cov(LF + ∈) (2)

Where the term μ the mean of all items of features, and LF corresponds to lower
bound of the features set, ε indicates the + or − values of bounding errors.
The process also involves certain principles of converting the several similar
values of grouped variables into few variable with common features eliminated
and keeping few factors that are responsible for estimation of factors with data
to be predicted.

Mann Whitney-U. Here, the non–parametric independent of the sample test

to be conducted with respect to the existing values of variables, test that uses
certain techniques, such as comparing the sample mean of two different indepen-
dent variables with respect to the simple mean. It is used to test whether two
sample tests are equal or not. This process of eliminating the sample means of
two different variables with similar characteristics together forms a keeping of
single variables instead of multiple variables with multiple similar values, which
may lead to redundant data. Further,
n1 (n1 + 1)
U1 = R1 − (3)
2
n2 (n2 + 1)
U2 = R2 − (4)
2
The smaller the value of U1 and U2 , lesser the accuracy when consulting
significance tables. Thus sum of the two values is considered for estimation of
Mann Whitney U by
U1 + U2 = n1 n2 (5)
 Routine Statistical Framework to Speculate Kannada Lip Reading 31

The values of U1 and U2 are calculated as per (3) and (4).

Mann Whitney-U proposes the method of finding simple mean of n distinct
variables with different values within it to form a sample data values into a set
of linear representation of values. There by the system becomes stable and keeps
the redundant data out of the system and maintains the distinctive data in the
system, which also makes the system more robust and simple towards achieving
better statistical results.

Mean Square Weighted Deviation (MSWD). The method relies on three different
values between <1 and >1.
N N
W wi .(xi − x̄i )2
M SW D = N i=1 N (6)
i=1 − i=1 Wi i=1
2 (σxi )

By referring to Eq. 2, MSWD = 1, the data fits into univariate normal dis-
tribution, MSWD <1, the data is said to be undispersed, MSWD >1 indicates
as over dispersed. The arithmetic mean is first determined and then standard
deviation calculated to perform the task of values that ranges between −1 and
>1. The mean of the values of considered from all data variables are squared to
obtain the weighted deviation that fall into the values of deviation between <1
and >1.

Pearson Product Moment Correlation. Measure of the strength of linear rela-

tionship between two variables, depending on the value of Pearson correlation
coefficient the value of r will take either +1 or −1. The value +1 is used when
the Pearson correlation coefficient is maximum and the value is −1, the Pearson
correlation coefficient is minimum.
The PPMCC does not distinguishes the variables into either dependent
or independent variables, it considers all values of variables into same form
while evaluating the product of correlation coefficient. The correlation coeffi-
cient makes a contribution of whether the system has certain values of r closer
to maximum or closer to minimum. Based on these values of r, the decision
of classification is carried out to check the effectiveness of moment correlation
coefficient.
E[xy] − E[x]E[y]
ρx,y =   (7)
E[x ] − [E[x]]2 E[y 2 ] − [E[y]]2
2

The PPMCC also relies on the values of variables that may supersede the sys-
tem with correlation existence with other system coefficients. Since, the system
makes use of certain principles like determining the values of qualitative analysis
and its effectiveness towards achieving the strength of correlation coefficient.

Regression Analysis. The powerful statistical routine has able to determine the
relationship between two or more variables that may be of interest. Here the
dependent variables are under the influence of one or more independent variables.
These independent variables are more interested towards achieving the values of
dependent values of variables formed from it. The practice makes use of certain
32 M. S. Nandini et al.

principles like determining the correlation of independent variable values to form

values of dependent variables.

Y = θ1 + θ2 .X (8)

where θ1 and θ2 corresponds to the information extracted from the statistical

features.

Spearman Rank Correlation Coefficient (SRCC). The SRCC is more prone

towards finding the process with non-parametric version of PPMC. The purpose
of using this spearman rank correlation is to order the correlation coefficients
into different order of ranks that corresponds to linear distribution of values of
variables. These values may be linearly or non-linearly scattered, but the system
performs the task of calculating the independent variables and its correlation
towards finding the system with maximum relevance. There by the dependent
variables are more prone to a closer values of independent values of variables.
0.6325
σr = √ (9)
n−1
where n indicates the number of features extracted with respect to the maximum
features of the featured value.
The rank of a system is determined by considering the correlation exists
between independent and dependent variables formed from independent vari-
ables. The correlation rank is completely dependent on correlation of values of
independent variables together with dependent values of variables. If these corre-
lation is more towards +1, the system considers that the rank is 1, the correlation
of independent variable closer to +1 with slight margin towards a values next to
rank 1 is considered as rank 2 correlation coefficient.

5 Results and Discussions

The statistical perusal of results of various parameters shall be verified with the
presence of different numerical study that is more powerful towards determining
the effectiveness of independent and dependent variables. These variables are
more likely to be used for analysis of true versus false positive and various other
metrics are too considered for evaluation of the system.

Results of Precision versus Recall. The precision is obtaining the true positive
of predicted results with respect to sum of True positive and false positive rate.
TPR
P recission = (10)
TPR + FPR
TPR
Recall = (11)
TPR + FPR
 Routine Statistical Framework to Speculate Kannada Lip Reading 33

Table 1. Indicates the precision versus recall results produced from different statistical
manner.

Sl. no Methods Precision Recall

1 Analysis of variance 39.8 44
2 Chi-square test 32.4 25
3 Factor analysis 36.5 38
4 Mann Whitney-U 39.5 50
5 MSWD 37.6 34
6 PPMCC 36.1 37.5
7 Regression analysis 32.8 36.9
8 SRCC 39.1 40.1

It is clear from the Eqs. (10) and (11) that the statistical enumerations
are assessed with discrete modus operandi. Further based on true positive rate
yielded from predicted results with reference to false prediction results from
enlarged statistical framework (Table 1).
In addition to other, the statistical parameter have not yielded much opti-
mized results.

Table 2. Sensitivity and specificity of statistical methods.

Sl. no Methods Sensitivity Specificity

1 Analysis of variance 43.21 41.12
2 Chi-square test 45.12 40.31
3 Factor analysis 41.22 42.23
4 Mann Whitney-U 39.46 43.41
5 MSWD 39.98 44.16
6 PPMCC 41.23 45.21
7 Regression analysis 43.48 46.12
8 SRCC 46.19 41.21

It is evident from Fig. 3, the precision and recall of different statistical pro-
cesses have shown their best of ability to detect the shape of lips or extract
much detailed information from lips in term of true positive and false positive
rate. These metrics together with other false negative rate also defines the recall
strategy.

Results of Sensitivity versus Specificity. The sensitivity and specificity are two
additional metrics plays a very important role in accurately determining the fea-
tures to be used for prediction of Lip reading. The performance of the statistical
approach has been assessed with metrics enforced visibility are shown in Table 2.
34 M. S. Nandini et al.

Fig. 3. Comparision of non-identical stastical methods

By employing the sensitivity and specificity mentioned in Eqs. (12) and (13)
respectively.
TPR
Sensitivity = (12)
TPR + FPR

The sensitivity and specificity defines the metric used to assess the proposed
statistical methods. Overall, the Statistical methods have given better informa-
tion as to how the frames is to be processed. The result of processing of informa-
tion using different statistical methods yielded better outcome while mapping
the lip shapes with annotated values at the time of testing.
TNR
Specif icity = (13)
TNR + FPR
It is evident from the Eqs. (11) and (12) respectively that the sensitivity is a
metric defines the statistical features in terms of true positive rate versus sum of
true positive rate and false negative rate. Further, the specificity is determined
by employing Eq. (13) defined by true negative rate versus sum of true negative
rate and false positive rate. The graphical representation is shown in Fig. 4.

Results of Accuracy. The accuracy of the proposed task is determined by using

Eq. (14) as
TPR + TNR
Accurancy = (14)
TPR + FPR + TNR + FNR
 Routine Statistical Framework to Speculate Kannada Lip Reading 35

Fig. 4. Comparison of contrasting statistical methods with respect to sensitively vs

specificity

Table 3. Accuracy of statistical methods.

Sl. no. Accuracy Accuracy

Methods
1 Analysis of variance 40.14
2 Chi-square Test 39.21
3 Factor analysis 38.44
4 Mann Whitney-U 41.28
5 MSWD 47.15
6 PPMCC 40.43
7 Regression analysis 41.26
8 SRCC 40.15

The Eq. (14) determines the accuracy of Lip Reading strategy. The perfor-
mance has been verified with accuracy based on Eq. (14) and the results of it
are shown in Table 3.
36 M. S. Nandini et al.

Fig. 5. Juxtaposition of desperate statisticals with respect to accuracy.

The accuracy is another metric used to determine the most accurate values of
predicting the Lip Reading using sundry statistical machine learning strategies
(Fig. 5).

6 Conclusion

The research work has attempted to contribute towards predicting the lip move-
ment by analyzing the features of simple statistical boundaries and imposed
certain metric as a rough yardsticks towards predicting the correctness of var-
ious statistical method that are very much helpful in predicting the language
spoken by a person. The sensitivity and specificity of the proposed statistical
methods have yielded 41.21% and 46.19% respectively. The accuracy of these
statistical methods have been presented with 40.18%. As statistical methods
are computationally incapable to produce good performance, hence the outcome
accuracy is 40.18%. The performance can be improved with hybrid techniques.
The effort made here to signify the sophomore rate of outcomes. Further, the
attempt made with the effortless and manageable parameters as a rehearsal to
reveal lip reading accomplishment.
 Routine Statistical Framework to Speculate Kannada Lip Reading 37

References
1. Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous dis-
tributed systems. arXiv preprint arXiv:1603.04467 (2016)
2. Assael, Y.M., Shillingford, B., Whitestone, S., de Freitas, N.: LipNet: sentence-level
lipreading. arXiv:1611.01599 (2016)
3. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning
to align and translate. In: Proceedings of ICLR (2015)
4. Bengio, S., Vinyals, O., Jaitly, N., Shazeer, N.: Scheduled sampling for sequence
prediction with recurrent neural networks. In: Advances in Neural Information
Processing Systems, pp. 1171–1179 (2015)
5. Chan, W., Jaitly, N., Le, Q.V., Vinyals, O.: Listen, attend and spell. arXiv preprint
arXiv:1508.01211 (2015)
6. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the
details: delving deep into convolutional nets. In: Proceedings of BMVC (2014)
7. Chorowski, J., Bahdanau, D., Cho, K., Bengio, Y.: End-to-end continuous speech
recognition using attention-based recurrent NN: first results. ArXiv Preprint
arXiv: 1412.1602 (2014)
8. Chorowski, J.K., Bahdanau, D., Serdyuk, D., Cho, K., Bengio, Y.: Attention-based
models for speech recognition. In: Advances in Neural Information Processing Sys-
tems, pp. 577–585 (2015)
9. Chung, J.S., Zisserman, A.: Lip reading in the wild. In: Lai, S.-H., Lepetit, V.,
Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10112, pp. 87–103. Springer,
Cham (2017). https://doi.org/10.1007/978-3-319-54184-6 6
10. Chung, J.S., Zisserman, A.: Out of time: automated lip sync in the wild. In:
Chen, C.-S., Lu, J., Ma, K.-K. (eds.) ACCV 2016. LNCS, vol. 10117, pp. 251–
263. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54427-4 19
11. Cooke, M., Barker, J., Cunningham, S., Shao, X.: An audio-visual corpus for speech
perception and automatic speech recognition. J. Acoust. Soc. Am. 120(5), 2421–
2424 (2006)
12. Czyzewski, A., Kostek, B., Bratoszewski, P., Kotus, J., Szykulski, M.: An Audio-
visual corpus for multimodal auto-matic speech recognition. J. Intell. Inform. Syst.
49, 167–192 (2017)
13. Feichtenhofer, C., Pinz, A., Zisserman, A.: Convolutional two-stream network
fusion for video action recognition. In: Proceedings of CVPR (2016)
14. Galatas, G., Potamianos, G., Makedon, F.: Audio-visual speech recognition incor-
porating facial depth information captured by the Kinect. In: 2012 Proceedings
of the 20th European Signal Processing Conference (EUSIPCO), pp. 2714–2717.
IEEE (2012)
15. Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal
classification: labelling unsegmented sequence data with recurrent neural networks.
In: Proceedings of the 23rd International Conference on Machine Learning, pp.
369–376. ACM (2006)
16. Graves, A., Jaitly, N.: Towards end-to-end speech recognition with recurrent neu-
ral networks. In: Proceedings of the 31st International Conference on Machine
Learning (ICML-14), pp. 1764–1772 (2014)
17. Graves, A., Jaitly, N., Mohamed, A.-R.: Hybrid speech recognition with deep bidi-
rectional LSTM. In: 2013 IEEE Workshop on Automatic Speech Recognition and
Understanding (ASRU), pp. 273–278. IEEE (2013)
38 M. S. Nandini et al.

18. Hermansky, H.: Perceptual linear predictive (PLP) analysis of speech. J. Acoust.
Soc. Am. 87(4), 1738–1752 (1990)
19. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regres-
sion trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pp. 1867–1874 (2014)
20. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–
1758 (2009)
21. Koller, O., Ney, H., Bowden, R.: Deep learning of mouth shapes for sign language.
In: Proceedings of the IEEE International Conference on Computer Vision Work-
shops, pp. 85–91 (2015)
22. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep con-
volutional neural networks. In: NIPS, pp. 1106–1114 (2012)
 Incepting on Language Structures
with Phonological and Corpus Analysis
Using Multilingual Computing

Bidisha Choudhury1(B) and Anupam Das2

1
Visva Bharati University, Santiniketan, West Bengal, India
[email protected]
2
Cotton University, Guwahati, Assam, India

Abstract. In this paper it is tried to incept into the structures of differ-

ent languages around the world. The languages are selected on the basis
of the four directions of this part of the world. Initially, the structures of
the languages are explained. A model for machine translation is proposed
in which the audio of a language is taken as input and subsequently con-
vert the data into text and finally in binary to process the language.
During the process, the model finds the phonetic distances and phono-
logical distance and then blended both. The corpus generation is also
made the model more useful for the further language computing. There
are three languages such as Sanskrit Language (SL), English Language
(EL) and the Russian Language (RL) considered for the analysis.

Keywords: Phonetic distance · Phonological distance ·

Language-preservation · UG-Unigram · BG-Bigram · LCFG-Language
Computing for Feature Generation

1 Introduction

Languages spoken by the human beings are extinguishing at rates and it is affect-
ing the global bio-diversity as the disappearance of languages also dissociate us
from the dimensions of those cultures which are associated with those lost lan-
guages. According to the world report, there are about 7000 languages in the
world are in use today and it is expecting to have 3000 languages in next 100
years. The most challenging thing is that the 48% of the world’s population are
first language speakers of one of the world’s top 10 most spoken languages: Stan-
dard Chinese, English, German, Spanish, Portuguese, Russian, Bengali, Hindi,
Arabic and Japanese. The most alarming situation of the language extinguishing
is that out of 10000 languages before 100 years already 3000 languages are extin-
guished as it is forgotten by people. Now out of 7000 languages 3500 languages
are spoken by a little group of people consisting of around 10000 and around
2100 languages are spoken by a mere 1000 people [4]. Therefore it is our utmost

c Springer Nature Singapore Pte Ltd. 2020

A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 39–52, 2020.
https://doi.org/10.1007/978-981-15-3666-3_4
40 B. Choudhury and A. Das

duty to make some way to preserve such languages. This study focuses on dis-
cussing some parameters of language computing and also gives the theoretical
idea on grammar for the major languages spoken in the world which will give
for further researches on this language preservation.

1.1 Problem Definition

The problem lies in language preservation is that language changes its feature
aspect in the direction of longitude as well as latitude with respect to period.
Unfortunately, due to these changes many languages already abolished from
the set of languages of the world, and many are in the endanger position. So
in this work we propose a model through which we can try to preserve the
languages by keeping the analogue or digital records of the chunks of audio files
of different languages and through the LCFG (proposed model), the features are
synthesized for identifying the class of the language and also the name of the
language. Subsequent paragraphs, however, are indented (Fig. 1).

Fig. 1. Countries along the longitude and latitude.

 Incepting on Language Structures with Phonological and Corpus Analysis 41

1.2 Purpose

This work deals with the location based study and an attempt is made to find the
variations of spoken languages of the different regions spread over a range of lon-
gitude and latitude of the world. In our study we had taken languages: along the
longitude-India (initial), Uzbekistan (immediate up from India), Mongolia (next to
Uzbekistan), Russia (next to Mongolia) and Sri-Lanka down from India. The lan-
guages along the latitude are Saudi Arabia left of India and China, right of India.

1.3 The Theory

One of the most evidential elementary languages of Indo European IE family is

Sanskrit [2]. The proposed theory thus based on the ancient Sanskrit Grammar
that deals with a concrete pattern of word formulation. Like any other IE group
of languages Sanskrit has marker and basic word, combination of whom a word
is formulated. Like any other IE group of languages Sanskrit has marker and the
basic word, combination of whom a word is formulated. In Sanskrit, it is called
“prakriti” and “pratyaya”. Panini defined the crude or elementary form of word
as “prakriti” or “prātipadika”. He also termed them as base or root or word
free from inflection [1]. In the word “parting” there is a word contains “part”
as prakriti and “ing” as pratyaya or marker. Both of them combined to make a
single word which defines a name in noun form. In this way we can define all the
existing languages in basic two forms. They are the languages that have markers
are Agglutinative and Inflected languages. Examples-Sanskrit, Hindi, Arabic,
Bask, Hibru languages and the languages that don’t have markers are called
Positional or Isolating languages. Examples-Chinese, Sudani, Tibetan, Barmese
languages etc. [3].
The above arguments are explained with the counter examples
as below:

(i) Indian Languages: All Indian languages have marker as they are rooted
to Sanskrit. Those languages that have influence of Chinese or Tibetan lan-
guage are usually behaved like no-marked languages.
(a) Arunachal Pradesh, a state situated in the north-east part of India and
bordered by neighboring country China. So its language is influenced by
Chinese language. Having 30 mostly popular and 50 distinct languages
Arunachal Pradesh uses the language named Tani that belongs to Sino-
Tibetan family. The peculiarity of this particular language is the missing
of cognate and overload of reconstructed roots. In respect of vocabulary
the phonemic strategy of this language is completely depended on its
phonetic structure.
(b) Assam, a state is situated in the south east of Arunachal Pradesh and
bordered by Bangladesh. The language Assamese is the official native lan-
guage of this state and it is highly influenced by Silet dialect of northern
region of Bangladesh and Mising, Tani languages of Arunachal Pradesh.
With the effect of isolated phonology and morphology the Assamese lan-
guage has more the 50% single-lettered suffixes. Such as, ė, o, ni etc.
42 B. Choudhury and A. Das

Making resemblance with isolated morphological structure Assamese lan-

guage is intended to inflect affix with root verb and make a new verb
form. Such as, (Verb-root) khā + uvā = khuvā. Again Assamese verb
root can associate suffixes in numerable numbers on the basis of mean-
ing prediction, gender, time etc. and they can be inflected or associated.
Example: Killed = mar + isil/ isilu = marisil/marisilu. Did (in past tense,
1st person) = kor + isilō + ne (empathic marker) = korisilōne [7].
(c) Bihar: Amongst the sub-division languages of Hindi, Bhojpuri is the
mostly used native language of Bihar. Bhojpuri is familiar with Indo-
Aryan language family. As it is spread throughout the eastern- middle
and northern area and historically influenced by English lexicography and
Dutch language, so the intension of suffix and affix to use with phonologi-
cal variation is prominent and categorised as the markers that associated
with the verb or noun roots. Thus it is evidentially noticed that distance
and heredity plays a huge role of influence in the phonetic structure espe-
cially on the addiction of suffixes with verb-roots.
(ii) Languages across the world: Considering the countries along the globe
we have the following observations:
(a) Characteristics of Uzbekistan Language: Turkic is the native language of
Uzbekistan. Turkic is a newly born (around 7th Century A.D.) language
that has link with its ancestor of 25000 years old. Through the minute
study of isoglosses of this language it can be presumed that this language
is a family of those languages which were running across the centum-satam
boundary; such as, Tocharian, Anatolian languages etc. The peculiarity of
Turkic language are - inflexions, vowel harmony and lack of grammatical
gender. In vowel harmony and agglutination suffix and prefix; i.e. markers
play a huge role. In that sense Turkic falls in that category of languages
who have marker. Here according to the rule of inflection suffix can main-
tain its separate identity also after adjoining with the verb or noun. The
difference from English language of this language is that Turkic language
uses fewer words than English. The variety of meaning laid on the usage
of suffix which also makes words longer than the other. But suffix with
its meaning remain unchanged in the word of Turkic language. Example,
verb root gelmek means “come”. It is used as present continuous form as
geliyom means “coming”. Now for distinguishing person, Turkic language
uses suffix “um”, “sun” etc. For example geliyor with suffix “um” becomes
geliyorum which means “I am coming”. Geliyor with suffix “sun” geliy-
orsun which means “you are coming”. In this way a pattern can be seen.
That is verb with suffix “um” represents first person singular number.
Verb with suffix “uz” represents first person plural number. Verb with
suffix “sun” represents second person singular number. This formation is
quite similar to proto-indo European languages where suffix maintains its
pattern and remain same after agglutination. Example, market(e) geliy-
orum means “I am coming to the market”. Market(e) geliyorsun means
“you are coming to the market”.
Incepting on Language Structures with Phonological and Corpus Analysis 43

(b) Characteristics of Mongolian Language: Mongolian is the official native

language of Mongolia. Like Turkic or Uzbek language, this language
has vowel harmony and it is also a typical inflected language. Unlike
English, Mongolian verb-suffix and sentence construction follows the rule
and structure of Proto Indo-Aryan language family; i.e. subject-object-
predicate structure and verb are marked as per voice, aspect, tense and
linguistic expression, such as empathetic tone, moods etc. Example, to
make a non acquisitive verb from an acquisitive verb, there is a suffix
“rah” but this remains unchanged after joining the verb root. Such as
verb root “zuuh” means bite. After joined with suffix “rah” it becomes
zuurah which means lock of jaws to clutch at etc. In this way some suf-
fixes form those verbs from verb roots which come from adjectives and
finally denote the meaning of “to become more”. Such as suffix “sah”
with adjective verb root “tom” and made word tomsah that means “to
grow bigger”, “to act big” etc.
(c) Characteristics of Russian Language: Belonging to the Indo-European
language family Russian is a fourth living branch of Balto-Slavic group.
The grammatical peculiarity is laid in this language in the formation of
verb-word. Following the basic Indo-European verb formation Russian
language grammar separates its verb into two parts-imperfect or infinite
verbs and perfect or finite verbs. The unique conception is that these two
section use to be adjoined with prefix and form a new verb in an inflected
rule. Again in Russian language verb does not appear in a single way. It
always appears in the conjugated form. Alike old Indo-European gram-
mar verb has perfective stem that imposed from aorist or infinite form
of verb. It is noticeable that in Russian language, markers play much
more multi-disciplinary role than any other languages. They are - Suffix
that indicates the number and person of the subject. Example Ya chitayu
mean “I read”. Here -yu suffix stands for first person. On chitayet means
“He reads”. Here et suffix stands for third person.
On the other hand, there are particular suffixes or noun ending to indi-
cate masculine, feminine and neuter gender. They are - masculine gender
usually has consonant letter as ending. Feminine gender usually ends up
with suffix -a. Neuter gender ends up with -o and -e suffixes.
(d) Characteristics of Sinhalese language: Sinhalese is the largest spread
native languages of Sri Lanka. It belongs to Indo-Aryan Family. Alike
Japanese, Korean and other longitudinal languages centering India, in
Asia, Sinhalese language also has SOV (Subject-Object-Verb) syntactical
structure. Verbs are assimilated in this language. Besides this the lim-
ited number of suffixes eases the grammatical application in the spoken
Sinhalese. Such as, in Sinhalese language cry means anna, look means bal-
anna. So –nna is such a suffix which makes lots of finite verbs. Sinhalese
verbs consist of stems. They are, “-bo” denotes drink, -ka” denotes eat,
“-duva” denotes run. They mostly adjoined with suffix “-nna” for present
tense and “-va” for indicative form of verb. In Sinhalese verb-suffix combi-
nation, agglutination is not the only way of verb formation. Usually, when
44 B. Choudhury and A. Das

the verb represents the past tense then the suffix has been inflecting with
the verb. In the past tense suffix –u is inflected by replacing the stem
vowel “a” of the present finite verb. For example baeluvā means looked
instead of balanna. Denna in present form of verb root “de” means give.
But in the past form suffix -nā is inflected with the verb root and makes
new word Dunnā.
(e) Characteristics of Chinese Language: Chinese language is the only exam-
ple of root or isolating classification of language. In such languages word
has been decoding by its position in a sentence. Thus a word can be sub-
ject, predicate as well as verb also. The intension of the word is depended
on the tonal verification of the speaker and its usage in a particular sen-
tence. And so it is called analytic language. In Chinese language there is
no separate formation or conjugation of verb according to the tense. The
only way to intend past, present, future is to add “today”, “yesterday”
and “tomorrow” before the verb. For example, “eat” is used as “chi” in
all the forms of tenses. Zuótiān wǒ chı̄le Ork means in English Yesterday
I ate pork. Pitch intonation is therefore a primary equipment to deci-
pher the intension of morphemes. There are five major tone structures in
Standard Chinese language. Every Chinese character has different tone.
Amongst them first four are basic pitch tones and fifth is neutral tone.
It is as followed as first dā is high and level, second dá starts medium
in tone, then rises to the top, third dǎ starts low, dips to the bottom,
then rises toward the top, fourth dà – starts at the top, then falls sharp
and strong to the bottom, and neutral da – flat, with no emphasis. Exam-
ple, ‘tāng’ with a high tone means ‘soup’, but ‘táng’ with a rising tone
means ‘sugar’. ‘Āi’ with a high tone means ‘hey’, but ‘ǎi’ with curvy low
to rising high tone means ‘short and ‘ài’ with sharp fall tone from top to
bottom means ‘love’. Similarly ‘bā’ means ‘eight’, ‘lā’ means ‘pull’, ‘bǎwò’
means ‘grasp’ and ‘Fùqı̄n ’ means ‘father’.

2 The Proposed Model: Language Computing for Feature

Generation (LCFG)

The languages of different parts of the world are different in acoustic features
with the variation of phonetics, phonemes etc. The proposed model LCFG is
designed with the following features: phonological analysis, corpus generation
and text-mining. It measures the following metrics like: phonetic distance, phono-
logical distance, combined distance and also generate the corpus of the languages.
The phonetic analysis is done by applying the phonetic distance, phonological
distance, combined distance, the corpus generation is done by applying an easy
mechanism with the help of the huge repository of web. Finally find some per-
plexity of some corpus of the languages (Sanskrit, English, and Russian): The
below diagram shows the flow of the work of LCFG.
 Incepting on Language Structures with Phonological and Corpus Analysis 45

Fig. 2. The proposed model: LCGF.

2.1 Phonological Parametric Measures

To measure the phonological parameters of a language we need to find the fol-
lowing: phonetic distance, phonological distance, combined distance. Phonetic
Distance: The phonetic distance is measured by taking the distance between two
phonemes. Here, each phoneme is measured as vector of binary values which will
characterize the features like: voice, position of articulation and the behaviour
of articulation. A phoneme is denoted by Ph * L, where L lies between 1 to k.
Ph * L represents the language that includes the phoneme, and i lies between 1
to Lk which indicates the index of the phoneme in language L (Fig. 2).
Phonological Distances: A phoneme is identified by /a/ in a language that has
vowels /a e i o u/, and also in another language if /a/ is in any case a set of
vowels then in storing /a/ for the second language will be a sub phoneme in the
storage. This kind of variations is identified with feature applications of the /a/
as phoneme and /a/ as a sub-phoneme. The features are then stored in two kind
of phonemes as mono-phoneme and bi-phoneme of languages.
Combined Distance: The combined distance is computed by blending the pho-
netic and phonological distances. As these distances are different, therefore
before combining, the normalization is applied before making it as a unit. The
process is so done that the two distances are calculated and blended in to another
function with the features from the vectors of phonetic distance as well as phono-
logical distance.
The study of the model: Since the different languages have its own grammar and
structure so decoding a language into machine learning code is quite a difficult
task. The task is simplified by considering the audio files and then converting
them into the digital file which is then considered as the input file for the model to
get some feature analysis of phonetic, phonological and finally to generate corpus.
Let us see the treasure of sets from the grammar of the three testing languages
(SL-Sanskrit-language, EL-English-Language and RL-Russian-Language).
46 B. Choudhury and A. Das

Fig. 3. The wave format of the EL, RL and SL of the sentence I EAT RICE three
times.

2.2 Phonological Study

This phase of the model works out for finding the phonetic distances as vectors,
phonological distances of characters (with two categories: vowels and consonants)
and finally blends the two to find the resultant effect in the repository. Let us
take an example to explain how we derive the phonetic and phonological analysis
from a sentence. The sentence in English “I eat rice” is spoken in Sanskrit
and Russian as: “aham annang khaadaami” and “ya yem ris”. The sentences
are now taken as the strings by eliminating the white spaces in between the
words. Now the sentences in the three languages will be looked as the following
string: “ieatrice”, “ahamannangkhaadaami” and “yayemris”. Now we may get
more than one phonetic strings in this process but for more precision we will
take the larger strings. Thus finally we take the strings as: “(ai)eatr(ai)ce”,
“ahamannangkhaadaami” and “y(aa)y(a)mr(ii)s”. The following table gives the
details (Fig. 3).
Sentence Corpus: In general definition, the corpus is a collection of written or
spoken material stored in a computer and they are used to find the usage of a
language. Thus they are the units stored in a computer to use for a language [6].
The corpus may be stored for a single language or two or more languages. If
it stores for one language it is termed as monolingual and for two or more
it is known as multilingual corpus. Multilingual corpora are used for side-by-
side comparison which is called aligned parallel corpora. The corpora can be
prepared for linguistic research by applying the annotation process like: PoS
Incepting on Language Structures with Phonological and Corpus Analysis 47

Fig. 4. The corpus of words from a randomly selected text in SL.

Fig. 5. The corpus of words from a randomly selected text in EL.

Table 1. The sets of basis of grammars of SL, EL and RL

Basis of language SL (Sanskrit Language) EL (English Language) RL (Russian Language)

Character set (alphabet) 42. 26 33
No. of vowels 09. 05 10
No. of consonants 33. 21 21
Number 03. 02 02
Order of Sentence 03. 01 01
Tense 10. 03 02
Verb mood 04. 05 03
Gender 03. 03 03
Part of Speech (PoS) 05. 08 07
48 B. Choudhury and A. Das

Fig. 6. The corpus of words from a randomly selected text in RL

Table 2. languages with its vowel and consonants used in the strings

Languages Vowels Consonants

Sanskrit (SL) 2(a, i) 6(h, m, n, ng, kh, d)
English (EL) 3(a, i, e) 3(t, r, c)
Russian (RL) 2(a, i) 4(y, m, r, s)

Table 3. The Uni-Gram (UG) and Bi-Gram (BG) of the vowels and consonants

Languages Vowels Consonants Vowels Consonants

UG UG BG BG
Sanskrit (SL) “a” and “i” 6(h, m, n, ng, kh, d) NIL “ah”, “ha”, “am”, “ma”,
“aa”, “an”, “nn”, “na”, “ad”,
“da”,“ng”,“kh”,“mi”
English (EL) “a”, “i”, “e” 3(t, r, c) NIL “ai”, “ie”, “ea”, “at”, “ra”, “ic”,
“ce”
Russian (RL) “a” and “i” 4(y, m, r, s) NIL “ya”, “ay”, “am”, “ri”, “ii”, “is”,
“aa”, “mr”

(part of speech) tagging, lemma (base), interlinear glossing etc. (Figs. 4, 5 and
Tables 1, 2, 3, 4).
 Incepting on Language Structures with Phonological and Corpus Analysis 49

Table 4. The weights calculated of UG and BG of vowels and consonants of the strings

Languages Vowels Consonants Vowels Consonants

WUG WUG WBG WBG
Sanskrit (SL) 10.52632 31.57895 00 68.42105
English (EL) 30 30 00 70
Russian (RL) 20 40 00 80

2.3 Efficiency Measurement of Corpus

The perplexity is the measurement of the wellness of a corpus according to the

probability distribution of the words, sentences or texts [5]. The word, sentence
or text can be taken as sample. The low perplexity implies good probability
distribution of the sample. The perplexity can be computed by using the math-
ematical expression:

perplexity(word) = P (w1 , w2 .w3 , ....wN )−1/N (1)

by chain rule
n

P (word) = P (w1 |w2 , w3 , ....wi−1 )−1/N ) (2)
i=1

In NLP, more precisely given expression using entropy is:


2H(P ) = 2− p (x)log2 p(x) (3)
where H(P) is the entropy, p(x) is the probability distribution and x is a random
variable. Therefore, 
H(P ) = − p(x)log2 p(x) (4)
x

Similarly, we can find the perplexity of sentence, if S contains m sentences

and N words, then corpus per word is given as:

P (S) = P (s1 , s2 , s3 , ....sm )1/N (5)

it can further mathematically simplified as:

N
 N
 N

P (S) = (P (s1s2.s3, ....swm ) − 1/N ) = p(si) 1/N
= 2log2 p(si)1/N
i=1 i=1 i=1
N
 m

= 2 − 1/N log2 p(si) = 2−1/N p(si )log2 p(x) (6)
i=1 x
50 B. Choudhury and A. Das

Table 5. The frequency wise occurrences of words in EL, SL and RL from randomly
selected text

Frequency EL (out of 115) SL (out of 43) RL (out of 70)

1 41 41 52
2 07 02 07
3 02 00 00
4 04 00 01
5 02 00 00
7 01 00 00
10 01 00 00
21 01 00 00

Prefix measurement of corpus:-1-gram or (uni-gram), the simplest case of

Markov assumption is the case when the size of prefix is 1. P (w1, w2. w3, . . . .
wN) = product of P (wi), it gives us grammar that only consider one-word. It
generates a set of unrelated words.
From Fig. 6, it is observed that with frequency 1, there are 41 words in EL
as well as SL, out of 115 and 45 respectively; and 42 out of 70 in RL. This
shows that around 35.65% in EL, 74.28% in RL and 91.11% in SL unique words
are used to express a particular topic in their respective languages. Texts are
selected at an unbiased manner. Also, with frequency 2, there are 7 words in EL,
2 words in SL and 7 words in RL. It represents respectively, 6.086%, 4.44% and

Fig. 7. Word corpus in EL, SL and RL from randomly selected text

 Incepting on Language Structures with Phonological and Corpus Analysis 51

10% in EL, SL and RL. More than frequency 2, there is no word found in SL,
whereas in EL and RL some words are present. Interestingly there are two words
in EL, viz. “to” and “the” the frequencies are 10 and 21 respectively (Table 5
and Fig. 7).
The generation corpus of a language is really a tedious task. But here one
easy mechanism is shown for generating the corpus of any language. The steps
are:
(i) By using AI-application web-crawler the searching is done from www, and
it returns the list of all possible websites which contains the desired texts,
for our test we searched Sanskrit, English and Russian. The application
downloads the information of each language into a specific method.
(ii) The text collected from step-1 is stored according to fonts. The information
will be stored as per the different folder names to get an easy access for fur-
ther process. Next all the language texts are converted to its corresponding
UNICODE by applying the converter.
(iii) We can use Google search engine for searching by setting the keywords
from the test set languages like: Sanskrit, English, and Russian etc. The
searched documents are then stored in the selected folders local drives of
the computer.
(iv) Then the proper storing in the machine is done for generating the corpus
of the language. Thus the corpus of the language is prepared.

3 Findings
From the above study the following observations are found:
(i) In phonetic and phonological distances, though the Sanskrit language has
less number of vowels still this language produces maximum number of BG
(Bi-Gram) strings. Thus Sanskrit is more phonetically and phonologically
strong language than the other two languages.
(ii) Sanskrit Language is free from ordering of words.
(iii) In Sanskrit, the utterance and the phonetic and phonological strings are
same as that of the actual string of the sentence, but in case of other two
languages some changes are observed.
(iv) A minute observation of UG and BG would show that if any language
is much closer to its proto or primitive family language then its Mora of
morpheme would behave likewise. Here East Slavic existing language Rus-
sian is much closer to Indo-European primitive language Sanskrit. Thus
the audio graph of this language shows some similarities in morphological
Mora with Sanskrit rather than English. Because as per antiquity English
is much younger than Russian and Sanskrit.
(v) Features of characteristics of languages those discussed in this paper would
denote the facts that not only borrow words from different languages
would change the structural periphery of a particular language but char-
acter of neighbouring language also makes some noticeable variations in
that language.
52 B. Choudhury and A. Das

(vi) The immediate impact of neighboring language is on the phonological vari-

ation of that language. Sometimes the impact would be much strong that
one language could behave differently from its family tree. For example,
being a part or daughter language of Sanskrit, the languages of Assam
and Arunachal Pradesh behave alike with Chinese language which is free
from marker. Thus it can be said that language family tree is not enough
to detect the characteristics of the existing or extinguished languages.
(vii) The corpus of languages shows that the uniqueness in vocabulary in San-
skrit is much higher than the other two.
(viii) Statistical analysis of these three languages shows that the theorem and
rules to put a word in sentence, or to put characters (letters) in order to
make word is much elaborate in Sanskrit compared to other two languages.
In the order of ranks among these three languages are 1st Sanskrit, 2nd
Russian and 3rd English.
(ix) The free word format and elaborate grammatical format made a huge
scope to make unique words to express same meaning in Indo-European
language family.
(x) In Indo-European group, the basic characteristics of a language are that it
has free sentence making structure with limited markers. It leads to make
words easily in the vocabulary. But the younger languages like English
and Russian these characteristics are absent which reduces the chances of
enriching the vocabulary in these languages.
(xi) If the extinguished languages have the same kind of primitively in their
grammar like Indo-European language family then the endangered lan-
guage can be restored by studying the pattern of vocabulary harmony
through the words and sentence corpus.

References
1. Katre, S.M.: Astadhyayi of Panini. Motilal Banarasidas, Delhi (2015). (2nd Edition
of 1st edition 1989)
2. Chatterjee, S.K.: The Origin and Development of the Bengali Language. Rupa Pub-
lication India Pvt. Ltd., New Delhi (2011). 4th Impression of, 1st edn. 1926
3. Das, A., Choudhury, B.: A study on the process of Supra segmental preservation
of IE Language and computational data analysis of acoustic phonetics. J. Emerg.
Technol. Innov. Res. 7(3), 898–903 (2019). E-ISSN 2347–2693
4. Das, A., Choudhury, B.: A study on language computations by preserving them as
audio documents. Int. J. Comput. Sci. Eng. 5(8), 209–214 (2018). ISSN 2349-5162
5. Das, B., Mandal, S., Mitra, P.: Bengali speech corpus for continuous speech recog-
nition. IEEE (2011)
6. Sarma, S.K., et al.: A structured approach for building Assamese corpus: insights
applications and challenges. In: Proceedings of the 10th Workshop on Asian Lan-
guage Resources, COLING 2012, Mumbai, December 2012 (2012)
7. Baruah, B.K.: History of Assamese Literature, 1st edn. Sahitya Akademy Publica-
tion, New Delhi (1984)
 Morphotactics of Manipuri Verbs:
A Finite State Approach

H. Kenny Devi1 , Gaurav Raghav1 , S. Bobita2 , and L. Sarbajit Singh1(B)

1
Indian Institute of Information Technology, Manipur, Imphal, India
{kenny,g.raghav,sarbajit}@iiitmanipur.ac.in
2
Tezpur University, Tezpur, India
[email protected]

Abstract. The present paper is to study the morphotactics of Manipuri

verbs using Finite State Approach. Manipuri is a highly agglutinating
and morphologically complex Tibeto-Burman language. It has only two
word forms- noun and verb. The other word forms are derived from
verb by means of affixation. For instance, negation, noun, adjectives and
adverbs are derived from verbs and all the verbs in Manipuri are bound
in nature that means they cannot stand on its own. Affixes play a very
significant role in the word formation process. Verb covers the main mor-
phology of the language. Hence, it becomes mandatory to study Manipuri
verbs since it plays the major role in word formation process and set out
rules for various Natural Language Processing applications.

Keywords: Agglutinating · Morphotactics · Finite State Automata ·

Finite State Transducers

1 Introduction
Manipuri is a Tibeto-Burman language as well as a Scheduled Language of India
mainly spoken in the state of Manipur and other states like Assam, Tripura,
Mizoram, Nagaland and other countries like Bangladesh and Myanmar. Accord-
ing to [1] in the classification of T.B. languages Manipuri language comes under
the Kuki-Chin subgroup of the Burmese division. Shafer [2] places Manipuri in
the Meitei branch of Kukish section while [3] puts Manipuri in Kuki-Naga of
Kukish section. It is the medium of instruction in schools, colleges, universities,
governmental and non-governmental organizations and institutes. However, the
status of the language in the field of Natural Language Processing is not so
advanced as compared to other languages like Hindi, Bengali, Tamil and Kan-
nada. This is because of the fact that the language is highly agglutinating and
morphologically complex. A chain of string can be concatenated to derive a new
word which creates ambiguity in determining the Part of Speech of the given
word. Manipuri has extensive verb morphology and extensive suffixes with lim-
ited prefixation [4]. There are some works done in Manipuri verb but the setting
of morphotactics rule is not discussed so far or is not utilized. The paper is
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 53–63, 2020.
https://doi.org/10.1007/978-981-15-3666-3_5
54 H. K. Devi et al.

the introductory step towards because it deals with the designation of various
automata setting from the various rules. Testing the result is not covered but
saved for later work.

1.1 Objective

The main objective of the paper is to construct the theoretically motivated

computational model to represent the structure of Manipuri verbs by setting the
various rules of word formation by means of Finite-State-Technique. Finite-State
Techniques are known to be the most suitable techniques to represent the mor-
phosyntactic rule for the agglutinating languages and Manipuri is one of them.
The usefulness of an automaton for defining a language is that it can express
an infinite set in a closed form [5]. Their uses range from the compilation of
morphological and phonological rules [6]. The paper is divided into 5 parts. The
first part introduces the language in question with its features stating the main
objective of the paper. The second part discusses the role of morphotactics, one
of the essential techniques applied by the computer scientists in the formation
of morphological analyzer which is again one of the main applications of Nat-
ural Language Processing (NLP). The third part covers the main part of the
paper which discusses the Manipuri verb morphology stating the various affixes
responsible for the formation of negation, adjective, adverb and derived noun.
The fourth part covers designing the Finite State Automata for the above rules
with further suggestion to improve the work.

2 Morphotactics

The word morphotactics is formed by the combination of two words; morph and
tactics which means the possible combinations of morphemes. In other word, it
can be defined as the permissible and non-permissible combination of morpheme.
Morphotactics is one of the most useful techniques used by the linguists as well
as the computer scientists for developing the morphological analyzer of the word.
As for instance, the suffix -ity produces a noun from an adjective (capable >
capability), and -able creates adjective from verbs (eat > eatable). The reversed
order may violate the morphotactics of the language. All the languages of the
world have morphotactic rules or constraints. For Manipuri, it has two negative
allomorph suffixes /-te/ and /-de/. /-te/ is used after voiceless ending verbal root
(k@k ‘cut’+ te ‘NEG’ > k@kte ‘not cut’) and /-de/ after voiced ending sound such
as ca ‘eat’+ de ‘NEG’ > cade ‘didnt eat’.

3 Manipuri Verb Morphology

Manipuri verbs cover 80% of vocabulary and they are bound in nature. As stated
by [7] that words in Manipuri can consists of stems or bound root with suffixes
from one to ten. For instance, the word ca-rm–d-b-ni-ko-ne, where ca is the
 Morphotactics of Manipuri Verbs: A Finite State Approach 55

bound root and the rest are the suffixes. It is mandatory to identify all the
verbal roots available in the language along with the affixes. David [8] defines a
root as the base form of a word which cannot be further analysed without total
loss of identity. Negations are also derived from the verbs. As for instance, nt
(Not)+ -te (NEG) > nt.te (not true) and na (sick)+ -de (NEG) > na.de (not
ill). -te∼-de are negative marker added to the verbal roots to form negation. -te
is used after /t/, /k/ and /p/ sounds which are all voiceless and -de is used after
the following verbal roots ending with N, n, m, a, a, i, u sounds.
Rule 1: VR + -te∼-de > Negative
For instance,
jeN+de > jeNde ‘didnt see’
mun+de > munde ‘not ripe’
ph @m+de > ph @mde ‘not sit’
na+de > nade ‘not ill’
irué@+de > irué@de ‘not taken bath’
i+de > ide ‘not writen’
u+de > ude ‘not seen’

Q0 Q1
Q2
Start Root word -te ∼ -de

Fig. 1. Transition diagrams representation of the Rule 1

The Fig. 1 shows the Transition diagrams of Rule 1. This Rule is consisted
of 3 states. State Q0 is the starting state(intial state), on getting root word as a
input the FSA move from Q0 to Q1 state. On getting the input ∼ -te,∼ -de as
a input the FSA accept the string and identify as the Verb. Q2 is the final state
in the FSA (Tables 1 and 2).

Table 1. Table representation of above transition diagram

States Root word ∼te, ∼de

Q0 Q1 Φ
Q1 Φ Q2
* Q2 Φ Φ

Exception: There are only three verbal roots which take –te to make the
negative even though the verbal roots are ended with voiced sounds such as hau,
l@i, h@i as shown below: verbal roots ended with voiced sound + -te suffix to
form the negation.
56 H. K. Devi et al.

Table 2. Affixes used in negative formation.

Sl.no Affix Allomorphs Environment Affix type Category

1 -te -de After the following verbal Suffix Negative marker
roots ending with /t/,
/k/, /p/ sounds
2 -de -te After verbal roots ending Suffix Negative marker
with /ng/, /n/, /m/, /a/,
/a/, /i/ and /u/sounds
3 -k@nu -g@nu After voiceless sound Suffix Negative prohibitive
4 -g@nu -k@nu After voiced sound Suffix Negative prohibitive
5 -kum -gum After voiceless sound Suffix Let negative
6 -gum -kum After voiced sound Suffix Let negative
7 -suk Prefix Negative polarity Item
8 -ph at After voiced sound, occurs Suffix Negative Polarity Item
in negative environment
only
9 paN- Can occur in both positive Prefix
and negative environments
with different meanings.
verbal roots get
reduplicated
10 -tri -dri After voiceless sound Suffix Unrealized negative marker
11 -dri -tri After voiced sound Suffix Unrealized negative marker
12 Oi loi,roi,poi,doi,moi,Noi After /k/ ending Suffix Unrealized negative marker
morpheme only
13 -roi After vowel ending Suffix Unrealized negative marker
morphemes
14 poi After /p/ ending Suffix Unrealized negative marker
morpheme
15 moi After any morpheme that Suffix Unrealized negative marker
ends with /m/ sound only
16 -Noi After /N/ ending Suffix Unrealized negative marker
morpheme only

hao + te > haute ‘not delicious’

l@i + te > l@ite ‘do not have’
h@i + te > h@ite ‘unable to do’

Manipuri verbs are not inflected for number, person, gender and tense (or aspect
for Manipuri).
For instance,

tomb@gi hui m@Na lei

Tomba-POS dog five ASP
(Tomba has five dogs)

tomb@gi hui @m@l@i

Tomba-POS dog one ASP
(Tomba has one dog)
 Morphotactics of Manipuri Verbs: A Finite State Approach 57

There is no change with the change of number in the verb. Again,

m@hak Na ca-i
he fish eat-ASP
‘He eats fish’ (Habitual or only today) (to indicate Present aspect)

m@hak N@raN Na ca-i

he yesterday fish eat-SAM (to indicate Past aspect)
‘Yesterday he ate fish’

[email protected] Nasi Na l@i-da-rabadi hajeN hawai ca-i

you-NOM today fish buy-NEG-if tomorrow pulse eat-SAM
if you do not buy fish today, then tomorrow we will naturally eat pulse’ (Future
aspect).

The above three examples have been excerpted from the thesis done by Pre-
mabati [9]. The verb ca-i remains in all the three sentences.

3.1 Verbal Suffixes

Verbal suffixes are numerous than nominal suffixes due to the agglutinating
nature of the language. The lists of the verbal suffixes are given below:
1. Aspect: /i/ (Simple Aspect), /-ri/ (Progressive Aspect), /-re/ (Perfective
Aspect), /-g@ni ∼ k@ni/ (Unrealized Aspect)
2. Mood: niN as in @i waN-niN-i I Tall-wish-ASP (I want to be tall)
3. Negative Suffixes: As shown above/
4. Imperative Suffixes: -o/-mo/-po, -u/-mu/-pu/-nu/-ju, -ro/-lo and –ru/-lu
5. Deictic Suffixes: -r@/-ru, -r@k and -kh i.
6. Intensive Suffixes: -je/-ke
7. Suggestive: -si
8. Concessive: -si
9. Adverbial: -n@but there are other adverbial suffixes which force the verbal
root to reduplicate in order to intensify the meaning such as: -p@k, -s@n, -sin,
-drin etc.
10. Nominalizing: -p@/-b@to the verbal roots
11. Copula:-ni
12. Directional: -sin (inwards), -th ok (outward), -kh @(upward) and -
th @(downward).
13. Suffixes of Destruction: -kh ai (break into pieces), -th @t, -t@t, -th @k, -dek, -tet
14. Reciprocal Suffixes: -n@is the reciprocal suffix. It is used with the dynamic
verbs such as ph u-n@-b@Beat-REC-NOM ‘to beat each other’
15. Excessive Suffix: -m@n or -m@l as in ca-m@n-b@ ‘to eat excessively’
16. Causative Suffix: -h@n/-h@l as in ca-hen-b@ ‘to make him or her eat’
17. Reflexive Suffix: -c@/-j@as in th @k-c@-b@ ‘to drink oneself’
18. Honorific Suffix: -bi/-pi as in t@u+bi+b@> t@ubib@ ‘to help for someone’
19. Endearment suffix: -ko/co
58 H. K. Devi et al.

20. Habitual Suffix: -k@n/g@n

21. Superiority Suffix: -h@t as in m@pu-n@minai-siN-bu tu-h@t-li owner-NOM-
servant-PL-ACC-exploit-ASP ‘The owners are exploiting the servant’

Rule 2: If ∼p@ ∼b@ are added to these above dynamic verbal roots, it becomes a
derived noun. The Fig. 2 shows the Transition diagram of the Rule 2. This Rule
is Transition diagram using the 3 states. State Q0 is the starting state(intial
state), on getting root word as a input the FSA move from Q0 to Q1 state. On
getting the input p@ b@as a input the FSA accept the string and identify as the
Noun. Q2 is the final state. For example:

Verbal Root + Process of Nominalization > Derived nominals

Ca- ‘eat’ + -b@> cab@ ‘eating/to eat’
th @k- ‘drink’ + -p@> th @kp@ ‘drink/to drink’
tum- ‘sleep’ + -b@> tumb@ ‘sleeping/to sleep’
pok- ‘birth’ + -p@> pokp@ ‘birth’

Q0 Q1
Q2
Start Root word

Fig. 2. Transition diagrams representation of the Rule 2

Rule 3: But if we want to derive noun again from the negative verb, -te -de
changes into -t@∼ d@with the addition of a nominalizer -b@as shown below: The
Fig. 3 shows the Transition diagram of the Rule 3. This Rule is demonstrated
using the 4 states. State 0 is the starting state(intial state), on getting verbal
root as a input the FSA move from Q0 to Q1 state. On getting the input t@ d@the
FSA move from state Q1 to the state 2(Example 2). On getting nominalizer as
input from state Q2 or from state Q1 the FSA move to the final Step. This
accept the string which is derived noun (Tables 3 and 4).

Table 3. Table representation of above transition diagram

States Root word ∼p@, ∼b@

Q0 Q1 Φ
Q1 Φ Q2
* Q2 Φ Φ
 Morphotactics of Manipuri Verbs: A Finite State Approach 59

Nominalizer(NZM)

Q0 Q1 Q2 Q3
Start NZM
Verbal Root

Fig. 3. Transition diagram of the Rule 3

Table 4. Table representation of above transition diagram

States Verbal root ∼t@, ∼d@ NZM

Q0 Q1 Φ Φ
Q1 Φ Q2 Q3
Q2 Φ Q3 Φ
* Q3 Φ Φ Φ

Verbal root +Negative Suffixes +Nominalizer > Derived Noun

Ca- + -d@+b@> cad@b@ ‘one who doesn’t eat’
k@p- + -t@+b@> k@pt@b@ ‘one who doesn’t cry’

The various Stative bound roots are given below: Cao-, waN,-nem, -saN, -pak,
ph @, -mot, -nan, -mu, -sin, -ka etc. Nouns can be derived from these verbal roots
by suffixing nominalizer -p@/-b@as shown below:
Verbal root + Nominalizer > Derived Noun
Cao + -b@> caob@ ‘big/ name of a person’
waN + -b@> waN b@ ‘tall’
saN + -b@> saNb@ ‘long’ pak + -p@> pakp@ ‘broad’
ph @+ -b@> ph @b@ ‘good’

Nominalizer(NZM)

Q0 Q1 Q2 Q3 Q4
Start NZM
Verbal Root

Fig. 4. Transition diagram of the Rule 4

60 H. K. Devi et al.

Table 5. Table representation of above transition diagram

States ∼@ Verbal root ∼t@, ∼d@ NZM

Q0 Q1 Φ Φ Φ
Q1 Φ Q2 Φ Φ
Q2 Φ Φ Q3 Q4
Q3 Φ Φ Q4 Φ
* Q4 Φ Φ Φ Φ

Rule 4: From these derived nouns as shown above adjectives are derived by
prefixing @-. This is for the case where the verbal root is monosyllabic in nature.
The Fig. 4 shows the Transition diagram of the Rule 4. This Rule is represented
using the 5 states. State Q0 is the starting state, on getting @∼ as a input the
FSA move from Q0 to Q1 state. when followed by the verbal root the FSA look
for the rule 3. the final state in the FSA is Q4. By going through the state Q0-
Q1-Q2-Q4 (Rule 4.1), Q0-Q1-Q2-Q3-Q4 (Rule 4.2) (Table 5).

For instance,

@+ caob@> @caob@ ‘one that is big’

@+ waNb@ > @waNb@ ‘one who is tall’
@+ saNb@ > @saNb@ ‘one that is long’

N.B. Adjectives in Manipuri are mostly derived from stative verbs than
dynamic verbs.
Rule: 4.1 If the verbal root is disyllabic, only nominalizer is added followed by
a noun to modify the quality of the noun such as
nuNsi (2 VR) + b@(NMZ) is@i > nuNsib@is@i (sweet song)
So the syntax is Compound + NMZ > Adj
Rule 4.2: However, there are exceptional cases which is after suffixing the neg-
ative marker t@∼ d@ to the verbal root followed by the nominalizer -p@∼-b@, the
prefixation of @- is no more required to form the adjectives. For example,
*@+ca (VR)+d@(NM)+ b@(NMZ) > @cad@b@is not a valid word.
Rule 4.3: Again the prefix /@/ is not required if the verbal root is immediately
followed by an aspect marker and a nominalizer as shown below
Ca (VR) + ri (SAM) + b@(NMZ) cak > carib@cak (the food which is eating)

Rule 5: Again adverbs are derived from verbs with the addition of suffix -n@to
the verbal root as shown below:
 Morphotactics of Manipuri Verbs: A Finite State Approach 61

Q0 Q1 Q2
Start Verbal Root

Fig. 5. Transition diagram of the Rule 5

Table 6. Table representation of above transition diagram

States Verbal root ∼n@

Q0 Q1 Φ
Q1 Φ Q2
* Q2 Φ Φ

The Fig. 5 shows the Transition diagram of the Rule 5. This Rule is imple-
mented using the 3 states. State Q0 is the starting state(intial state), on getting
verb as a input the FSA move from Q0 to Q1 state. On getting ∼n@as input the
FSA accept the string and identify as the Adverb. VR + -n@(suffix) > Adverb
i.e. t@m (slow) + n@> t@mn@ ‘slowly’
ca (to eat) + n@> can@ ‘by eating’
Rule 5.1: But in order to express the adverb of manner, the verb will be redu-
plicated. There are two ways to express it. One is the complete reduplication
and the other is the partial reduplication. That is (i) t@m (VR) +n@+ t@m (VR)
+ n@> t@mn@t@mn@ ‘slowly slowly’
And, (ii) t@m (VR) + t@m (VR) + n@> t@m t@mn@ ‘slowly’ (Table 6).

The first one is the case for complete reduplication and the second one for
partial reduplication.
Now, the finite-State Machine (FSM) is constructed to represent the con-
struction of the negation, adjective, adverb and noun to identify each path that
takes the machine to a final state from its initial state as shown below: The
Fig. 6 shows the combination of all above 5 Rules. This FSA accept Verb, noun,

epsilon -(te,de,p ,b ,n ),Nominalizer(NZM)

Q0 Q1 Q2 Q3 Q4
Start NZM
Verbal Root

Fig. 6. Left to Right Transition diagram of FSA of Negation, adjective, adverb & noun
from verb
62 H. K. Devi et al.

adjective and adverb. State Q4 represent the final state(Verb), adjective, derived
noun and the adverb. There is two transition from Q0 to Q1 that is via epislon
or ∼@.

4 Designing the Finite State Machine for the Above

Rules

The use of morphotactics or the knowledge of morphotactics is responsible for the

morphological analysis of words which will be helpful for the various applications
of NLP such as spell-checking, Information Retrieval, machine translation etc.
The various affixes which are responsible for the formation of negation, adjective,
adverb and noun from the verbs are stored with a unique identity of numeric
value in the database. All the affixes responsible for the formation of negation,
adjective, adverb and derived noun are collected along with the syntax of their
formation. Finite State Automata or FSA for the various rules are constructed.
Non-Deterministic Automata is not possible for this case because there is no
single finite state. In other word, it can be said that we cannot identify the
validity of verbs but how verb is responsible for the formation of new word class.

5 Conclusion and Future Work

The morphological rules for the derivation of negation, adjective, adverb and
derived nouns are presented by using finite- state techniques. A hand crafted
rule automaton has been developed to implement the system in order to verify
the validity of verbs in Manipuri. Since verb covers the major part of Manipuri
Vocabularies. Hence, it is not possible to list down and identify all the verbs
available in the language. Again all the verbs in Manipuri are bound in nature.
Further it can be studied into two ways - Structurally and semantically [9]. For
this a huge amount of corpus is required. The present paper is restricted to the
formation of negation, adjective, adverb and derived noun by means of affixation.

Symbols and Abbreviations

Adj - Adjective
ASP - Aspect
DR - Derived Noun
NMZ - Nominalizer
NOM - Nominative case
PL - Plural
POS - Possesive Marker
VR - Verbal Roots
 Morphotactics of Manipuri Verbs: A Finite State Approach 63

References
1. Grierson, G.A.: Linguistic Survey of India, Part III, vol. III, p. 8. Motilal Banarasi-
das, Delhi (1927). (Reprinted 1967)
2. Shafer, R.: Classification of the Sino-Tibetan languages. Word 11, 94–111 (1955)
3. Benedict, P.K.: Sino-Tibetan: A Conspectus, pp. 4–11. Cambridge University
Press, Cambridge (1972)
4. Singh, C.Y.: Manipuri Grammar, pp. 31–55. Rajesh Publications, New Delhi (2000)
5. Jurafsky, D., Martin, J.H.: Speech and Language Processing, p. 16. Pearson Edu-
cation, Delhi (2000)
6. Kaplan, R.M., Kay, M.: Regular models of phonological rule systems. Comput.
Linguist. 20(3), 331–337 (1994)
7. Chelliah, S.L.: A Grammar of Meithei, p. 26. Mouton de Gruyter, New York (1997)
8. Crystal, D.: A Dictionary of Linguistics and Phonology, p. 267. Basil Blackwell,
Oxford (1985)
9. Premabati. The Syntax of Manipuri Verbs. Unpublished Ph.D Thesis, Dept. Lin-
guistics, Manipur University, p. 105 (2009)
10. Singha, K.K.B., Singh, K.R., Purkayastha, B.S.: Morphotactics of Manipuri adjec-
tives: a finite state approach. Int. J. Inf. Technol. Comput. Sci., 96 (2013). https://
doi.org/10.5815/ijitcs.2013.09.10
11. Abbi, A.: Reduplication in South Asian Languages: An Areal, Typological and
Historical Study, p. 12. Allied Publishers Ltd., New Delhi (1992)
 Proportional Analysis of Feature
Extraction for Tamil Handwritten
Characters Using Centroid Based Method

D. Shashi kala1,2 and M. P. Indra Gandhi1(B)

1
Department of Computer Science,
Mother Teresa Women’s University, Kodaikanal, India
[email protected], [email protected]
2
Department of Computer Science,
St. Joseph’s College (Autonomous) Bangalore, Bangalore, India

Abstract. A system for recognizing offline handwritten Tamil charac-

ters, using Feed Forward neural network, the primary objective of OHR
is enhancement of a software for processing the images with different
scripts and writing styles with high accuracy. The quality of the image
is enhanced using diverse pre-processing operations; the feature of the
character image is extracted using Centroid Based Distance Measure.
In this paper a general description of the Algorithm is presented and
classified using the Neural Network classifier. The proposed method is
used for testing of handwritten Tamil character and has made a com-
parative analysis of different Distance Measure like Euclidean distance,
Manhattan distance and Chessboard distance.

Keywords: Chessboard distance · Classification · Euclidean distance

and Manhattan distance

1 Introduction

Handwritten character recognition is a subfield of Optical character recognition;

it’s an encoded format of the scanned character image handwritten or printed,
which is understood by electronic device. The character image is considered as a
static representation for handwritten character. It is rigid to trace pen movement
of offline handwritten character compared to online character recognition. Offline
handwritten character recognition for Indian scripts is a remarkable research
as millions people use scripts [2]. India is multilingual country uses different
scripts for the documentation of its regional languages. They are Devanagari,
Guajarati, Bangla, Oriya, Gurumukhi, Malayalam, Telugu, Kannada, Tamil and
Urdu (Nastaliq). There is large variation in the writing styles of individuals such
as size shape, speed of writing, thickness of characters curves in the characters,
number of strokes and holes etc, the same character differ with each individu-
als which has made recognition of offline handwritten character difficult these
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 64–73, 2020.
https://doi.org/10.1007/978-981-15-3666-3_6
 Proportional Analysis of Feature Extraction 65

issues limit the accuracy of OHR systems. The proposed method is based on
feature extraction technique to recognize offline handwritten Tamil characters.
It’s a Dravidian and official language of Tamil Nadu belongs to south India
and also widely used in Pudhucherry, Singapore and Srilanka [5]. The structure
of language is complex due to its cursive shapes and loops. The Tamil script
embodies 12 vowels, 18 consonants and 1 special character named ayutha and
an additional 216 characters its derived from vowels and consonants all together
is about 247 characters.

2 Literature Survey

Raj and Abirami [8] proposed 8 directional Chain Code and Zone based Features
approach, calculate their area is additional feature used Support Vector Machine
classifier to achieve 80% accuracy for Tamil Handwritten Character.
Raj and Abirami [7] proposed a Zone based Statistical Features approach
and Eight directional Chain Code for each zone Axis based, pixel location And
Point counts, Used Support Vector Machine classifier to achieve 89% accuracy
for Offline Tamil Handwritten Character.
Rajashekararadhya [10] presented a Zone and Distance metric based feature
extraction method for the recognition of Kannada and Telugu numerals. Feed
forward back propagation neural network is used for classification and recogni-
tion with a recognition rate of 98% for isolated handwritten Kannada numerals
and 96% for isolated handwritten Telugu numerals.
Sangame [4] proposed a system for unconstrained handwritten Kannada vow-
els recognition based upon invariant moments. A Euclidian distance criterion and
K-NN classifier is used for classification of the Kannada vowels. The system had
overall accuracy of 85.53%.
Rajashekararadhya [9] proposed an offline handwritten numeral recognition
technique for four south Indian languages, based on zone and image centroid used
two different classifiers nearest neighbor and back propagation neural network
to achieve 99% accuracy for Kannada and Telugu, 96% for Tamil and 95% for
Malayalam.

3 Motivation and Objectives of the Proposed Method

The existing methods are based on 8 directional chain code, distance metric and
zone based the accuracy found for offline Tamil Character from Literature survey
was 80% and 89%. In order to increase the accuracy the proposed algorithm is
implemented based on Euclidean distance measure, Manhattan distance measure
and Chess board distance measure for Tamil vowels character to extract feature,
the method uses 8 directional chain code for feature selection.
66 D. S. kala and M. P. I. Gandhi

The main objective of proposed work:

1: Comparatively less work has been reported of offline Tamil character.

2: To prepare a dataset for Tamil character as no standard database is available.
3: In general accuracy rate is given for individual character or numerals. In our
experiment we have proposed overall accuracy rate for random three character
from Tamil Vowel letters.

4 Data Set for Our Experiments

A well structured and standard database of isolated handwritten character

images is developed by collecting samples from different age category and source
like college student, school and in house, as there is no cursive writing, each
symbol can be considered as a separate class for recognition. Total of about 100
writers and 3 samples of each class are collected. The data set of 100 writers has
been split into two group: The training set comprises 80% of samples which arbi-
trarily are chosen from our data set for each category of sample; the remaining
20% for testing set, a few of training data set is also used as testing. During image
acquisition process the image is acquired through a scanner, digital camera or
any other suitable digital input device, the scanned input image is converted to
JPEG format and resized into a standard size 60 * 60 pixels, without affecting
the real shape Fig. 1 shows few sample character from our real data set.

Fig. 1. Presents few samples of the Tamil character set used

5 Feature Extraction Using Centroid Based Distance

Measure

Recognition of Off-line Handwritten Character constitutes three basic

approaches they’re image preprocessing, feature extraction and classification.
First and foremost, quality of the acquired image can be enhanced using prepro-
cessing approaches, here it is assumed that the given character image is prepro-
cessed. Feature extraction is a very important field of image processing, an image
is stored in from a large matrix. Feature extraction is a method that is carefully
chosen to identify relevant attribute that is expected to be the feature set and
remove irrelevant and redundant attributes from data that do not contribute to
the accuracy. In order to extract features, the proposed method uses structural
feature which is measurement of the geometrical quantities of the character such
as ratio between width and height of bounding box, relative distance between
 Proportional Analysis of Feature Extraction 67

the horizontal and vertical, distance between two point etc. In the proposed work
8 directional chain code is used for feature selection and centroid based distance
measure is considered to extract features, we use the methods such as Euclidean
distance measure, Manhattan distance measure, Chessboard distance measure
to extract features [6]. Finally training of extracted features for recognition of
characters is given as input to a neural network for further classification, the
algorithm to extract feature is given below:

Input: Characters
Output: Classification

5.1 Centroid Based Distance Measure Method

Step 1: For each character find out the Centroid of character.

Step 2: From the centre we travel in 8 direction of chain code
Step 3: Find out the First Black pixel and Last Black pixel for each direction
Step 4: From the centroid to the coordinate of Black pixels for each direction
is calculated using different distance measure given below.

This work is used to calculate the distance from the centroid to the pixels
in each direction using Euclidian distance formula, Manhattan Distance formula
and Chessboard Distance formula. The image is rotated in different angle 0◦ ,
90◦ , 180◦ and 270◦ the image is rotated to prove that it is invariant to rotation.
The above steps are repeated for each degree. Our feature vector consists of 16
features to recognize the character.

5.1.1 Distance Measure

Euclidean Distance: Euclidean distance is the ultimate distance measure
method, It is used to measure the length segment between two points. which
is also known as L2 norm or nearest neighbor classifier.

Edis = Sqrt((x1 − x2)2 + (y1 − y2)2 ) (1)

Manhattan Distance: The Manhattan distance is the absolute sum of the

difference between two point. It is also known as City Block distance or Taxicab
distance (“Taxicab geometry,” 2016).

Mdis = |x1 − x2| + |y1 − y2| (2)

Chessboard Distance: The Chessboard distance is the maximum distance

between two points. It is also known as Chebyshev distance or maximum metric
(“Chebyshev distance,” 2016)

Cdis = M ax(|x1 − x2|, |y1 − y2|) (3)

68 D. S. kala and M. P. I. Gandhi

6 Experimental Results
The classification plays a special role in OHR the extracted feature are given
as an input to the neural network. An artificial neural network trained with
feed forward algorithm is used here to classify and recognize the handwritten
characters.
A multilayer neural network classifier comprises of three layer: hidden layer,
input and output layer. The hidden layer uses log sigmoid, bias is found both in
output and hidden layer. Input layer is connected to output layer through hidden
layer and output layer is connected to the output. They are connected by means
of weight whose output is always one. In this work the total of neurons in the
output layer and input layer is 3 and 16 respectively. Figure 2 shows Multilayer
neural network.

Fig. 2. Architecture of multilayer neural network

This work uses MATLAB R2015a as tool for implementing the algorithm
and designed using neural network, this data stored in Microsoft excel and fed
in MATLAB, where the data was loaded in Neural Network function to get the
accuracy rate for different distance measure. The experiment result and over-
all accuracy achieved for different character samples (100 * 3 = 300) are imple-
mented using Euclidean distance, Manhattan Distance and Chessboard distance
are respectively tabulated in the following Table 1.
A confusion matrix is often used to illustrate the performance of a classifier,
it is used to list correct classifier as true positives or true negatives, incorrect
classifier as false positive or false negatives. Error rate and accuracy are most
common predict, it also gives imminent into the types of errors made by classifier.
For the experiment a confusion matrix of 3 * 3 is considered.
 Proportional Analysis of Feature Extraction 69

Table 1. Show the experimental results for different character and accuracy percentage
for different degree.

6.1 Parameters Considered for Proposed Algorithm

Accuracy (ACC): Total true Positive +true negative divide by Total positive
+Total negative.
T rue Positive Rate(TPR): Total true positive divide by Total true Positive
+ Total false negative.
F alse Negative Rate(FNR): Total false negative divide by Total false nega-
tive + Total true positive.
T rue Negative Rate (TNR): Total true negative divide by Total false Posi-
tive + Total true negative.
F alse Positive Rate (FPR): Total false positive divide by Total true negative
+ Total false positive.
70 D. S. kala and M. P. I. Gandhi

Fig. 3. Outcomes of a classifier

Confusion matrix shows the classification classes for the dataset, the value
given diagonally in 3 * 3 matrix is True positive rate for class 1, class 2, class 3
and the other rate (FN, FP, TN) are calculated based the Fig. 3. The average
of diagonal value gives the Accuracy of the each degree and Error Rate is (1-
Accuracy) the Fig. 4 shows the confusion matrix for different distance measure
and Rotation. The image is rotated at an angle of 90◦ to prove that algorithm
is invariant to rotation.
The TP rate is calculated for each class that is class 1, class 2 and class3
separately and the average TPR of all the classes is taken. Similarly the other
rates TNR, FPR and FNR are calculated. The best rate for TPR and TNR
is 1.0, whereas the worst rate is 0.0. The best rate for FNR and FPR is 0.0
and worst rate is 1.0. This means most positive classes are recognized correctly.
Table 2 show the Performance Rate of Euclidean, Manhattan and chess board
distance.
 Proportional Analysis of Feature Extraction 71

Fig. 4. Shows the representation of confusion matrix for rotation for (a) Euclidean
Distance rotation of 0◦ , 90◦ , 180◦ , 270◦ respectively (b) Manhattan Distance rotation
0◦ , 90◦ , 180◦ , 270◦ respectively. (c) Chess Board Distance rotation 00 , 900 , 1800 , 2700
respectively in Vertical wise
72 D. S. kala and M. P. I. Gandhi

Table 2. Shows the TPR, TNR ,FNR and FPR rates for distance measure

Distance measure Degree of rotation and performance

Degree TPR FNR TNR FPR
Euclidean distance 0 0.92 0.08 0.96 0.03
90 0.92 0.08 0.96 0.03
180 0.90 0.09 0.95 0.05
270 0.93 0.07 0.95 0.03
Manhattan distance 0 0.92 0.07 0.96 0.03
90 0.90 0.09 0.95 0.05
180 0.95 0.06 0.97 0.03
270 0.93 0.06 0.95 0.04
Chess board distance 0 0.93 0.06 0.97 0.02
90 0.93 0.07 0.96 0.03
180 0.95 0.04 0.97 0.02
270 0.96 0.04 0.97 0.02

7 Conclusion
The proposed features are found to be effective for Tamil offline handwritten
characters. This work has been implemented using centroid based distance mea-
sure by choosing 3 letters from Tamil real character dataset which was generated
by us and compared the experimental results for Euclidean distance, Manhattan
distance and Chessboard distance. It was found that results comparable to the
best result of Chess board distance was 95.3%, 93.3%, 96.0% and 96.3% when
the image was rotated to different angles of 90◦ such as 0◦ , 90◦ , 180◦ , and 270◦
respectively. It is observed that rotation accuracy vary in Euclidean and Man-
hattan distance measure and better in chess board distance measure. The TNR
and TPR value of chess board distance is better than other distance measure
The error rate is basically due to unusual writing style, which arise haziness
among similar shaped characters. Future work our aim is to consider all Tamil
characters for recognition. Also hoping to develop method to find relative error
in terms of distance measure.

References
1. Jain, A.K., et al.: Feature extraction methods for character recognition-a survey.
Pattern Recogn. 29(4), 641–662 (1996)
2. Kudo, M., Sklansky, J.: Comparison of algorithms that select features for pattern
classifiers. Pattern Recogn. 33(1), 25–41 (2000)
3. Jagadeesh Kannan, R., Prabhakar, R., Suresh, R.M.: Offline cursive Tamil hand-
written recognition. In: IEEE International Conference on Security Technology,
vol. 4, pp. 159–164 (2008)
 Proportional Analysis of Feature Extraction 73

4. Sangame, S.K., Ramteke, R.J., Benne, R.: Recognition of isolated handwritten

Kannada vowels. Adv. Comput. Res. 1(2), 52–55 (2009). ISSN: 0975-3273
5. Sampath, A., Tripti, C., Govindaru, V.: Freeman code based online handwritten
character recognition for Malayalam using backpropagation neural networks. Adv.
Comput.: Int. J. 3(4), 51–58 (2012)
6. Tokas, R., Bhadu, A.: A comparative analysis of feature extraction techniques for
handwritten character recognition. Int. J. Adv. Technol. Eng. Res. 2(4), 215–219
(2012)
7. Raj, M.A.R., Abirami, S.: Offline Tamil handwritten character recognition using
statistical features. Adv. Nat. Appl. Sci. 9(6), 367–374 (2015)
8. Raj, M.A.R., Abirami, S.: Offline tamil handwritten character recognition using
chain code and zone based features. Indian J. Sci. Technol. 8(S7), 110–116 (2015)
9. Rajashekararadhya, S.V., Vanaja, R.P.: Efficient zone based feature extraction
algorithms for handwritten numeral recognition of Indian scripts. Int. J. Inf. Pro-
cess. 2(4), 15–28 (2008)
10. Rajashekararadhya, S.V., Vanaja Ranjan P.: Zone-based hybrid feature extraction
algorithm for handwritten numeral recognition of two popular Indian scripts. In:
Proceedings of the IEEE, World Congress on Nature and Biologically Inspired
Computing, pp. 526–530 (2009)
 Mobile Supported Interaction Modeling
to Find Engagement of Acolyte
in Live Classroom

Barnita Debnath(B) , Suman Deb, and Pooja Kumari

National Institute of Technology Agartala, Agartala, Tripura, India

[email protected], [email protected], [email protected]

Abstract. The affordable small devices have changed people’s lives in

the last decades exponentially, especially mobile phones because peo-
ple are now moving with mobile devices in a pervasive manner. These
changes influenced human attention in positive and negative both man-
ner. This work, try to explore how these small devices can affect people’s
attention in the live classroom. Mobile can be used as a device to identify
how mobile usage effects the level of engagement during the live class-
room. During learning and listening activity maintaining the concentra-
tion of acolyte through mobile devices has become a vital issue. Research
on maintaining attentiveness during learning can offer new insights into
the selection of m-learning topics. This study used a quasi-experimental
design to investigate the concentration and understanding of knowledge
of the student in the traditional classroom as well as the classroom where
everyone used mobile phones for learning purposes. Ninety numbers of
undergraduate engineering students participated in the study who were
divided into an equal number of unbiased groups to be treated as a con-
trol and treatment group. After performing an independent sample t-test,
results are obtained like: correlations existed between the interest and
attention also in between attention and success in the mobile learning
concept. Research findings entail the correlation between the student’s
score and engagement which can be identified using mobile devices can
give a formative influence on the teacher and student to identify the
time gap among students in understanding coursework that can stair
the teacher to drive for a better class. The limitation of the study also
identified in this work.

Keywords: Attention model · Interaction sequence · Live classroom

interaction · Mobile interaction · Mobile learning

1 Introduction

In the present era where virtualization and inter connectivity have covered the
world in multi-layer interaction, which prevails more in virtual than in actual.
Digital devices have more latent to enhance the performance in the classroom,
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 74–86, 2020.
https://doi.org/10.1007/978-981-15-3666-3_7
 Mobile Supported Interaction Modeling to Find Engagement of Acolyte 75

[5] increasing the ability of the students, making the classroom more interac-
tive, engaged students in exciting ways. In the area of the online environment,
Problem-solving capability is also prolonged. Wireless devices i.e., mobile devices
stimulate learners to learn and maintain their interest in learning. Nowadays
learning through mobile devices at different education levels is more popular.
Mobile learning [9,12] field also refers to the use of mobile devices for learning
purposes. Mobile devices facilitate students to instant interaction with real con-
texts and also allow them to join in different learning activities. Several studies
[1,5,12] suggest that mobile learning gives a positive result in concentration, atti-
tudes, behavior, and learning achievements. On the other hand, many researchers
[18–20] give negative responses about mobile learning and they also suggest
that there is also a need for further investigations. Maintaining the attention of
learners during the learning period through a mobile device is also a big issue.
This study mainly focuses on the learning attention of college students who use
m-learning [9] concept in the classroom and also entail that correlation exist
between the students score and engagement.

2 Literature Review

2.1 Classroom Learning with Mobile Devices

Mobile devices can support the flexible and real-time learning environment. The
traditional classroom can be transformed into a new form by adapting the mobile
devices in the classroom. Different studies [1–3] find the actual use of student
response system(SRS) in classroom. By using mobile devices students can eas-
ily response to the question-answer and also asked questions to their teach-
ers. Thus SRS can improve classroom interactivity, and also motivate students.
Mobile devices used in the classroom do not mean students understand every-
thing or fully rising the attention of the students. But mobile devices had that
much potential that they can promote the active learning technique in classroom
Mobile technology types which make sense for the classroom are e-readers: In
the traditional classroom, students use textbooks for study purpose. But the
main issue about this fact is they are so swiftly outdated regarding the subject
matter. But e-readers eliminate that issue because they allow real-time updates
for the learners. Individual mobile modules: Educational games and apps allow
the user for individual log in. Students can take extra time according to their
need and it also gives the chance at their own speed. Seamless cloud learning:
Mobile technology allow students to direct connection with the cloud. This facil-
ity allows them to transition from working, it may be the classroom, home or
anywhere else. Thus time is saved and skill also developed.

2.2 Learning Concentration in e-Learning and m-Learning

Concentration means the capability of a person to direct his/her thinking toward

those directions where they want to intends. This ability is performing a great
76 B. Debnath et al.

role in processing information efficiently and successfully. Learning concentration

means how much attention giving by an acolyte to under each and everything
of the learning task. This is also measured by the learning outcome. According
to (Kydd & Ferry) [4], e-learners should give more attention compared to the
traditional learner to understand any topic and obtain a better outcome. Con-
centration is the most important factor for the learner for finding themselves in
the state of flow. Different studies have been conducted for better improvement in
the concentration of learners in the e-learning area. (Han, Jo, park) [5] developed
the first e-learning home robot, which can automatically update the e-contents
and also assist the learning to improve the learning performance. Several stud-
ies also used eye-tracking technology for evaluating the attention when they are
viewing some websites or board work. This eye-tracking technology capture data
accurately. For example tracking the movement of the eyes, eye blinking. These
can help researchers to asses the efficiency of learning tools and objects. However
several studies are only focused on analyzing the concentration of the learners
using mobile devices. E-learning provides the learner’s mobility, more litheness
so that students can learn from anywhere and anytime. In contrast, many other
researchers (Hwang and Wu 2014; Sharples [5]) also said that it is difficult to
maintain concentration in the case of mobile learning.

2.3 Smartphone Technology

Mobile phones mainly consist of the user interface, external interface, and service
interface. User Interface refers to hardware factors, input methods, detachable
parts, etc. The external interface is those factors that are not the physical part of
the device but they promote Smartphone usage. The external interface includes
user support, accessories, supporting software, etc. Lastly, the service interface
contains service coverage, availability, utility, and interoperability. (Parhi, Karl-
son, Bederson et al.; Sanchez, Branaghan; Molina, Mahan and IIIingworth) [6–8]
they showed in their paper that smartphones had advantages as well as disad-
vantages in interface contexts. The advantages of smartphones in the interface
field are discussed below.

Portability: Mobile phones are portable devices. For these mobility characteris-
tics, it can be moved or carried anytime and anywhere.

Screen size: Smartphones have smaller screens, which makes them pocketable
and convenient. But a smaller screen size can be used as a disadvantage since
phone screens put up less number of contents compared to a personal computer
or notebook.

Touchscreen: Smartphone has a finger driven touch interface. By touching the

contents it will be accessible.

Connectivity: By using the 4G and wifi technology of smartphone users can

connect easily with nearby peer devices.
 Mobile Supported Interaction Modeling to Find Engagement of Acolyte 77

Ram and processing power: These two core components helps to store program
and run program easily.

Accessories: S-pen is under the accessories part, it helps the user to create much
work easily which are usually takes more time. By using this work will has done
a very quick and friendly manner.

Software factors: Mobile device allows much educational software, which allows
students to do some educational work.

2.4 Cell Phone Learning Strategies:

– Recording Lectures: The “Flipped” Classroom: In the flipped classroom, stu-
dents recorded using audio or video what the teacher tries to explain or teach.
Those recording they can listen again and again outside the classroom while
they try to do their homework. This can also help them to clarify their doubts.
In this strategy student can collaborate in online conversation, online video
lectures they can listen at home.
– Delivering materials: Nowadays everything is available digitally. The teacher
can directly send materials to all the students digitally by using a cell phone,
One of the examples of this type of system is School Town. This system makes
it possible for learners to take part in collaborative learning and improving
their problem-solving skills.
– Teacher Apps: Evernote: One of the coolest apps for the classroom is Ever-
note. This app allows students to take notes by one click and it also allows
students to find the web pages. Dropbox: The teacher can put the class mate-
rials into the dropbox and it will be accessible for all the devices those who
belong to that dropbox group.

3 Problem Statements of Traditional Classroom

– For functioning brain and holding information in mind it is necessary to take a
few minutes to break after a certain period. Sitting in one position for a long
time students get loose their focused. In the traditional classroom system,
students are more concentrating on taking notes rather than understanding
new concepts.
– In traditional system students give more effort on theory and less on the prac-
tical. In the classroom, times are also bounded professor teaches those things
what needs to be and fail to realize that students need to apply practically
what they understand and their concepts also. Traditional classroom gives a
better understanding of solving problems but cant gives practical knowledge.
The skill and knowledge which they learn through practical experience, a
traditional classroom cant give them.
– Every student is dependent only on teachers in a traditional classroom. One
single person can not give equal attention at a time to all the students. For
introvert students, traditional classrooms maybe not useful as they are not
participating in classroom activities.
78 B. Debnath et al.

– In the traditional classroom, if the teacher didn’t give equal attention to all
the learners in the classroom then it will be the cause of the destruction of
their self-confidence. As a single person, it can’t be possible to give equal
attention all the time.

4 Purpose of Study and Research Questions

This study aims to investigate the effect of mobile phones in the classroom and
also try to find out the learning concentration of acolyte who use mobile phone
in classroom for learning purpose.

Table 1. The questionnaire about concentration for mobile learning.

Item Description
(1) I try to give my best in mobile learning
(2) I can fully concentrate on my learning contents by ignoring the surrounding noises
(3) During mobile learning I lose my concentration
(4) I can easily concentrate on my topics in case of mobile learning
(5) In the mobile learning activity my mind wandered
(6) During mobile learning it is hard to maintain concentration after 25 min
(7) In mobile learning I can easily grip everything
(8) In the mobile learning activity I easily forgot what I learn or listen

(1) What is the relationship among mobile learning,interest, concentration, and

achievement?
(2) Do mobile learning, interest, and learning concentration generate an inter-
action effect on learning achievement?

5 Method

This method consists of participants and instruments. All the participants were
belongs to NIT Agartala campus and instrument contains m-learning devices.

5.1 Participants

The participants comprised 90 bachelor of technology 2nd students at NIT Agar-

tala. Mean age of the participants are 17.3833 years (SD = .59817) and randomly
divided all the students into two groups with equal participants. Everyone had
smartphones and they are very well about the operation of smartphones. One
group name is control group and another group name is treatment group. We
assigned 10 question to each of the group as a part of their course study. After
 Mobile Supported Interaction Modeling to Find Engagement of Acolyte 79

analyzing their responses we find that both group had same output. So, by con-
sidering their result we can say that there is no difference between both the
group. This was our pre test.
After this survey we took post test. Assigned same question again but that
time smartphone was allowed to one of the group, which is our treatment group.
Another group also did the same task, but were not allowed to use smartphone.
After analyzing the result of the post test found that control group result was
poor compare to the treatment group (Tables 2 and 3).

Table 2. Independent samples pre test

t df p Mean Difference SE Difference 95% CI for Mean Difference Cohen’s d

Lower Upper
Marks 0.000 88.000 1.000 0.000 NaN −5.980 5.980 0.000

Table 3. Test of equality of variances (Levene’s)

F df p
Marks 5.751e−32 1 1.000

Table 4. Group descriptives pre test

Group N Mean SD SE
Marks 1 45 65.022 14.274 2.128
2 45 65.022 14.274 2.128

5.2 Instruments
By some m-learning questions, questions on learning concentration and perform-
ing a pre-test and post-test this quasi-experiment is designed.

M-Learning Questionnaire: questions was tested for finding the interest of

the students. Every question carry some responses. Responses are “Strongly
agree”, “agree”, “neutral”, “disagree”, “strongly disagree”. Generally for relia-
bility purpose Cronbach’s alpha should be greater than or equal to 0.70. And for
this study reliability was 0.86. Thus it implies that study questions are reliable
(Table 4).

Concentration Questionnaire: By using concentration questionnaire stu-

dents concentrations was examine while they use mobile phone for learning pur-
pose. Table 1 show the all the questionnaire which were used. Questionnaires
were evaluated by five point scale and it was scored 0.82 by Cronbachs alpha.
80 B. Debnath et al.

Fig. 1. A diagrammatic representation of the literature search and review process

Post Test: Post test was conducted between the 90 students. All the students
are divided into the two groups one group used mobile phones and others are
not. Table 5. Shows the result of the post test.

Fig. 2. Interpolation for data 1 Fig. 3. Interpolation for data 2

Table 5. Independent samples post T-Test

t df p Mean SE 95% CI for Mean Cohen’s d

Difference Difference Difference
Lower Upper
Marks −0.660 88.000 0.511 −1.889 2.863 −7.578 3.800 −0.139

Figures 4 and 5: shows the histograms of the students marks for group1
(Control group) and in group 2(Treatment group). Visually, This diagrams rep-
resent that Students in treatment group means who were use mobile phone in
 Mobile Supported Interaction Modeling to Find Engagement of Acolyte 81

Fig. 4. Interpolation for data 1 Fig. 5. Interpolation for data 2

Table 6. Test of equality of variances (Levene’s)

F df p
Marks 0.707 1 0.403

Table 7. Group descriptives post test

Group N Mean SD SE
Marks 1 45 65.022 14.274 2.128
2 45 66.911 12.845 1.915

Fig. 6. A diagrammatic representation of the literature search and review process

82 B. Debnath et al.

classroom for study purpose getting better marks on average compare to Control
group who were not use mobile phone in classroom (Tables 6 and 7).
This (Fig. 6) plot diagram shows the mean result of the students who are in
Control Group as well as Treatment group. Error bars shows 95 % confidence
intervals approximately the mean value. Visually, it shows that there is a differ-
ence between the groups.

6 Motion Detection
By using smartphone motion can be detected. Accelerometer, the rotation-
sensing gyroscope can be used as the main tool for collecting data. This study
also used the sensor lab app for collecting data. Nowadays every student is having
a mobile phone and they also bring them in the classroom. We have examined
the data by using the mobile phone for some classroom activities as well as by
keeping it on the table. After analysis, those data key taps reveal that in which
direction and how much a smartphone rotates. Here the value “X” shows the
movement along the horizontal and “Y” Shows the vertical movement. Table 8
shows the data when the mobile phone was used for different classroom activi-
ties during the classes and Table 9, shows the data when the mobile phone was
kept on the table during the classes. It is clearly shown that while the phone
was used for doing classroom activity maximum rotation has happened. After
analyzing those value we conduct a controlled experiment, we ask participant
(NIT Agartala student) to perform different study-related activities during class
for five minutes time stamp during the class. A sensor lab app is installed in
the smartphone which was provided by us for data collection. The app runs in
the background of the smartphone for collecting the values of the three sensors(
accelerometer, gyroscope, and magnetometer). After that, a small test was taken
from the topic that was covered during the class. In which he scored 6 out of
10. The same experiment was again done but this time that smartphone was
kept in the table. Again test was taken from the covered topic, now this time he
scored 8 out of 10. All the collected values are manually labeled and analyzed in
the JASP 0.10.2.0 statistical tool. By analyzing the collected data observed that
their some distinct patterns for both the experiment as shown in Figs. 7 and 8.

Table 8. Data captured while mobile phone was used for educational activity

x y z
Valid 15300 15300 15300
Missing 0 0 0
Mean 0.005 −0.769 9.777
Std. Deviation 0.470 0.586 0.285
Minimum −9.596 −10.285 3.486
Maximum 6.522 11.042 16.290
 Mobile Supported Interaction Modeling to Find Engagement of Acolyte 83

When the smartphone was placed in the table there is less movement happened
so the density of the x,y,z axis also less as compare to the graph which shows
the smartphone used as an activity.
The above observation affirms that it is possible to identify that there is a
relationship present between the motion and attention.

Table 9. Data captured while mobile phone was on table

x y z
Valid 387 387 387
Missing 0 0 0
Mean 2.818 3.185 8.303
Std. Deviation 2.518 2.074 0.919
Minimum −4.434 −1.015 2.662
Maximum 7.604 8.092 13.005

Fig. 7. A diagrammatic representation of data while smartphone used for educational

activity

7 Limitation
There is a few limitation in mobile learning concepts
84 B. Debnath et al.

Fig. 8. A diagrammatic representation of data while smartphone were keptin table

– Screen size: Mobile devices have smaller screen size where student is required
to strain their eyes for long time in the concept of mobile learning.
– Cost: For mobile learning mobile devices are needed. As we know nowadays
most of the people carry mobile phone but still there are so many poor people
exist they can’t effort mobile phone since it is relatively expensive.
– Limited storage: Memory storage of mobile devices is not enough for students
to store all the classroom courses, offline data.
– Unexpected distraction: In the Mobile learning concept distraction can hap-
pen in many ways e.g., in the form of notification, call, messages etc.
– Short battery life: Battery life of mobile devices is very short. Since many
apps are active all the time in the mobile devices so, they discharge quickly.
This is one of the big limitations in mo-learning concept.
– Limited connectivity: Mobile devices have limited bandwidth; it may be create
difficulties to access online contents.
– Technological Problem: Mobile devices have different operating system and
different hardware. This may create difficulties in learning concept.
– Safety: One of the major problem is that safety for users. Preventing users
from cyber-bullying, personal bio hack, preventing them from the access of
illegal information in classroom is one of the key challenge.

8 Conclusion

This study investigate the interaction effects of learning interest and improve-
ment in learning of all the students who use their mobile phones for learning
purposes in the classroom. The major finding of this paper as follows: essential
 Mobile Supported Interaction Modeling to Find Engagement of Acolyte 85

correlations present between the interest and concentration. This paper also find
the good result for mobile learning concept in the classroom. First, the learning
progress of the acolyte also measured by conducting the pretest and posttest.
Second, the learning concentration of the students also measured by the result
of the test. Third, it also shows that during the m-learning concept there maybe
a relationship present between the movement of the mobile phone and the con-
centration of the students. Actually concentration during study may differ in
various factors like educational content, motivational speech, superiority of the
course design, gender, formative assessment technique etc.

References
1. The Interaction of Pedagogical Approach, Gender, Self-Regulation, and Goal Ori-
entation Using Student Response System Technology. J. Res. Technol. Educ.
(2019). https://www.tandfonline.com/doi/abs/1
2. Gauci, S.A., et al.: Promoting student-centered active learning in lectures with
a personal response system. - PubMed - NCBI. https://www.ncbi.nlm.nih.gov/
pubmed/192617
3. Internationalization testing for a multilingual based wireless response system: a
case study - University of Huddersfield repository (2019). Accessed 2 Sept 2019
4. Kydd, C.T., Ferry, D.L.: Managerial use of video conferencing. Inf. Manage. 27(6),
369–375 (1994)
5. Using mobile phones in college classroom settings: effects of presentation mode
and interest on concentration and achievement - PDF Free Download (2019).
kundoc.com. https://kundoc.com/pdf-using-mobile-phones-in-college-classroom-
settings-effects-of-presentation-mode-a.html. Accessed 2 September 2019
6. Parhi, P., Karlson, A., Bederson, B.: Target size study for one-handed thumb use
on small touchscreen devices. In: Proceedings of the 8th Conference on Human-
Computer Interaction with Mobile Devices and Services - Mobilehci 2006 (2006)
https://doi.org/10.1145/1152215.1152260
7. Sanchez, C.A., Branaghan, R.: Turning to learn: screen orientation and reasoning
with small devices. Comput. Hum. Behav. 27, 793–797 (2011). https://doi.org/10.
1016/j.chb.2010.11.004
8. Molina, N.A., Mahan, R.P., Illingworth, A.J.: Establishing the measurement equiv-
alence of online selection assessments delivered on mobile versus nonmobile devices.
Int. J. Sel. Assess. 22(2), 124–138 (2014)
9. Garcı́a-Cabot, A., De-Marcos, L., Garcı́a-López, E.: An empirical study on m-
learning adaptation: learning performance and learning contexts. Comput. Educ.
82, 450–459 (2015). https://doi.org/10.1016/j.compedu.2014.12.007
10. Lee, H., Parsons, D., Kwon, G., Kim, J., Petrova, K., Jeong, E., Ryu, H.: Coop-
eration begins: encouraging critical thinking skills through cooperative reciprocity
using a mobile learning game. Comput. Educ. 97, 97–115 (2016). https://doi.org/
10.1016/j.compedu.2016.03.006
11. Shu, H., Gu, X.: Determining the differences between online and face-to-face
student-group interactions in a blended learning course. Int. High. Educ. 39, 13–21
(2018). https://doi.org/10.1016/j.iheduc.2018.05.003
12. Pedro, L.F.M.G., Barbosa, C.M.M.O., Santos, C.M.N.: A critical review of mobile
learning integration in formal educational contexts. Int. J. Educ. Technol. High.
Educ. 15(1), 1–15 (2018). https://doi.org/10.1186/s41239-018-0091-4
86 B. Debnath et al.

13. Heflin, H., Shewmaker, J., Nguyen, J.: Impact of mobile technology on student
attitudes, engagement, and learning. Comput. Educ. 107, 91–99 (2017). https://
doi.org/10.1016/j.compedu.2017.01.006
14. Balta, N., Perera-Rodrı́guez, V.-H., Hervás-Gómez, C.: Using socrative as an online
homework platform to increase students’ exam scores. Education and Information
Technologies 23(2), 837–850 (2017). https://doi.org/10.1007/s10639-017-9638-6
15. A user-friendly voting and quiz system for classroom use with connected devices.
In: IEEE Conference Publication (2019). https://ieeexplore.ieee.org/document/7.
Accessed 22 July 2019
16. Influence of Digital Equipment on Interaction Quality in Technology-rich
Classroom. In: IEEE Conference Publication (2019). https://ieeexplore.ieee.org/
document/7757022. Accessed 22 July 2019
17. Sung, Y., Chang, K., Liu, T.: The effects of integrating mobile devices with teach-
ing and learning on students’ learning performance: a meta-analysis and research
synthesis. Comput. Educ. 94, 252–275 (2016). https://doi.org/10.1016/j.compedu.
2015.11.008
18. Jumoke, S., Blessing, O.: Analysis of mobile phone impact on student aca-
demic performance in tertiary institution (2015). https://www.semanticscholar.
org/paper/Analysis-of-Mobile-Phone-Impact-on-Student-Academic-Jumoke-
Blessing/00ad8a3fdad62581f83d0a8cf46. Accessed 22 July 2019
19. Uğur, N., & Koç, T.: Mobile phones as distracting tools in the classroom: college
students perspective. Alphanumeric J. 3(2) (2015). https://doi.org/10.17093/aj.
2015.3.2.5000145549
20. Osei Frimpong, K., Asare, S., Otoo-Arthur, D.: The effects of mobile phone usage
on the academic performance of Ghanaian Students, a case of Presbyterian Uni-
versity College Asante-Akyem Campus (2016)
 Contribution of Application of Deep
Learning Approaches on Biomedical Data
in the Diagnosis of Neurological
Disorders: A Review on Recent Findings

Waseem Ahmad Mir1(B) , Izharuddin1 , and Iqra Nissar2

1
Aligarh Muslim University, Aligarh 202002, Uttar Pradesh, India
[email protected]
2
Jamia Millia Islamia, New Delhi 110025, India

Abstract. Recent developments in Artificial intelligence and their

applications in Medical health diagnosis have shown promising results
especially in the diagnosis of brain disorders, thereby paving the way for
better healthcare and early detection of Neural disorders. Several diag-
nostic tests are routinely performed on humans to evaluate the nature
of brain signals and images. These diagnostic tests typically rely on the
physiological signals and neuroimaging analysis commonly performed by
an expert manually. Large databases of clinical data of recorded brain
signals and images are available. Secondary use of these medical records
for precision medicine and predictive analysis involves machine learning
which has fueled the excitement of medical experts and data scientists
equally. For an effective medical diagnosis, intelligent behavior is the
prerequisite for learning, and there has been a deep insight in this par-
ticular area for small diagnostic problems. Symbolic learning, statistical
methods and neural networks in a broader perspective have contributed
to the development of machine learning in general and medical diagno-
sis in particular. Deep learning, a recent trend in machine learning has
been able to produce exceptionally accurate results in the diagnosis and
treatment of various diseases. This review article covers the exciting con-
tributions of the many researchers involved in this field and highlights
the challenges considered critical in this area of research.

Keywords: Deep learning · Epilepsy · Alzheimer’s disease ·

Parkinson’s disease · Neural disorder

1 Introduction
The brain is the most vital part of the human body that receives input from the
sensory organs of the body and sends output signals to the muscles thereby act-
ing as the command center for the human body. Different regions of the brain are
spatially distributed but continuously share information with each other form-
ing a functionally connected network [36]. At present, people with neurological
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 87–97, 2020.
https://doi.org/10.1007/978-981-15-3666-3_8
88 W. Ahmad Mir et al.

disorders are numbered in hundreds of millions. Approximately 50 million peo-

ple have been affected with epilepsy alone [37]. Neural disorders may comprise
of different debilitating conditions that involve the malfunction, deterioration,
or death of neurons that can at the beginning manifest forgetfulness, cognitive
impairments like seizures, dementia, and loss of coordination. Generally, neuro
imaging and signal analysis are carried out to measure the neural activities. The
unusual patterns in the images and signals give an insight about the malfunctions
of the brain. Physiological signals provide data in the form of signals which on
analysis assist in the diagnosis and treatment of the disorder [13]. These signals
represent the electrical activity of a particular body organ [12]. A medical prac-
titioner who analyses these signals and diagnoses the disease, if any, based on
his/her experience. For example, a medical practitioner may diagnose a patient
with a coronary artery disease upon recognizing a suspicious or unusual wave-
form of an ECG signal. From the machine learning perspective, the ECG signal
is an input, the analyzing medical practitioner is the system, and the system
state is represented by the suspicion of a practitioner. This entire knowledge
can be expressed as mathematical rules, and computer programs can use this
explicit knowledge to create expert systems [27]. Computer-intensive methods
for the analysis of biomedical signals have paved the way for experts in artificial
intelligence to make predictions about unseen data after making the machine
to learn on a particular data set [14]. One of the limitations of these systems
is that these systems encompass very small and specific domains, operating in
environments that are highly controlled and in addition suffer the problem of
over fitting, which at times may not be immediately apparent to the investiga-
tor [13]. Deep learning as a change in basic assumptions of artificial intelligence
algorithm design is being seen as a savior to overcome these limitations as it
considers all the available information in the training data set [30]. Deep learn-
ing as a machine learning approach is based on parallel processing and entities
networked together. Deep learning approaches have been extensively used in
EEG, ECG, EMG, EOG and Neuroimaging data but due to the vast diversity
of the health care applications very small benefits are being derived. Therefore,
for proper interpretation of the physiological signals and images, diversification
in the research area is crucial. It can be carried out by training the learning
algorithms with voluminous and varied data sets. This paper highlights some
of the common neurological disorders and later discusses their diagnosis using
neurological signals and images. Applications of deep learning on this biomedi-
cal data for disease diagnosis and prediction have been discussed. Deep learning
models from literature for the detection and diagnosis of these disorders have
been pointed out and it is demonstrated how these models have outperformed
the conventional machine learning approaches.

2 Background
Symbolic approaches of Artificial Intelligence in medical health diagnosis can
be traced back to the 1980s. Although Artificial systems that can learn from
 Contribution of Application of Deep Learning Approaches 89

the trial and error method had surfaced in the early days of the 1960s [31].
There has been a tremendous development in health data analysis using machine
learning. Accuracy rates of modern classifiers, noise filters, and feature selection
approaches have advanced beyond limits. The availability of systematic knowl-
edge sources, well defined, reasonably sized subdomains, efficient and powerful
processors are the reason behind this success [30]. The concept of inferring a
result after making a system to learn from data samples marks a branch of Arti-
ficial Intelligence known as Machine Learning [7,28]. This learning process may
be divided into two parts (a) from a known data set we estimate unknown depen-
dencies in the system and (b) based on the estimated dependencies; we predict
the new results from unknown data [22]. It is imperative to stress upon that
Machine Learning (ML) offers a highly sophisticated, computer-aided system of
analyzing high dimensional and multimodal biomedical data with the help of
developed and objectively oriented algorithms [32]. ML has shown interesting
results in the area of biomedical research. There are two commonly known ML
techniques (a) Supervised Learning and (b) Unsupervised learning. In case of
supervised learning, a function is derived that takes a specific input and pre-
dicts output values in close approximation to the desired output. A training
data set is taken as input, and an algorithm is designed to derive the required
function which produces output on unknown data to a close degree of fidelity to
the actual outcome. For the Unsupervised learning, the data set consists of only
input values, and an algorithm is designed to identify patterns, trends, and sim-
ilarities in the data thereby clustering the similar inputs [8,21]. A combination
of the supervised and unsupervised learning approaches widely applied is known
as semi-supervised learning. When unlabeled data sets outnumber the labeled
data sets, a more accurate learning model can be constructed with the help of
semi-supervised learning [22]. In the last few years, Deep Learning has helped
in improving the quality of diagnosis [38]. Physiological conditions concerning a
particular individual and the hidden relationships between a particular neural
disorder and the associated brain electrical activity can be more accurately pre-
dicted by the deep learning approaches. Deep learning has the capacity of faster,
reliable and accurate diagnosis and drifts from currently used decision support
systems to deep learning approach is on the cards, despite deep learning being in
its infancy. Deep Learning effectively combines the feature extraction and classi-
fication processes [34]. A robust convolutional neural network (CNN) model can
be developed after extracting features from the input data which improves the
diagnostic performance of the model during the testing phase [29].

3 Deep Learning
Deep learning methods are learning methods with multiple levels of representa-
tion that have become popular for effectively analyzing unstructured data [15].
These multiple levels are obtained by composing different nonlinear modules
that transform the representation from simpler to some abstract levels. These
transformations can be used in systems to learn very complex functions [23].
90 W. Ahmad Mir et al.

One of the important features of deep learning is that the features of the layers
are not designed by human engineers but are learned after following a general-
purpose learning process from the data itself. Deep Learning methods are com-
posed of multiple layers of neurons stacked together to produce discriminating
classes after proper feature selection procedure. Deep learning, a new trend in
machine learning has evolved as a de-facto approach for applications spread over
a vast spectrum of research areas like medical data analysis, social media analy-
sis, natural language processing, showing tremendous accuracy, robustness and
generalizability over the conventional approaches [11].

3.1 Application of Deep Learning on Neurological Data Analysis

Deep learning has been appealing to the neurologists because of the power-
ful models that have been proposed which in actual can approximate input to
output mapping provided there is abundant data for analysis [17]. In the pres-
ence of abundant data functional genomics, image and medical diagnosis have
attained pioneering gains. Application of deep learning in biomedical research
is expected to help in early diagnosis, treatment, and monitoring of patients. In
this paper, we have selected three Neurological diseases namely Epileptic Seizure,
Alzheimer’s disease, and Parkinson’s disease and reviewed some of the contribu-
tions of the deep learning in the diagnosis of these three particular disorders.

3.2 Epileptic Seizures

Epilepsy is one of the common neurological disorders which affects over 50 mil-
lion people worldwide [2]. The presence of Epileptic seizures in the long EEG
records of the patients is generally detected by Visual inspection performed by
the clinicians. As this process is cumbersome various techniques of signal pro-
cessing for classification and detection of seizures have been developed, based on
various feature extraction techniques. In the time domain analysis, techniques
of linear prediction, fractional linear prediction, and principal component based
radial basis function neural networks have been proposed in the literature for
classification purposes [20]. Some of the recent contributions in the detection
and diagnosis of Epileptic seizures using deep learned networks are as follows.
In other significant research, Avcu, et al. [5] developed a Deep Convolutional
Neural Network namely BPsvm and Sizenet which have shown promising results
in diagnosing Epileptic seizures. These models have been trained using 18-chn,
and 2-chn networks and a comparative analysis was carried out. It was found that
Seiznet 2-chn outperforms 18-chn BPsvm. Further, seiznet attained a sensitivity
of 95.8% and has only 25% false alarm rate for the 18-chn network.
Acharya et al. [1] have employed recently improved Convolutional neural
Network on EEG data for Epilepsy diagnosis. The authors have employed 13
layers deep convolutional neural network and categorized the EEG into normal,
preictal and seizure class. They have proposed 13 layer CNN to provide good
convergence and high accuracy. Although deep neural networks can be tuned for
complex and nonlinear functions, yet they face the problem of convergence. In
 Contribution of Application of Deep Learning Approaches 91

comparison to the other works published recently, the proposed model does not
provide the best of the performance. This model has managed to produce only
88.67% of accuracy, 95% of sensitivity and 90% specificity. The authors have
proposed a bagging algorithm and enhancement in sample space for improving
the performance metrics of the model.
Ullah, et al. [35] have proposed an automatic system for prediction of epilepsy
using a standard deep CNN which performs on EEG signals using an ensemble
of P-1D-CNN model. The whole system consists of three modules (a) an Input
module which takes EEG signals and breaks the EEG signal into parts and then
passes these fragments to P-1D CNN model (b) Signal segments are classified
by base P-1D CNN model using an ensemble approach (c) Decision making as
per majority vote by fusing local decisions. Although deep CNN models require
a huge amount of data for training purposes, the authors have introduced the
scheme of data augmentation where deep ensemble classifiers are built after
training the P-1D CNN model using the copies as base models, examining dif-
ferent parts of the input signal. An ensemble takes the input and performs the
process of classification. Local decisions are fused together and a final decision
is taken using the majority vote. The model was tested on University of Bonn
Datasets, which is a benchmark dataset for epilepsy-related cases. The model
has outperformed various previous machine learning models and has produced
an accuracy of 99.1%. Further, the model requires less memory and space and
produces results in less than 0.048 s.
Yuan et al. [39] introduced a novel channel-aware attention framework (Chan-
nelAtt) which is an end to end multi-view deep learning model which uses a
multi-channel approach in high-level space. The authors have proposed tech-
niques for learning attentional representations based on EEG signals in the
time-frequency domain. The authors have demonstrated that the system model
outperforms ordinary machine learning techniques in many ways. The perfor-
mance metrics of the proposed system showed an accuracy of 99.61%, sensitivity
of 96.51% and specificity of 97.85%. The comparison of various deep learning
techniques discussed above has been summarized in Table 1.

Table 1. Comparison of performance metrics.

Author Techniques Used Accuracy (%) Sensitivity (%) Specificity (%)

Avcu, et al. BPsvm 90 90 88.4
Sizenet 95 95.8 97.5
Acharya et al. 13 Layered CNN 88.67 95.00 90
Ullah, et al. P-1D-CNN model 99.1 96.00 97.50
Yuan et al. ChannelAtt 96.61 96.51 97.85

3.3 Alzheimer’s Disease

Alzheimer’s disease (AD) causes problems of memory, behavior and hinders
thinking process. A person faces severe interference in accomplishing daily tasks.
92 W. Ahmad Mir et al.

This interference keeps on developing and gets worse with time. In the US alone
5.7 million people are suffering from Alzheimer’s dementia [4]. Alzheimer’s is
a progressive and degenerative mental disorder caused due to the death of the
neuron cells thus causing shrinkage of the hippocampus [19]. This number is
expected to increase in the next 30 years. In the US the number of deaths due to
Alzheimer’s disease have shown a steep increase of 123 in the last decade caus-
ing emotional distress to caregivers and puts negative impacts on their mental
health. Presently, there is no cure to Alzheimer’s disease but early detection may
help prevent the progress of the disease [19]. Machine learning approaches that
can help in the diagnosis of AD have been proposed for early detection of the
disease. As the disease has many manifestations and the mild cognitive impair-
ments associated with it may progress to AD or may render the patient stable,
therefore multiple classification techniques are required to access the severity of
the disease properly. In the presence of these constraints, Deep learning algo-
rithms have shown promising results. Some of the recent contributions in this
regard are summarized as follows:
Liu, et al. [26] have introduced a novel method which contains stacked
autoencoders and a softmax output layer based on deep learning to overcome
the bottleneck of conventional machine learning methods and aids in diagnosis
Alzheimer’s disease. Unlike binary classification methods based on support vec-
tor machines (SVM), the authors have introduced multiclass classification which
uses small prior knowledge, has enhanced dimensionality reduction capability
and can reverse the synergy between data modalities simultaneously at the time
of data fusion. The authors were able to produce an accuracy of 87.76 %, the sen-
sitivity of 88.57% and specificity of 83.75%. The cost of misclassifying between
different groups usually varies; therefore, the greater sensitivity figures are very
beneficial for diagnosis.
In a very significant contribution, Bi and Wang [6] have introduced a novel
method for early detection of Alzheimer’s disease. The model is based upon a dis-
criminative deep probabilistic approach which has the capability of multi-task
learning and can classify EEG spectrum image into three classes. It employs
a deep convolutional generative model for multi-task learning. The interesting
feature of this model is that it has been able to bridge the connection between
feature extraction and classification. The authors have employed the system as a
demonstration of Discriminative contractive Slab and Spike Convolutional Deep
Boltzmann Machine (DCssCDBM). The authors are of the view that DCss-
CDBM may be used as a standalone nonlinear classifier with other classifiers.
The proposed method has been able to produce the accuracy of 95.04% and
sensitivity of 90.15% which is better than some known discriminative models
like CDBN. However, the discriminant version of the DCssCDBM is yet to be
introduced to the diagnosis of other neurological disorders like Epilepsy which
might show some positive results.
Spasov, et al. [33] have developed a deep learning architecture which has been
able to classify Mild Cognitive Impairment (MCI) patients and patients who have
a high likelihood of developing Alzheimer’s disease as it has been found that
 Contribution of Application of Deep Learning Approaches 93

clinical precursors of AD remain stable over time and do not turn to AD. It is
therefore imperative to classify patients based on the severity of the clinical con-
ditions. The authors have introduced deep learning methods which in addition to
structural magnetic resonance imaging (sMRI) have combined neuropsychologi-
cal and demographic data as input to the system. The deep learning algorithm
proposed by the authors has been able to differentiate between the MCI patients
developing AD and the patients who will remain stable and will not develop AD
over three years. The proposed system was able to produce an accuracy of 86%,
the sensitivity of 87.5% and specificity of 85%. Lee, et al. [24] have applied a
deep learning approach using Recurrent Neural Network (RNN), an architec-
ture specially designed for sequential data. The authors believe that RNN can
help in finding hidden temporal patterns from variable-length sequences. The
authors have applied multiple GRUs for longitudinal multidomain data. The
proposed model was able to predict correctly the conversion of MCI to AD in
only 75% of cases. The model achieved its best when it incorporated longitudinal
multi-domain data with an accuracy of 86%. In another significant contribution,
Hosseini-Asl et al. [18] have adopted a 3D deep convolutional network (3D-CNN)
which can learn generic features of AD. The authors have validated their model
on the ADNI data set. This model outperforms some of the state-of-the-art con-
ventional machine learning models despite using single image modality (MRI)
without any previous skull stripping. The model has been able to produce an
accuracy of about 90.82%. The comparison of various deep learning techniques
discussed above has been summarized in Table 2.

Table 2. Comparison of performance metrics.

Author Techniques used Accuracy (%) Sensitivity (%) Specificity (%)

Liu, et al. Softmax 87.76 88.57 83.75
Bi and Wang DCssCDBM 95.04 90.15 NA*
Spasov, et al. Parameter efficient deep 86 87.5 85
learning
Lee, et al. Recurrent Neural 86 NA* NA*
Network (RNN)
Hosseini-Asl et al. 3D-CNN 90.8 NA* NA*

3.4 Parkinson’s Disease

Parkinson’s disease (PD) is a neuro degenerative disorder of insidious onset which
is characterized by the presence of late-onset motor symptoms that may cause
postural instability, tremors, rigidity, and freezing of gait and speech abilities.
Around 1% population of the 60 industrialized countries are affected by this
disorder [10]. Based on voice recordings deep neural networks have been used to
detect the Parkinson’s disease [3]. Although the literature is rich with the studies
of the application of machine learning on detection and diagnosis of severity of
the disease, deep learning which is still in infancy has also experimented in this
94 W. Ahmad Mir et al.

regard. Some of the recent contributions in this area of research are reviewed as
as follows.
Grover, et al. [15] have devised a method using deep learning to diagnose the
severity of Parkinson’s disease based on the voice of the patients. The authors
have deployed the “TensorFlow” deep learning library of python for the diag-
nostic purpose. The experimentation has been done based on Motor UPDRS
score and on the input data set of 16 voice features, the classification accuracy
obtained was 81.6%. The authors have found that motor UPDRS score is a
better classification basis than total UPDRS score and hence it is regarded as
a better metric for predictive analysis. However, the experimentation has been
done on a small data set that too on voice features only. The results would have
been better if gait and handwriting features were also taken into consideration.
In a significant contribution Choi, et al. [9] have developed a deep learn-
ing based FP-CIT SPECT interpretation system which works on image data
and can classify normal and PD images and can accurately interpret variability
in the images which cannot be done by human evaluation. The authors have
developed a deep Convolutional Neural Network and validated it for discrimi-
nating Parkinsonian and non-Parkinsonian tremor on independent SPECT data.
The data used for validation of the model has been independently taken from
the PPMI database (www.ppmi-info.org/data) and SNUH cohort. The accuracy,
sensitivity, and specificity obtained is 96%, 94.2%, 100% and 98%, 98.6%, 100%
respectively for the PPMI and SNUH dataset The authors suggested that the
proposed model can be applied to the reclassification of SWEDD group and can
further be extended to imaging analysis of other diseases and can further help
in making imaging biomarkers.
In another study conducted by Hammerla, et al. [16], the authors have devel-
oped an assessment model based on deep learning approach which can clas-
sify disease state in naturalistic settings, taking into consideration the practical
usability constraints. The model has been validated on a large data set that con-
tains 5,500 hours of movement data that has been collected from 34 participants
in a natural setting. On a HOME data set after training the system, it was able
to obtain an f1 score of 58.1% and on the LAB data set it was able to produce an
accuracy of 59.4%, specificity of 88% and sensitivity of 57%. The results cannot
be compared to prior results as no other systems have followed similar training
and methodology.
Hosseini-Asl, et al. [18] have devised a model to predict the PD with the help
of deep 3D convolutional neural network (3D-CNN) capable of learning generic
features and adaptable to different domain datasets. The proposed model is built
on 3D convolutional autoencoder capable of capturing variations in sMRI scans.
The model has been experimented on CAD Dementia MRI dataset without skull
stripping and has been able to outperform various classifiers viz accuracy. For
enhancing the generality of the features, the authors have used pre-training and
layer freezing strategies. Classification performance has been done after ten-fold
cross-validation on 210 subjects of the ADNI data set. The proposed model was
able to produce an overall accuracy of 92.56%. The comparison of various deep
learning techniques discussed above has been summarized in Table 3.
 Contribution of Application of Deep Learning Approaches 95

Table 3. Comparison of performance metrics.

Author Techniques used Accuracy (%) Sensitivity (%) Specificity (%)

Grover, et al. DNN 81.66 NA* NA*
Choi, et al. FP-CIT SPECT (PPMI Dataset) 96 94.2 100
FP-CITSPECT (SNUH Dataset) 98 98.6 100
Hammerla, et al. DNN 59.4 57 88
Hosseini-Asl, et al. 3D-CNN 92.56 NA* NA*

4 Conclusion
Deep learning as a powerful learning method has much to offer to the data
scientists in general and neurologists in particular. This paper has reviewed
some major contributions of deep learning methods that have been applied to
medical data in recent times. The current trends in deep learning are finding
its relevance particularly in unsupervised learning and training strategies in this
area of learning has regained traction [25]. Deep learning as an experienced
modeling approach is a valuable addition to healthcare decision making. This
procedure can do a large body of work and can reveal unexpressed knowledge
during the diagnosis process. In future, deep learning is expected to play an
important role in improving the quality of diagnosis and understanding numerous
clinical events. It can do accurate and reliable diagnosis, thus making physicians
and healthcare providers smarter in the future.

References
1. Acharya, U.R., Oh, S.L., Hagiwara, Y., Tan, J.H., Adeli, H.: Deep convolutional
neural network for the automated detection and diagnosis of seizure using EEG
signals. Comput. Biol. Med. 100, 270–278 (2018)
2. Acharya, U.R., Sree, S.V., Suri, J.S.: Automatic detection of epileptic EEG signals
using higher order cumulant features. International J. Neural Syst. 21(05), 403–414
(2011)
3. Jabardi, M.H., Al-Fatlawi, A.H., Ling, S.H.: Efficient diagnosis system for parkin-
son’s disease using deep belief network. IEEE Congress on Evolutionary Compu-
tation (CEC), Vancouver, Canada (2016)
4. Association, Alzheimer’s, et al. 2018 alzheimer’s disease facts and figures.
Alzheimer’s Dement. 14(3), 367–429 (2018)
5. Avcu, M.T., Zhang, Z., Chan, D.W.S.: Seizure detection using least eeg channels
by deep convolutional neural network. In: ICASSP 2019–2019 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1120–1124.
IEEE (2019)
6. Bi, X., Wang, H.: Early alzheimer’s disease diagnosis based on eeg spectral images
using deep learning. Neural Networks 114, 119–135 (2019)
7. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York
(2006)
8. Butler, K.T., Davies, D.W., Cartwright, H., Isayev, O., Walsh, A.: Machine learning
for molecular and materials science. Nature 559(7715), 547 (2018)
96 W. Ahmad Mir et al.

9. Choi, H., Ha, S., Im, H.J., Paek, S.H., Lee, D.S.: Refining diagnosis of parkinson’s
disease with deep learning-based interpretation of dopamine transporter imaging.
NeuroImage Clin. 16, 586–594 (2017)
10. de Lau, L., Breteler, M.M.B.: Epidemiology of parkinson’s disease. Lancet Neurol.
5(6), 525–535 (2006)
11. Deng, L., Yu, D., et al.: Deep learning: methods and applications. Found. Trends R
Sign. Process. 7(3–4), 197–387 (2014)
12. Devasahayam, S.R.: Signals and Systems in Biomedical Engineering: Signal Pro-
cessing and Physiological Systems Modeling. Springer, Boston (2012). https://doi.
org/10.1007/978-1-4614-5332-1
13. Faust, O., Bairy, M.G.: Nonlinear analysis of physiological signals: a review. J.
Mech. Med. Biol. 12(04), 1240015 (2012)
14. Foster, K.R., Koprowski, R., Skufca, J.D.: Machine learning, medical diagnosis,
and biomedical engineering research-commentary. Biomed. Eng. Online 13(1), 94
(2014)
15. Grover, S., Bhartia, S., Yadav, A., Seeja, K.R., et al.: Predicting severity of parkin-
son’s disease using deep learning. Procedia Comput. Sci. 132, 1788–1794 (2018)
16. Hammerla, N.Y., Fisher, J., Andras, P., Rochester, L., Walker, R., Plötz, T.:
PD disease state assessment in naturalistic environments using deep learning. In:
Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
17. Hornik, K.: Approximation capabilities of multilayer feedforward networks. Neural
Networks 4(2), 251–257 (1991)
18. Hosseini-Asl, E., Keynton, R., El-Baz, A.: Alzheimer’s disease diagnostics by adap-
tation of 3D convolutional network. In: 2016 IEEE International Conference on
Image Processing (ICIP), pp. 126–130. IEEE (2016)
19. Ahmad, M.O., Swamy, M.S., Jabason, E.: Deep structural and clinical feature
learning for semi-supervised multiclass prediction of alzheimer’s disease. In: 2018
IEEE 61st International Midwest Symposium on Circuits and Systems (MWS-
CAS), pp. 791–794. IEEE (2018)
20. Joshi, V., Pachori, R.B., Vijesh, A.: Classification of ictal and seizure-free EEG
signals using fractional linear prediction. Biomed. Sign. Process. Control 9, 1–5
(2014)
21. Kireeva II, N., Baskin, H.A.G., Horvath, D., Marcou, G., Varnek, A.: Genera-
tive topographic mapping (GTM): universal tool for data visualization, structure-
activity modeling and dataset comparison. Molecular Inform. 31(3–4), 301–312
(2012)
22. Kourou, K., Exarchos, T.P., Exarchos, K.P., Karamouzis, M.V., Fotiadis, D.I.:
Machine learning applications in cancer prognosis and prediction. Comput. Struct.
Biotechnol. J. 13, 8–17 (2015)
23. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444
(2015)
24. Lee, G., Nho, K., Kang, B., Sohn, K.-A., Kim, D.: Predicting alzheimer’s disease
progression using multi-modal deep learning approach. Sci. Rep. 9(1), 1952 (2019)
25. Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image
Anal. 42, 60–88 (2017)
26. Liu, S., Liu, S., Cai, W., Pujol, S., Kikinis, R., Feng, D.: Early diagnosis of
alzheimer’s disease with deep learning. In: 2014 IEEE 11th international sym-
posium on biomedical imaging (ISBI), pp. 1015–1018. IEEE (2014)
27. McAndrew, P.D., Potash, D.L., Higgins, B., Wayand, J., Held, J.: Expert system
for providing interactive assistance in solving problems such as health care man-
agement, 14 May 1996. US Patent 5,517,405
 Contribution of Application of Deep Learning Approaches 97

28. Mitchell, T.M.: The discipline of machine learning, vol. 9. Carnegie Mellon Uni-
versity, School of Computer Science, Machine Learning (2006)
29. Oh, S.L., et al.: A deep learning approach for parkinson’s disease diagnosis from
EEG signals. Neural Comput. Appl. 1–7 (2018)
30. Peek, N., Combi, C., Marin, R., Bellazzi, R.: Thirty years of artificial intelligence
in medicine (aime) conferences: a review of research themes. Artif. Intell. Med.
65(1), 61–73 (2015)
31. Quaglini, S., Barahona, P., Andreassen, S. (eds.): AIME 2001. LNCS (LNAI), vol.
2101. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-48229-6
32. Sajda, P.: Machine learning for detection and diagnosis of disease. Annu. Rev.
Biomed. Eng. 8, 537–565 (2006)
33. Spasov, S., Passamonti, L., Duggento, A., Liò, P., Toschi, N., Initiative, A.D.N.,
et al.: A parameter-efficient deep learning approach to predict conversion from mild
cognitive impairment to alzheimer’s disease. Neuroimage 189, 276–287 (2019)
34. Tan, J.H., Acharya, U.R., Bhandary, S.V., Chua, K.C., Sivaprasad, S.: Segmenta-
tion of optic disc, fovea and retinal vasculature using a single convolutional neural
network. J. Comput. Sci. 20, 70–79 (2017)
35. Ullah, I., Hussain, M., Aboalsamh, H., et al.: An automated system for epilepsy
detection using EEG brain signals based on deep learning approach. Expert Syst.
Appl. 107, 61–71 (2018)
36. Van Den Heuvel, M.P., Pol, H.E.H.: Exploring the brain network: a review
on resting-state fmri functional connectivity. European Neuropsychopharmacol.
20(8), 519–534 (2010)
37. WHO. Neurological disorders public health disorders. Neurological disorders
38. Yang, H.-C., Islam, M.M., Li, Y.-C.J.: Potentiality of deep learning application in
healthcare. Comput. Methods Programs Biomed. 161, 1 (2018)
39. Yuan, Y., et al.: A novel channel-aware attention framework for multi-channel eeg
seizure detection via multi-view deep learning. In: 2018 IEEE EMBS International
Conference on Biomedical & Health Informatics (BHI), pp. 206–209. IEEE (2018)
 Integration of BYOD Technology
in Traditional Classroom: A Statistical
Approach

Pooja Kumari(B) , Suman Deb , and Barnita Debnath

National Institute of Technology Agartala, Agartala, Tripura, India

[email protected], [email protected], [email protected]

Abstract. Today we are moving from the world of cellular communi-

cation to the era of internet. Mobile technology has paved a way into
our pockets. We use mobile phones as an extension of our body. With
the advent of these pervasive devices MOOCs, m-learning, e-learning has
found a way into our classroom. Smart learning technology is growing
exponentially with smartphones as a major part. In this work, we try to
model the traditional classroom with Bring Your Own Device (BYOD)
concept. We statistically find if these mobile devices are suitable for
classroom or not. The data collected by the survey was used to find the
adaptability of mobile devices in the traditional classroom using JASP
Tool. A descriptive statistics analysis, linear regression techniques, and
Bayesian sample paired t-test are done on sample survey data to find
the compliance of smart devices in the classroom. The vast expansion
of mobile technology has not only enhanced student-teacher interaction
but has also improved the quality of education.

Keywords: Traditional classroom · Regression · Bayesian sample

paired t-test · BYOD · Statistical analysis

1 Introduction
A survey in 2005 [1] showed that there was an exponential rise in the use of
tablets by the students compared to the previous years. Today, there are more
mobile devices as compared to the number of humans in the world. We can
utilize this increment in mobile devices for the good of education and learning.
BYOD concept gives a cost effective solution for the problems rising in traditional
classroom. In a classroom, teacher plays an important role encouraging and
motivating student along with course work. Introduction of mobile devices inside
the classroom can give teachers freedom to interact and attain things which
were difficult before. It not only enhances classroom interaction, but also gives
a platform to students as well as teacher to interact and collaborate whenever
they want. With the advancement of mobile technologies, MOOCs, m-learning,
Supported by National Institute of Technology Agartala.
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 98–110, 2020.
https://doi.org/10.1007/978-981-15-3666-3_9
 BYOD Technology in Traditional Classroom 99

e-learning has grown rapidly giving a base for improving quality of education.
In this study we have utilized statistical way to prove the usability of mobile
devices in the classroom. These mobile devices hold an immense possibility in
changing the way we look at the world. Though, it is still in its initial phase but
with the rise of mobile devices it will surely find its way into the classroom.

1.1 Mobile Technology

Mobile devices are handheld, portable devices like smart phones, tablets, lap-
top, etc. Earlier mobile devices were used just for the cellular communication,
but now it has recreated its meaning by implementing innovative ideas and
application which can be utilized for teaching-learning process. Today a mobile
phone has a processing capacity like laptops with 6 GB and 8 GB of RAM. Many
researchers have analyzed the effect of mobile devices on learning and defined
mobile technology as the portable, light weight, spontaneous learning [2]. Over
the last few years many researchers have been trying to study the implication of
mobile devices inside the classroom with BYOD concept. For making a mobile
compatible to classroom, various attributes of the mobile phones should be kept
in mind. The phone should be powerful enough to serve all the functionality
required for classroom teaching, it should have a screen of optimal size i.e. it
should neither be too small nor too large. Various attributes which is to be
checked while designing mobile phones for classroom are as follows:

– RAM and processing speed of the phone

– Screen resolution
– Screen size
– Battery life of mobile device
– Storage Capacity
– Camera quality
– Weight
– Adaptability

1.2 Rise of MOOCs

The term MOOCs was coined by David Cormier in 2008, meaning Massive Open
Online Course. It started with a small course named Connectivism and Connec-
tive Knowledge with just 25 paid students [3]. It was that day, and it is today that
thousands and thousands of students join these online courses. Later, Coursera
and Stanford launched its course in 2011. Soon this became a new business, and
various others started this type of learning through internet. Harvard joined this
march by the name of edX, and now we have several others like Udemy, Udacity,
etc. The New York Time named 2012 as the year of MOOCs. This concept using
mobile technology has opened the gates for student worldwide. Now a person
from one part of globe can learn from the teacher of different part of globe.
Now a day, students are more attracted towards e-books and video lectures. It
also provides with cheap and easily accessible learning materials. On combining
100 P. Kumari et al.

MOOCs with traditional classroom with the involvement of mobile technology,

we can see the rise of new kind of learning. It will not only reduce pressure of
teacher and student but will also guide them towards quality education.

2 Literature Survey: Intelligent Teaching and Mobile

Devices

Today in the era of being smart, education and technology go hand in hand.
Various researchers have explored the domains on the use of mobile technology
in the traditional classroom. Few works that matched with our idea of incorpo-
rating mobile technology in traditional classroom are as follows:

1. In a study [4] by Wang, Xiao, Kong, and Huang in 2016, the influence of
digital devices on classroom was being analysed. The authors observed 54
lectures in Hong Kong, Beijing, and Shenzhen. It was found that the class-
room when equipped with digital mobile devices gave better understanding of
learning and interaction between teacher and student. The study emphasised
on the use of mobile devices inside the classroom for better interaction and
understanding.
2. A research on the use of mobile devices for teacher education [5] by Evrim
explores the domain of mobile technology for student as well as teacher. The
author had six findings in her study including variation that existed between
practical and theoretical use of mobile device, existing trends of mobile tech-
nology, etc. In the end of the study, it was suggested to use these technologies
for the betterment of education system.
3. A case study [6] that suggested BYOD as the cost effective solution for learn-
ing in Australian Regional University explores over the use of mobile tech-
nology inside the classroom. They proposed some practical, low cost solution
for classroom learning and interaction using BYOD concept.
4. A study [7] in England in year 2012 compared the progress and quality of
learning of two different school. One school was allowed to use mobile tech-
nology in their classroom, while the other school was not allowed to use
mobile devices. In conclusion author recommended to use mobile technology
inside the classroom. The school which was equipped with mobile technology
showed better performance result along with increased attention and interac-
tion among students and teachers.
5. A study on the role of iPad in primary school [8], it was found that with
the incorporation of mobile technology i.e. iPad inside the classroom, learn-
ing can be made interactive. Author integrated iPad in primary classroom
for around eight months and found these devices as the facilitator inside the
classroom. They also analysed the behaviour of students while using mobile
technology in classroom. Later they suggested to use mobile devices in class-
room for enhancement of education in terms of classroom interaction and
better understanding.
 BYOD Technology in Traditional Classroom 101

6. A work on e-assessment [9] where different mobile based technologies were

incorporated in classroom shows that there was increased interaction between
teacher and students. This paper also explained the importance of mobile
technology as an educational aid inside the classroom.

3 Statistical Analysis

This statistical work started with a survey among undergraduate students of

NIT Agartala, comprising some questions about the use of mobile technology
inside the classroom. This survey was done to check the thoughts of students on
introduction of technology inside the classroom, and to collect the sample data
for our analysis. Students were provided with some questions regarding smart
phones as an educational aid, and were asked to answer those questions. Later
we divided these students into two group as control group and treatment group.
Control group was restricted from using mobiles and other technology inside the
classroom for educational purpose, while treatment group was given full freedom
to use this mobile technology for enhancing quality of education. Later, a test
was conducted for both set of students and their marks were analyzed. We used
JASP 0.10.2 tool for all statistical work.
Our work started by considering two hypotheses:
NULL Hypothesis, H0: With the introduction of mobile technology inside the
classroom, there is no change in the quality of education.
Alternate Hypothesis, H1: Due to involvement of mobile technology inside
the classroom, marks of students as well as the quality of education has gone up.
We started with descriptive analysis of the marks data of two different groups.
This analysis allows us to get basic statistics including histogram, correlation
plot, frequency table, etc. Figure 1 represents descriptive statistics with total
number of valid cases, mean, standard deviation, maximum and minimum value
of both the groups. In Fig. 2 we have descriptive plots i.e. histogram of treatment
group. Also, Fig. 3 represents the histogram of control group. It is clearly visible
that the mean of treatment group is more than the mean of control group, which
shows that there is an increment in the marks of students who were allowed to use
the mobile phones. Mobile phones not only enhance student-teacher interaction,
but also affect the grade of students. This technology is taking over world so
fast that today almost 70% of the teenagers use mobile devices, including smart
phones, tablets, laptops, etc.
Figure 4 represents the correlation plot which shows a matrix of plot between
continuous parameters, with scatter plot between the variables in non diagonal
entries. This plot shows the correlation between two parameters.
Figures 5 and 6 represents the box plot, which is a way of representing data
through rectangle in second and third quartile. The vertical line inside the rect-
angle represents the median value. Horizontal lines on either side of the rectangle
are upper and lower quartiles. We can see that the median value of control group
is lower than the median value of treatment group as shown in the figure. It
shows that treatment group has better result as compared to the control group.
102 P. Kumari et al.

Fig. 1. Statistic analysis of treatment and control group

Fig. 2. Histogram for Treatment group

Fig. 3. Histogram for Control group

 BYOD Technology in Traditional Classroom 103

Fig. 4. Correlation plot between two groups

Smart phones when joined with education system can give exponential growth
to education system.
Figures 7 and 8 represents a Q-Q plot which stands for quantile-quantile plot.
It is formed by plotting two different set of quantities against each other [10]. In
both of the figure we see the group of points forming approximately straight line.
This method calculates and compares the two different probability distribution.
The Q-Q plot for treatment group has almost all its points on the straight line
forming a better representation, where as in control group points are not that
much concise as in the case of treatment group i.e. points are a bit far from the
straight line. Hence, from the Q-Q plot we can easily infer that the treatment
group has much more effective result than the control group.
While analyzing for the sample data collected from the classroom we also
used linear regression to find if mobile phones are feasible inside the classroom
or not. Linear regression is used to model the relationship between some depen-
dent variable and independent variable [11]. Here, we consider that the data of
treatment group is dependent on the use of mobile phones inside the classroom.
We follow certain assumptions like, the response parameter is continuous, all
the predictors are linearly related to response variable, and the error variance
is constant across each value. Figure 9 represents the summary of the regression
model. R is the multiple correlation coefficients; R square is the proportion of the
regression model and RMSE is the root mean square error. Also, ANOVA stands
for analysis of variance [12], it is used to find the difference between the group
mean of a particular data value used in Fig. 10, representing the sum of square
value for regression model and then residual model, and next it calculates the
total sum of square for both the models. Degree of freedom, represented by df is
calculated for both the models separately and then the total df of both regres-
sion and residual model. F value is used as the criteria for adding or removing
104 P. Kumari et al.

Fig. 5. Box plot for treatment group

predictors i.e. independent variable in regression. For adding a predictor we check

if the F-value t2 of regression coefficient is greater than x, then we add predictors
where default x value is 3.84. For removing we check if F-value t2 of regression
coefficient is less than x we then we remove the predictors, here default value of
x is 2.71. The t-value used is for testing the null hypothesis that the regression
coefficient is equal to zero, p-value also checks if we need to consider the null
hypothesis or alternate hypothesis. The F value after the analysis is 0.390 and

Fig. 6. Box plot for control group

 BYOD Technology in Traditional Classroom 105

Fig. 7. Q-Q plot for treatment group

p value is 0.535 which says that the alternate hypothesis yields better result. As
per alternate hypothesis, mobile technologies should be implemented inside the
classroom. Mobile technologies along with laptop, tablet and smart phone can
take teaching-learning process to a new level.

Fig. 8. Q-Q plot for control group

106 P. Kumari et al.

Fig. 9. Linear Regression model summary

Fig. 10. ANOVA values

To support our analysis strongly we also used Bayesian paired sample t-test
to interpret the probability of adoption of mobile devices inside the traditional
classroom. It considers the two hypotheses H0 and H1 to calculate and analyze
the procurement of data. T-test is a type of statistical model that takes two
groups and find distribution and difference between the mean. It takes into
account the data of both the groups and finds if the null hypothesis is accepted
or rejected. In our work we have used Bayes factor as the function of statistics
to compare mean and other relevant data. BF10 collects the truthfulness of
alternate hypothesis over null hypothesis, whereas BF01 measures the evidence
for null hypothesis over alternate. In Fig. 11, BF10 can be seen as 31.140, this
acts as the evidence for treatment group i.e. alternate hypothesis, which means
that Bayes factor is 31 times in favor of alternate hypothesis. There is an error of
3.313e-4, which is approximately equal to 5. We can neglect this part till it is less
than 10 Fig. 12 shows various descriptive including N as the sample size, which is
the total number of student considered for this analysis. It also represents mean,
standard deviation (SD), standard error of mean (SE). Credible interval is the
gap where the factors that we have considered have given certain probability.
Figure 13 represents the graphical form of the mean between the two control and
treatment group.

Fig. 11. Graphical Representation of mean with credibility percentage

 BYOD Technology in Traditional Classroom 107

Fig. 12. Basic descriptive of Bayesian t-test

Figure 14 represents the prior and posterior distribution under alternative

hypothesis. Cauchy prior distribution of 0.707 is considered. The dash line in
the figure represents prior distribution and solid line represents posterior distri-
bution. Two dots in the curve represent height of the curve at effect size delta.
We see that the dot on the dotted line is above the dot of solid line, it means
that the model support alternate hypothesis. Also we see a graphical plot of
Bayes factor BF10 = 31.140, this is the evidence for alternate hypothesis over
null hypothesis. Also, the horizontal line in figure represents the width of 95%
credible interval of posterior distribution. Figure 15 represents the sequential
analysis with X-axis as the number of data points, and Y-axis as the Bayes fac-
tor. Here we see the variation in the data points as it moves through the Bayes
factor. The points on or above the line supports H1, while the points below line
support H0. We can see that there is a strong evidence for alternate hypothe-
sis saying mobile technology should be incorporated inside the classroom. The
BF10 is 31.140 while Bayes factor for null hypothesis BF01 is 0.032, which is
very less as compared to the Bayes factor of alternate hypothesis. The horizon-
tal line on the figure separating H1 and H0 at BF10 = 1 is the place below

Fig. 13. Graphical Representation of mean of control and treatment group

108 P. Kumari et al.

Fig. 14. Prior and Posterior distribution of null and alternate hypothesis

which the points accounts for null hypothesis and above it the points give the
evidence of treatment group i.e. alternate hypothesis. Since we have more data
points accounting for treatment group, it proves with a strong point that mobile
devices are good for classroom. Incorporating these technologies inside the class-
room with enable enhanced interaction, better learning, quality education, and
collaborative teaching learning approach.

Fig. 15. Sequential analysis of data points against Bayes factor

 BYOD Technology in Traditional Classroom 109

4 Limitation
Smart mobile technology involving BYOD in the classroom do come up with cer-
tain limitations and challenges. These mobile devices unknowingly invite various
distractions and threat inside the classroom. Some limitations of mobile devices
inside the classroom are as follows:

– Mobile phones can act as a source of distraction [13] for learning process going
on inside the class. Any vibration due to calls or notification can distract
student.
– Mobile devices with network connection are a gateway for many dreadful
things. Students will have access to all the data over the internet.
– There can be health issues due to large use of mobile phones as these devices
use radio frequency for communication, and these radiations can affect our
body turning out to be harmful [14].
– Since traditional classroom is implemented using mobile devices, battery issue
of these devices is also a limitation.
On one hand mobile technology in classroom can provide us with ample of advan-
tages, but on the other there are so many disadvantages attached to it. We can
limit the use of mobile phones just for the educational purpose. Various rules and
norms can be made on the use of this technology inside the classroom. If handled
properly this implementation can mark the beginning of new era in classroom
teaching learning methodology.

5 Conclusion
Our world is moving at a very fast scale; today we have more mobile devices as
compared to total number of humans on the earth. Mobile technology not only
allows us to access learning data from any geographical location, but also has
enhanced teacher-student interaction. These pervasive devices have turned out to
be a paramount in traditional classroom. This research work analyses the imple-
mentation of BYOD technology along with mobile devices inside the traditional
classroom. This work is presented from the statistical point of view, analyz-
ing class data using regression, Bayesian t-test and other statistical method. In
the course of our study we found that our treatment group got more evidence
compared to our control group proving alternate hypothesis H1 to be true. On
analyzing the result of Bayesian paired t-test, we found that maximum points
that we studied support mobile devices in the classroom with positive results.
This study recommends the use of mobile technology inside the classroom
on the basis of statistical result obtained during analysis. With the growth of
mobile technology, enhancement of traditional classroom should also take place.
Incorporating this mobile technology in the classroom will result better interac-
tion among the students along with quality education, with the power of learning
new and more every day.
110 P. Kumari et al.

References
1. Traxler, J.: Defining mobile learning. In: IADIS International Conference Mobile
Learning, pp. 261–266, June 2005
2. Mbabazi, B.P., Ali, G., Geoffrey, A., Lawrence, N.: Mobile devices for learning in
universities: challenges and effects of usage (2018)
3. The Rise of MOOCs: How Massive Open Online Education Started and Where it
is Now (2019). Examined Existence
4. Wang, X.C., Kong, S.C., Huang, R.H.: Influence of digital equipment on interaction
quality in technology-rich classroom. In: 2016 IEEE 16th International Conference
on Advanced Learning Technologies (ICALT), pp. 455–459. IEEE, July 2016
5. Baran, E.: A review of research on mobile learning in teacher education. J. Educ.
Technol. Soc. 17(4), 17–32 (2014)
6. Sher, V., Hatala, M., Gašević, D.: On multi-device use: using technological modal-
ity profiles to explain differences in students’ learning. In: Proceedings of the 9th
International Conference on Learning Analytics & Knowledge, pp. 1–10. ACM,
March 2019
7. Walker, R.: “I don’t think I would be where I am right now”. Pupil perspectives
on using mobile devices for learning. Res. Learn. Technol. 21 (2013)
8. Mann, A.M., Hinrichs, U., Read, J.C., Quigley, A.: Facilitator, functionary, friend
or foe?: Studying the role of iPads within learning activities across a school year.
In: Proceedings of the 2016 CHI Conference on Human Factors in Computing
Systems, pp. 1833–1845. ACM, May 2016
9. Crisp, G., Guàrdia, L., Hillier, M.: Using e-Assessment to enhance student learning
and evidence learning outcomes (2016)
10. Turner, S.D.: qqman: an R package for visualizing GWAS results using QQ and
manhattan plots. Biorxiv, 005165 (2014)
11. Linear regression. En.wikipedia.org (2019). https://en.wikipedia.org/wiki/Linear
regression. Accessed 5 Sept 2019
12. St, L., Wold, S.: Analysis of variance (ANOVA). Chemometr. Intell. Lab. Syst.
6(4), 259–272 (1989)
13. Elgan, M.: Smartphones make people distracted and unproductive. (2019).
Computerworld. https://www.computerworld.com/article/3215276/smartphones-
make-people-distracted-and-unproductive.html. Accessed 18 July 2019
14. “Mobile Phones And Your Health” (2019). www.betterhealth.vic.gov.au. Accessed
22 Aug 2019
 Statistical Analysis of Road Accident
Data of UK Using Operational
Intelligence Tool - Splunk

Tapajyoti Deb(B) , Niti Vishwas , and Ashim Saha

NIT Agartala, Agartala 799046, Tripura (West), India

[email protected], [email protected], [email protected]

Abstract. In general, people believe speeding and alcoholism are the

two main considerations that impacts on road accidents. We suspected
that it would wise to investigate the impact information to guarantee the
rightness of this conclusion and furthermore to get additional data like,
which zones are prone to accidents, at what time most accident occurs,
what were the road conditions, and so on. These encounters can perhaps
make general populace aware of the explanations behind accidents made
by impacts. To break down in million records, we embraced Splunk for
speedier handling of this enormous information and analytics. In this
paper, we are showing certainties based on data and analytics which
lead to conclusions like the rate of accidents increased or diminished
in between 2005 and 2014 in UK. Pushing forward in our exploration,
we tended to complex investigation like areas in UK more inclined to
accidents and severity of accidents occurred.

Keywords: Accident data · Big data · Geo-spatial analysis · Splunk

1 Introduction

There are colossal number of impacts happening around the globe and the demise
and fatal injury rates are essentially developing as demonstrated by assessments
and statistics distributed by various organizations over the world. Impacts can
happen due to many reasons like liquor, drugs, weather conditions, careless-
ness, and so forth. Traffic engineers are accused of influencing transportation
frameworks to keep running as effectively as could be expected under the cir-
cumstances, however the assignment appears to be unmanageable. In any case,
with mechanical advancements and complex data analytics, governments are
better ready to fight gridlock now than any time before. In the long run big data
analytics will make driving considerably more secure. Thus, ample opportunity
is there to adequately deal with the road accident analysis. Prescient analytics
gives an all new way to extract valuable urban mobility data from networked
infrastructure, associated vehicles and cell phones, for real-time evaluation of
traffic patterns, and deploying necessary management strategies. Data sources
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 111–119, 2020.
https://doi.org/10.1007/978-981-15-3666-3_10
112 T. Deb et al.

within our cities means analytics offer potential to open up a whole new era of
smart transport.
We have utilized an operational intelligence tool called Splunk [1,2] to dissect
our information in view of the accompanying reasons:

– We generally search for processing our huge information as quickly as possible.

Splunk empowers applications to run very fast compared to other tools.
– Splunk has real time data processing capability which most other tools lack.
The input data can be in any format, for e.g., json, log, txt, csv, etc.
– We can configure Splunk to give Alerts/Events notification at the onset of a
machine state.
– We can precisely anticipate the assets required for scaling up the framework.
– We can make learning objects for Operational Intelligence.

Features of Splunk

– Analyze system performance

– Troubleshoot any failure condition
– Monitor Business metrics
– Search and investigate particular outcome
– Create dashboards to visualize and analyze results
– Store and retrieve data for later use.

The UK government amassed movement information from 2005 and 2014,

recording more than 1.6 million accidents and making this a standout amongst
the most exhaustive traffic dataset out there. On analysis, it came out to an
average of 177778 number of crashes each year and around 494 crashes every
day. The analysis we performed is basically centered around information gath-
ered from kaggle.com [3] for the years 2005–2014 for United Kingdom with a
specific end goal to provide insight into the sorts of impacts, zones with higher
recurrence of crashes, and time window of high recurrence impacts and so on.
This can conceivably empower law implementation divisions in crash compre-
hension, impact reduction, and in turn take proactive measures to control the
crash rate and spread awareness among drivers thus ensuring security and safety
of people.

2 Importance of Analytics in Road Traffic Accident

Scenario

There is a need to manage traffic in a smart way using data analytics and
machine learning as the management of traffic with the conventional way such
as the signaling system is not having a major effect in the road accident pre-
vention and also to ensure the smooth flow of traffic at peak times and during
emergency situations, such as following an accident, for instance. The analysis
information and the predictions can provide researchers with insights as what
 Statistical Analysis of Road Accident Data of UK 113

factors contribute most to causing accidents and how to tackle accidents in the
urban areas. This will ensure safety to the people out there. Hence, a smart
living experience.

3 Literature Survey
Urban areas have a few difficulties for us to handle; one of them is the issue of
urban portability. The United Nations anticipated that half of the total populace
would live in urban zones toward the finish of 2008. The developing populace
and absence of accessible physical space have made traffic administration pro-
gressively difficult in Singapore. So with the increasing traffic, the probability of
road accident is getting increased. By 2020, travel demand is supposed to ascend
from 8.9 million trips for each day to around 14.3 million, implying the stamped
increment in the city-state’s populace in Singapore. Simultaneously, Singapore
confronts real imperatives in space, with 12% of land officially possessed by the
3,300 km-street arrange and another 15% dedicated to lodging [4,5]. These days,
forecasts say that by 2050, 86% of the developed world and 64% of the devel-
oping world will be urbanized. The Fig. 1 demonstrates the accident scenario in
the world for the year 2010.

Fig. 1. Road traffic scenario of 2010 in the world [6]

We make use of different types to predict the causes of highway traffic acci-
dents by analyzing the data. Beshah’s and his team [7] analyzed 14,254 accidents
that happened in Ehiopia between May 2005 and September 2008 to figure out
their patterns and predict the role of commuters on possible injury risks. Yan
et al. [8] analysis revealed the causes of highway traffic collisions to be human
error, vehicles, environment of the traffic and traffic management. They set up
114 T. Deb et al.

a dynamic system wherein if one of the factor goes out of order, an accident will
occur. R Geetha Ramani’s et al. researched various data mining techniques to
help in the prediction of road accidents due to the commuters [9]. They ana-
lyzed a dataset of about 159,417 casualties in 157,463 accidents of Great Britain
to find out road accident patterns related to pedestrian characteristics. Sharaf
Alkheder et al. used artificial neural network [7] to predict the injury severity of
traffic accidents based on 5973 traffic accident records occurred in Abu Dhabi
over a 6-year period (from 2008 to 2013). They used a tool called WEKA to
build ANN classifier. These papers endeavored to make sense of traffic accident
patterns based on human behaviour. However, it is hard to discover relationship
between static information about people and impacts since human conduct is
difficult to foresee. What we center around are the conditions that surround with
the traffic accidents. This includes road surface conditions, climate conditions,
number of casualties and other factors.

4 Analysis of Accident Data in Splunk

Data taken from kaggle.com contained 3 data sets, viz, 2005–2007, 2009–
2011 and 2012–2014. We combined all the 3 CSV files together and named it
all accidents.csv. It was observed that the total number of accidents reported
from 2005 till 2014 has decreased over the time and this can be visualized in
Fig. 2.

Fig. 2. Accident rate decreasing over the years

Further more, in Fig. 3, all of the accidents are pictured that happened in
between 2005 to 2014 in a cluster map which basically shows aggregated values
in a geographic region.
Data Visualization and Statistics. Data visualization is projecting the rele-
vant information in pictorial or graphical form. For instance, iPython queries can
be visualized using matplotlib plotting library and seaborn visualization library.
Apart from these, there are other powerful tools available for data visualization
like Cognos, Clickview, Tableau and R for raw data statistics. Splunk on its own
has the capability for data visualization as well as raw data statistics.
 Statistical Analysis of Road Accident Data of UK 115

Fig. 3. Cluster map of accidents

Graphical Representations of Data Analyzing Results. The other vital

parameters in our analysis were day and time as in Fig. 4, where we investigated
impacts that happened day-wise and on time basis too. We watched that crashes
were higher between Saturday and Friday than the rest of the days.

Fig. 4. Weekday analysis of accidents reported

Something else to observe was that the most number of crashes were
accounted in between 2:00 PM to 7:00 PM where top time of impacts is between
4:00PM to 6:00PM. The slightest is amid 8:00 PM to 8:00 AM. This is appeared
in Fig. 5.
116 T. Deb et al.

Fig. 5. Hourly time analysis of accidents reported

We performed a level of accident severity analysis on day basis and it was

intriguing to find that most of the impacts that have been accounted for through-
out the years were at level 3 and level 1 accident severity was the least that
occurred and most of them occurred on day 6 i.e., Saturday. The Fig. 6. below
shows this analysis.
s o u r c e = ‘ ‘ a l l a c c i d e n t s . csv ’ ’ h o s t = ‘ ‘ l o c a l h o s t . l o c a l d o m a i n ’ ’
i n d e x = ‘ ‘ web ’ ’ s o u r c e t y p e = ‘ ‘ csv ’ ’ | c h a r t count o v e r
Day of Week by A c c i d e n t S e v e r i t y
We also did an analysis on the number of casualties over lighting conditions
on the accidents reported. There were mainly 6 lighting conditions that were
reported when accident occurred. These are - ‘Daylight: Street light present’,
‘Darkness: Street lights present and lit’, ‘Darkeness: No street lighting’, ‘Dark-
ness: Street lighting unknown’, ‘Darkness: Street lights present but unlit’ and
‘Daylight: Stree high winds’.

Fig. 6. Accident severity over the days of week

 Statistical Analysis of Road Accident Data of UK 117

More than 70% of the impacts happened in the lighting condition - “Day-
light: Street light present” and least in - “Darkness: Street lights present but
unlit”. Figures 7 and 8 shows number of casualties from 1 to 8 in various lighting
conditions.
s o u r c e = ‘ ‘ a l l a c c i d e n t s . csv ’ ’ h o s t = ‘ ‘ l o c a l h o s t . l o c a l d o m a i n ’ ’
i n d e x = ‘ ‘ web ’ ’ s o u r c e t y p e = ‘ ‘ csv ’ ’ | c h a r t count o v e r
L i g h t C o n d i t i o n s by N u m b e r o f C a s u a l t i e s

Fig. 7. Number of casualties VS lighting conditions

Fig. 8. Number of casualties VS lighting conditions

118 T. Deb et al.

Fig. 9. Count of accidents based on road surface conditions

As per the analysis, Figs. 9 and 10 shows most severe accidents occurred in
dry road conditions followed by damp road surface conditions.

Fig. 10. Accident severity VS road surface conditions

5 Future Work

The statistical analysis on the traffic data can be put on a machine learning
model so that we can get an idea of which factors could have any sort of impact
or influence on the road traffic accident. Machine learning is a cutting edge field
in computer science that seeks to get computers to carry out tasks without being
explicitly programmed to carry out a given task [10]. Machine learning uses many
techniques to create algorithms to learn and make predictions from data sets. It
is used in data mining which is a technique to discover patterns and models in
data sets where relationships are previously unknown.
To create good machine learning systems, we need to focus on 4 things -
Data preparation capabilities, Algorithms basic and advanced. Automation and
iterative processes and Scalability.
Predictive analytics for traffic accidents information will provide capability
of integrating different technologies with the existing communication infrastruc-
tures that can help diminish casualties; increase safety across the capacity and
adequacy of street networks. We can achieve a unique approach of analyzing
accident data using Operational Intelligence and Machine Learning. We can do
horizontal scaling as well to increase our computational power using big data
tools like Apache Hadoop and Spark framework that can help in distributed
processing, sorting the outputs.
 Statistical Analysis of Road Accident Data of UK 119

6 Conclusion
In this paper, we have adopted the approach to leverage the Splunk platform
for road accident data analysis. We have analyzed the data in a single platform
compared to using multiple platforms for data storage for statistics and analysis.
In our thorough analysis, we projected and visualized information about impacts
that happened in UK in between 2005–2014. Through our investigation and by
looking at the visualizations we can infer that:
– Road accidents have decreased over the years gradually from 2005 to 2014.
– Most of the accidents happened in between Thursday and Saturday than the
remaining days.
– Peak time for accidents is between 4:00PM to 6:00PM
– Around 80% of the collisions reported were of level 3 i.e. severe.
– Dry road surface conditions accounted for the most number of accidents fol-
lowed by wet road surface.
– More than 70% of the accidents happened in “Daylight:street light present”
lighting condition.
– Also the accidents reported were most severe in dry road surface conditions.
Further machine learning techniques can be applied on this data analysis for
predicting accident hotspots, accident severity and much more.

References
1. Splunk. https://www.splunk.com/. Accessed 4 Sept 2019
2. Splunk features. https://www.quora.com/in/What-are-the-benefits-of-using-Spl
unk. Accessed 1 Aug 2019
3. https://www.kaggle.com/daveianhickey/2000-16-traffic-flow-england-scotland-
wales
4. Intelligent Transportation Systems - Next Wave of the Connected Car. https:
//www.intellimec.com/insights/intelligent-transportation-systemsnext-wave-conn
ected-car/
5. Urbanization of World. http://www.civilservicesexpert.com/geography/urbanisat
ion-of-world/
6. Road Traffic Accident status. http://www.armstrongforensic.com/blog/index.
php/2010/04/12/south-asia-leads-world-in-road-traffic-accidents/
7. Beshah, T., Ejigu, D., Abraham, A., Snasel, V., Kromer, P.: Pattern recognition
and knowledge discovery from road traffic accident data in Ethiopia: implications
for improving road safety. In: 2011 World Congress on Information and Commu-
nication Technologies (WICT), pp. 1241–1246 (2011)
8. Yan, Z. Wang, X., Du, L.: Design method of highway traffic safety analysis model.
In: 2011 IEEE International Conference on Transportation, Mechanical and Elec-
trical Engineering (TMEE) (2012). https://doi.org/10.1109/TMEE.2011.6199168
9. Ramani, R.G., Shanthi, S.: Classifier prediction evaluation in modeling road traffic
accident data. In: 2012 IEEE International Conference on Computational Intelli-
gence and Computing Research (ICCIC), pp. 1–4 (2012)
10. Machine Learning - Brilliant Math Science Wiki. https://brilliant.org/wiki/
machine-learning/
Security
 Detection of Routing Infrastructure
Attack in TCP Connection

Asem Debala Chanu(B) and Bobby Sharma

Department of Computer Science and Engineering, School of Technology,

Assam Don Bosco University, Guwahati, Assam, India
[email protected]

Abstract. From the last few years, routing infrastructure attacks like
Distributed Denial of Service Attack (DDoS) and Denial of Service (DoS)
Attack have been the most trending topic in the domain of Network secu-
rity. And these days most of the people prefer online payment, online
shopping, online class etc. So the attackers tries to flood the router dur-
ing this time. And We are focusing on Distributed Denial of Service
Attack. In this paper, it implement a method to detect (DDoS) Dis-
tributed Denial of Service Attack in router by using Ns3 simulator. For
this simulation we used unbalanced dumbbell topology. And the process
of imposing attack scenario in dumbbell topology is defined in section
VI. This simulated DDoS attacks are presented in graph by using the
proposed algorithm to detect the routing infrastructure attack in the
network.

Keywords: DDoS · Routing infrastructure attack · Ns3

1 Introduction
Routing is one of the most important parts of infrastructure to keep the network
running. Its main function is to forward the packets from one network to another
network. And this is usually done by the device called router [1]. Router does not
just route the traffic to another networks, they provide a database that stored
all the information of the network and choose the shortest route to forward the
packet to their destination. It consists of three components. They are IP address,
metrics and topology. IP address comprises of the IP address of the sender’s and
the receiver’s. Metrics provides the details of cost values used by the routers to
route a packet. By using metric values, the router determines the best path to
forward the packets in a network. And topology is the arrangement of the sender
nodes and the receiver nodes through the lines of connection. In [2], routing table
stores all the route information of the nodes (senders, receivers). From that route
information the sender will choose the best route i.e. shortest route to send the
packets to another host. During this transmission of packets from one host to

Supported by organization x.
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 123–131, 2020.
https://doi.org/10.1007/978-981-15-3666-3_11
124 A. D. Chanu and B. Sharma

another host, an attacker tries to disrupt one or more primary function of the
router. Routing infrastructure attack is a kind of attack that occurred in router
to disrupt in the peering session [11]. During this process, several attacks try
to forge the network. As mentioned in, for exchanging the route information
among the neighbors, a router needs to create a peering. During this peering
session the attacker attempts to break the established session by resetting the
TCP connection. Some of the attacks in router are Denial of service attack,
Packet mistreating attack, Routing poisoning attack, Hit and run attack and
Persistent attack. In denial of service (DoS) attack, it attempts to prevent the
users from attempting network information or services by sending more numbers
of ICMP packets from multiple sources. Packet mistreating attack aims to inject
packet with malicious code to confuse and disrupt the network and router. In
Hit and Run attack, the attackers inject the malicious code into the router and
check whether the network is running properly or not. It is very easy to spot this
kind of attack. Whereas in persistent attack, the attackers inject the malicious
packets to the router until they achieved their goal.

2 DDoS Attack
Distributed Denial of service (DDoS) attack is also a kind of attack that happen
in router, where a group of connected devices try to flood the bandwidth or
resource of target systems such as server, website or the network [9,12]. This
flooding of connection messages or connection request to the target system may
shut down or crash, thereby denying services to legitimate users or system.
Figure 1 shows the structure of DDoS attack. It consists of four components
such as Attacker, Master, Handler and the Victim. The attackers will handle the
port number, TTL value of the victim. Whereas the handler will check which
agents are up and running, it can be TCP, UDP or ICMP. And the agent will
generate powerful packet streams to the victim to flood the bandwidth of the
network or the resources of the victim system [10]. To detect the DDoS attack,
first it is mandatory to know about router’s functionality. The DDoS (Router
Infrastructure attack) attack can be described as follows:-
– At first the network is stable and everything works fine.
– Hackers start to control some host for generating traffic from the controlled
host to the router.
– When the attack on the router is initiated, then some network connections
to that router started dropping the data packets.
– During that attack, some connections are affected and finally all the connec-
tions to the routers are shut down completely.

2.1 SYN Flooding Attack

SYN Flooding Attack is also a kind of Distributed Denial of Service attack that
affects the normal TCP three-way handshake to consume the bandwidth of the
 Detection of Routing Infrastructure Attack in TCP Connection 125

Fig. 1. The structure of DDoS attack

network and render it unresponsive. This can be done by sending faster connec-
tion request than the targeted machine. In SYN flood attack, the attacker starts
sending repeated SYN packets to every port on the targeted server or system to
flood their bandwidth. The server unknowingly started receiving multiple legit-
imate requests to establish the three-way handshake connection. And this will
wait the legitimate users to establish the connection. During this time no new
connection cannot be establish (Fig. 2).

Fig. 2. SYN Flooding attack

2.2 Characteristics of DDoS Attack

– Slow network performance.
– Unavailability of targeted website.
– Usually the connection gets disconnected or frequent link failure

3 Related Works
In [1], authors proposed a scheme to detect and mitigate against the TCP SYN
flooding attacks using three counter measures algorithm to detect the spoofed
126 A. D. Chanu and B. Sharma

IP packets. The three counter measures are C-1, C-2, C-3. C-1 is used to record
the initial SYN packets of each connection. C-2 is used to record the completed
SYN packets. C-3 is used to record the other flooded SYN packets. From this
mechanism they are capable of detecting various SYN flooding attack with high
accuracy in short response time. This mitigation scheme starts working once it
detects the SYN flood. And the received SYN packet is monitored by using three
C’s. The results are
– If the SYN packet is not in any of the three C’s then drop the SYN packet
– If it is in C-2 then pass the packet to C-3.
– If the packet is in C-3. TCP connection is completed; pass this packet from
C-3 to C-2.
As mentioned in [2], authors developed a scalable method for detecting the
routing attacks and locating the malicious nodes called slip. In slip it includes the
synchronization to provide synchronization for the updates of the routing table.
And they undergo a simulation to detect the attack under different network.
As mentioned in [3], the authors proposed an approach called hop count
based packet processing to counter DDOS attack. This DDOS attacks are very
difficult to detect in the source node since the attackers used spoofed IP address.
So, in this approach they marked all the packets which came from the same hop
count and through the same router with the same identification number. At the
receiving side, it checked the hop count value for the incoming packet with that
stored value. Thus, attack packets are identified early and reduced the spoofing
threats also. As mentioned in [4] nowadays, most of the commercial activities like
online shopping, online banking depends on the internet. And denial - of - service
attacks become a main threat for this internet infrastructure. They assumed all
clients are outside the LAN in which the server is present so that the proposed
system can monitor all the incoming packets connecting to the server. In the
detection architecture they divide it into three part i.e. flow classifier, object
module, filter. In flow classifier it consists of IP address of the source, IP address
o f the destination and the IP port number and it maintained the arrival time
of the packets at the edge router. Whereas in object module it will monitor the
flow unti it is considered normal. It changed its parameter to detect different
kinds of attacks. And lastly the filter is used to block malicious flow by the
object module. If the burst length is greater than or equal to RTT (round trip
time) of other connections with the same server and time period is equal to fixed
minimum RTO (Recovery time objective) then they conclude that there is an
attack in edge router.
As mentioned in [5], Distributed denial of service has become a serious prob-
lem to a network. They proposed a defense mechanism to detect the malicious
traffic mainly the SYN flooding attack. SYN flooding attack is a kind of attack in
which the SYN packet is much higher than the RST or FIN packets. In defense
mechanism they use four cases. Case 1: To establish a connection between a
client and a server SYN, SYN ACK, ACK packets are required. During this con-
nection if a server received SYN FIN packet then they conclude that the packet
is malicious. Case 2: The attacker source port or destination port should not be
 Detection of Routing Infrastructure Attack in TCP Connection 127

considered as 0. If the ACK flag is set then the acknowledgement number should
not be considered as 0. Case 3: If a hacker uses some IP address to send the SYN
packets to the server. And the server will reply SYN+ACK packet with ICMP
message to the client to get the information like whether the client receive the
SYN+ACK or not. Case 4: By tracing the route from where it is sent and where
to receive.
As mentioned in [6] DDOS attack is an attempt to prevent the legitimate
users from using services provided by the service providers. So, to detect such
kind of attacks the author proposed an approach based on entropy. Entropy is
the measurement of the randomness and uniformity of the IP address. Some
researchers examined that higher volume of traffic, incomplete connections,
flooding of packets are characteristics of the attack. It can be calculated by ana-
lyzing a series of packets, refer as the window. For normal network connection
the value of the entropy may vary up to some extent but in case of DDOS attack
it will have vast difference in result. They calculate normalized entropy to find
out the overall probability distribution in packet window W. If the normalized
entropy < threshold value then, mark window as suspected.
As mentioned in [7], the authors proposed an algorithm to detect DDOS
attack in the network using Entropy Based Anomaly Detection Algorithm.
Entropy value is calculated with respect to time and packet windows. For identi-
fying the attacks, they mixed the attack traffic with legitimate traffic. And again,
the entropy of the mixed traffic is calculated. By using PERL programming lan-
guage, they used the retrieved data as input to the program for calculating
entropy. Two different approaches are used for detecting DDOS attack. In first,
they calculate the entropy with respect to time window. In time window app-
roach, entropy of the traffic is calculated with respect to equal time stamps. In
the second approach, entropy value is calculated with respect to packet window.
For packet window, equal numbers of flowing packets are taken from network
traffic.
As mentioned in [8], A Distributed Denial of Service attack generates mali-
cious packets from different computers to shut down the communication between
the legitimate users and the servers. In order to mitigate the DDoS attack, it
includes several steps like detection, characterization and trace back. So, the
author used entropy approach to detect the flooding attacks. First, they calcu-
late the entropy for all the clients connecting to a router. Secondly entropy-based
trace back method is applied to find the edge routers from where the whole attack
traffic is entering into the ISP domain of the data center. From this they plot a
graph between the entropy based on trace back with the time. If the values of
entropy get fluctuate then they concluded that there is a DDoS attack in that
period of time.

4 Experimental Setup for Topology

The goal is to implement SYN flood attack agent which sends only the SYN
packet to disrupt the new connections (Figs. 3 and 4).
128 A. D. Chanu and B. Sharma

Fig. 3. Dumbbell structure

Fig. 4. Dumbbell Topology using Ns3 simulator

5 Imposing Attack

Create a new socket with TCP interface by sending packets only with SYN flag
by using the following process,

– Using Factory
• CreateSocket (): Ptr<Socket>
– Using Helper
• Install (node: Ptr<Node>)
• Install the new socket factory class instances to the nodes.
– Using SYN flood socket
• GetType ID
• Bind()
• Connect()
• Listen()
• Set allow broadcast with the SYN flag

6 Proposed Methodology to Detect Routing

Infrastructure Attack
Input: The trace file (Filter) containing packet details after attack.
Output: Counting number of RST packet (RSTpkt), SYN packet (SYNpkt)and
 Detection of Routing Infrastructure Attack in TCP Connection 129

confirming the presence of attack.

Process: Step 1: Open the Filter Step 2: While not EOL Step 3: Begin Step
4: if reg expression(‘SYN’) matches ACK Step 5: increment(SYN count) Step 6:
if reg expression(‘RST’) matches FIN Step 7: increment(RST count) Step 8: if
SYN count > RST count: Step 9: then print “There is no attack” Step 10: elif
SYN count == RST count: Step 11: then print “The connection is good” Step
12: else: Step 13: print “There is an attack” Step 14: Exit

7 Results and Analysis

Above mentioned algorithm has been implemented in a topology using ns3 simu-
lator. Dumbbell topology having two routers are used and traced all the connec-
tion information for all the nodes. From the trace file, it calculates the number of
RST (reset) packets and SYN (synchronization) packets as shown in the figure
below using the proposed algorithm (Figs. 5 and 6).

Fig. 5. Calculation of RST packet and SYN packet

Fig. 6. Calculation of SYN packet and RST packet

For the same topology with attack scenario, gradually the time of simula-
tion is increasing. It has been observed that number RST packets are gradually
increasing with high numbers in comparison to the same topology with same sim-
ulation time without attack. Efficiency of detection methodology is also observed
in both cases (Table 1 and Fig. 7).
130 A. D. Chanu and B. Sharma

Table 1. Trace table of attack packets and non-attack packets

Simulation No. of RST packet No. of RST packet

time (without attack) (with attack)
0–5 2 9
0–10 5 29
0–15 8 35
0–20 19 55
0–25 23 63

Fig. 7. Calculation of RST packets and SYN packeGraph of attacks and non-attacks
for RST packets.

8 Conclusion

Routing Infrastructure attack like DDoS attacks are very complex problem, and
numerous approaches have been proposed to detect this DDoS attack. In this
paper, a simple method is proposed to detect the routing infrastructure attack
i.e. DDoS attack in router by using Ns3 simulator. We used dumbbell topology
for the attack scenario and non-attack scenario and it has been observed that
number of RST packet for attack scenario is more than the RST packet of non-
attack scenario with respect to simulation time.

References
1. Gavaskar, S., Surendiran, R., Ramaraj, E.: Three counter defense mechanism for
TCP SYN flooding attacks. Int. J. Comput. Appl. 6(6), 0975–8887 (2010)
2. Prathap, A., Sailaja, R.: Detection and Prevention of Denial of Service Attacks
Using Distributed Denial-of-Service Detection Mechanism
 Detection of Routing Infrastructure Attack in TCP Connection 131

3. KrishnaKumar, B., Krishna Kumar, P., Sukanesh, R.: Hop count-based packet
processing approach to counter DDoS attacks. In: 2010 International Conference
on Recent Trends in Information, Telecommunication and Computing (ITC). IEEE
(2010)
4. Shevtekar, A., Anantharam, K., Ansari, N.: Low rate TCP denial-of-service attack
detection at edge routers. IEEE Commun. Lett. 9(4), 363–365 (2005)
5. Kumarasamy, S., Gowrishankar, A.: An active defense mechanism for TCP SYN
flooding attacks. arXiv preprint arXiv:1201.2103 (2012)
6. Sharma, S., Sahu, S.K., Jena, S.K.: On selection of attributes for entropy-based
detection of DDoS. In: 2015 International Conference on Advances in Computing,
Communications and Informatics (ICACCI). IEEE (2015)
7. Singh, J., Sachdeva, M., Kumar, K.: Detection of DDOS attacks using source IP
based entropy. Int. J. Comput. Sci. Eng. Inf. Technol. Res. (IJCSEITR) 3, 2249–
6831 (2013)
8. Bhandari, A., Sangal, A.L., Kumar, K.: Destination address entropy based detec-
tion and traceback approach against distributed denial of service attacks. Int. J.
Comput. Netw. Inf. Secur. 7(8), 9 (2015)
9. Zhang, J., et al.: An advanced entropy-based DDOS detection scheme. In: 2010
International Conference on Information Networking and Automation (ICINA),
vol. 2. IEEE (2010)
10. https://www.cisco.com/c/en/us/td/docs/solutions/Enterprise/Security/
Baseline Security/securebasebook/sec chap3.html
11. Navaz, A.S., Sangeetha, V., Prabhadevi, C.: Entropy based anomaly detection
system to prevent DDoS attacks in cloud. arXiv preprint arXiv:1308.6745 (2013)
12. Jun, J.-H., Oh, H., Kim, S.-H.: DDoS flooding attack detection through a step-by-
step investigation. In: Networked Embedded Systems for Enterprise Applications
(2011)
 Optimizing e-KYC Process Using
Distributed Ledger Technology
and Smart Contracts

Hrushikesh Hanbar, Varan Shukla, Chirag Modi(B) , and C. Vyjayanthi

National Institute of Technology Goa, Farmagudi, Ponda, Goa, India

[email protected], [email protected],
{cnmodi,c.vyjayanthi}@nitgoa.ac.in

Abstract. The current know-your-customer (KYC) process in banks

needs the customer to go through the entire process of KYC in every
bank. This process is redundant, time consuming and costly. To address
this problem, we propose a an optimized solution for e-KYC process
using distributed ledger technology and smart contracts, which aims at
reducing the overall cost of KYC verification process in banks. With
the help of the proposed smart contracts, the KYC verification needs
to do once for a customer. The result of this KYC verification process
is securely shared among other banks with the customer’s consent. It
ensures that the relation between a customer and a bank is kept hidden
from other banks. The proposed e-KYC solution offers efficiency, overall
cost reduction and the improved trust with the increased transparency
and throughput. The proposed solution is implemented on hyperledger
fabric blockchain, and throughput and latency are analyzed to test its
feasibility.

Keywords: e-KYC · Optimization · Blockchain · Smart contracts ·

Hyperledger

1 Introduction
Know Your Customer (KYC) is the process of a business or organization verify-
ing the identity of its clients or customers. It is required to assess the suitability
of customers. Especially, banks and other financial organizations are employing
the KYC process to make sure that their customers provide the due diligence
information required to fulfill the anti-money laundering regulations. Therefore,
KYC has become a mandatory and crucial procedure for most of the businesses
[1]. However, there is a lot of efforts and cost involved in the KYC verifica-
tion. In the current system, when a customer opens first account with a bank,
the process cost recurs (please refer Fig. 1 [2] as this process is labor and time
intensive. Some banks don’t have enough staff to attend the number of Anti-
Money Laundering (AML) alerts, and it has more than 85% false positive rate.
Hence, an adaptive and time efficient system is needed which should offer more
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 132–145, 2020.
https://doi.org/10.1007/978-981-15-3666-3_12
 Optimizing e-KYC Process Using Distributed Ledger Technology 133

granular data. An inefficient system costs banks as they are failing to fulfill KYC
AML compliance and thus, fined by governing authorities. In 2018, Dutch bank
ING was failed to fulfill Dutch AML compliance, and as a result, it was fined by
900 million dollars [3]. Thus, there is a need of an optimized system which can
achieve an efficient KYC and AML, while creating the least amount of friction
for banks and its customers.

Fig. 1. KYC process and cost structure [2].

To address the above problem, the blockchain/distributed ledger technology

(DLT) can play a vital role. A blockchain is a distributed database, in which list
of records forming a block are cryptographically linked together and maintained
through the consensus of the entire network of transacting peers. These peers
can have smart contracts (decentralized replicated applications) which help to
carry out the interactions among parties without the involvement of a trusted
third party and publishes the results to the blockchain [4]. Blockchain offers
data immutability, transparency and trust building features. As KYC verification
process needs privacy along with auditability, blockchain can better help to share
the customer data among the banks and to reduce the overall effort/cost involved
in it [5]. However, the relationship between a customer and a bank must be
hidden from other banks while sharing this information due to competitiveness.
As a result, customers need to repeat the KYC process at every banks. At the
same time, customers should know about sharing of their data in such network.
Thus, with the blockchain, customers should do KYC process only once and the
result of the same should be shared with other banks. In addition, the cost borne
by the first bank for customer verification should be shared by the later banks.
In this paper, we propose the optimized e-KYC process by leveraging the
key features of the blockchain and smart contracts. We propose the crypto-
graphic solution to anonymously share customer’s data among the banks with
the customer’s consent over blockchain. Here, the verified KYC documents for
a customer are shared with other banks by considering the customer’s consent.
The proposed e-KYC process addresses the problem of repeated on board of
customers in the due diligence process for KYC in different banks. The KYC
process cost is shared among all the banks which wish to do the KYC verification
for a customer. The privacy of customer-organization relationship is maintained.
134 H. Hanbar et al.

The proposed solution is implemented using the hyperledger fabric blockchain

with the proof-of-authority consensus method. We analyze the experimental
results on throughput and latency of the proposed e-KYC process.
The rest of this paper is organized as follows: Sect. 2 discusses the existing
e-KYC solutions with research gaps, while the proposed e-KYC solution is dis-
cussed in Sect. 3. Section 4 analyzes the experimental results of the proposed
solution. Section 5 concludes our work with references at the end.

2 Background and Related Work

2.1 Existing Efforts on e-KYC Process

In literature, a very few attempts have been made towards simplifying the KYC
process and increasing its efficiency without compromising security. We have
observed that the e-KYC process needs immutability of the records. The cost
of e-KYC should be further reduced and if multiple banks are involved for a
customer, then the overall cost should be shared among them. In addition, it
should have resistance to compromise capability. We have investigated a role of
the national regulator such as RBI in India for a DLT-based e-KYC solution.
However, the identity of the individuals is revealed during KYC approval pro-
cess and storing the documents. Although blockchains offer data transparency
and tracking, the privacy and auditability of a customer is also important for
many applications. The proposed solution attempts to achieve anonymous KYC
details sharing among organizations with the consent of users and cost distribu-
tion of KYC process among the involved organizations. To offer anonymity of
transactions, zero knowledge proofs are used in most of the proposals. Zerocash
[6] offers the decentralized anonymous payments. It guarantees strong privacy
for transactions. However, it lacks in auditability. As many applications needs
auditability, later Zerocash [6] was enhanced to offer privacy and accountability.
In addition, it focuses more on payments and thus, it is very difficult to use in
applications which require data or asset transfer.
IBM has developed “Proof-of-Concept Blockchain-based Shared KYC” in
January 2018 in collaboration Deutsche Bank and HSBC [7]. Shyft [8], a
blockchain-based network allows users to transfer data with regulatory com-
pliance. It has a creditability feature which offers users with a reputation score
based on compliance and historical transaction activity. Here, customers, data
attesters and data consumers interact with each other via the shyft network, and
thus the anonymity of customer relationship is not maintained. Other solutions
like kyc-chain.com [9], kyc.legal [10], and Fides (norbloc.com) [11] are proposed
use blockchain for KYC verification. However, a successful implementation is not
reported yet for document validation by the trusted participants in the network.
In addition, it is very much important to offer data validation among the banks
with the privacy of the customer-bank relationship. Some solutions such as like
KYC services [12] attempts to offer data standardization and harmonization in
KYC processes among banks. However, it relies on a trusted third party [1].
 Optimizing e-KYC Process Using Distributed Ledger Technology 135

2.2 Blockchain and Inter Planetary File System (IPFS)

A blockchain is a distributed database, in which list of records forming a block
are cryptographically linked together and maintained through the consensus of
the entire network of transacting peers. Bitocin [13], a cryptocurrency is the
first and widely used application of blockchain. Later, many other blockchains
are evolved. Ethereum [14] is also a cryptocurrency which leverages some features
of Bitcoin. It supports smart contracts which can be used for many applications.
Bitcoin and Ethereum are considered as public permissionless blockchain tech-
nology. However, they lack in offering sufficient level of performance in many
applications which require to identify participants, permissioned network, high
throughput, low latency and privacy of transactions and data pertaining to busi-
ness transactions [15].
Hyperledger Fabric [15] is a permissioned distributed ledger technology
(DLT) platform which offers access control and can be used for many applications
such as banking, healthcare, supply chain etc. It supports smart contracts writ-
ten in general purpose programming languages. It supports pluggable consensus
protocols that make it feasible for various use cases and trust models. It allows to
deploy and run different consensus protocols which can address scalability issues
and costly mining as in bitcoin and ethereum. A smart contract (chaincode in
Fabric) is the self running or custom running program/code to run business
logic of an application. Most of the existing blockchain frameworks follow order-
execute method. Here, all the transactions are validated through the consensus
method. The validated transactions are ordered and propagated in a blockchain
network in the form of a block. At the end, each peer executes the block sequen-
tially as per the given criteria and put it on blockchain, once the execution is
completed from any of the peer. However, it affects the performance and scala-
bility. In contrast to this, hyperledger Fabric performs execute-order-validate on
transactions which addresses the problem of scalability and performance as in
order-execute model.
Inter Planetary File System (IPFS) [16] is a distributed system for storing
and accessing files, websites, applications, and data. Instead of being location
based, IPFS addresses a file by its contents. The address of a content is hash of
contents and hence unique to that content only, and thus it offers to store large
amounts of data in IPFS, and its links on a blockchain. It secures contents as
only hash of the content is recorded on the blockchain. In the proposed e-KYC
solution, we use the IPFS as permissioned database and hash of the encrypted
documents is recorded in hyperledger.

3 Proposed e-KYC Solution

3.1 Objective and Design Goals
The objective of the proposed e-KYC solution is to optimize e-KYC process
using distributed ledger technology and smart contracts with following design
goals:
136 H. Hanbar et al.

Anonymity of relationships: The proposed solution must keep the relation-

ship between the customer and bank confidential. The banks need to access
customer’s data. However, the identity of the bank must be hidden, while shar-
ing the data from one bank to another. The consent of a customer is must for
the sharing of data from one bank to another.
Minimizing KYC cost: The cost incurred for conducting the KYC process
must be shared among all the banks which wish to do the KYC verification for
a given customer.
Creditability: Every customer must have a creditability measure which helps
banks to build confidence on the customer.
High throughput and low latency: The proposed solution should be able to
perform high number of transactions in a given time with low latency.

3.2 Design of the Proposed e-KYC Solution

In the proposed e-KYC solution, we use a permissioned blockchain-Hyperledger

fabric whose access is controlled by a central regulator such as RBI in case of
bank’s KYC in India. The financial organizations such as banks are enrolled
in this network of blockchain by the central regulator and are issued identity
certificates, as given in [14]. It includes two types of certificates: (1) Enrolment
Certificate - Long term and to identify bank. (2) Transactional certificates - Short
term and unique to each customer bank relationship. The regulator/identity
provider maps all the transactional certificates to their corresponding enrollment
certificates as transactional certificates have pseudonyms of the banks instead
of their original identity. This helps in maintaining the anonymity of interacting
banks. In e-KYC, several entities need to interact with each other through smart
contracts. The interaction of entities with the smart contracts, database and each
other is given in Fig. 2. The customers interact with the network through the
customer portal which acts as an entry point for the customers. It is used to
transfer customer’s KYC data and credentials, and to interact with banks for
giving KYC verification consent and data. The customer is also provided with
the enrollment certificate which is a unique identification number given by the
regulator/identity provider (e.g. Certification authority) when he/she sign up on
the customer portal. Banks have APIs to interact with the customers through
their portal. Banks interact with off-chain database to store and retrieve the
encrypted documents.

Recording Documents. For the first institute recording the KYC documents,
the customer must provide the documents in person and have to be verified
by the bank. It is a tedious process and involves a cost. After the customer has
been verified for those documents and the consent for these documents and bank
has been recorded, the bank then selects a symmetric encryption key such as
AES, encrypts these documents and store them in an off-chain database. This
off-chain database is an IPFS database. After this, the bank records the hash
of these documents along with the link encrypted using the same key on the
 Optimizing e-KYC Process Using Distributed Ledger Technology 137

Fig. 2. Entity interaction in the proposed e-KYC process.

blockchain through record document contract. This link can be later used to
share these documents with other banks and hash helps to prove their integrity.
The contract for recording the document is as follows:
recordDocument (customerID, documentsList, transactionCertificate, docu-
mentsHash, encryptedLink)
– Validate customer signature
– Check for the presence of consent on the ledger by querying
– Records the hash and encrypted link of the encrypted documents on DLT
As shown in Fig. 3, the data stored on blockchain includes the Customer
Profile Table, Customer Consent Table, Encrypted Key Table, Hashed Document
Table. The customer ID is a primary key in each table. The consent record is also
linked by type of document and bank’s transactional certificate. Each customer’s
profile is addressed by a globally unique ID with type, and a list of such IDs
and types are recorded in a profile table. The document hash with the link to
off-chain is stored by encrypting it using symmetric key (AES). Such AES keys
are stored in a separate table by encrytping using the public keys as given in
the transactional certificates. Here, the encrypted documents of a customer are
138 H. Hanbar et al.

stored in this IPFS database. This encrypted data is sent to the permissioned
database with a corresponding link. This link can later be used by other banks
to get the encrypted data.

Fig. 3. Data model in the proposed e-KYC solution.

Consent. A customer gives the consent to a bank for accessing KYC details
by recording “the consent” on the blockchain. A consent is a structured record
including customer ID and the transactional certificates of the bank to which
the customer has given consent along with the list of documents to access. The
smart contract for recording consent is as follows:
recordConsent (consent, transactionCertificate, customerCertificate, signT,
signC)

– Validate customer signature using Hyperledger Fabric CA server API

– Validate bank signature using Hyperledger Fabric CA server API
– Build consent object to be recorded on the ledger
– Write Consent and order the transaction using putState Fabric API

As shown in Fig. 4, through recording consent, the consent is recorded for

each bank wishing to share KYC details of the customer before they start the
process of borrowing this information from other banks. This consent presents
with the first bank which has actually uploaded the documents for the first time.
The procedure of recording this consent involves following steps:

1. The customer selects the list of documents through the portal and sends a
request to a bank for which he/she wants to give consent.
2. The bank selects a unique transaction certificate for this bank customer rela-
tion and replies to customer with this transaction certificate signed with
enrollment certificate secret key and consent signed with transaction certifi-
cate’s secret key. The bank must maintain a map of customers to transaction
certificates and key used to encrypt documents for later exchange of key.
 Optimizing e-KYC Process Using Distributed Ledger Technology 139

Fig. 4. Customer’s consent for bank created through one round trip.

3. The customer then validates these signatures to make sure he/she is interact-
ing with correct bank and then invokes the record consent smart contract on
blockchain to record consent and corresponding transaction certificate. The
smart contract records the consent on the ledger after validating customer
and bank sign.
When other banks with consent from customer wishes to borrow the verified
credential, the exchange of key between these two banks happens without know-
ing each other’s identity through requestKey and fulfillKey contracts as given
below:
requestKey (customerID, documentsList, transactionCertificate, signT)
– Check for consent and sign
– Select another Banks’ Tcert at random
– Ask all banks to fulfil key if Tcert matches
fulfillKey (customerID, documentsList, transactionCertificate A, transac-
tionCertificate B)
– Check the presence of consent for the requestor and provider
– Select another Banks’ Tcert at random
– Record the AES key encrypted using requestor’s TCerts’ public key
Figure 5 shows the process of exchanging the key between banks. It performs
following steps:
1. The second bank first selects a unique transaction certificate for this relation
with the customer and record a request on the blockchain for the key of
those documents after signing this request with its transaction certificate’s
secret key. The requestKey contract selects a random transaction certificate
holder, among all those having consent for this customer, who must satisfy
this request of key. This is done to ensure that not all banks try to fulfil this
request at a time.
140 H. Hanbar et al.

Fig. 5. AES encryption key sharing in the proposed e-KYC solution.

2. The success of this request key notifies all banks, including the bank with key
to customer credentials selected by the contract. This bank realizes that it is
its transaction certificate which has been selected and has key for the creden-
tials of the customer asked in this request. It then looks for the corresponding
symmetric key and encrypt this key with the second banktransaction certifi-
cate’s public key and invokes fulfilKey contract after signing request with its
transaction certificate’s secret key.
3. The smart contract validates the presence of consent for both transaction
certificates and then records this encrypted key on the blockchain.
4. Using this recorded key, second bank is able to verify KYC for a customer
who has given consent.

Here, the cost sharing mechanism works as follows: An average cost per
customer is decided by unanimous agreement of all the parties involved. A fixed
value token system (T ) is used to meet such a requirement. To get the customer
data from a bank, the requesting bank has to pay his share of cost via this token.
This cost is shared among all the banks which currently hold the customer’s
identity. This ensures no bank single handily exploits the systems for its own
benefit and the overall cost of this process is shared among all the banks which
desire the KYC data of that particular customer. Suppose, bank A’s cost is T for
customer C’s KYC verification. When bank B receives confirmation from bank
A for customer C’s KYC verification, it pays cost T/2 to bank A. Thus, cost to
bank A and B becomes T/2. Now, bank C needs to do KYC verification for same
customer and bank A has given confirmation, then bank C has to pay T/3 cost
to bank A. As a result, bank C’s cost is T/3 and bank A’s cost becomes T/6.
Thus, the overall cost of KYC verification of a customer is shared among the
banks. In addition, the bank fulfilling the key request is provided with an extra
fractional reward later to keep them motivated towards this business network.
 Optimizing e-KYC Process Using Distributed Ledger Technology 141

4 Experimental Results and Analysis

4.1 Experimental Setup

For the performance evaluation of the proposed e-KYC solution, we have imple-
mented a prototype using Hyperledger Fabric [15]. As shown in Fig. 6, we set up
a virtual prototype network using Docker on a computer with following spec-
ifications: Operating System: Ubuntu 18.04 (Linux), Processor: Intel core i5
8250U, Memory: 8 Gb. We have considered two organizations, namely: Banks
and Customer portal with one peer for each organization. Peers are owned by
organizations and considered as nodes in the network which host the ledger and
smart contracts. Each organization also have a certification authority (CA) and
membership service provider (MSP) to issue certificates and to provide role to
identities in network.

Fig. 6. Experimental setup of the proposed e-KYC solution.

We have generated the required certificates and keys for the banks and cus-
tomers, with the genesis block which bootstraps the service. We have created a
number of transactions to configure the channel. All the Docker containers are
launched along with the couchDB containers for each peer. CouchDB serves as
the state database to store key value pairs of the ledger and allows rich query
against the chaincode data. Docker starts a virtual network, e-KYC network
for the services running in container to communicate. The peers are then made
to join the channel. A client container is launched to execute commands in the
peer containers, which is used to install the proposed smart contracts (KYC-
chaincode) in both the peers. Here, chaincodes are written in node.js, through
which blockchain is accessed. We have deployed different chaincodes viz; record-
Consent, recordDocument, requestKey and fulfillKey in node.js (as discussed in
Sect. 3.2). APIs of Banks and customer portals are implemented in node.js. These
RESTful APIs are implemented with express.js to record consent and facilitate
142 H. Hanbar et al.

exchange of keys. They wait for consent record request from a customer and
reply with the transaction certificates. We use MongoDB database for banks to
store the mapping between transaction certificates, customerID, and the key for
the encrypted documents. This key is encrypted using a master key of bank.
Bank’s API also waits for any key request made for their transaction certificates
and invokes the fulfil key smart contract. New users register by Fabric Node
SDK through node.js. An admin is enrolled first and his/her ID is used by the
customer portal to register new customers. This returns a wallet ID which is used
by a customer to interact with the blockchain and perform transaction such as
record consent. Same SDK is used to enroll banks and to provide them with
enrollment certificates. In addition, it is used at bank API side to get new trans-
action certificates. All these certificates are X.509 and SHA-256 with ECDSA
used for digital signature generation.
The chaincode mentioned above in the proposed e-KYC solution are tested
using Hyperledger Caliper, a blockchain benchmark tool [17]. Its functions are
subjected to a large number of test transactions at different sent rates. We
have derived the results in terms of throughput and latency in executing the
transaction at different sent rates.

4.2 Results and Discussion

Figure 7 represents the number of the customer’s transactions over chaincode

served by the blockchain network with different sent rate of transactions. It
shows that the maximum throughput with a transaction over blockchain on the
RecordConsent function of chaincode is 5 tps on average. We have observed
that the throughput initially increases with the sent rate, but then saturates
around 5 tps. Similarly, the latency increases with the increase in sent rate as
the transaction requests pile up due to high send rate (please refer Fig. 8).

Fig. 7. Throughput of the customer’s chaincode in the proposed e-KYC solution.

 Optimizing e-KYC Process Using Distributed Ledger Technology 143

Fig. 8. Latency of the customer’s chaincode in the proposed e-KYC solution.

As shown in Fig. 9, the maximum throughput with the bank’s transactions

over blockchain on fulfillKey chaincode is 9 tps on average. We have observed
that the throughput initially increases with the sent rate, but then saturates
around 9 tps. In addition, the latency increases with the increase in sent rate as
transactions tend to queue up (please refer Fig. 10).

Fig. 9. Throughput of the bank’s chaincode in the proposed e-KYC solution.

Fig. 10. Latency of the bank’s chaincode in the proposed e-KYC solution.
144 H. Hanbar et al.

In the proposed e-KYC solution, the transaction certificates act as pseudon-

ames which are used among the banks to share information without their iden-
tity. This helps in achieving anonymity of relationships constraint as the cus-
tomer bank relationship is not exposed while sharing the key for those encrypted
KYC documents. The smart contracts make sure that this information is not
shared through the network without the consent of the customers. The smart
contract validates customer’s certificates and signed consent for transactional
certificate. This process ensures that a man-in-the-middle attack is not possible
and the bank cannot forge a fake consent as it is using digital signature on each
transaction.
The overall cost of KYC process is shared among the banks, which wish to do
a KYC verification for a given customer. Data transparency and immutability
features of the blockchain help in achieving the user’s trust. This helps banks to
improve confidence of the users.
As per experimental results, the throughput of the proposed e-KYC solution
is comparable with the existing process and latency is affordable.

5 Conclusion

The KYC revolution has addressed the problem of money laundering and other
unidentified funding. The current KYC process in banks is redundant, time con-
suming and costly for every customer. To address these problems, we have pro-
posed an optimized e-KYC solution which leverages the benefits of the blockchain
and smart contracts. With the help of proposed smart contracts, a customer need
to do one time KYC verification process only and results of same is shared among
other banks with the consent from a customer, while maintaining the confiden-
tiality of bank-customer relationships, improving the accountability, cost sharing
among the involved banks, sufficient throughput and affordable latency as com-
pared to current solutions. We have implemented the prototype of the proposed
solution using hyperledger fabric blockchain and evaluated the results in terms of
throughput and latency in performing the transactions. The experimental results
and analysis of the proposed solution are very encouraging to make the proposed
solution standardized and widely utilized in financial institutions. In future, the
throughput of the proposed solution can be further improved by incorporating
lightweight crypto algorithms and better performance databases in peer nodes
for the fast execution of chaincode.

References
1. Rajput, V.U.: Research on Know Your Customer (KYC). Int. J. Sci. Res. Publ. 3,
541–546 (2013)
2. Parra Moyano, J., Ross, O.: KYC optimization using distributed ledger technology.
Bus. Inf. Syst. Eng. 59(6), 411–423 (2017)
 Optimizing e-KYC Process Using Distributed Ledger Technology 145

3. Munserman, R., Blackman, A.: ING to Pay $900 Million to End Dutch
Money Laundering Probe. https://www.bloomberg.com/news/articles/2018-09-
04/ing-to-pay-784-million-in-fines-to-settle-dutch-criminal-case. Accessed 13 Aug
2019
4. Martens, D., Van Serooskerken, T., Alexander, V., Steenhagen, M.: Exploring the
potential of blockchain for KYC. J. Digit. Bank. 2(2), 123–131 (2017)
5. Lootsma, Y.: Blockchain as the newest regtech application-the opportunity to
reduce the burden of KYC for financial institutions. Bank. Finan. Serv. Policy
Rep. 36, 16–21 (2017)
6. Ben-Sasson, E., et al.: Zerocash: decentralized anonymous payments from Bitcoin.
In: IEEE Symposium on Security and Privacy, pp. 1–56 (2014)
7. Curry, M.: Blockchain for KYC: game-changing RegTech innovation, IBM
RegTech In-novations. https://www.ibm.com/blogs/insights-on-business/
banking/blockchain-kyc-game-changing-regtech-innovation/. Accessed 13 Aug
2019
8. Shyft. White paper. https://www.shyft.network/pdfs/Shyft-Network-Inc-
Whitepaper V3.1.pdf. Accessed 13 Aug 2019
9. KYC-CHAIN. https://kyc-chain.com/. Accessed 13 Aug 2019
10. KYC Legal. https://kyc.legal/. Accessed 13 Aug 2019
11. Fides. https://www.norbloc.com/. Accessed 13 Aug 2019
12. KYC Services. https://ihsmarkit.com/products/kyc-services.html. Accessed 13
Aug 2019
13. Nakamoto, S.: Bitcoin: A Peer to Peer Electronic Cash System. https://bitcoin.
org/bitcoin.pdf. Accessed 13 Aug 2019
14. Buterin, V.: A Next-Generation Smart Contract and Decentralized Applica-
tion Platform. https://github.com/ethereum/wiki/wiki/White-Paper. Accessed 13
Aug 2019
15. Hyperledger Fabric v1.4: Protocol Specification. https://github.com/hyperledger/
fabric/tree/release-1.4/docs. Accessed 13 Aug 2019
16. Benet, J.: IPFS Content Addressed, Versioned, P2P File System. https://
github.com/ipfs/ipfs/blob/master/papers/ipfs-cap2pfs/ipfs-p2p-file-system.pdf.
Accessed 13 Aug 2019
17. Hyperledger Caliper. https://www.hyperledger.org/projects/caliper. Accessed 13
Aug 2019
 Detecting Fake Image: A Review
for Stopping Image Manipulation

Jahanara Islam Lubna(B) and S. M. Abrar Kabir Chowdhury

University Malaysia Sarawak, Kuching, Malaysia

[email protected], [email protected]

Abstract. Data manipulation getting bigger threat day by day with the
dynamic tech touch for the time being. Image is represented by underly-
ing pixelated data consisting by its area elements. By the blessings and
high availability of smart technology and device, images took important
part of humans memorable life events. This is the evidence with most
consideration for significance by human eye view. A true image can be a
big game player both in social and practical situation. Moreover the tech-
nological manipulation of an image named fake image can make violation
in major perspective consideration rather than any thinking flow or data
to obtain the difference of right or wrong. Here the detailed information
obtained from the conducted a literature review on the fake detection
identification techniques is presented. The review paper contains infor-
mation related to different fake image detection techniques instead of
making detection true and false image. Several detection techniques had
been studied like iris recognition, Support Vector Machine (SVM) and
Purkinje image-based. Simultaneously we have considered biometric sys-
tems for security aspects as well as 2D to 3D image transformation prob-
lems. For web based applications demosaicing detection method and a
colour image change splicing technology we have analyzed. Moreover we
found underwater dam methods can be used for crack detection, where
we focused on fake colorized image detection. Most importantly we have
studied on fake smile identification to enrice image forgery technology
stronger.

Keywords: Fake image detection · Fake smile · GLCM

1 Introduction

The strongest and honest source of demonstration or illustration is pho-

tographed. The evidence-based domains such as news reporting, forensics, army
intelligence, research, crime detection, legal proceedings, insurance claim pro-
cessing, medical imaging, publishing, etc. used to utilize photographs as authen-
ticating document. With time, the technology used for photography has devel-
oped from ordinary photographs to digital ones in each and every domain. Even
though digital imagery has developed to be pocket-friendly or even free, it has
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 146–159, 2020.
https://doi.org/10.1007/978-981-15-3666-3_13
 Detecting Fake Image: A Review for Stopping Image Manipulation 147

downgraded in its reliability aspect over the course of time. The ordinary pho-
tographs were more trustworthy than digital ones. The development of photo
improvement programs such as Adobe Photoshop, Corel Paint Shop, GIMP,
Pixelmator, Photoscape, and Photoplus has played a major role in digital pho-
tography’s credibility demise. With the use of these software’s, it is very easy to
alter any photograph like iris or smile or both alternation [5,10] of an image can
change the output significantly. In addition, if there is a shadow [1] alteration
occurred the view of the considered image will be changed completely. Due to
this, it has become very difficult to decide whether a photograph is original, or
it is altered. This, in turn, has lost its trustworthiness as a piece of evidence in
every important field. The ‘digital image tamper detection’ field is blooming as
a result, to authenticate the digital photographs. It helps to filer the original
images from the altered photographs.
With the great ease of computer graphics and digital imaging, the content
of the image can be altered very easily. None can judge the image if it is real or
a fake one. To make a forgery picture, two resources are usually used, these are
copy-move an image and splicing. In this case, a part of the image is duplicated
and then pasted on different other areas to coat up any unwanted part in the
identical picture. The capabilities of multimedia devices and mobile photography
have increased image fakeness. This activity spoils the name of the deserving
persons. It also influences legal decisions. So, there is a strong need for developing
a healthy framework to analyze the originality of the images. This paper brings
an overview of the current research progress on fake image identification. This
review paper provides information about the work carried out by the different
authors. Here the work is classified based on the image classification technique.
It shows the most effective and most used techniques.

2 Literature Review
2.1 Fake Iris Detection
Precise iris segmentation and localization are the most important objects dur-
ing the implementation of the iris recognition method. The efficiency of the iris
segmentation also matters a lot from an eye reflection that determines the rate
of success for a featured extraction algorithm. With the specific method, we
can easily calculate the accurate outer iris boundary through the tracing feature
of the object all the time irrespective of the shape & size. Inner iris boundary
gives us the result after comparing the two eye images of equal titles at various
intensities so that it can efficiently detect the pupil size. Moreover, the variation
in pupil size is implemented at a time of iris detection. Hence, this approach is
proved more useful than any other based on the fake iris spoofing. This method
has highly encouraging techniques for the iris recognition system. The vital algo-
rithm is implemented through the Phoenix database of 384 images both eyes of
64 subjects. Furthermore, the profit rate in the particular iris localization from
eye image which is more than 99% always. It also removes the possibilities of all
losses of iris texture characteristics in the spatial domain that makes it distinct
148 J. I. Lubna and S. M. A. K. Chowdhury

from other conventional methods. And processing time is also great compared
to these techniques [24].
An analysis on [12] the usage of Support Vector Machine (SVM) for detecting
fake iris in biometric authentication systems. Iris recognition technology can be
certain drawbacks like usage of an artificial eye or printed contact lens. So,
detection of the fake iris is important. This paper proposes the usage of Support
Vector Machine (SVM) as it has high classification ability and is user-friendly.
It is suggested to categorize lower half of Iris patter as Region of Interest (ROI)
to avoid corruption of image. Properties like contrast, correlation, energy and
homogeneity in a Grey Level Co-occurrence Matrix (GLCM) can be useful to
extract textural features. Once the feature values are obtained, a vector is formed
and then classified by Support Vector Machine (SVM). If the output is positive,
it means that it is a live iris. The experiment is carried out using MATLAB.
The total sample size is 2250 images, 1100 are used for testing and the rest for
training. The main issue with the Grey Level Co-occurrence Matrix (GLCM)
approach is the high amount of computing resources required. To solve this,
during the scaling process, grey levels are set at a lower number. Even with this
change, the rate of correct classification is 100% and the average execution time
is 31.3 ms for feature extraction and 14.7 ms for classification [27]. Hence the
scheme can have practical applications.
In order to detect fake iris, a new method, based on Purkinje image has been
proposed. Although previous research has been done on this topic, it has serious
drawbacks [14]. This study proposes to use collimated Infrared Light Emitted
Diode (IR-LED) in order to calculate theoretical positions between the images
and enhance the fake detection algorithm. Based on the focus value, we can
discern if the image is focused enough. The proposed structure for the iris camera
includes dual Infrared Light Emitted Diode (IR-LEDs), two collimated Infrared
Light Emitted Diode (IR-LEDs) and a Universal Serial Bus (USB) camera with
a Charged Coupled Device (CCD) sensor. Designing a shaping model of the
Purkinje image using Gullstrand eye model makes it difficult to use a fake iris
to bypass the system. Based on theoretical values, it is possible to calculate the
exact location where the images are formed. For the experiment, 30 people tried
to recognize 10 times each, resulting in a total of 300 images. In addition, 15
counterfeit samples were used [13].
Several programs have been launched by the government for national iden-
tification like Aadhar in India. It includes iris biometric that presents a unique
identity to millions of people. With the passage of time and development of
modern technologies, iris recognition has become vulnerable to advanced sensor
spoof attacks [30]. In this paper, researchers have investigated another way to
deal with consequently distinguish artificial eye/iris pictures displayed by the
client to the genuine iris imaging sensor for trading off the trustworthiness of
sent iris acknowledgment framework. Recent research has revealed the use of cos-
metic texture lenses to deceive iris recognition devices. Experimental data has
been achieved by performing tests on record obtained from the publicly altered
iris image database. The development of an anti-spoofing technology is based on
 Detecting Fake Image: A Review for Stopping Image Manipulation 149

the global and localized area surrounding the iris, the iris created an application
at the same time uses the highlights from the iris area and the whole eye picture
so as to carry out a solid examination of genuine or fake iris images [14]. The
component considered in this technology utilizes powerful descriptors for the
force conveyance, surface irregularity, edge quality, and surface range to assess
the validness of the iris pictures.

2.2 Fake Biometric Detection

The security of biometric systems is a very concerning topic these days [26].
By using various sophisticated techniques like face masking and gummy fingers
spoofing attacks are being conducted on the biometric systems to gain illegal
access to systems and data. It has been an area where a lot of research has
been done since such fraudulent attacks have been on the rise [19]. A robust and
efficient system is needed to meet the challenges that this threat poses. A new
method to detect fraudulent biometric impression uses an assumption that the
image quality and properties from a real sample will be different from a fake
image sample collected during a hack attack on a biometric system and uses a
very basic but robust level of complexity to ensure it is practical for use in real-
time [25]. It uses livens detection via quality assessment as well as many other
traits of biometrics. The system uses Support Vector Machine (SVM) Classifier
to differentiate between a real and fake sample. This system can also in the
future use the sample parameters to test hand geometry, finger or vein prints by
using further detailed quality parameters for testing.
Biometric Detection of persons has become one of the main tools for the orga-
nizations to know the legitimacy of the photo. For the real authentication of the
images, some organizations have also started the implementation of some authen-
tic protection steps which is a valuable process in reconstructed samples. Novel
software-based fake detection method will be useful during the multi-biometric
systems which will work like obstacles for the frauds and fraudulent cases will
be detected efficiently in less time. Hence, the introduction motive proves which
will drastically work like a reliable ally for organizations enhancing their security
all round the clock [4,9]. Besides, by adding the feature of enhancing the secu-
rity of the biometric recognition framework, organizations also add the action
assessment swiftly and most importantly they’ll be user-friendly. Hence, after the
inclusion of the specific method, we can also neglect the complexity by which
method is drastically implemented in the real-time applications with the most
commonly used 25 images with their quality features extracted from any single
image which is needed during the authentication. This specific image need for
these common images will also identify the legitimate or impostor samples at a
time. The analysis of the common image quality through a particular method
will also assist organizations in getting highly valuable information and thus it
also discriminates itself from the fake characteristics of the obtained result [22].
150 J. I. Lubna and S. M. A. K. Chowdhury

2.3 Image Change Detection

Data about image change detection is very crucial in many areas like satellites.
They use image matching algorithm to provide quick, convenient and cost effec-
tive data to detect change. Image change detection is very useful for urban &
rural planning and monitoring environmental changes [18]. One of the biggest
bottlenecks in this area of research is a 2D format of images that make it very dif-
ficult to separate man-made constructions like buildings, highways, and bridges
from natural changes in the landscape. For that, this paper discusses a new
system which obtains 3D images via stereo technology to resolve this problem.
Kullback Leibler Divergence (KLD) is used to compare the similarity of two
images, the results are combined with the Digital Surface Model (DSM) images
using Dempster Shafer Fusion theory (DS Fusion). Radiometric information at
pixel level does not suffice to the cause thus object-based descriptions of the
images are used to get the accurate data. This new method uses the Digital
Surface Model (DSM) technology and algorithms and the data generated from
the stereo data derived from the space can be a very useful and reliable source.

2.4 Demosaicing Detection

The development of false information among communities is spreading through

Image manipulation techniques. This research [2] aims to create web based appli-
cations to be developed is the main target of this research by putting together
an algorithm and web technology in order to detect false images. The outcome of
this research is, the accuracy of 72% in detecting forged images is managed to be
produced by the web application. Few could be the recommendations can be as
follows: Application of fake image detection techniques by executing enforceable
programs run in server-side comprising of false image detection algorithms with
scripting languages.

2.5 Image Splicing Detection

A color image joining splicing technology dependent on dim level co occurrence

frame-work (GLCM) of thresholded edge picture of picture chroma. There are
two methodologies of image fabrication discovery: active and passive recognition.
The active methodology requires pre-handling (for example watermark inserting)
when producing an image or before disseminate picture. The passive methodol-
ogy does not require this task, be that as it may, and could make an investigation
on different images dependent on regulated learning. This paper has highlighted
the passive method joining identification strategy dependent on the examination
of picture chroma part. The test results have demonstrated that the proposed
highlights of Cb(or Cr) part are more compelling than that of the Y segment.
After the element separating, highlight choice (boosting highlight choice) has
been done so as to lessen highlight measurements [20].
 Detecting Fake Image: A Review for Stopping Image Manipulation 151

2.6 Crack Detection

The detection and classification approach in the novel underwater category pro-
posed. For lack of the underwater dam image detection, an algorithm is intro-
duced [28]. With the assistance of the intensity values of the two-dimensional
image to create a three-dimensional spatial surface, the algorithm can be imple-
mented efficiently. This is also perceived as a concave-convex ground with various
pits and ditches. The pits can be used for the noise pixels and the ditches are for
the crack pixels [7]. Cracks may seem arduous in describing the two-dimensional
image, which is observed as ditches, particularly in the 3D spatial surface envi-
ronment. For getting characteristics of ditches space shapes, the space methods
prove great results to us with the space detected method drastically implemented
to fetch the ditches data that is mapped with the two-dimensional surfaces [28].
Due to the collection of noise and forged cracks, BP neutral system is taken that
can recognize the specific crack object, where the crack data can be obtained
greatly.

2.7 Fake Colorized Image Detection

Detection of manipulation of images can be done by colorization. i.e., the col-
oring of grayscale images to confuse object recognition algorithms. Colorized
images have different hue and saturation channels and dark and bright channels.
The research proposed two ways of detecting these images: Histogram based
Fake Colorized Image Detection (FCID-HIST) and Feature Encoding based Fake
Colorized Image Detection (FCID-FE). This methods have the ability to detect
manipulations directly. Some of these schemes include a comparison of statistical
differences in hue, saturation, dark and bright channels, and modeling the created
4D samples and encoding them. There are three basic types of forgery detection:
Copy-move, which depends on identifying copied areas, splicing detection which
finds altered regions on the final product that are sourced from different images
and finally, image retouching detection. There are three types of colorization
techniques: (1) Scribble based: colors are assigned to pixels on grayscale, (2)
Example-based: Reference color images are provided, similar to grayscale, and
(3) Fully automatic: No supervision is required as a neural network is trained
and images are colorized using end to end network.
Colorization can be detected by examining statistics. Colorized images are
less saturated and favor certain colors. Using Hue-Saturation-Value (HSV), nor-
malized histograms can be calculated. These values differ significantly between
colorized and normal images, which allows us to tell them apart. Histogram
based Fake Colorized Image Detection (FCID-HIST) method can also be used
to detect colorized images [11]. This is done by using the four detection features
to find forgeries. The images that have the most divergence from normal values
are the fake colorized images. After using the divergence to calculate a detec-
tion feature, Supporting Vector Machine (SVM) is used by FCID-HIST to train
and detect fake images. Feature Encoding based Fake Colorized Image Detec-
tion (FCID-FE) method is more effective to fully utilize statistical differences. It
jointly models data distribution and exploits divergences inside the distribution.
152 J. I. Lubna and S. M. A. K. Chowdhury

2.8 Region Duplication Detection

Copy move is one type of forgery to create a fake image. It is a complex issue
and very difficult to detect. In this paper, the researcher proposed a method
to identify copy-move forgery [17]. Despite leaving no trail to detect forgery,
tampering an image to do region duplication damages the image at the pixel
level. Their method uses the robust color moments derived from the image and
makes it possible to detect forgery. The method uses the HSV Histogram based
method and the features retrieved from the image detects duplicated regions in
the image, since any region duplication or copy-move forgery requires duplicated
regions to be used from the same image and thus the field of the copied image
properties are going to be similar to one part or another of the same image due
to duplication. This is a very robust and efficient method to catch duplicate
elements. The proposed system is very efficient in working over two separate
databases and has shown great results in the detection of the copied section. In
a large database of non-tampered and forged images, this system will successfully
detect images which have been subject to region duplication and thus will be
very useful in detecting multiple attacks of copy-move forgery.

2.9 Face Spoofing Detection

Biometric systems in place at the moment are not capable enough to withstand
spoofing attacks which gain illegal access to systems & data by falsifying the
identity. This developed system uses image quality assessment & texture analysis
of an image to determine whether the face on the other end is a person or just a
face print [16]. The system is based on image quality assessment and differences
between the two samples’ artifacts. The difference between a no rigid 3D human
face and a rigid planar photograph of a face is exploited by this technique and
the difference between image pigments, light reflections & shadows are used to
identify fake images [29]. By using Local Binary Patterns (LBP) technique, the
method identifies printing quality defects to make the distinction & it’s a robust,
quick & can be done without end-user approval. This method can also be used
for face recognition & has given successful results on testing is done on a publicly
available database which consisted of several real and many fake images and the
testing results were very accurate. This system can be successfully implemented
in other spoofing attacks that use masks or 3D models on the basis that human
skin has a unique texture which is almost impossible to replicate in that level of
detail from a mask [21].

2.10 Object Detection

It is critical to improving the uprightness and exactness of sonar picture tar-

get identification, which is huge for submerged location. An assortment of sonar
picture denoising calculations and division calculations are considered, and a
denoising calculation dependent on quick bend change is proposed. Affected by
 Detecting Fake Image: A Review for Stopping Image Manipulation 153

submerged condition commotion, the ship’s very own clamor and resonation sig-
nal, Doppler move and engendering misfortune during the time spent acoustic
transmission brings the sonar picture issues as low complexity, solid dot commo-
tion, fluffy objective shape, etc. Low Resolution, poor picture quality, less valu-
able data, and different issues, genuinely influencing the submerged discovery
and tasks. The picture division calculation dependent on k-implies bunching is
examined, and the ideal grouping number screening and sonar picture subsurface
division are figured it out. The sonar picture quick division calculation dependent
on the Independent Chip Model (ICM) calculation and the item form location of
sonar picture dependent on level set strategy is acknowledged in MATLAB [27].
The outcomes demonstrate that the proposed calculation can improve the com-
motion decrease impact of the sonar picture under resonation obstruction and
get a superior picture recognition impact. Joined with the underlying division
aftereffects of k-implies grouping, the last division is accomplished by refresh-
ing the markings through the iterative restrictive model (ICM). The guideline
and imperfections of the C-V model division steady four-stage in the level set
are talked about in detail. Considering the improved calculation proposed by
Professor Li Chumming, we can separate the framework of the objective in the
picture precisely, which have tremendous flexibility.

2.11 Fake Smiles Detection

Fake smiles can indicate negative emotions, discomfort, and secrecy. The purpose
of this research is to detect fake smiles [11]. On a fake smile, certain muscles in
the eyelids do not contract [10]. Using segmentation, wrinkle density is calculated
and based on these variables, smiles can be classified. Accuracy is 86%. Although
it is difficult to discern between real and fake smiles, Duchenne proposed to
observe contractions of muscles in the eye area as these indicate real smiles.
So, real smiles are also called Duchenne smiles. Previous research is focused on
determining whether the person in the picture is smiling based on factors like
facial symmetry and distance between the corners of the mouth. This research
focuses on detecting fake smiles based on detection of wrinkles (a contraction
of muscles near the mouth) and squinted eyes (contraction of muscles near the
eyes) Mouth segmentation is done using color segmentation. To avoid a Region of
Interest (ROI) emerging outside of the mouth, a block is used. Within this block,
corners of the mouth can be detected and based on these, mouth segmentation
can occur. The sample size of the experiment is 100 images, out of which 50
contain real smiles. Using MATLAB, eye elongation and wrinkle density are
calculated [23]. Data is verified 4 times, with 25 images as training data, and the
rest as test data. In the end, the test had an accuracy rate of 86% and an error
rate of 14%. Errors occurred due to inaccurate segmentation and narrow shape
of the subject’s eyes. Based on this study, cheek segmentation can be improved
in the future and improve fake smile classification. Compared to real smile, a
fake smile does not include elongation of eyes. The difference is found by the
calculated value of elongation. The accuracy of this classification is measured by
the exactness of segmentation on cheeks and eyes.
154 J. I. Lubna and S. M. A. K. Chowdhury

2.12 Other Methods

In image processing Edge detection is an essential process and as the detection

results, it has a direct effect on the image analysis [3]. With the detection of the
first derivative, the traditional edge detection algorithms are done. The first-
order differential has few benefits such as speed, simple computation, and how
easy it is to implement. With the help of different algorithms and techniques,
the schemes for counterfeit detection are being evolved. This paper discusses the
image sensing and image enhancement with the help of Canny Edge Technology.
Though the counterfeit detection which is done on a few chosen Philippine ban-
knotes was received with the incorporation of a dissimilar security trait called an
Optically Variable Device (OVD). A MATLAB GUI program is being created
which does the processing of the image with Canny Edge Technology. A robust
method is taken into consideration for the Canny edge algorithm. Thus in image
sensing and Image enhancement, the technology which is used is Canny Edge
Technology. At the same time, the MATLAB program was also being designed.
Though the study showed great improvements in counterfeit detection and also
there was a successful implementation of four-way detection. Optically Variable
Device which is a security feature being used for the counterfeit detection which
is done on opted Philippine banknotes. Thus with these updated technologies,
things have become easier for people to perform.
Computer digital imaging and graphics makes much easier to change the
content of images without any visually traces to identify such manipulations [5].
Numerous false images are developed that are created artificially. Therefore, it
becomes difficult to judge, as if they are visually real or unreal. Singular Value
Decomposition (SVD) is used to detect fake images to make a binary decision
as a classifier to find whether a picture is false or real. This method improvised
the current process of detecting false image using SVDs. These speculations
prove the working of this algorithm, even with one dot of real. Image detect any
minute changes in an image. Authors used the SVD technique in this paper for a
false image detection Scheme. Before the publication of the images to publicare,
certain helpful information inserted into them. The work in recurring to the
hidden information introduced the mathematical SVD operation in the false
image. Improvisation of earlier work in this paper particularly in is achieved.
Two secret vectors are calculated in a new way. Scalar factor expanding from
0.01 to 0.0001 is the important improvisation in this work. The threshold in was
stable for all images which are also another improvement, while in this work, each
image threshold works differently, for each image Instead of constant threshold
(0.01), an auto threshold is chosen. The detection efficiency and removing false
detection get enhanced due to these improvisations. When checking 1000 image
the fake positive rate was 0.8% in while the rate decreases to 0.0% in this paper.
It must be noted that all techniques of detecting false image need the real image
and the false image to make a comparison between them but nowadays the
original image no longer is needed. Comparatively previous work and this work
is analyzed where the sample test taken as color images. They compared and saw
 Detecting Fake Image: A Review for Stopping Image Manipulation 155

that their SVD scheme is sensitive for any minute area modified in any image
and it is great and effective in detecting the false image.
Efficiency & automation in the process of detecting wheel treat defects is the
need of the hour and traditional methods like matching key points are inadequate
to detect the wheel tread defects and that is why a new method was proposed
that uses wheel tread extraction & registration [15]. This system, first of all,
detects the defects of the tread. By using Helmholtz Principle, lines & ellipses
in images are detected, registrations are then performed between the template
& under detecting tread image. Finally, the differences in pre and post-testing
images identify tread defect candidates which are comparing two images in ques-
tion for the tread defects. The testing shows that results of up to 96% could be
achieved between under detected treads & the template. The defect can be iden-
tified accurately according to the results of this method. This solves the problem
of speed, automation & accuracy of tread defect location identification. The next
stage of testing & development plans to recognize the type of defects as well. By
using SVM classifiers, detailed information of the defect like the depth and the
area can be retrieved along with the type of the defect. Labeled images may also
be used to train a CNN model that predicts the type of defects.
Medical science drastically depends upon the digital images these days which
has also increased the query solving efficiency during the authentication process
of the images [8]. Authenticity verification is one of the most vital processes for
scientific research of the digital image. With the help of authentication, we can
also obtain the purpose like forensic investigations and governments documents
verification greatly. Robust image editing software like Microsoft paint & Pho-
toshop helps us in tampering through the digital image kept for spiteful usage
anytime. Furthermore, fake insurance claim cases in the medical sector have
also been increasing in the past few years so the medial sector has also started
implementing medical imaging. Through the past process, Researchers applied
the algorithm that could address the problem of detecting and localize imaging.
Besides, the algorithm drastically depends upon the firmly based representation
of images implementing the discrete wavelet transform method that will also
assist in detection and localization in tempering. Our study shows the algorithm
is thoroughly based on some robust procedures which will remove the threat of
harmful manipulation and thus these are also sensitive for the tempering process
occurring. The proposed algorithm is great in many aspects as well, where the
less computational resource consumption has also been discussed consequently.
As of late, the railroad business in China demonstrates a quick improvement.
Railway technology in china is not that advanced like European countries and
Japan. This paper centers around the detection of track hurdle based on image
processing, canny edge detection algorithm. They are focused on railroad imped-
iment location strategies dependent on the qualities of the tracks; additionally
advanced the identification window idea to lessen the obstruction of the mind-
boggling foundation to disentangle the rail deterrent discovery. In conclusion,
they affirm through tests that the multi-step recognition program undergoes a
compelling impact in obstacle automatic detection. Through an examination of
156 J. I. Lubna and S. M. A. K. Chowdhury

the three techniques referenced above, we arrive at the resolution that “Strat-
egy dependent on the trustworthiness of the Rails and Sleepers” is most precise,
notwithstanding, somewhat, the other two strategies rely upon the light condi-
tion and deterrent hues. For rapid trains, the braking separation is a lot bigger
than the separation that the camera can identify; essentially utilizing the pic-
ture acknowledgment may go out on a limb. In any case, as an optional discovery
technique, picture recognition can consequently perceive harms of train tracks
rather than the human eye, simultaneously, it additionally provides the funda-
mental pieces of information to encourage investigation of the tracks and train
exemptions (Fig. 1).

Fig. 1. Figure shows some comparison and accuracy of methods year wise.

The edge identification examinations of two genuine pictures are led by meth-
ods for two calculations. The similar trial results demonstrate the new calculation
of programmed edge is powerful [30]. The outcomes are additionally superior to
 Detecting Fake Image: A Review for Stopping Image Manipulation 157

the old style Otsu techniques. The improved Sobel administrator and heredi-
tary calculations are used to upgrade division edge of inclination picture and
proposed another programmed ideal limit calculation. The recent calculation
defeated numerous deficiencies of traditional Sobel administrators, for example,
over-segmentation and affect ability. Experiments demonstrate that the estima-
tion speed and hostile to commotion capacity of the new calculations get more
grounded. Be that as it may, we may likewise locate the identified edges are not
fine enough, and a superior refining administrator will be created later on.

3 Conclusions

Research on fake image identification is one of the most preferred topics among
researchers. A number of researchers have come forward to carry on the research
on this topic. This kind of research helps to human mankind to resolve the prob-
lems related to the image forgery. The review of different classification methods
is identified. Mainly the researcher’s used some techniques for detecting fake
image identification. And they are Histogram of Oriented Gradients (HOG) [1],
Fake Iris Detection, Fake Biometric Detection, Image Change Detection, Demo-
saicing Detection, Crack Detection, Fake Smile Detection, Fake Colorized Image
Detection, Region duplication detection, Face Spoofing Detection and etc. [6].
The detailed comparison of the different methods and their relative pros and
cons are given in this paper. The MATLAB 9.0 algorithm for iris detection was
developed. It is tested with 1 GB ram on the 2.4 GHz CPU. And used the CASIA
Iris database was chosen for tests, which is accessible in the public domain. The
database comprises of 30 pictures (320 * 280) photographic and each picture con-
sists of 3 distinct pictures placed. For segmentation based methods Daughman’s
algorithm can give better results. SVM, PCA, and ICA can give better feature
extraction facilities and can give much better fake image identification. In IRIS
based fake image identification the quality evaluation can be done by Laplacian
of Gaussian (LoG). IRIS recognition is the best method in fake image identifi-
cation. Fake smile identification methods can be combined with the above fake
image identification methods to combine emotional effects with image identifi-
cation.

References
1. Arulananth, T., Sujitha, M., Nalini, M., Srividya, B., Raviteja, K.: Fake shadow
detection using local histogram of oriented gradients (HOG) features. In: 2017
International Conference of Electronics, Communication and Aerospace Technol-
ogy (ICECA) (2017). https://doi.org/10.1109/iceca.2017.8212765
2. Bachtiar, M., Gusti, D., Wijaya, I., Hidajat, M.: Web-based application develop-
ment for false images detection for multi images through demosaicing detection.
In: 2018 International Conference on Information Management and Technology
(ICIMTech) (2018). https://doi.org/10.1109/icimtech.2018.8528175
158 J. I. Lubna and S. M. A. K. Chowdhury

3. Ballado, A., et al.: Philippine currency paper bill counterfeit detection through
image processing using Canny Edge Technology. In: 2015 International Conference
on Humanoid, Nanotechnology, Information Technology, Communication and Con-
trol, Environment and Management (HNICEM) (2015). https://doi.org/10.1109/
hnicem.2015.7393184
4. Bhakt, N., Joshi, P., Dhyani, P.: A novel framework for real and fake smile detection
from videos. In: 2018 Second International Conference on Electronics, Communi-
cation and Aerospace Technology (ICECA) (2018). https://doi.org/10.1109/iceca.
2018.8474594
5. Bodade, R., Talbar, S., Batnagar, A.: Dynamic iris localisation: a novel approach
suitable for fake iris detection. In: National Conference on Signal and Image Pro-
cessing Applications (2009). https://doi.org/10.1049/ic.2009.0123
6. Bulla, A., Shreedarshan, K.: Fake shadow detection using local HOG features. In:
2016 IEEE International Conference on Recent Trends in Electronics, Information
& Communication Technology (RTEICT) (2016). https://doi.org/10.1109/rteict.
2016.7808043
7. Chen, C., Wang, J., Zou, L., Fu, J., Ma, C.: A novel crack detection algorithm of
underwater dam image. In: 2012 International Conference on Systems and Infor-
matics (ICSAI2012) (2012). https://doi.org/10.1109/icsai.2012.6223399
8. Gadhiya, T., Roy, A., Mitra, S., Mall, V.: Use of discrete wavelet transform
method for detection and localization of tampering in a digital medical image.
In: 2017 IEEE Region 10 Symposium (TENSYMP) (2017). https://doi.org/10.
1109/tenconspring.2017.8070082
9. Galbally, J., Marcel, S., Fierrez, J.: Image quality assessment for fake biometric
detection: application to iris, fingerprint, and face recognition. IEEE Trans. Image
Process. 23(2), 710–724 (2014). https://doi.org/10.1109/tip.2013.2292332
10. Gunadi, I., Harjoko, A., Wardoyo, R., Ramdhani, N.: Fake smile detection using
linear support vector machine. In: 2015 International Conference on Data and Soft-
ware Engineering (ICoDSE) (2015). https://doi.org/10.1109/icodse.2015.7436980
11. Guo, Y., Cao, X., Zhang, W., Wang, R.: Fake colorized image detection. IEEE
Trans. Inf. Forensics Secur. 13(8), 1932–1944 (2018). https://doi.org/10.1109/tifs.
2018.2806926
12. He, X., An, S., Shi, P.: Statistical texture analysis-based approach for fake iris
detection using support vector machines. In: Lee, S.-W., Li, S.Z. (eds.) ICB 2007.
LNCS, vol. 4642, pp. 540–546. Springer, Heidelberg (2007). https://doi.org/10.
1007/978-3-540-74549-5 57
13. Hou, B., Wei, Q., Zheng, Y., Wang, S.: Unsupervised change detection in SAR
image based on gauss-log ratio image fusion and compressed projection. IEEE J.
Sel. Top. Appl. Earth Obser. Remote Sens. 7(8), 3297–3317 (2014). https://doi.
org/10.1109/jstars.2014.2328344
14. Lee, E.C., Park, K.R., Kim, J.: Fake iris detection by using purkinje image. In:
Zhang, D., Jain, A.K. (eds.) ICB 2006. LNCS, vol. 3832, pp. 397–403. Springer,
Heidelberg (2005). https://doi.org/10.1007/11608288 53
15. Lv, S., Zhou, F., Wei, Z.: Train wheel tread defects detection based on image reg-
istration. In: 2017 IEEE International Conference on Imaging Systems and Tech-
niques (IST) (2017). https://doi.org/10.1109/ist.2017.8261509
16. Maatta, J., Hadid, A., Pietikainen, M.: Face spoofing detection from single images
using micro-texture analysis. In: 2011 International Joint Conference on Biometrics
(IJCB) (2011). https://doi.org/10.1109/ijcb.2011.6117510
 Detecting Fake Image: A Review for Stopping Image Manipulation 159

17. Malviya, A., Ladhake, S.: Region duplication detection using color histogram and
moments in digital image. In: 2016 International Conference on Inventive Computa-
tion Technologies (ICICT) (2016). https://doi.org/10.1109/inventive.2016.7823199
18. Patil, R., Pete, D.: Image change detection using stereo imagery and digital surface
mode. In: 2015 International Conference on Information Processing (ICIP) (2015).
https://doi.org/10.1109/infop.2015.7489376
19. Pravallika, P., Prasad, K.: SVM classification for fake biometric detection using
image quality assessment: application to iris, face and palm print. In: 2016 Interna-
tional Conference on Inventive Computation Technologies (ICICT) (2016). https://
doi.org/10.1109/inventive.2016.7823189
20. Pritam, D., Dewan, J.: Detection of fire using image processing techniques with
LUV color space. In: 2017 2nd International Conference for Convergence in Tech-
nology (I2CT) (2017). https://doi.org/10.1109/i2ct.2017.8226309
21. Rebhi, A., Abid, S., Fnaiech, F.: Texture defect detection method based on H-
image and Hotteling model T2 . In: 2014 1st International Conference on Advanced
Technologies for Signal and Image Processing (ATSIP) (2014). https://doi.org/10.
1109/atsip.2014.6834589
22. Reno, A., David, D.: An application of image change detection-urbanization. In:
2015 International Conference on Circuits, Power and Computing Technologies,
ICCPCT 2015 (2015). https://doi.org/10.1109/iccpct.2015.7159368
23. Rosario-Torres, S., Velez-Reyes, M.: Speeding up the MATLABTM hyperspectral
image analysis toolbox using GPUs and the jacket toolbox. In: 2009 First Work-
shop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing
(2009). https://doi.org/10.1109/whispers.2009.5289089
24. Sanchez, C., Niemeijer, M., Suttorp Schulten, M., Abramoff, M., van Ginneken,
B.: Improving hard exudate detection in retinal images through a combination
of local and contextual information. In: 2010 IEEE International Symposium on
Biomedical Imaging: From Nano to Macro (2010). https://doi.org/10.1109/isbi.
2010.5490429
25. Tan, C., Kumar, A.: Integrating ocular and iris descriptors for fake iris image
detection. In: 2nd International Workshop on Biometrics and Forensics (2014).
https://doi.org/10.1109/iwbf.2014.6914251
26. Wang, W., Dong, J., Tan, T.: Effective image splicing detection based on image
chroma. In: 2009 16th IEEE International Conference on Image Processing (ICIP)
(2009). https://doi.org/10.1109/icip.2009.5413549
27. Xinyu, T., Xuewu, Z., Xiaolong, X., Jinbao, S., Yan, X.: Methods for underwater
sonar image processing in objection detection. In: 2017 International Conference
on Computer Systems, Electronics and Control (ICCSEC) (2017). https://doi.org/
10.1109/iccsec.2017.8446701
28. Yao, T., Dai, S., Wang, P., He, Y.: Image based obstacle detection for automatic
train supervision. In: 2012 5th International Congress on Image and Signal Pro-
cessing (2012). https://doi.org/10.1109/cisp.2012.6469703
29. Zhang, L., He, X.: Fake shadow detection based on SIFT features matching. In:
2010 WASE International Conference on Information Engineering (2010). https://
doi.org/10.1109/icie.2010.58
30. Jin-Yu, Z., Yan, C., Xian-Xiang, H.: Edge detection of images based on improved
Sobel operator and genetic algorithms. In: 2009 International Conference on Image
Analysis and Signal Processing (2009). https://doi.org/10.1109/iasp.2009.5054605
 A View on the Blockchain as a Solution
to the Healthcare Industry: Challenges
and Opportunities

Sharmila(B) , Krista Chaudhary, Umang Kant, and Pramod Kumar

Krishna Engineering College, Ghaziabad, Uttar Pradesh, India

[email protected]

Abstract. Due to the advancement of technology, the large volume of

digital data increases rapidly in every field. One of the most sensitive dig-
ital data field is the healthcare system. Healthcare information contains
high delicate private data, and it needs to be shared among the peoples
such as health specialists, pharmacist, family members, insurance compa-
nies, hospitals etc. Healthcare organization have not only their patient’s
medical details which include insurance and financial account informa-
tion. The breach of healthcare information is one of the most important
concern nowadays. Recently, the research studies are going on Blockchain
technology to secure the health care system. Blockchain technology is an
open and distributed online database system which comprises the list of
blocks that are related to each other. This paper summarizes the impact
of blockchain in the healthcare system and enfranchises the privacy of
a patient’s health data. This paper also addresses the issues, challenges,
and research onset of blockchain in the area of the healthcare system.

Keywords: Blockchain · Breaching · Data storage

1 Introduction
Generally, the modern healthcare systems provide more facilities to the doc-
tors to gather, analyze, and monitor the health information of remote patients
due to the advanced computing approaches such as the Internet of Things (IoT)
and clouds. Technology development has eroded the privacy and security protec-
tion of our data. The privacy and integrity of the healthcare systems information
must be protected from an unauthorized user, the internal, and external attacks.
The extensive research work is going on to secure the health care systems such
as Electronic Medical Report (EMR), Electronic Health Records (EHR), Health
Information System (HIS), and Personal Health Records (PHR) [1]. Recently,
researchers have been proposed many cryptographic algorithms to ensure the
privacy and security of the healthcare system. Healthcare system consists of
a large diversity of data’s are generated, scattered, stored and retrieved. For
example, consider the patient undergoes to treatment; the patient needs to take
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 160–169, 2020.
https://doi.org/10.1007/978-981-15-3666-3_14
 A View on the Blockchain as a Solution to the Healthcare Industry 161

various tests, scans, CT scan, etc. The data will be available at the test cen-
ter as well as with a physician. The data could be accessed by a physician in
the same hospital as well as a physician in another hospital. The advancement
of technology plays a vital role to provide security, privacy, and integrity of
health care data management system. Blockchain is a tamper-proof distributed
or decentralized online database which is undisputable, non-rewritable, read a
dissimilar data structure, consists of the large number of blocks which are linked
together using non alterable key referencing mechanism. Blockchain is a network
of independent entities which work together and form a peer to peer network.
The blocks of data which is stored in the network are synchronized together
using the consensus mechanism and offers a validity of data over the network.
The blockchain [2] consists of a list of blocks called a data structure. The data
structure of blockchain consists of the following elements: timestamp, the hash
value of the previous block, transaction data. The properties of data structure
are as follows: (1) secured blocks (data records) (2) peer to peer network (3)
consensus mechanism (4) security mechanism which offers unchallengeable of
the data. These four properties play vital role in blockchain mechanism for the
important consequences. The data availability of blockchain is either permission
or permissionless. The block-chain is distributed ledger, which does not depend
on a third party to perform transaction between two entities. The transaction
between the two parties can be carried out using consensus rules to ensure the
trustworthy of the parties. The trustworthy can be ensured by the participants
in the network instead of any centralized parties. The inherent characteristics
of the blockchain technology are the transaction cannot be changed anymore.
The security of blockchain [3] depends on the cryptographic encryption func-
tion and consensus mechanism. The large resources are needed to carry out
the malicious attack on the block of transaction. The complexity of attacks as
increases whenever the new block of the transaction is attached to the chain of
an existing transaction block. The aforementioned features of blockchain offer
the following services such as integrity, privacy, security, traceability, and non-
repudiation of data while storing the information in a decentralized way. The
exhaustive research works have been carried out in the field of the banking
sector, real estate, government bid, and finance using blockchain. Recently, the
researchers started to address the potential of using blockchain technology in the
field of healthcare. The blockchain technology provides the solution to the exist-
ing challenges in healthcare applications. The main contribution of this paper
is to addresses the challenges, possible solution and its of blockchain technology
in the healthcare applications. The organization of this paper as follows: Sect. 2
describes the literature of the healthcare system. Section 3 discusses the litera-
ture of blockchain. Section 4 discusses the uses of blockchain for the healthcare
industry. Section 5 concludes this paper.
162 Sharmila et al.

2 Healthcare System Literature

Healthcare system needs a unique security requirement due to the sensitive and
legal information of patients. In this modern era, the risk of malicious attack and
private information being compromised increases day by day because of ease data
sharing with cloud computing using the internet. The sharing of information is
the main concern due to the development of technology, the patient can use
smart devices to carry their health information to the different doctors. One
of the most significant field is big data which can be applied in the healthcare
data to improve the patient’s outcome, predict the epidemic, avoid preventable
diseases, and minimize the cost of healthcare delivery and improve the quality of
life [3]. The security and privacy of healthcare data is a difficult task. Big data
which can be useful. In order to ensure the security and privacy of data, it is
necessary to identify the limitations of existing systems and envisage directions
for research in future. The security requirement of the healthcare is as follows:
access control, authentication, interoperability in terms of centralized storage of
data, mobility, transfer of medical data, etc. Figure 1 shows the requirements of
healthcare system. Medical information [6] consists of patient’s personal details,
medical records as well as medical data which received from wireless body area
network or sensor nodes implanted in the body [7]. The medical records of the
patients are transferred from traditional paper to electronic documents i.e., digi-
tal medium. Electronic records are stored in the form of databases. It needs more
security as well as authorized access control of data. The risk of replication or
modification of data can be reduced by means of access control. The traditional
encryption techniques are not suitable for the medical record due to the different
standards of encryption used in various systems.
Interoperability is generally defined as the process of sharing and transmit-
ting the data between the various sources. In a centralized database, all the
medical records are stored in one database. Centralized data storage is the main
limitation of interoperability which causes data to need to be fragmented, diffi-
cult to perform data sharing and limit the speed of data access. The advancement
of MEMS technology, the mobility of healthcare data is an important require-
ment of the health industry. The real-time healthcare data is collected from
sensor nodes, Body Area Sensor Network (BASN), pervasive smart devices, and
Internet-enabled devices. It is a challenging task to secure the real-time data
collected from wireless devices or an IoT. The privacy and security of mhealth
care [8] suffer from the centralized healthcare system. Wireless Sensor Networks
(WSN) or IoT is resource constraint devices which need to perform more compu-
tation to provide security and privacy. The malicious attacker compromises the
sensitive health data which affects the future vision of healthcare applications.
 A View on the Blockchain as a Solution to the Healthcare Industry 163

Fig. 1. Healthcare requirements.

3 Blockchain Architecture and Its Requirements

Blockchain is decentralized ledger in which transaction among a node to node in
the network is carried out without any intervention of centralized authority. Gen-
erally, the two version of blockchains [6] such as cryptocurrencies version 1.0 and
version 2.0. Zheng et al. [9] address four key parameters of blockchain method-
ology which will directly influence the healthcare industries in terms of decen-
tralization, auditable, persistence, and obscurity. Figure 2 shows the blockchain
architecture. The architecture of blockchain consists of the following components:
blocks, header, transaction counter, and transaction data. The algorithms used
in blockchain are the hash algorithm, digital signature, forking, and consensus
algorithm. Figure 3 shows the requirements of blockchain. The components of
blockchain as follows: Block: Each block stores the transaction information. It
maintains an immutable and permanent record of data. It consists of three blocks
as follows: 1. Header 2. Transaction Counter 3. Data Transaction
164 Sharmila et al.

Header: Each header section maintains the following information.

• Version Number: It represents which regulation the block follows.

• Hash Block: It maintains and points the parent block hash values and their
corresponding parent’s block.
• Merkle tree root hash: It keeps blockchain values after double hash function
using SHA256.
• Time Stamp: It records the time at which the particular block approved.
• Nonce: On time variable use.
• Threshold: It represents the mining difficulties.

Transaction counter: It sums the total number of the transaction completed

successfully and maintain the details about the current transaction block.
Transaction Data: The transaction data varies depends on the applications
such as Bitcoin, Smart contract, business data, and healthcare data.
Hash function: Hash function is used to provide the fixed length hash value.
Forking: Forking happens in blockchain whenever there is any contradict in
consensus algorithm or any software changes. Depending on the problem, the
forking is divided into the hard, soft fork, and usercentric fork.
Types of Blockchain: Blockchain mechanism is divided into three types
depending on permission of the user such as public, private, and Consortium.
Consensus algorithm: Each node in the distributed network agree on the com-
mon rule based on consensus algorithm for a successful transaction. Therefore,
it needs consensus algorithms to ensure the consistent data/state of transac-
tion among the nodes. Some of the consensus models are Proof of work (PoW),
Byzantine fault tolerance, Proof of Stake (PoS), Proof of Authority (PoA), rip-
ple, and tender mint.
Digital Signature: Digital signature plays an important role to improve the
security of sensitive data, increase the efficiency of administrative processes,
and treatment, as well as E-prescribing and admission in hospitals. The digital
signatures in healthcare need to obey with the Health Insurance Portability and
Accountability Act of 1996.

3.1 Blockchain Features

Blockchain has many features that can be used for healthcare. The features of
blockchain are as follows:

• Decentralized storage
• Authentication
• Disintermediation
• Cost reduction
• Immutability
 A View on the Blockchain as a Solution to the Healthcare Industry 165

Fig. 2. Block chain Architecture.

3.2 Blockchain Limitations

In spite of all the advantages of Blockchain, still, it has limitations need to
over-come. The limitation of blockchain as follows:

• Privacy leakage
• Lack of Standardization
• Key Management
• Scalability issues due to IoT
• Software Vulnerabilities
166 Sharmila et al.

Fig. 3. Blockchain requirements.

4 Blockchain for the Healthcare Industry

Blockchain technology brings more advantages for the healthcare industry.
Blockchain offers secure and trustworthy digital electronic data among health-
care data stakeholders. Figure 4 shows the impact of blockchain on the healthcare
industry. Blockchain provides solutions for the challenges of the healthcare sys-
tem. The existing software solutions for blockchain enabled healthcare solutions
will be discussed in the following sections.

4.1 Gem Health Network

Healthcare industry deals with more number of records and medical documents
which follow the Health Insurance Portability and Accountability (HIPAA) act
of 1996 and offers confidentiality. Earlier, the data’s are stored in a central-
ized system. The main issues of centralized health care database are security,
interoperability, difficult to share the confidential record [6,7] among different
physicians and hospitals. If the patient consults different physicians, it creates
overhead in terms of resource rigorous authentication and consumes more time
due to the creation of a duplicate copy or new records, communication pro-
tocols between the doctors need to change, and records need to update. The
aforementioned problem can be solved by Gem health network [3] which used
 A View on the Blockchain as a Solution to the Healthcare Industry 167

the concept of Ethereum Blockchain technology. It creates a shared network

infrastructure and overcomes the limitation of a centralized database. It facili-
tates the patients and doctors to access the real-time data at any place and any
time. Estonia country has collaborated with healthcare platform named Guard-
time, which uses blockchain technology to provide healthcare service. In Estonia,
every citizen, physician, healthcare provider, and health insurance companies
can retrieve the patients’ medical treatment records by using blockchain tech-
nology. The major limitation of this technology is not addressed the scalability
and key replacement. OmniPHR is designed to share patient up to date medical
records distributed among healthcare care providers and addressed the scalabil-
ity issues. OmniPHR [4] was developed by Roehrs, Costa, and Righi. OmniPHR
addressed the difference between Electronic Health Records (EHR) and Personal
Health Record (PHR). Electronic Health records are maintained and updated
by a doctor without the intervention of patients. In PHR, medical records are
maintained by patients. OmniPHR offers an integrated view of health records
which are distributed across the different health care organization. The main
limitation of OmniPHR is that the data needs to follow the standards reinforced
by the model. It would not share the data if it does not support the scope of the
standards. In the case of key lost or leaked, the key management and recovery
problems are not addressed by OmniPHR.

4.2 MedRec

MedRec [5] is decentralized data management system used to handle EMRs using
blockchain which was proposed by Azaria, Ekblaw, Vieira, and Lippman. In this
MedRec, the stakeholders of the healthcare industry participated as the miners
and access, aggregate, mining data as an anonymous and securing network using
PoW. The main limitation not addressed by MedRec is a key replacement and
legal issues.

4.3 Research Scope and Solutions

Blockchain technology for healthcare applications has been widely acknowledged

by means of many of the organizations. Though, blockchain technology for the
healthcare industry still in their primary stages. Many existing challenges have
been not yet addressed. Figure 5 shows the solution offered by blockchain for the
healthcare industry. This section gives several challenges and its future research
directions of blockchain technology for healthcare applications.

• Need to focus on the scalability of blockchain enabled healthcare.

• Required to conduct and verify the result on analysis of real-time datasets.
• More focus on key management and security(lost key and key recovery).
• More focus needed on the privacy of patients and identity verification.
• Need to work on doctor access authorized data in an emergencies situation
without authorization.
168 Sharmila et al.

Fig. 4. Blockchain requirements.

Fig. 5. The solution offered by blockchain for the healthcare industry.

 A View on the Blockchain as a Solution to the Healthcare Industry 169

5 Conclusion
In conclusion, blockchain offers a solution for some of the challenges faced by the
healthcare industry. Blockchain facilitates secure data access and sharing, secure
the privacy of the patients, patients can keep track of medical records, etc. In
spite of complex design of healthcare system, the blockchain resolves the key
challenges of the healthcare industry including data dissemination, concurrent
access, a huge amount of data processing, access control, limit the storage of data
and cost of data processing. Blockchain not only offers a wide range of applica-
tion in the healthcare industry but it also offers vast research opportunities on
employing blockchain in the healthcare industry.

References
1. Abouelmehdi, K., Beni-Hessane, A., Khaloufi, H.: Big healthcare data: preserving
security and privacy. J. Big Data 5(1), 1–18 (2018). https://doi.org/10.1186/s40537-
017-0110-7
2. Mettler, M.: Blockchain technology in healthcare: the revolution starts here. In:
IEEE 18th International Conference on e-Health Networking, Applications and Ser-
vices (Healthcom) (2016)
3. Zhang, J., Xue, N., Huang, X.: A secure system for pervasive social network-based
healthcare. IEEE Access 4(1), 9239–9250 (2016)
4. Torre, I.: A framework for personal data protection in the IoT. In: 11th International
Conference for Internet Technology and Secured Transactions, ICITST (2016)
5. Cai, Y., Zhu, D.: Fraud detections for online businesses: a perspective from
blockchain technology. Finan. Innov. 2(1), 20 (2016). https://doi.org/10.1186/
s40854-016-0039-4
6. Kiyomoto, S., Rahman, M.S., Basu, A.: On blockchain-based anonymized dataset
distribution platform. In: 2017 IEEE 15th International Conference on Software
Engineering Research, Management and Applications (SERA) (2016)
7. Sharmila, Kumar, D., Som, K., Kumar, P., Chaudhary, K.: General outlook of wire-
less body area sensor networks. In: Singh, M., Gupta, P., Tyagi, V., Flusser, J.,
Ören, T., Kashyap, R. (eds.) ICACDS 2019. CCIS, vol. 1046, pp. 58–67. Springer,
Singapore (2019). https://doi.org/10.1007/978-981-13-9942-8 6
8. Swan, M.: Blockchain: Blueprint for a New Economy. O’Reilly Media Inc., Newton
(2015)
9. Zheng, S., Xie, H., Dai, X., Chen, H.: An overview of blockchain technology: archi-
tecture, consensus, and future trends. In: 2017 IEEE International Congress on Big
Data (BigData Congress), pp. 557–564. IEEE (2017)
 An Approach Towards Encrypting Paired
Digits Using Dynamic Programming
and Diffie-Hellman Key Exchange

Abhilash Kumar Das(B) , Abhrajyoti Das, and Nirmalya Kar

Department of Computer Science and Engineering, NIT Agartala, Agartala, India

[email protected], [email protected], [email protected]

Abstract. Cryptography has gone through tremendous evolution in

recent years. But the concept of enciphering text was started in 1900 BC.
There have been many modifications to older cryptography as well as pro-
found discoveries have been made in this field. Likewise, this paper pro-
poses a cryptographic technique that uses the idea of the Diffie-Hellman
key Exchange to share a common key. This key-value behaves as a seed
value in the formation of key sequences. A methodology has been pro-
posed to encrypt paired digits using dynamic programming. Based on the
shared common key, a table is generated on both sides of the sender and
receiver to have a common data set. This data set helps in encryption
and decryption both. The dynamic table uses the Fibonacci series and
recursive functions to fill the table. Decryption needs searching operation
in the receiver’s table which would search for (x, y) pair. This (x, y) pair
is concatenated to produce plain text.

Keywords: Cryptography · Diffie-Hellman key exchange · Fibonacci

sequences · Dynamic programming · Non Redundant Cipher (NRC) ·
Key Sequence (KS) · Table Generation (TG) · Common Key (CK)

1 Introduction

With the advancement in security mechanisms the level of attacks have also
improved significantly. In cryptography the algorithm is as strong as the key asso-
ciated to it. Hence protecting the integrity of the key is very much essential. Public
key Cryptography like RSA uses the discrete logarithm problem to encipher the
plain text. Many equal blocks of plain text are taken separately and applied with
RSA in each block. RSA inherently uses discrete logarithm problem to encipher
each block of plain text. Such discrete logarithm problem takes enough amount
of time to encipher each block of data. Unfortunately, there are many thousands
block of data in a plain text. Then, ultimately we will take huge amount of time
and demand high computation power or require upgraded architecture.
One of the mechanisms of secure key transfer is the Diffie Hellman Key
Exchange. This section briefly discusses about the idea and methodology that has
been used in this cryptography technique. In [5] the author has discussed about the
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 170–181, 2020.
https://doi.org/10.1007/978-981-15-3666-3_15
 An Approach Towards Encrypting Paired Digits 171

key exchange to multiple users using Logical Key Hierarchy (LKH) and stated the
reliability of Diffie-Hellman key exchange to multiple peers by broadcasting. In [6]
the paper basically discusses about the improvement of Kerberos protocol based on
Diffie-Hellman key exchange and they have shown the results, demonstrating that
the proposed system gives better results by reducing password guessing attacks. In
[7,8] the author has proposed a better modification of Diffie-Hellman key exchange
for Zero Knowledge Proof (ZKP) and has designed an algorithm in order to achieve
it. They have proved ZKP concept by using various situations of key exchanges and
analyzed them. Similarly, many authors have analyzed different situations of key
exchange to demonstrate that Diffie-Hellman key exchange is reliable and easy to
implement. Diffie Hellman is vulnerable to man-in-the-middle attack in wireless
communication, as it is not able to verify the communication. In [9] the author
proposes a improved version of Diffie Hellman Key Exchange using String com-
parison mechanism. Diffie Hellman key exchange can be used in embedded devices
[10] this is achieved by using socket communication through the network. In [11]
the author presents a secure cryptographic method of Key Generation and Shar-
ing using Diffie Hellman Key Exchange Protocol, after the key generation the mes-
sage is encrypted using AES and sent to the recipient. Elliptic Curve Cryptography
has the capabilities of encryption, Digital Signature and Key exchange, in [12] the
author presents a Elliptic Curve Diffie Hellman key exchange. Gowda in [13] pro-
posed a variation of Diffie Hellman that determines the pixel position of the image
to hide data, the public key exchange is done using Diffie Hellman, where the pub-
lic key is a product of the public key and the private key. In this [14] the author
uses a shared secret key for encryption and decryption, this key is shared using
Diffie Hellman Key Exchange, DNA Hybridization technique is used to generate
cipher text from DNA sequences. Joshi [15] uses the Diffie Hellman key exchange
for node to node authentication in a Wireless Sensor Network(WSN) they used
cryptographic concepts and cluster head mechanism to eliminate the weaknesses
of Diffie Hellman. In this paper [16] proposed three different that securely integrate
Diffie Hellman Key Exchange into Digital Signatures.
The Diffie Hellman key exchange [1] has been used here only to exchange the
common key which in turn will generate a sequence of values in tabular form.
These values in the sequence are referred as codes. These codes are considered
to be a cipher of paired digits. In order to encrypt a 2-digit number, we take
half of the digits corresponding to x-axis and the other half to be in y-axis. The
value corresponding to the intersection of the x-axis and y-axis is the required
code. By this simple idea, we will encipher the entire plain-text into cipher of
larger length. During deciphering we take back the reverse process by searching
the cipher in the tabular data set.
After implementation and analysis, it has been inferred that the cryptography
with dynamic programming enhances the speed of execution of the algorithm.
The table formed dynamically fetches the codes (refer Sect. 3) in a easy and
faster manner. Even though the table formation uses enormous system stack
if implemented through recursion but then it decreases the running time when
implemented by dynamic programming in cost of space.
172 A. K. Das et al.

The rest of the paper is segmented as follows: Sect. 2 discusses about the
preliminary assessments of Diffie-Hellman Key exchange. In Sect. 3 the proposed
methodology, the encryption and decryption mechanism is elaborated. Section 4
describes the results and analysis of the proposed method. The comparison with
RSA has been done in Sect. 5. Lastly, the scope and area of application has been
stated in Sect. 6.

2 Preliminaries
2.1 Diffie-Hellman Key Exchange
We have used this protocol in order to arrive at a common key [2]. This com-
mon key will help us in building the sequence table. The algorithm takes few
requirements into considerations. A random integer XA < q and XB < q along
with the primitive root of a prime number p whose powers modulo p generates
integers from 1 to (p − 1). Here YA and YB are published to one another for key
exchange.
The key is generated as:

K = (YB )XA modq (1)

K = (YA )XB modq (2)

The key exchange uses the fact that computing the higher powers (x) of
larger number (g) modulo large number (p) is feasible by using fast modular
exponentiation algorithm [3].
y = g x modp (3)
On the other hand, it is computationally infeasible to solve for x when g and p
are very large numbers. The problem is inherently discrete logarithm problem.

2.2 Modulo Finding Problem

In our cryptography technique, we have used Modulo function to find out y. It
uses the fact of reducing the dividend when it becomes greater than divisor. The
following algorithm finds out (modp).

Modulo(g,x,p):
1. assign i = g
2. do
(a) if (i > p)
i. i = imodp
(b) else
i. i = ai
ii. i = imodp
3. k = k − 1
4. while(k! = 1)
5. return i
 An Approach Towards Encrypting Paired Digits 173

The dividend is squared each time and checked through a condition if it is greater
than divisor. If it is greater, (modp) is taken into consideration but if not the
case, the dividend is squared again. This procedure is repeated until the power
of g get reduced to 1.

2.3 Security Threat in Diffie Hellman Key Exchange

In this section, we shall discuss the security issues in this proposed cryptography
technique. The most common attack in Diffie-Hellman Key Exchange is Man-
in-the-middle-attack (MITMA).

Man-in-the-Middle-Attack. In this attack, the intruder firstly generates a

private key and waits for the first request from any of the two parties to begin
communication. Now, the intruder intercepts the first request from the party
A and sends two data packets giving ACK to party A and sending request to
party B. By now party A has exchanged a common key with A. Furthermore,
the intruder waits for the ACK from party B. At the end of the whole process,
the intruder and both the parties have learned a CK for data exchange. In some
of the cases, the intruder can use only a single private key for getting the CK
for both of the parties. The security of such of the type of the attack relies
completely on the secure connection. Such attacks can tackled by authenticating
the sender and receiver. Such protocols includes MACs and Message Encryption.
In Fig. 1 party A initializes ‘a’ to 3, private key Pr(A) to 13 and prime number
‘p’ to 31 which is relatively prime to ‘a’. Similarly, the intruder generates a
private key Pr(C) whose value is 11 and party B generates its private key Pr(B)
whose value is 17. For instance, party A wants to communicate with party B.
In the beginning, party A sends ‘a’ and ‘p’ to the party B. The intruder in the
middle learns the value of ‘a’ and ‘p’. Now, party A calculates 313 mod31 = 24

Fig. 1. Diagram showing key exchange during MITMA

174 A. K. Das et al.

and sends 24 through the channel. The 24 is received in the intruder’s buffer and
the intruder calculates 2411 mod31 = 11 and sets 11 as CK for party A. Now,
the intruder calculates 311 mod31 = 13 and sends 13 to both the parties A and
B. In return the party B calculates 317 mod31 = 22 and sends 22 to the party A.
The intruder, using the integer 22 calculates 2211 mod31 = 10 and arrives 10 as
CK for party B. This whole procedure can be done by two private keys of the
intruder.

Overcome Against MITMA. The Man-In-The-Middle-Attack is a very com-

mon risk in the security aspect. This kind of attack is unavoidable but somehow
we can stop it to some extent. This can be done by using Message Authentication
Code(MAC) and hash functions where they can provide the authorization of the
message. The authorization can be checked when the intruder replays the mali-
cious messages to the destination as if the messages are coming from the source.
The MACs uses hash function which are one way functions. If any intruder tries
to modify the messages, it would change the output of hash function. When the
receiver receives the message it again calculates the hash function of the received
message. Now, the receiver compares the new hash value with the previous hash
value sent by the sender. If the two hash value mismatches then it signifies that
someone has modified the messages. If matches, then the integrity of the message
is maintained.

3 Proposed Methodology

3.1 Key Sequence (KS) and Table Generation (TG)

The KS (also called codes) are generated by the initial seed value obtained
from the common key (K). A table in a two-dimensional matrix which is filled
up dynamically by a recursive equation as shown in Table 1. The cell[x,y] is
represented in Table 1 as value corresponding to the intersection of x and y. The
x-axis is the rows where as the y-axis is the columns of the table. The cell[0,0]
and cell[0,1] are filled by half of the digits of the common key. For Ex. CK = 136,
then cell[x] = 13 and cell[y] = 6, CK = 1234, then cell[x] = 12 and cell[y]= 34.
It uses floor function after dividing the CK. So, we can use partition function
for splitting the CK into two half partitions. The lef tP artition function divides
the key for cell[0,0] and rightP artition for cell[0,1]. The value of c and d are
referred as seed values for Fibonacci series. In Table 1 the first row with 0th index
is filled upto 9th Index by the consecutive Fibonacci numbers [4] with the base
numbers as c and d, i.e. 1, 2, 3, 5, 8...89. The first column with 0th index is filled
by (Fibonacci number from the previous row of 9th ) + 1 as shown in Table 1.
The rest is filled by the sum of the adjacent upper, left and diagonal left cell.
For example, cell[1,1] is filled up by the sum of cell[0,1], cell[1,0] and cell[0,0]
respectively. The recursive equation is given as:
 An Approach Towards Encrypting Paired Digits 175
⎧
⎨=fib(y,c,d) ; if x = 0
Cell[x, y, c, d] = =cell(x-1,9,c,d)+1 ; if y = 0
⎩
=cell(x,y-1,c,d)+cell(x-1,y-1,c,d)+cell(x-1,y,c,d); else
(4)
Where c is the left partition of CK where as d is the right one. For Ex. CK =
12, then c = 1 and d = 2.

Table 1. Table showing KS with CK = 12

Index
* 0 1 2 3 4 5 6 7 8 9
0 c=1 d=2 3 5 8 13 21 34 55 89
1 (89)+1 (1)+(2)+(89+1) ... ... ... ... 318 462
2 ..
3 ..
4 ..
5 ..
6 ..
7 19745070
8 167084937
9 ..

The Table 1 is generated as shown by using CK=12. Now this can only be
generated if and only if c and d are known to both the sender and the receiver.
If the adversary somehow learns the key ‘c’ and ‘d’ then he can generate the KS
and can search for the codes in the table. The cipher text (codes) are generated
in ascending order of its value in row-wise manner. The proceeding rows get
higher values than the previous rows. Thus, providing non-redundant codes in
column wise fashion. The strength of the algorithm depends on the variability of
partitioning the CK. So, it is quite obvious to say that the large is the CK, larger
the possibilities of partitioning and more the difficulty in breaking the common
key. The recursive equation as shown in Eq. (4) is the most promising part of
table generation. So, the table generator uses the concept of Non Redundant
Cipher (NRC). The NRC gives us unique values in the table so that a plain text
can have at most one cipher. In order to achieve this, the first column of the
table is filled up by next proceeding integer of the previous row corresponding
to 9th column. For Ex. cell[1,0] is filled up by cell[0, 9] + 1.
The encryption model in discussed in the following lines. To understand
the proposed model better we will refer Fig. 2 for our reference. The proposed
model starts with an input of cipher text. The cipher text is in paired form. So,
the model takes 2-digit numbers as a part of plain text as input. The message
176 A. K. Das et al.

length defines the pair of 2-digit numbers. For Example if message length is
4, there will be 4 pair of 2-digit numbers. The input is then divided into two
partitions the left-partition and the right-partition. Out of these two partitions,
the left partition is mapped to x-axis(horizontal axis) and the right partition
is mapped to y-axis(vertical axis) of the table. The values (codes) in the table
(refer Table 1) is the cipher text. It contains all the cipher text corresponding
to plain text 2-digit integer at index [0,0], [0,1], [0,2],...[9,7], [9,8] and [9,9] from
the table. Therefore, tenth place of the 2-digit integer will get mapped to x-axis
where as unit place of 2-digit number will get mapped to y-axis. The intersection
of indexes of x-axis ans y-axis is value that we want for the cipher of respective
plain text 2-digit integer. For example if one pair of the plain text is say 76
then we shall find the cipher at position [7,6]. Similarly, we can place the same
methodology for other plain message pairs as well. After calculating for all pairs,
the cipher text will be of equal or larger length than the plain text.

Fig. 2. Block Diagram for encryption showing the procedure of key splitting and taking
it into account for key generation.

3.2 Algorithm Used for Encryption

The function ENCIPHER(M, CK) takes two parameters, first is the message M
and second is the Common Key(CK). The message M is a sequence of paired
digits. Here, the ’loc’ refers to the location of digit’s place where the division is
to be made for left partitioning and right partitioning. CipherBuffer is the buffer
which is present in the sender’s local machine. At the end, the CipherBuffer is
filled with cipher text and sent to the destination through the network.
 An Approach Towards Encrypting Paired Digits 177

ENCIPHER(M, CK)

1. Assigning the location of the partition at loc

2. Assign one part to the function LeftPartition
c = LeftPartition(CK,0,loc-1)
3. Assign another part to the RightPartition function
d = RightPartition(CK,loc,length(CK))
4. Generating the Cipher Text for each paired digits
for each paired-digits(x,y) in the message string M:
(a) cipher[x,y] = cell[x,y,c,d]
(b) append cipher[x,y] in cipherBuffer
5. append loc in cipherBuffer
6. send cipherBuffer to the socket

Initially, location of the partition of the CK is assigned. The LeftPartition func-

tion combines the digits of CK which are at the left of loc to form a number. Sim-
ilarity, RightPartition form a number with digits of right of (loc-1). Cipher[x,y]
is the table which is formed dynamically when recursive calls are made.

3.3 Algorithm for Decryption

DECRYPT(CIPHER, CK)

1. Extracting the location loc from the CipherBuffer.

2. Finding the cipher text in the table and returning the plain text from the
corresponding indexes of x and y.
for each cipher[i, j] in cipherBuffer:
(a) for each (x, y) in row-major:
i. if (cipher[i, j]==cell[x, y])
ii. store (i, j)
iii. if (cipher[i, j]>cell[x, y])
A. change row = row + 1
B. change column = 0
C. break
(b) append (i, j) pair in PlaintextBuffer
3. return PlaintextBuffer as paired digits

The decryption process includes extensive searching in the dynamically created

table if the CK is large enough. The cipher values are taken from the cipherBuffer
and sent into DECRYPT function with CIPHER and the CK as arguments. The
Plain Text recovered from cipher are appended to PlainTextBuffer. From this
buffer the intended user can extract the message. NOTE: - The searching should
be in row-major order otherwise, the element to be searched can get skipped.
178 A. K. Das et al.

4 Results and Analysis

4.1 Complexity Analysis
Complexity in Encryption
Complexity of encryption includes formation of dynamic table. Each value in the
table requires O(k) running time to compute. But there is a trade-off between
space and time. So, total space required to accommodate all the non-trivial values
is 100 cells. Since, paired digits can have values from 0 to 99 in decimal. Thus total
cost of computation is O(space x time) = O(100 × 1 × k) = O(100k) = O(k).
The functions LeftPartition and RightPartition needs O(length(CK)) to get
the seed values i.e. c and d of the table. So, it can concluded that complexity
of encryption is O(length(CK)length(message)). The efficiency of the algorithm
depends on length of the CK.

Complexity in Decryption
Complexity of decryption includes computation in searching the required cipher
from the table. It takes O(100 entries) x O(k) running time in worst case. Since,
there are only 100 entries that have to be searched. So, in worst case running
time complexity is O(k).

4.2 Implementation Details

In this section, details of execution time in both the encryption and decryption
process in Windows 7, Intel(R)Core(TM) i3-4005U CPU @ 1.7GHz, 64-bit sys-
tem having 4GB RAM and 64-bit High Performance Computing Systems(GHz)
has been shown. An example is given which depicts the working model of the
proposed algorithm.

Sample Experiment for Encryption

Input
1. Message Length: 8
2. Input Message Buffer: 34 41 56 64 24 25 20 42
3. Key Exchange details: a = 122, P rkeyA = 145, P rkeyB = 23, p = 123
Output
CipherBuffer: 37143 144414 4096653 21627588 4544 5474 1662 183338
The Fig. 3 shows the outputs of the encryption.

Sample Experiment for Decryption

Input
CipherBuffer:
37143 144414 4096653 21627588 4544 5474 1662 183338

After decryption
Output: 34 41 56 64 24 25 20 42
 An Approach Towards Encrypting Paired Digits 179

Fig. 3. Output showing encryption

Fig. 4. Output for decryption

The Fig. 4 shows the outputs of the decryption. It has been found that the run-
ning time during encryption is 0.08709 s where as during decryption it has taken
1.485 s. The time difference lies in procedure of encryption and decryption. The
encryption uses dynamic programming to fill up the table in O(k) time for each
entry. But on the other hand while encryption, it have used extensive searching
algorithm for finding the cipher in the table in row-major order. Then, it returns
the corresponding x-axis and y-axis value index.

As this encryption technique is using symmetry key cryptography, it pro-

vides confidentiality and some degree of authentication [2]. Apart from this, it
is vulnerable to man-in-the-middle attack because it does not authenticate the
participants. In order to reduce such vulnerabilities, some countermeasures can
be taken such as Message Encryption, MACs and Hash Functions can be used.

5 Comparison with RSA Algorithm

In this section, the analysis of then running time of both the proposed scheme
and the RSA is done. Unlike the RSA scheme, there is no factorization problem
where the encryption depends on the product of very large prime numbers.
According to paper [17] the author have shown their results of execution.

5.1 RSA Based Scheme

In paper [17] the author have stated the following details.
Number of Bytes processed = 256 Bytes
Number of blocks = 4
Threads per block = 64
Total number of threads = 64 × 4 = 256 threads

As the number of threads and processing time is inversely proportional to each

other. Since, 256 threads takes 12.56 s. Then, single thread will take 3215.36 s.
180 A. K. Das et al.

5.2 Proposed Scheme

In our paper, the following details have been stated.
Number of threads = 256

The message length of 8 is used which gives 8 x (4 Bytes) in total. Since, 32

Bytes of input takes 0.08709 s for producing output. Then 256 Bytes will take
0.69672 s. Since, our program uses 256 threads, then single thread will achieve a
processing time of 178.36032 s. There has been improvement in the performance
when considering the execution on a single thread. Overall the complexity of the
encryption procedure is significantly reduced.

6 Conclusion
This paper proposes an innovative method of Shared Key Cryptography using
the Diffie Hellman Key Exchange mechanism. Our algorithm uses dynamic pro-
gramming and recursion that speeds up the key exchange process, this has per-
formed well with higher range of key values. The performance has been improved
and the complexity has been reduced. This approach has a very wide application
in the field of financial sectors or the commercial sector. For example credit card
or debit card information or the pin for your card can be transmitted to and from
the bank servers using this mechanism as it provides security in a very efficient
manner. Information like OTP or phone numbers can be exchanged in private.
These critical sectors can use our algorithm for a better security infrastructure.

References
1. Diffie, W., Hellman, M.: New directions in cryptography. IEEE Trans. Inf. Theory
22(6), 644–654 (1976)
2. Stallings, W.: Cryptography and Network Security, 4th edn. Pearson Education,
India (2006)
3. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms.
MIT Press, Cambridge (2009)
4. Horadam, A.F.: A generalized Fibonacci sequence. Am. Math. Mon. 68(5), 455–
459 (1961)
5. Bodur, H., Kara, R.: Implementing Diffie-Hellman key exchange method on logical
key hierarchy for secure broadcast transmission. In: 2017 9th International Con-
ference on Computational Intelligence and Communication Networks (CICN), pp.
144–147. IEEE (2017)
6. Hu, Z., Zhu, Y., Ma, L.: An improved Kerberos protocol based on Diffie-Hellman-
DSA key exchange. In: 2012 18th IEEE International Conference on Networks
(ICON), pp. 400–404. IEEE (2012)
7. Ibrahem, M.K.: Modification of Diffie-Hellman key exchange algorithm for zero
knowledge proof. In: 2012 International Conference on Future Communication Net-
works, pp. 147–152. IEEE, April 2012
8. Velten, M.: Zero-Knowledge, the Magic of Cryptography. Saarland University,
August 2006
 An Approach Towards Encrypting Paired Digits 181

9. Taparia, A., Panigrahy, S.K., Jena, S.K.: Secure key exchange using enhanced
Diffie-Hellman protocol based on string comparison. In: 2017 International Confer-
ence on Wireless Communications, Signal Processing and Networking (WiSPNET),
pp. 722–726. IEEE (2017)
10. Deshpande, P., Santhanalakshmi, S., Lakshmi, P., Vishwa, A.: Experimental study
of Diffie-Hellman key exchange algorithm on embedded devices. In: 2017 Interna-
tional Conference on Energy, Communication, Data Analytics and Soft Computing
(ICECDS), pp. 2042–2047. IEEE (2017)
11. Yusfrizal, Y., Meizar, A., Kurniawan, H., Agustin, F.: Key management using
combination of Diffie-Hellman key exchange with AES encryption. In: 2018 6th
International Conference on Cyber and IT Service Management (CITSM), pp.
1–6. IEEE (2018)
12. Mehibel, N., Hamadouche, M.H.: A new approach of elliptic curve Diffie-Hellman
key exchange. In: 2017 5th International Conference on Electrical Engineering-
Boumerdes (ICEE-B), pp. 1–6. IEEE (2017)
13. Gowda, S.N.: An advanced Diffie-Hellman approach to image steganography. In:
2016 IEEE International Conference on Advanced Networks and Telecommunica-
tions Systems (ANTS), pp. 1–4. IEEE (2016)
14. Aieh, A., Sen, A., Dash, S.R., Dehuri, S.: Deoxyribonucleic acid (DNA) for a shared
secret key cryptosystem with Diffie Hellman key sharing technique. In: Proceedings
of the 2015 Third International Conference on Computer, Communication, Control
and Information Technology (C3IT), pp. 1–6. IEEE (2015)
15. Joshi, P., Verma, M., Verma, P.R.: Secure authentication approach using Diffie-
Hellman key exchange algorithm for WSN. In: 2015 International Conference
on Control, Instrumentation, Communication and Computational Technologies
(ICCICCT), pp. 527–532. IEEE (2015)
16. Harn, L., Mehta, M., Hsin, W.-J.: Integrating Diffie-Hellman key exchange into the
digital signature algorithm (DSA). IEEE Commun. Lett. 8(3), 198–200 (2004)
17. Mahajan, S., Singh, M.: Analysis of RSA algorithm using GPU programming.
arXiv preprint arXiv:1407.1465 (2014)
 Detecting Malicious Twitter Bots
Using Machine Learning

Tanu Satija(B) and Nirmalya Kar

Computer Science and Engineering Department, NIT Agartala, Jirania, India

[email protected], [email protected]

Abstract. Cybercrimes and phishing scams have increased multi-folds

over the past few years. Now a days, hackers are coming up with new tech-
niques to hack accounts and gain sensitive information about people and
organisations. Social networking site like Twitter is one such tool. And due
to its large audience hackers use such sites to reach large number of people.
They use such sites to circulate malicious URLs, phishing mails etc. which
serve as the entry point into the target system. And with the introduction
of Twitter Bots, this work got even easier. Twitter bots can send tweets
without any human intervention after a fixed regular interval of time. Also
their frequency of tweets is much more than humans and therefore they
are frequently used by hackers to spread malicious URLs. And due to large
number of active members, these malicious URLs are reaching out to more
people, therefore increasing the phishing scams and frauds. So this paper
proposes a model which will use different algorithms of machine learning,
first to detect twitter bots and then find out which of them is posting mali-
cious URLs. In the proposed model, some features have been suggested
which distinguishes a twitter bot account from a benign account. Based on
those statistical features, model will be trained. The model will help us to
filter out the malicious bots which are harmful for legitimate users.

Keywords: Malicious bots · Twitter bots · Twitter mining ·

Malicious URL

1 Introduction

Twitter is a very famous social networking service where users from all over the
world post and interact with messages which are known as “tweets”. Founded
in 2006 by Jack Dorsey, Noah Glass, Biz Stone, and Evan Williams [1] twitter
has seen some remarkable growth over the years. According to statistics, twitter
had around 30 million users which shoot up to 335 million active users in the
2nd quarter of 2018 (Fig. 1) [2].
Twitter has become one of the most used social media platforms because it is
both personal and rapid. It gives people from different spheres of lives to express
themselves and build relationship with their followers.
Different news channels have twitter accounts to share latest news and devel-
opments. Celebrities use Twitter to build a personal connection with their fans.
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 182–194, 2020.
https://doi.org/10.1007/978-981-15-3666-3_16
 Detecting Malicious Twitter Bots Using Machine Learning 183

Fig. 1. Comparison between active users from 2010 to 2018 on Twitter [2]

Twitter is being used by many brands to market their products and get feed-
back from people about their brand. It helps them to improve their product.
People with same taste and interest form communities on twitter and share their
ideas and hold discussions online. Also people share interesting photographs, new
researches. Authors use twitters to promote their books and the list is endless.
Another interesting feature of twitter is bots. A bot software which controls a
twitter account with the help of Twitter Application program interface is called
Twitter bots [3]. It is like the automation of a twitter account. Automations rules
are set by twitter only. These bots can tweet, re-tweet, like, follow, unfollow,
or direct message any accounts without any human intervention. According to
study conducted by University of Southern California and Indiana University
around 15% of the active twitter accounts are bots [4]. These bots behave just
as humans and therefore it is difficult to identify them. These bots chats with
random users or posts poetry, news or photographs at regular intervals. Some of
the famous bots are:

1. @HundredZeros: Twitterbot that posts URLs to the eBooks that are freely
available on Amazon.
2. @DearAssistant: Gives detailed response to any question asked quickly.
3. @netflix bot: Tweets about new shows on Netflix.
4. @nicetipsbot: Twitter bot which posts fun little life tweets multiple times a
day and many more [5].

Owing to the popularity of twitter, its huge active audience and such inter-
esting features, it also grabs the attention of malicious and unethical users like
hackers, cyber criminals, cyber bullies. There are many malicious and political
bots on twitter which post sensitive contents and malicious URLs. Cyber crim-
inals use these bots to circulate malicious and Phishing URLs through automa-
tion and regular intervals in order to gain sensitive information of various people
leading to phishing scams and frauds. Twitter has seen an increase in the number
of malicious bots in recent years that have distributed fake news and distorted
images, malicious URLs.
184 T. Satija and N. Kar

These malicious twitter bots regularly post malicious URLs, fake news and
try to reach as many people as possible. These malicious URLs can be of some
malicious website or it can be a part of phishing scam asking for some person
sensitive information. These bots serve as a very helpful tool for hackers to
carry out data breaches. Since 2013 till today, total data breaches recorded is
14,717,618,286 [6]. Twitter has been trying to identify such accounts but due to
their human like behaviour, it is difficult to identify them and filter them out
(Fig. 2).

Fig. 2. Number of accounts challenged to be bots by twitter from Sept 2017 to May
2018 [7]

In order to detect such malicious bots, machine learning can be considered

to be a useful aid. Machine learning is a part of artificial intelligence. It works
on the idea that system can be trained to learn and identify patterns and then
take decisions based on that knowledge with very less human intervention [8].
There are like millions of twitter bots and identifying them manually is nearly
impossible. But with machine learning algorithms, we can make models which
will automatically analyse the twitter accounts and help in identifying twitter
bots. So in this paper, a model is being proposed which will, with the help of
machine learning algorithms, filter out those twitter accounts which have a bot
like activity and are posting malicious URLs.

2 Background
Machine learning is considered to be a fast evolving branch which has increase the
pace of automation. Machine learning algorithms like logistic regression, SVM
helps in handling and understanding large amounts of data. These algorithms
help use to develop models which can identify patterns and take decisions with
 Detecting Malicious Twitter Bots Using Machine Learning 185

less human involvement [9]. There are several machine learning algorithms which
are used to train models according to our requirements like linear regression is
used when we have one dependent and one independent variable [10]. Logistic
regression is used when more than one independent variable is there [11].
In order to implement machine learning algorithms and train our models,
we need large amount of dataset. In this paper, we are taking datasets to train
our model from Kaggle [12]. It is an online community which provides many
different kinds of datasets in different formats which can be used by machine
learners and data scientist. It allows its members to use published dataset to
build different machine learning models. Users can even publish their own dataset
on this platform which can be used by others.
Since machine learning is powerful and useful tool, researchers have been
trying to incorporate it with security. Lee, Sangho, and Jong Kim have pro-
posed a suspicious URL detection system for Twitter, WARNINGBIRD. They
have considered correlated redirect chains of URLs in a number of tweets. As
hackers have fewer resources and therefore they will have to use them again , a
portion of their redirect chains will be shared. It’s a real time system. WARN-
INGBIRD consists of 4 modules: data collection, feature extraction, training, and
classification. Data was collected from the collection of tweets with URLs and
crawling for URL redirections. From collected data, useful and unique features
were extracted which were used for training the classifier to identify malicious
URLs. In WARNINGBIRD 12 features has been identified for classifying suspi-
cious URLs on Twitter. WARNINGBIRD uses a static crawler implemented in
Python [13]. In a survey paper published in IJRASET, authors have used the
WARNINGBIRD mechanism and implemented it and have given the results [14].
Short URLs (Uniform Resource Locators) are now a days very popular in
Social media networks but at the same time detecting whether they are mali-
cious or not becomes even difficult. Raj Nepali, et.al. proposed a mechanism
using machine learning to develop a classifier to detect malicious short URLs
with visible content features, tweet context, and social features from Twitter.
The system will be implemented as Firefox’s web browser extension and will be
programmed with XML User Interface Language and Javascript to automate
the processes. The system will fetch tweets on its own and extract the features
from the data and for classification submit it to the classifier. The result will be
displayed to the user [15].
Another paper published in 2017, provides basic understanding of how
machine learning can be used to detect malicious URLs and presents information
about the related work which has been done in this domain [16]. Nikan chavoshi
et.al. proposed a warped correlation finder to identify correlated user accounts
in social media websites such as Twitter. The authors suggested that humans
are not highly synchronous for a long duration, therefore highly synchronous
accounts are most likely to be bots. The proposed model works on activity
correlation and does not require a labeled data set as compared to twitter sus-
pension technique and per-user technique [17]. Authors Novotny and Jan pro-
posed a machine learning approach to distinguish between sophisticate and less
186 T. Satija and N. Kar

sophisticated twitter bots. They defined 4 categories in which twitter accounts

can be separated: social bots, traditional spambots, and fake followers and actual
human accounts. They used 4 machine learning methods to train different mod-
els and showed a comparison between them. As per their work,random forest
performs slightly better than the rest in all performance measures used [18].
Clayton A. Davis et al. proposed a supervised machine learning based model
to identify twitter bots. Since it is a supervised learning based model therefore
authors used Twitter’s REST API to extract data from twitter and converted
into labelled training data. The features taken into consideration in the pro-
posed model were divided into 6 main classes: Network features, User features,
Friends features, Temporal Features, Content Features, Sentiment features [19].
Zhouhan Chen et al. proposed an approach to detect malicious bot groups on
Twitter using features like the use of URL shortening services by account, dupli-
cate tweets and content coordination between accounts over extended periods of
time. Since the approached proposed is unsupervised, therefore there is no need
for labelled data for training [20].

3 Proposed Model and Discussion

There are millions of bots operating on twitter and not all of them are malicious.
The goal of this paper is to develop a system which is able to detect those
twitter bots which are posting malicious URLs. The proposed system, called
Find Malicious Bot can be divided into 3 modules. Figure 3 gives a brief about
the 3 modules.

Fig. 3. Proposed system Find Malicious Bot with its 3 modules

 Detecting Malicious Twitter Bots Using Machine Learning 187

Following are the details about each module:

3.1 MODULE 1: Mining Twitter Data to Extract Tweets

Under this module a Twitter API is created and trending tweets are extracted
according to the required location using WOEID. A WOEID (Where On Earth
IDentifier) is a unique 32-bit reference identifier, assigned by Yahoo!, that iden-
tifies the place on Earth [21,22]. In this paper,tweets related to India and New
York were collected from Twitter using WOEID of India : 23424848 and that of
New York is 2459115.
For over a week, top 1000 treading tweets were collected daily using both
WOEID, resulting in a database of 7000 tweets for each. From all those
tweets,tweets containing any kind of URL were filtered out and all the details
of those tweets were saved in a separate database. And the URLs present in the
tweets were extracted and saved in a separate database. Details extracted from
the tweets are given in Table 1.

Table 1. Details extracted from tweets

Details extracted from Tweets

Id
id str
screen name
created at
Location
Description
url
followers count
listed count
friends count
favourites count
Verified
statuses count
Lang
Status
default profile
default profile image
has extended profile
Name
188 T. Satija and N. Kar

Proposed Algorithm
Step 1: Create Twitter API
Step 2: WOEID = 23424975 for India
Step 3: WOEID1 = 2459115 for New York
Step 4: Extract tweets using trends place(woeid) function for India and New
York
Step 5: Now from Extracted tweets for India
Search for tweets which contain any URLs
If URL present in tweet
Extract all the details mentioned in Table 1 from tweet
Step 6: Save it in a csv file.
Step 7: Also save the URL in another file with twitter id of that
Step 8: Function to identify URL in tweet text
def Find(string):
url = re.f indall( http[s]? : //(? : [a − zA − Z]|[0 − 9]|[$−
@.&+]|[!∗, ]|(? : %[0 − 9a − f A − F ][0 − 9a − f A − F ]))+ , string)
return url
The trending tweets were collected from Twitter because bots post tweets very
frequently (sometimes 700 times in a day) as compared to humans and therefore
those tweets become a trend. As per the tweets extracted, tweets from India
have more URLs as compared to that of New York.

3.2 MODULE 2: Model Trained to Recognize Twitter Bots Using

Machine Learning
In order to identify bot like behaviour several factors need to be considered.
There are three most important factors which can help to identify any bot [23]
(Fig. 4).

Fig. 4. The 3 A’s to identify the bot like activity

 Detecting Malicious Twitter Bots Using Machine Learning 189

3.2.1 Activity
Since bots are automated software, so the frequency of their tweets is much
more as compared to humans. There are some bots which are posting tweets
every minute. This is not human pattern of behaviour. Therefore by tracking
the number of tweets posted by account, bot like activity can be detected. For
example: Netflix Bot(@netflix bot). It will post a new tweet every minute about
some new show, gives updates about going on shows and posts URLs of the same
whereas a legitimate human account will post 2–3 tweets in a day.

3.2.2 Anonymity
Another factor is the degree of anonymity maintained by account. Usually the
less personal information the account gives, the more likely it is to be a bot. If it
will be a legitimate human account, then it will have personal information like
reading their education, jobs and profile pictures whereas bots won’t have such
information. For example: Museum Bot(@MuseumBot). The profile of this bot
doesn’t contain any profile photo.

3.2.3 Amplification
Bots tend to boost the signal from other users by retweeting, liking or quoting
them. Therefore, if timeline of an account consists of a procession of retweets
and word-for-word quotes of news headlines and very few or no original posts,
then there is a very strong possibility that it is a bot. For example: Dear Assis-
tant(@DearAssistant). The timeline of this bot will be filled with retweets and
replies to the questions asked. It contains very less original posts. Another exam-
ple: Museum Bot(@MuseumBot). Its timeline contains URLs and images from
museum always. It will not post anything personal.
Keeping above three points in mind and using the details extracted from
twitter a model is trained to identify whether the twitter account is a bot or not.
The machine learning algorithm used is Bag of Words. This algorithm counts
how many times a word appears in a document. Then these counts are used in
comparison of documents and find their similarities. It can be used in application
like searching, document classification and topic modelling [24]. This algorithm is
used to extract features from text and then those features are used for modelling
purpose. In this algorithm, occurrence of each word is used as a feature for
training a classifier.
Proposed algorithm
Step 1: Dataset about twitter bots is taken from Kaggle and split in 75:25
Step 2: Using 75% of data model is trained on given parameters:
1. Whether account is verified: if verified chances of being bot being mali-
cious is less
2. Tweets name, screen name, description and status is check for some spe-
cific words like
bot—bot—cannabis—tweet me—mishear—follow me—updates—every—
gorilla—yes ofc—forget etc. – these words are usually used in name,
description etc. of bots
190 T. Satija and N. Kar

3. Bag of words algorithm is used to check how many times these words
appear
If frequency is more than 50 ; possibly a bot
4. Then listed count is check. If greater than 16000; less chances of a bot
5. Check for number of retweets; if more than 10,000 and followers less than
200 ; chance of being a bot
Like this several other features are tested and data is saved.
Step 3: Then this trained model is tested on rest 20% data and comparison
is made with predicted data and original data to calculate accuracy.
Step 4: True positive rate (TPR) and false positive rate (FPR) calculated
Step 5: A ROC curve is plotted between true positive rate (TPR) and false
positive rate (FPR) with threshold value of 45◦ .
The data set for malicious bots and legitimate human accounts is taken from
Kaggle and divided in ratio of 75:25 ratios to get optimal result for training
and testing purposes. The accuracy of this model is: 95.44% as per the data set
used. A Receiver Operating Characteristic (ROC) curve is plotted between the
true positive rate (TPR) and false positive rate (FPR) with threshold value of
45◦ . ROC is a curve which is dependent on probability and tells how well model
can distinguish between two classes. Figure 5 shows the ROC plot for the model
trained. Area under curve is near to 1 which means model has a good measure
of separability Equations to calculate TPR and FPR:

T P R = T P/P = T P/(T P + F N ) = 1 − F N R (1)

F P R = F P/N = F P/(F P + T N ) = 1 − T N R (2)

Where
condition positive (P): the number of real positive cases in the data
condition negative (N): the number of real negative cases in the data
true positive (TP): hit
true negative (TN): correct rejection
false positive (FP): false alarm
false negative (FN): miss
false negative rate: FNR
true negative Rate: TNR [25]

3.3 MODULE 3: Model Trained to Recognize Malicious URLS

Using Machine Learning
Under this module, a model is trained using machine learning for detecting
whether the URL is malicious or not. The machine learning algorithm used
is logistic regression and dataset is taken from Kaggle. Logistic regression is
algorithm which is used when there is only one dependent variable but more
than one independent variable. The value of this dependent variable is calculated
using all these independent variables. It is used in this model because here there
 Detecting Malicious Twitter Bots Using Machine Learning 191

Fig. 5. ROC curve for between TPR and FPR

is only one dependent variable that is, whether the twitter account is bot or not,
which is dependent on many other parameters like twitter account name, when it
was created, account’s status, number of followers etc. which are all independent
of each other.
Proposed Algorithm
Step 1: Dataset from Kaggle is taken
Step 2: From sklearn.linear model package import LogisticRegression
Step 3: Data from CSV file is split into 80:20 ratio for training and testing
Step 4: Model is trained using features from csv file.
Step 5: Then the trained model is tested on the 20% of data.
Step 6: Then score is calculated based on test result which tells the accuracy
of model.
The data is divided in ratios of 80:20 in order to get optimal result. If the
dataset is divided in 60:40 ratio, then accuracy was coming to be 80.05%. There-
fore, Eighty percent of the data is used for training the model and twenty percent
of data for testing the accuracy of the model. As per the dataset, the model is able
to achieve 93% (approx.) of accuracy. It is a static model just like WARNING-
BIRD [13]. It works on the predefined dataset only. Figure 6 shows the complete
flow of the proposed system and how all modules shall work together.
Figures 7 and 8 shows the details about Module 2 and Module 3.
The proposed system can help in identifying twitter bots which are posting
malicious URLs and report them to twitter. Also it can help in reducing APT
attacks, phishing attacks etc. as the entry point for all these attacks is usually
the malicious URL circulated either through social media or emails. Once if the
source of these malicious URLs are detected then these attacks can be prevented.
192 T. Satija and N. Kar

Fig. 6. Flow of proposed system

Fig. 7. Module 2 of proposed system

Fig. 8. Module 3 of proposed system

4 Conclusion
In present day scenario, twitter bots can be very dangerous if they are broad-
casting malicious URLs. They can very easily reach out to many people and can
lead to many data breaches and phishing scams. So in this paper a model is
proposed, which by using machine learning algorithms can successfully identify
twitter bots which are posting malicious URLs. The proposed model has been
tested and so far it is achieving good accuracy. It is able to distinguish between
twitter bots and legitimate human accounts. Also, the model is able to detect
 Detecting Malicious Twitter Bots Using Machine Learning 193

malicious URLs. The proposed system can be used to prevent attacks where the
entry point is malicious URL. There is only one limitation. Proposed model is
trained to work with long URLs for now. But now a days these social networking
sites are using URL shortening facility which can reduce the URL to minimum
size possible. So in future, another module will be added to this system which
will be able to work on short URLS and detect whether they are malicious or
not.

References
1. https://en.wikipedia.org/wiki/Twitter
2. https://medium.com/the-startup-growth/how-many-people-are-on-twitter-
662d797d5958
3. Mønsted, B., Sapieżyński, P., Ferrara, E., Lehmann, S.: Evidence of complex con-
tagion of information in social media: an experiment using Twitter bots. PLoS
ONE 12(9), e0184148 (2017)
4. https://beebom.com/best-twitter-bots/
5. https://www.welivesecurity.com/2018/06/28/twitter-bots-disassemble
6. https://breachlevelindex.com/
7. Data from Twitter blog. https://blog.twitter.com/
8. Chang, S., Cohen, T., Ostdiek, B.: What is the machine learning? Phys. Rev. D
97(5), 056009 (2018)
9. Marx, V.: Machine learning, practically speaking. Nat. Meth. 16(6), 463 (2019)
10. Kan, H.J., Kharrazi, H., Chang, H.-Y., Bodycombe, D., Lemke, K., Weiner, J.P.:
Exploring the use of machine learning for risk adjustment: a comparison of standard
and penalized linear regression models in predicting health care costs in older
adults. PLoS ONE 14(3), e0213258 (2019)
11. Kuha, J., Mills, C.: On group comparisons with logistic regression models. Sociol.
Meth. Res. 47(1), 0049124117747306 (2018)
12. Kainkaryam, S., Ong, C., Sen, S., Sharma, A.: Crowdsourcing salt model building:
Kaggle-TGS salt identification challenge. In: 81st EAGE Conference and Exhibi-
tion 2019 (2019)
13. Lee, S., Kim, J.: WarningBird: detecting suspicious URLs in Twitter stream. NDSS
12, 1–13 (2012)
14. Chaudhary, M., Hingoliwala, H.A.: Warning Tweet: a detection system for suspi-
cious URLs in Twitter stream. Int. J. Res. Appl. Sci. Eng. Technol. (IJRASET) 2,
297–305 (2014). ISSN: 2321–9653
15. Alshboul, Y., Nepali, R., Wang, Y.: Detecting malicious short URLs on Twitter
(2015)
16. Sahoo, D., Liu, C., Hoi, S.C.H.: Malicious URL detection using machine learning:
a survey. arXiv preprint arXiv:1701.07179 (2017)
17. Chavoshi, N., Hamooni, H., Mueen, A.: DeBot: Twitter bot detection via warped
correlation. In: ICDM, pp. 817–822 (2016)
18. Novotny, J.: Twitter bot detection & categorization-a comparative study of
machine learning methods (2019)
19. Davis, C.A., Varol, O., Ferrara, E., Flammini, A., Menczer, F.: BotOrNot: a sys-
tem to evaluate social bots. In: Proceedings of the 25th International Conference
Companion on World Wide Web, pp. 273–274. International World Wide Web
Conferences Steering Committee (2016)
194 T. Satija and N. Kar

20. Chen, Z., Tanash, R.S., Stoll, R., Subramanian, D.: Hunting malicious bots on
Twitter: an unsupervised approach. In: Ciampaglia, G.L., Mashhadi, A., Yasseri,
T. (eds.) SocInfo 2017. LNCS, vol. 10540, pp. 501–510. Springer, Cham (2017).
https://doi.org/10.1007/978-3-319-67256-4 40
21. Devadoss, A.K.V., Thirulokachander, V.R., Devadoss, A.K.V.: Efficient daily news
platform generation using natural language processing. Int. J. Inf. Technol. 11(2),
295–311 (2019)
22. Chen, B.-C., Davis, L.S.: Deep representation learning for metadata verification.
In: 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW),
pp. 73–82. IEEE (2019)
23. https://medium.com/dfrlab/botspot-twelve-ways-to-spot-a-bot-aedc7d9c110c
24. Zeng, J., Liu, M., Xiang, F., Ruiyu, G., Leng, L.: Curvature bag of words model
for shape recognition. IEEE Access 7, 57163–57171 (2019)
25. https://en.wikipedia.org/wiki/Receiver operating characteristic
 Content Based Routing Algorithm
to Improve QoS in IoMT Networks

Shailja Agnihotri1(B) and K. R. Ramkumar2

1
Department of Information Technology, GGDSD College, Chandigarh, India
[email protected]
2
Chitkara University, Punjab, India
[email protected]

Abstract. The IoT combines the MANETs and WSNs, as it includes

the nodes which are stationary as well as dynamic in nature. The nodes
in IoT network may comprise of vehicles, smart devices, sensors, net-
works, other smart objects etc. This intercommunication between the
things also brings various routing challenges which may be posed during
the transmission of data. The data to be transmitted may not always be
scalar as in WSN but may be audio, video, image, text or multimedia as
in WMSN (Wireless Multimedia Sensor Network). The successor of WSN
is WMSN and the transmission of multimedia content in IoT leads to
the emergence of IoMT (Internet of Multimedia Things) networks. QoS is
extremely important during the transmission and the quality of the path
is calculated on the basis of noise on the channel. The congestion has
always been considered as the main cause of the packet loss and delay in
the transmission. The paper proposes the algorithm, CARA-IoT which
finds the reliable intelligent path which is also content aware. The path
is established using ACO (AntColony Optimization) approach, content
is checked and suitability of the routing path is confirmed. And finally,
the best intelligent reliable routing path is chosen, firstly based on the
noise present on the channel and secondly on the basis of the content to
be communicated.

Keywords: WMSN · IoMT · IoT · QoS · SNR · Content aware

1 Introduction
1.1 Internet of Things (IoT)
Smart objects need to communicate themselves forming IoT networks. But as
the devices are low powered and low energy in these networks making the process
of communication challenging. The routing issues are very important to be con-
sidered [1]. In this paper we have proposed an algorithm to choose the reliable
path for the specific type of content such as text, audio, video, image and multi-
media. The reliable path is chosen by taking ACO principle as the basis. ACO is
based on the swarm intelligence principle. It is mostly used to find the solutions
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 195–206, 2020.
https://doi.org/10.1007/978-981-15-3666-3_17
196 S. Agnihotri and K. R. Ramkumar

for the optimization problems. The value of pheromone tells the goodness of the
routing path. The shortest path is chosen on the basis of pheromone using the
Forward Ants and the Backward Ant. Based on this concept, the artificial ants
are used in the computer solvable problems.

1.2 Wireless Multimedia Sensor Network (WMSN)

There is also need to retrieve the different kinds of data such as audio, video,
multimedia apart from only scalar data. WMSNs are the networks which are
capable of sending and receiving multimedia content. These networks do not
only improve the existing sensor networks but also extend the functionality of
existing sensor networks. The presently available standard algorithms will not
be able to suffice the needs of WMSNs, the QoS [2] based efficient and reliable
algorithms are required [3].

1.3 Internet of Multimedia Things (IoMT)

For the pervasive and ubiquitous communication in the IoT networks, the com-
munication for the multimedia content is also required. In IoMT networks, the
various smart heterogeneous nodes can interact with each other using multime-
dia data. These networks have various stages like multimedia sensing, reporting
and addressability, multimedia aware cloud and multi agent systems. These net-
works are the subset of IoT networks. The architecture is not same as that of IoT
networks. As the IoMT networks contain WMSNs, the QoS is highly required to
be maintained.

2 Literature Survey
Palaniappan and Chellan [4] tell about the energy efficiency maintenance in
MANETs. In these types of networks, there is always dynamic topology and
are self-constructive. The ad-hoc networks have to face new challenges such
as real time communication, resource constraints, bandwidth management and
packet broadcast overhead. Due to these issues, the researchers have always
found difficult to design efficient and robust routing algorithms. The paper also
mentions that the link strength is very important and should be considered.
The authors have used the cross-layer approach and considered the SINR for
the routing layer. The best metrics for reliability of a wireless link are SNR
and BER (Bit Error Ratio), but these cannot be used for every packet received.
Therefore, another metric, LPER (Link Packet Error Rate) is used for the QoS
in MANETS.
Cevik et al. [5] mention the growth of WSN to the WMSN, which are able to
transfer the multimedia content instead of only scalar data. During the transmis-
sion, only scalar data is not sufficient to be processed and transmitted such as
humidity, temperature etc. The paper also mentions the different WMSN archi-
tectures such as Single tier, Single-tier clustered and Multi-tier. The QoS needs
 Content Based Routing Algorithm to Improve QoS in IoMT Networks 197

to be maintained while transmitting multimedia data such as packet latency have

to be considered and also the end-to-end delay. To improve the communication
performance, the quality metrics - reliability, delay etc. need to be considered.
The various challenges for WMSN are also mentioned. The authors express that
the standard protocols mainly focus on the congestion but not noise.
Singh and Verma [6] state that there are special ad-hoc networks for real time
applications. Along with the physical quantities, the real sense can be transmit-
ted to the sink or destination. The paper focuses on image transfer and mentions
some key points. The image transfer is a challenging task. The major drawback
here is the limited energy of the nodes and should not be affected for the perfor-
mance. The QoS should be maintained in multimedia communication networks.
Therefore, there is need for the routing protocols which ensures the robust, reli-
able and efficient transmission of data and also ensures limited delay and latency
requirement. In this paper, various QoS parameters have been analyzed and the
algorithm is proposed to deal with the issues in multimedia networks.
Kandris et al. [7] mention that the WSNs have large variety of multimedia
applications. The requirements of these applications are to maximize the QoS
and minimizing the energy requirements specifically for the video applications.
The paper proposes the scheme for the efficient video communication which
achieves both the requirements.
Hasan et al. in [8] mention that the WSMNs are useful in making real time
applications but QoS poses several routing challenges. The paper presents the
growth of various routing protocols for using multimedia content in the networks
and also highlights the need of other multipath routing protocols for the WMSNs.
Han et al. in the paper [9] state the requirement of different communication
standards due to the different bandwidth capabilities. The paper deals with the
heterogeneity in bandwidths and the proposes the Time Reversal (TR) app-
roach. It offers the energy efficiency of Terminal Devices and Access Points. The
paper states that there are varying requirements for different types of device
in the communication such as QoS, hardware capability, bandwidth etc. The
middleware approach has been proposed in this paper.
AlmalKawi et al. [10] has discussed the WMSNs in detail. The authors have
mentioned the trends and the future directions in the wireless multimedia sensor
networks area. The paper has the detailed survey of the various architectures,
protocols and algorithms for the various layers of WMSNs. The swarm nodes are
stated as better than the other sensor nodes in terms of networking and function-
ality. The standard TCP protocol cannot be used in the transport layer of WSNs
or WMSNs as there is connection establishment overhead and also TCP mainly
focuses on congestion which alone cannot be proved as good quality metric for
the connection establishment. The authors mention that the retransmissions are
on hop-by-hop basis in WMSNs.
Several approaches have been discussed but all these have several disadvan-
tages like these cannot be applied if the network area is large, more importance
has been given to the energy saving, less importance has been given to PDR and
packet prioritization has not been included. Proper utilization of the routing
198 S. Agnihotri and K. R. Ramkumar

protocols needs to be done to improve the QoS and the network performance.
The metrics such as data rate, end-to-end delay play a vital role for reliable data
transmission in WMSNs. In multimedia data transmission, not 100% reliability
is required, still we would need maximum reliability we can achieve. This paper
proposes a QoS based routing algorithm which is based on ACO to enhance the
reliability in case of the WMSN networks. The route is found out on the basis
of SNR and the predictive analysis of the route is done before transmission of
data.

3 QoS- Quality of Service

The reliability offered by WSNs can be improved further, if the packed drop
is decreased. This in turn increases the QoS [11]. Several studies have been
showed for the real time data transmission in WMSNs. The energy of the nodes,
bandwidth and other channel constraints make the transmission process more
challenging. There is a need to develop the solutions which increase the overall
network performance. Several cross-layered approaches have also been discussed
in literature. But the disadvantages are high number of packets drops and error
leading to low quality communication in the network. Measures need to be taken
to deliver the multimedia data without the packet loss as the packet loss will
affect the assembling of data at the receiver’s end. And if the data is in the
compressed form in the packet, the packet loss will lead to entire data loss. The
delays in the network or the packet drops significantly reduces the QoS.
The present routing algorithms are proving better results for the scalar data
provided by WSNs but the same has still some challenges for the WMSNs mul-
timedia content. The QoS has become a real challenge for such networks. There
are various factors which affect the QoS such as delay, packet loss, jitter, conges-
tion, SNR etc. The advancement in the technology field has led to the increase
in the multimedia traffic and also in the routing challenges.
Several context aware algorithms have been proposed for the IoT networks.
Along with these, the content aware routing algorithms are also required to deal
with the various routing challenges of IoMT networks. Various QoS metrics are

• PDR- Packet Delivery Ratio

• End-to End Delay
• SNR-Signal Noise Ratio
• Jitter
• Packet Loss Ratio

Where Packet Delivery Ratio is defined as the ratio of the number of packets
received by the receiver to the total number of packets sent by the sender.

P DR = N o.packetsreceived/N o.of packetssent (1)

The network performance is good if there is high PDR and vice versa. End-to-
End Delay is calculated as the time taken by the packet to reach from the source
 Content Based Routing Algorithm to Improve QoS in IoMT Networks 199

to the destination. The delay during the communication can be caused due to
several reasons, may be through interference, queuing, packet conversions etc.
It is actually the summation of transmission delay, queuing delay, propagation
delay and processing delay. It is the time difference between the sending time
and the receiving time of the respective packet.
The network performance is good if the end-to end delay is low and vice versa.
Types of content during Transmission: The payload to be transferred from the
source to the destination can be of any form such as audio, video, image, text
or it may be multimedia. And, also the audio or video data may not be in the
same packet.

• Audio
• Text
• Video
• Images
• Multimedia

The paper [12] mentions the delay limits for the different types of data to be
communicated on the network. The delay which can be considered for different
data is:

Table 1. Acceptable delay limit for different type of content on wireless channel

Type of data Limit

Textual data 2–5 s
Image data 155 ms
Audio data Less than 600 ms
Video data Less than 250 ms

4 Major Routing Issue

The SNR is defined as Signal to Noise Ratio, where low SNR means low signal
strength and high SNR means high signal strength. The higher the ratio in SNR,
it is easier to get the information communicated on the channel. It clearly shows
that the high SNR is always preferable. During the communication, various
noise factors exist. Even the signal transmitted can be noise to other entities
during communication. This term noise is taken from Radio Engineering. The
transmission demands the pure signal but pure signal means the signal which is
noise free and this pure signal never exists, whatever communicated is affected
by the noise. The efforts should be made in order to keep the noise factor low
to get the reliable results. The calculation for the SNR is either the difference
of the two logarithms or the logarithm ratio of the main and noise signals.
200 S. Agnihotri and K. R. Ramkumar

The signal can be treated as the information which is required and of interest
value whereas the noise is treated as the error which effects the accuracy and loss
of precision can be there. From the source end when the message is transmitted
to the receiver, the message should be optimized and interpreted clearly as much
as possible. SNR not only affects the message to be communicated, it also affects
the throughput in the communication.

SN R = P signal/P noise = (V signal/V noise)2 (2)

BER can be calculated from the given SNR. The formula to convert SNR to
BER is as follows:
BER = 1/2(SN R) (3)
The various factors are considered and the paper proposes algorithm CARA-
IoT, which is content aware. This algorithm takes ACO as the basic principle
and proposes the algorithms for Forward Ant-FANT, Backward Ant-BANT and
the packet delivery algorithm to deliver the payload on the channel.

4.1 FANT Algorithm

Step 1: Select neighbor node which is within the transmission range of current
node
Step 2: If no path information available initially, broadcast FANTs to the neigh-
bors of the current node
Else unicast the FANT to the best neighbor selected based on best path meric
such as goodness of the path using ACO [13] rule
Step 3: Increment hop count value
Step 4: If conversion of communication technology at the selected neighbor, set
the conversion value=1 otherwise 0
Step 5: Check for Noise in the channel. Calculate Bit Error Ratio (BER) and
update its RT (Routing Table) with the following entries
FANT ID Source Destination Prev Node Conv Y/N Node Type HCount
Next node Next Node Type Noise Y/N
Step 6. Choose the neighbor based having lesser number of packet drops
Step 7: Check RT entry if next node is the destination. If Yes, move FANT to
the next node, repeat step 4 and 5and GOTO step 8
Else, move FANT to the next node and repeat step 4,5 and 6
Step 8: Stop

4.2 BANT Algorithm

Step 1: Convert the first FANT to reach the destination as BANT
Step 2: Check RT entry of FANT visited for conv, hop count and distance values
Step 3: If conv=1, calculate convTime += conv*6.37 [14]
If Noise = ‘y’ , calculate delay on the channel
Step 4: Update the RT entry of neighbor for the time taken to reach the
destination
 Content Based Routing Algorithm to Improve QoS in IoMT Networks 201

Step 5: Visit the next neighbor stored in the RT of the current node
Step 6: If the next node is the source and update the pheromone value (consider
distance, hop count and conversion time and noise on the channel) at source RT
and GOTO step 5
Else, repeat step 2 and apply 70%-30% ACO rule for 70% of the current
pheromone value and the 30% of the previously calculated pheromone value
Step 7: Update the RT values
BANT ID Source Destination Pheromone value
Step 8: Stop pheromone calculation process

4.3 Packet Delivery Algorithm

Step 1: Start packet delivery process
Step 2: Check source node RT for the pheromone values and the PoGoM values
of the neighbor nodes
Step 3: Calculate PoGoM values using: PoG0M(A)=Ph(A)/Ph(b)+Ph(C)+
Ph(D)
Step 4: Check the delay value in all the PoGoMs for the type of packet to be
transmitted as given in the Table 1
Step 5: Start packet delivery through the best selected path having highest
PoGoM with the lowest delay value
Step 6: Send all the packets using the chosen path till 100% delivery including
retransmissions
Step 7: Stop packet delivery process

5 Simulation
The simulator has been developed in C# - IoTRSim. The CARA-IoT algorithm
(Content Awareness Routing Algorithm for IoT Networks) is the enhanced ver-
sion of ACO algorithm which finds the best reliable path not only based on
distance but the conversions made from one communication technology to other
communication technology and most importantly on the basis of delay limits for
the particular type of payload and the speed offered by the channel. The settings
for the IoTRSim are mentioned in Table 2.

5.1 Results and Analysis

The simulator implements the proposed algorithm CARA-IoT. The results
clearly show the improvement of CARA-IoT as compared to ACO based algo-
rithm and SPEED algorithm. The algorithm SPEED is considered as very effi-
cient for the sensor networks and also works on delays and QoS [15]. In this paper,
the parameters taken are link speed for setting the speed for different technolo-
gies, Full range is the maximum range providing the high signal strength, %
Drop is the total signal dropped for every n meter distance. Every node has the
delay factor, which is considered in calculating the best routing path. The route
202 S. Agnihotri and K. R. Ramkumar

Table 2. Simulation environment settings

Metric Setting
Simulation area 500 m × 500 m
No. of nodes 10–50
No. of ants 10–50
No. of iterations 10–20
Zigbee speed 75 mbps (full range is assumed to be 75 m) with
percentage drop of 25% for every 15 m
Bluetooth speed 20 mbps (full range is assumed to be 10 m) with
percentage drop 10% for every 5 m
Wi-fi speed 50 mbps (full range is assumed to be 50 m) with
percentage drop of 15% for every 10 m
Delay limit As mentioned in Table 1
Payload size Content Based-Text/Audio/Video/Image/Multimedia
Packet transmission size 512 KB

is found out considering the delay on every node and the speed. It may take more
time and as the congested nodes are not taken, the route is most reliable one.
The following graphs show the simulation results and compare the CARA-IoT,
ACO and SPEED algorithms.

Fig. 1. CARA-IoT vs ACO vs SPEED based on distance.

 Content Based Routing Algorithm to Improve QoS in IoMT Networks 203

The Fig. 1 shows the distance covered. The proposed algorithm CARA-IoT,
is covering longer distance in the routing path as compared to the ACO based
path and the SPEED. As the number of nodes in the simulation is getting higher,
so as the distance.

Fig. 2. CARA-IoT vs ACO vs SPEED based on time taken.

The Fig. 2 shows the comparison on the basis of time taken. The CARA-
IoT is taking less time as compared to ACO based approach and the SPEED
algorithm. The proposed algorithm proves to be more efficient as even if the
distance covered was high, still CARA-IoT has chosen the path which is taking
less time.
The packet delivery ratio (Fig. 3) has always been maximum in case of pro-
posed algorithm, CARA-IoT. Whereas in case of ACO approach, the PDR has
been varying. The CARA-IoT is having retransmissions to successfully deliver
the payload which makes the PDR as maximum as compared to ACO and
SPEED.
CARA-IoT considers the speed as well as the BER1 on the channel for cal-
culating the best possible path to deliver the different type of payload (that may
not be scalar data). Here, in the Fig. 4 shows that the average speed offered by
the CARA-IoT is higher as compared to ACO approach and SPEED algorithm.
204 S. Agnihotri and K. R. Ramkumar

Fig. 3. CARA-IoT vs ACO vs SPEED based on PDR.

Fig. 4. CARA-IoT vs ACO vs SPEED based on average speed.

6 Conclusion

One major problem of Internet is the lack of QoS, where there is always packet
loss, retransmissions and delays. So, reaching to the destinations becomes dif-
cult. The data packets may reach out of time, late, out of sequence or the packets
may not reach to the destination. To reassemble these packets again at the
destination is a major problem. In this paper, the routing path is calculated
based on the algorithm, which in turn finds the reliable, robust, efficient and
intelligent path which considers the type of content for the respective type of
 Content Based Routing Algorithm to Improve QoS in IoMT Networks 205

data to be transferred. The results clearly show that there is less delay and
high packet delivery ratio, when the proposed algorithm is applied. There is
certainly improvement in the QoS. The multimedia networks such as WMSNs
and IoMT has several routing challenges. There is need for the development of
such intelligent and efficient routing protocols to match the speed of the growth
of IoT devices and IoT networks. The standard algorithms need to be modified.
The protocol stack and the architecture should be modified to accommodate the
IoT and IoMT networks. Further work can be done by the researchers to resolve
the routing issues mentioned in the paper.

References
1. Abazeed, M., Faisal, N., Zubair, S., Ali, A.: Routing protocols for wireless multi-
media sensor network: a survey. J. Sens. 2013, 1–11 (2013)
2. Kalaivani, P., Sathya, G., Senthilnathan, N.: QoS parameters estimation in
MANET using position based opportunistic routing protocol. Am. J. Comput.
Sci. Eng. Surv. 3(1), 21–27 (2015)
3. Zwyssig, E., Renals, S., Lincoln, M.: On the effect of SNR and superdirective beam-
forming in speaker diarisation in meetings. In: 2012 IEEE International Conference
on Acoustics, Speech and Signal Processing (ICASSP), pp. 4177–4180 (2012)
4. Palaniappan, S., Chellan, K.: Energy-efficient stable routing using QoS moni-
toringagents in MANET. EURASIP J. Wireless Commun. Netw. 1(3), 13 (2015)
5. Cevik, T., Gunagwera, A., Cevik, N.: A survey of multimedia streaming in wireless
sensor networks: progress, issues and design challenges. Int. J. Comput. Netw.
Commun. 7, 95–114 (2015)
6. Singh, R., Verma, A.K.: Efficient image transfer over WSN using cross layer archi-
tecture. Optik 130, 494 (2017)
7. Kandris, D., Tsagkaropoulos, M., Politis, I., Tzes, A., Kotsopoulos, S.: Energy
efficient and perceived QoS aware video routing over wireless multimedia sensor
networks. Ad Hoc Netw. 9(4), 591–607 (2011)
8. Hasan, M.Z., Al-Rizzo, H., Al-Turjman, F.: A survey on multipath routing pro-
tocols for QoS assurances in real-time wireless multimedia sensor networks. IEEE
Commun. Surv. Tutorials 19(3), 1424–1456 (2017)
9. Han, Y., Chen, Y., Wang, B., Liu, K.R.: Enabling heterogeneous connectivity in
Internet of Things: a time-reversal approach. IEEE Internet of Things J. 3(6),
1036–1047 (2016)
10. Almalkawi, I.T., Guerrero Zapata, M., Al-Karaki, J.N., Morillo-Pozo, J.: Wireless
multimedia sensor networks: current trends and future directions. Sensors 10(7),
6662–6717 (2010)
11. Usman, M., Yang, N., Jan, M.A., He, X., Xu, M., Lam, K.-M.: A joint framework
for QoS and QoE for video transmission over wireless multimedia sensor networks.
IEEE Trans. Mob. Comput. 8, 746–759 (2017)
12. Chen, Y., Farley, T.R., Ye, N.: QoS requirements of network applications on the
internet. Inf. Knowl. Syst. Manage. 4, 55–76 (2004)
13. Agnihotri, S., Ramkumar, K.R.: A survey and comparative analysis of the various
routing protocols of Internet of Things. Int. J. Pervasive Comput. Commun. 13(3),
264–281 (2017)
206 S. Agnihotri and K. R. Ramkumar

14. Agnihotri, S., Ramkumar, K.R.: An enhanced routing algorithm for Internet of
Things that resolves heterogeneity issues. In: Proceedings of the 6th International
Conference, BVICAM, INDIACom 2019, New Delhi, pp. 1274–1279. IEEE (2019)
15. He, T., John, A., Chenyang, L., Abdelzaher, T.: SPEED: A Real-Time Routing
Protocol for Sensor Networks. Virginia University Charlottesville Department of
Computer Science, Charlottesville (2002)
 An Improved Chaos Based Medical Image
Encryption Using DNA
Encoding Techniques

Abhrajyoti Das(B) , Abhilash Kumar Das, and Nirmalya Kar

Department of Computer Science and Engineering, NIT Agartala,

Jirania 799046, India
[email protected], [email protected], [email protected]

Abstract. Medical Image protection is one of the primary concerns of

health care services. There has been a lot of research in this field in
the past decade. But unfortunately, in India, the implementation is very
poor and unstructured. In this paper, a secured model for MRI image
encryption and decryption using DNA Encoding and Logistic Chaotic
Map is proposed. Initially a key is generated using logistic map function
to generate a sequence K1,i this sequence is multiplied with a factor of
255, another sequence K2,i generated from Linear Feedback Right Shift
Register (LFRSR) using an initial seed value, then both the sequences are
XORed to form the key. After the key is generated the sequence is again
XORed with the 8-bit image pixel of the MRI Image. The generated
cipher is then converted into a DNA sequence using DNA Encoding.
The generated text of DNA sequence is then passed to the network. This
sequence is extremely secure and robust, the Root Mean Square Error
(RMSE) value measured between the original and encrypted sequence is
very high.

Keywords: Logistic map function · Linear Feedback Right Shift

Register (LFRSR) · DNA Encoding · Pseudo random sequence ·
Digital Imaging and Communications in Medicine (DICOM)

1 Introduction

In the digital era exchange of data in the form of images, videos and text are
very prevalent. Medical Image is one of the most prized possession. Leak of
medical data can lead to misuse of insurance benefits, wrong treatments, and
criminal intent to high profile personalities. In the recent years, medical image
security has been the prime concern of the health care sectors primarily due to
partnership with insurance agencies who provide medical benefits to patients.
Many Encryption and Decryption techniques have been proposed for securing
medical images from unauthorized access by changing the medical images into
some other form. Medical Image security should be of primary concern, but it is

c Springer Nature Singapore Pte Ltd. 2020

A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 207–220, 2020.
https://doi.org/10.1007/978-981-15-3666-3_18
208 A. Das et al.

neglected. The primary characteristics of information security like Confidential-

ity, Integrity, and Availability are also applicable to medical image encryption
and decryption. Since medical images are intended for a limited audience, there-
fore confidentiality needs to be preserved. Medical images are proprietary to
an individual hence tampering with them is not acceptable. Moreover, medi-
cal image access is very critical as it is required in emergency scenarios, hence
availability is essential.
The main purpose of medical image encryption is to securely transmit the
medical image from the sender to the receiver. A breach in the security of medi-
cal images will jeopardize the privacy of the patient. Moreover, if an intruder can
evade the implemented security and use unauthorized information, this may lead
to lawsuits and financial losses of both the patient and the insurance company.
In [1] Rohith has presented a method of image encryption and decryption using
chaotic key sequence using logistic map the author has taken a pseudo-random
sequence and have generated the logistic map for key generation. Ravichandran
in [2] has introduced few DNA encoding rules and few DNA Addition Rule and
Subtraction Rules to create confusion. In this paper different DNA based tech-
niques are applied on the Region Of Interest (ROI) of the image and later the
planes have been merged to create an encrypted Digital Imaging and Communi-
cations in Medicine (DICOM) image. This paper [3] combines the DNA Encoding
and Chaos-based logistic map for medical image encryption and decryption, the
author used permutation function and the inverse to disorient the image struc-
ture and has introduced few DNA encoding rules which helps in conversion of
bit sequences into DNA alphabets. In [4] the author has devised a method to
choose the optimal secret and public key of the system. It exposes the limitation
of a small keyspace and weak security of one-dimensional chaotic cryptosys-
tems. Prabhavathi [5] has introduced a Region Of Interest based Medical Image
Encryption using Multi Chaotic System, they used Lorentz Map and Logistic
Condition with different levels of diffusion. In this paper [6] the author uses DNA
cryptography to encrypt the medical records of the patient. This cipher is passed
to a verifiable secret algorithm which distributes the secret and cipher among
the participants and a check is done by combining the participant data to detect
tamper. This paper [7] uses Dual DNA cover for hiding data in a cover image,
it improves the existing Dual cover steganography by reducing noise bits in sec-
ondary cover. In this paper [8] the author has used scrambled logistic chaotic
encoding technique and DNA encoding of DICOM image for encryption, the
decrypted image is then watermarked to protect the integrity of the DICOM
image. In [9] the author presents a framework of dividing the DICOM image
into two sections one part is watermarked and other part is reversibly water-
marked and encrypted using AES, then both of them are combined to form
the encrypted watermarked image. This paper [10] presents a Dual DNA Addi-
tion and Encoding technique to produce a Partial Image Encryption (PIE), in
this a key is generated using a chaotic map and then passed through a Suduko
matrix then DNA Encoding is applied and XORed with the original image. The
author in [14] has used three types of chaotic sequences for scrambling of images.
 An Improved Chaos Based Medical Image Encryption Using DNA 209

They have used an unlimited one-dimensional folding map, two-dimensional

Henon map, and the one-dimensional logistic map.
The rest of the paper is segmented as follows: Sect. 2 describes the proposed
model, the encryption and decryption architecture. In Sect. 3 the experimental
results and analysis are highlighted. Section 4 concludes the scope and future
works for the proposed model.

2 Preliminaries and Proposed Model

In this section the outline of the proposed model for Medical Image encryp-
tion and decryption technique is discussed. The various methodology used to
structure the encryption and decryption model is shown here. The RGB Medi-
cal Image for Brain MRI is collected from the BraTS 2019 Challenge [17]. The
Image has been converted to a single integer pixel format and for encryption and
evaluation.

2.1 Chaotic Logistic Map

Chaotic Logistic Map [11] generates a pseudo-random sequence using some ini-
tial parameters, to generate values within the domain of 0–1. The function for
Chaotic Logistic Map is given by Eq. (1).

Xn+1 = rXn (1 − Xn ) (1)

The above equation represents r as the bifurcating parameter which determines

the randomness of the key. The value ranges between 0–4. The Initial value of X0
is done by using the random function of python library and set within the range
of 0–1. The equation is iterated based on the size of the image. On evaluation,
it was observed that the randomness was high when the value of r was in the
range of 3.5 to 3.99 which depends on the size of the image. Hence the value of r
is generated in random between the range of 3.5 to 3.99. The above Fig. 1 shows
us the random key generation space. The chaotic sequence that is generated is
then transformed into a unsigned integer value by multiplying with a factor of
255. The generated value is then rounded off to the nearest integer. The integer
sequence generated is then converted into a 8-bit sequence of binary matrix. The
Eq. (2) gives the us the Key sequence K1,i .

K1,i = Round(Xi ∗ 255) (2)

2.2 8-Bit LFRSR

A Linear Feedback Shift Register (LFSR) is a shift register with a feedback signal
[12]. A LFSR can be characterised using a Flip-Flop or can be visualized as a
Boolean function implemented using XOR function from the binary sequence.
A 8-stage Linear Feedback Right Shift Register (LFRSR) is characterised by a
210 A. Das et al.

Fig. 1. Logistic map function to calculate the value of Xn for various values of r with
a sequence of length equal to image size [18].

feedback of 8-degree polynomial, the sequence is generated by (28 −1), in this

case the polynomial is x7 +x5 +x3 +x. There are 255 possible states from the
polynomial for 8 degrees of x. This can be visualized by (28 − 1) = 255. Here
the odd parity checker has been used for odd position bits of the 8-bit word
and even position bit of odd parity in an alternate fashion. The 8-bits of the
binary sequence is taken and XORed to the bits at the odd position to create a
sequence and in the next iteration, the even position bits are taken respectively.
The resultant output is then placed at the 0th position or D0 bit before that a
right shift operation is performed then the generated bit is placed. The sequence
is generated to produce a sequence equal to the size of K1,i . The name of the
new sequence produced is K2,i . Figure 2 shows us the working for generating
this sequence. In Fig. 2 it can be seen that the bit are represented as D7 to
D0 . In the proposed algorithm XOR is taken of (D7 ,D5 ,D3 and D1 ) to produce

Fig. 2. 8-bit Linear feedback right shift register with Odd parity checker
 An Improved Chaos Based Medical Image Encryption Using DNA 211

K2,0 , K2,1 is produced by XORing (D6 ,D4 ,D2 and D0 ) subsequently and the
alternating pattern follows to increase randomness. K2,j = (Dj+6 ,Dj+4 ,Dj+2 and
Dj ) is general form of the evaluation for the Boolean function. After every right
shift the 0th position is affected.

2.3 DNA Encoding

DNA Encoding has been used in encryption mechanisms for quite a long time
now. DNA can be referred to as a molecule that is essential for the growth and
development of species. In the context of encryption we use the DNA nucleic acid
bases, A (Adenine), G (Guanine), C (Cytosine), and T (Thymine), to create con-
fusion in our encryption algorithm. These DNA bases follow the Watson-Crick
principle [13]. A and T are complementary and C and G are complementary.
DNA encryption can be used in various applications, [15] used DNA properties
to hide data in video frames, it show the strand being converted into binary for
evaluation. Table 1 shows us the DNA Encoding rules for our algorithm. We
apply these rule by initially choosing 2 bit per round from the cipher matrix
and the key sequence. From the DNA Encoding rules we choose the rules to be
applied in a circular fashion. Example Rule 1 is applied then Rule 2 to the next
two bits and so on.

Table 1. DNA Encoding rules

Rule
Base 1 2 3 4 5 6 7 8
A 00 00 01 01 10 10 11 11
T 11 11 10 10 01 01 00 00
C 01 10 00 11 00 11 01 10
G 10 01 11 00 11 00 10 01

2.4 Medical Image Encryption Algorithm

This section describes the Encryption procedure used to encrypt the Medical
Image. The Block Diagram for the Encryption Model is shown in Fig. 3. The
encryption model is further discussed in the following subsections:

Image Conversion
The algorithm segregates any medical image according to the type it belongs
to, some images had their pixel values loaded as tuples and some had unsigned
integer values for each pixel. Each image can be considered of the size M × N
this image is converted into binary sequences of 8-bit per block image matrix
Imat , where M and N are 1,2,3....n. The values of the image matrix range from
0–255.
212 A. Das et al.

Fig. 3. Block diagram for encryption model

Generate Keys
The generated key sequences K1,i and K2,i are converted into binary sequences
and then XORed to create the Key Ki for encrypting the MRI image.

Ki = K1,i ⊕ K2,i (3)

Generating Cipher Matrix

The generated key Ki is then XORed with the image matrix Imat . The produced
sequence is in binary form. Then each 8-bit binary sequence is converted into
unsigned integer values. These integer values can be considered as the pixel
values of the cipher image Ci .

Creating Cipher Image

The generated sequence of Cipher Matrix is then converted into Cipher Matrix
and this matrix is converted into Cipher Image. This step is done only for visu-
alization purposes to show how the random of pixels has generated a scrambled
Cipher Image.
 An Improved Chaos Based Medical Image Encryption Using DNA 213

DNA Sequencing
Here a different approach is taken. The binary sequences that were generated in
Step “Generating Cipher Matrix” are converted into a sequence of DNA strands
using DNA Encoding mechanism. The rule and mechanism have been discussed
in Sect. 2.3, this sequence has been saved into a text file. Figure 4 shows the DNA
sequence that is generated using the DNA encoding rules.

Fig. 4. DNA Sequence Generated.

2.5 Medical Image Decryption Algorithm

The Decryption mechanism for our proposed algorithm is discussed. The receiver
just receives a text file. The received key has been sent through a secure channel.
In this algorithm, one can send pixel by pixel in an ordered sequence to securely
transmit the key at the receiver end. Figure 5 shows us the Decryption Model in
our proposed algorithm.

Generate Binary Sequence from DNA Input

The received DNA text sequence is passed through the DNA encoding rule to
generate the binary sequence. The key is also converted to a DNA sequence and
is received from a secure channel.
214 A. Das et al.

Generating Image Matrix. The Cipher Image Matrix and the Key which is
received through a secure channel is then XORed to produce the Image Matrix.
This Image Matrix is in binary form.

Di = Ci ⊕ Ki (4)

Generating Decrypted Image

The Image Matrix Di ’ is now converted into integer value in order to get the
Image Idec . This reproduced image of M × N size is the Original Image after
Decryption. After Decryption Image Idec is found it is compared to the Original

Fig. 5. Block diagram for decryption model

Image Iorg . The Idec and Iorg does not have any visual difference as well as they
are of the same size. Hence it can be said that the transmission is lossless.

3 Result and Discussion

The simulation code for the model is written in python. The anaconda navigator
is used and the code is implemented in Spyder IDE. The algorithm is compatible
with any image of the size of M × N there is no restriction on the type or
size of the image. The algorithm automatically recognizes the image structure
and implements the algorithm accordingly. The proposed scheme compares the
results of other published papers with the proposed algorithm. The Histogram
Analysis of the Medical Images, Root Square Mean Value, The PSNR Value,
Correlation Coefficient value Analysis, etc. are computed to prove the efficiency
of the algorithm. The proposed approach uses the initial seed value of X0 as a
random value within the range of 0.1–0.5. The bifurcation value r is set in the
range of 3.5 to 3.99 and the LSRSR initial seed value is set as [1,0,1,0,1,0,1,0].
 An Improved Chaos Based Medical Image Encryption Using DNA 215

3.1 Attack Analysis

This section depicts what happens if an intruder uses some random key to deci-
pher the encrypted image. If the cipher is intercepted and the intruder say Trudy,
tries to decrypt the image using a similar algorithm as proposed with X0 as 0.2
and r = 3.99 and an L = [1,0,1,1,0,1,1,0] as initial seed and tries to generate the
key. The resultant will be as shown in Fig. 6(c). On the other hand when the
intended receiver who has received the key through a secure channel can decrypt
the cipher image to produce the image as shown in Fig. 6(d). The images Fig. 6
is of size 512 × 512.
Now a chest x-ray image of size 430 × 430 is taken Fig. 7(a) and analysis is
done and similar results are found as in case of Brain MRI Image.
It is observed that there is apparently no visual difference between the Orig-
inal Image and the Decrypted Image.

Fig. 6. (a) Original Image (b) Encrypted Cipher Image (c) Image Generated with
wrong Key by intruder (d) Decrypted Image generated by receiver using correct key Ki

Fig. 7. (a) Original Image (b) Encrypted Cipher Image (c) Image Generated with
wrong Key by intruder (d) Decrypted Image generated by receiver using correct key Ki

3.2 Histogram Analysis

A Histogram is used to plot the frequency of pixels in a gray level image. In

this paper, the histogram for the Original Image, Encrypted cipher image and
216 A. Das et al.

Decrypted Image has been plotted. The histogram for Fig. 7(a), (b) and (d) has
been plotted. It can be observed that the histogram plot for the Encrypted Image
is equalized than in case of original images. The Fig. 8(a) shows the chest Xray
image where the variation in the frequency of the pixels is very high, hence it
produces an irregular histogram formation. Whereas in Fig. 8(b) it is seen that
the cipher image is fairly equalized and the frequency of the pixels is uniform,
hence it can be said that the cipher has an even pixel density. In Fig. 8(c) it is
found that the decrypted image also has similar pixel frequency denoting that
the image is lossless. It is found that Fig. 6(a) produces similar results with a
high frequency of pixel 0 and with an even equalization of cipher image.

Fig. 8. (a) Histogram of original image (b) Histogram of encrypted cipher image (c)
Histogram of decrypted image

3.3 Root Mean Square Error Analysis

The Root Mean Square Error (RMSE) is the measure of how the pixel are spread
out in the image. This analysis considers the square root of difference between the
original image and the encrypted cipher image. This can be defined as follows:

 
 1
MxN −1
RM SE =  [F (i, j) − G(i, j)]2 (5)
M N i=0

Where F(i,j) is the original image and G(i,j) is the encrypted image, M × N is
the size of the image.
The higher the value of RMSE between the original image and the encrypted
image the higher is the randomness and the cipher image is more immune to
attacks. Now it is seen in Table 2 that the value for RMSE is much higher
for the proposed algorithm, compared to logistic map function only and other
standard algorithms.
It is observed that the RMSE values for both the Lena encrypted image
and the RMSE value for the Medical Image are higher than other proposed
algorithms. Please note that the medical image chosen in our case is different
than the one in [1]. Hence comparison values may differ in case of Medical Images.
 An Improved Chaos Based Medical Image Encryption Using DNA 217

Table 2. RMSE values of images

Image Logistic map Rohith et al. [1] Proposed algorithm

Lena image 84.2371 87.1292 87.8551
Medical image 87.3957 89.0932 127.7029

3.4 Peak Signal to Noise Ratio (PSNR)

PSNR in terms of image analysis can be explained as the ratio between the
maximum possible value (pixel) of a pixel and the mean value of distorting noise
that affects the quality of the image. This is always given in terms of a logarithmic
factor. Equation (6) gives us a mathematical representation to calculate PSNR.
 
R2
P SN R = 10log10 (6)
M SE
Where the MSE is the Mean Square Error and R gives us the maximum fluc-
tuation in the input image size. Here the PSNR values for Lena image and the
medical image Figs. 6 and 7 respectively are calculated. The values of the PSNR
to [1] and [3] are compared. The Table 3 depicts that the proposed algorithm
produces better results compared to the other models.

Table 3. PSNR values of images

Image Belazi et al. [3] Rohith et al. [1] Proposed algorithm

Lena image 32.8396 33.4099 36.3481
Medical image 30.0205 33.2163 33.9951

3.5 Entropy of Image

Entropy gives us the degree of unpredictability of information [16]. In the pro-
posed algorithm unpredictability of pixels occurrence in the encrypted image is
evaluated. Entropy of the image is higher when there is a uniform distribution
of pixels.
MxN −1
1
H(x) = P (Ci )log2 (7)
i=0
Ci
In a 8-bit image the maximum entropy achieved can by 8 bits/pixel value
this occurs when the histogram is flat. In the Table 4 the entropy of Lena
image Fig. 6(a) and medical image Fig. 7(a) and their cipher image entropy of
Figs. 6(b) and 7(b) respectively is shown. The comparison analysis is calculated
with [1] and [3].
From the Table 4 we find that our algorithm outperforms the existing tech-
niques for medical image encryption technique.
218 A. Das et al.

Table 4. Entropy of original and encrypted images

Image Entropy of Entropy of Entropy of Entropy of

original image Belazi et al. [3] Rohith et al. [1] proposed
algorithm
Lena image 7.3478 7.9993 7.9974 7.9983
Medical image 7.3454 7.9972 7.9970 7.9980

3.6 Correlation Analysis

Correlation Analysis is usually done by a researcher to determine the relationship
between two continuous numerical variables, arrays or images. The correlation
coefficient is determined using the Eq. (8) to establish a relationship between the
two images the original image and the cipher image. The coefficient is evaluated
by comparing the image pixel to pixel.
Cov(x, y)
CorrelationCoef f icient(ρ) = (8)
σx σy
Where Cov(x,y) is the covariance at (x,y) which can be given as shown in Eq. (9).
L
1
Cov(x, y) = [(xi − E(x))(yi − E(y))] (9)
L i=0

Here L is the total number of pixels of comparison, L = 512 × 512 and E(x) is
the mean value of x which is calculated as given in Eq. (11).

 
 1 L
σx =  (xi − E(x))2 (10)
L i=0

Equation (10) gives the step deviation at point x. Mean value of xi is given as,
L
1
E(x) = xi (11)
L i=0

Similarly, Step deviation at y and mean of y can also be evaluated. Table 5 shows
us the comparison between the Original image, Logistic Map Encrypted Image,
[1] and Proposed Algorithm.

Table 5. Correlation coefficient between original image and cipher image

Image Original image Logistic map Rohith et al. [1] Proposed

algorithm
Lena image 0.9970 −0.0133 0.0036 −0.00032
Medical image 0.9270 −0.0591 −0.0167 −0.00033
 An Improved Chaos Based Medical Image Encryption Using DNA 219

4 Conclusion
The proposed algorithm produced better results compared to many other stan-
dard algorithms used for medical image encryption and decryption. With high
RMSE value and promising results from histogram analysis the proposed method
can be implemented in health care industries. DNA based cipher text reduces
the cipher size without any loss in data and the efficiency of the algorithm is
very high. The proposed algorithm will provide a secure key exchange of med-
ical images within the hospitals, from hospitals to patients, from hospitals to
authorized insurance agencies, etc. This will lay the foundation for a better
Medical Image Encipherment process in the health-care industry, especially in
India where the infrastructure significantly needs to improve and implement such
systems. Because in the digital era data is of utmost importance hence privacy
in the medical system is the need of the hour. Further tree-based technique can
be used in the future to hide patient-related details in the Medical Image itself.
Data hiding of the report in the Medical image will increase the confidential-
ity of the patient’s data and also rather than sending two different files over
the network the system can consolidate information in a single document and
communicate securely.

References
1. Rohith, S., Hari Bhat, K.N., Nandini Sharma, A.: Image encryption and decryption
using chaotic key sequence generated by sequence of logistic map and sequence of
states of Linear Feedback Shift Register. In: 2014 International Conference on
Advances in Electronics Computers and Communications, pp. 1–6. IEEE (2014)
2. Ravichandran, D., Praveenkumar, P., Rayappan, J.B.B., Amirtharajan, R.: DNA
chaos blend to secure medical privacy. IEEE Trans. Nanobiosci. 16(8), 850–858
(2017)
3. Belazi, A., Talha, M., Kharbech, S., Xiang, W.: Novel medical image encryption
scheme based on chaos and DNA encoding. IEEE Access 7, 36667–36681 (2019)
4. Shankar, K., Elhoseny, M., Chelvi, E.D., Lakshmanaprabu, S.K., Wu, W.: An
efficient optimal key based chaos function for medical image security. IEEE Access
6, 77145–77154 (2018)
5. Prabhavathi, K., Sathisha, C.P., Ravikumar, K.M.: Region of interest based selec-
tive medical image encryption using multi Chaotic system. In: 2017 International
Conference on Electrical, Electronics, Communication, Computer, and Optimiza-
tion Techniques (ICEECCOT), pp. 1–5. IEEE (2017)
6. Kamble, P.R., Patil, S.: Enhancing security of electronic patient record with tamper
detection. In: 2018 3rd International Conference for Convergence in Technology
(I2CT), pp. 1–5. IEEE (2018)
7. Das, P., Deb, S., Kar, N., Bhattacharya, B.: An improved DNA based dual cover
steganography. Procedia Comput. Sci. 46, 604–611 (2015)
8. Bharghavi, G., Kumar, P.S., Geetha, K., Devi, N.S.: An implementation of SLICE
algorithm to enforce security for medical images using DNA approach. In: 2018
International Conference on Communication and Signal Processing (ICCSP), pp.
0984–0988. IEEE (2018)
220 A. Das et al.

9. Abdel-Nabi, H., Al-Haj, A.: Efficient joint encryption and data hiding algorithm
for medical images security. In: 2017 8th International Conference on Information
and Communication Systems (ICICS), pp. 147–152. IEEE (2017)
10. Parameshachari, B.D., Panduranga, H.T., Naveenkumar, S.K.: Partial encryption
of medical images by dual DNA addition using DNA encoding. In: 2017 Inter-
national Conference on Recent Innovations in Signal Processing and Embedded
Systems (RISE), pp. 310–314. IEEE (2017)
11. Yan-Bin, Z., Qun, D.: A new digital chaotic sequence generator based on logis-
tic map. In: 2011 Second International Conference on Innovations in Bio-inspired
Computing and Applications, pp. 175–178. IEEE (2011)
12. Deb, S., Bhuyan, B., Gupta, NCh.: Design and analysis of LFSR-based stream
cipher. In: Mandal, J.K., Saha, G., Kandar, D., Maji, A.K. (eds.) Proceedings of
the International Conference on Computing and Communication Systems. LNNS,
vol. 24, pp. 631–639. Springer, Singapore (2018). https://doi.org/10.1007/978-981-
10-6890-4 61
13. Cisse, I.I., Kim, H., Ha, T.: A rule of seven in Watson-Crick base-pairing of mis-
matched sequences. Nat. Struct. Mol. Biol. 19(6), 623 (2012)
14. Yu, H., Wu, G.-S.: The compounded chaotic sequence research in image encryption
algorithm. In: 2009 WRI Global Congress on Intelligent Systems, vol. 3, pp. 252–
256. IEEE (2009)
15. Kar, N., Mandal, K., Bhattacharya, B.: Improved chaos-based video steganography
using DNA alphabets. ICT Express 4(1), 6–13 (2018)
16. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3),
379–423 (1948)
17. Center for Biomedical Image Computing & Analytics (CBICA). http://
braintumorsegmentation.org/
18. Incompressible Dynamics. http://www.kierandkelly.com/what-is-chaos/logistic-
map
Internet of Things (IoT)
 Real Time Adaptive Street Lighting
System

Sahil Garg, Sahil Ahuja, and Sukhchandan Randhawa(B)

Computer Science and Engineering Department,

Thapar Institute of Engineering and Technology, Patiala, India
[email protected]

Abstract. With the aim to optimize the cost and performance of hard-
ware and software integration, this paper discusses and inculcates the
outcomes of various experiments that were conducted with different hard-
ware and software technologies available commercially. This Smart Street
light is designed with an aim of both performance and cost optimization,
and includes a built-in weather station, forest fire/general fire notification
and has the ability to continuously send data to the server for Analytics
and Data Acquisition, as well as to the Mobile Application server. The
data through the Mobile Application server can be fetched directly onto
an app with good performance and the data being sent onto the Ana-
lytics and Data Acquisition server can be used for analysis and further
research. Also, this project is sensitive to day and night and optimizes
energy when there is no motion or when the ambient light is higher than
a particular threshold determined experimentally. This paper opens the
need for a better IoT based Street Light project with additional features
and improved performance.

Keywords: Smart Street Lighting · Internet of Things · Gateway ·

Cloud computing · Machine Learning · Field devices · Sensor
technology

1 Introduction

The growth of Internet of Things (IoT) has brought about a revolution in our day
to day lives. Cheaper and improved sensor technology has provided an impetus
to this revolution of IoT. Each and everything can be connected to the internet
and synchronized to interact, collaborate and contribute. IoT has been responsi-
ble for machine-human inter-dependency. The major advantages that IoT offers
include improved efficiency, reduced interaction time and human effort. It also
provides support to Artificial Intelligence and Data Mining due to the involve-
ment of larger data-sets and the need for extracting useful information in order
to improve efficiency to the application and add ‘Smartness’ to it. IoT has a
great potential for Automation and Data research. The thing that we are con-
sidering in this paper is a Street Light. The main motivation of this research
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 223–239, 2020.
https://doi.org/10.1007/978-981-15-3666-3_19
224 S. Garg et al.

Fig. 1. Basic features of a Smart Street Lighting system illustrating various modules
involved in the system

work is derived from the fact that lighting accounts for more than 15% of elec-
tricity consumption of the world. This also contributes about 1.6 billion tons of
carbon emissions (per year) in the atmosphere. With the aim to cut-short the
energy wastage through the conventional street lighting systems and realizing
the growing demand of Automation, it is important to work in this area. For
making the Street Light smart, we need to define smartness. A ‘Thing’ is said to
be ‘Smart’ if it is able to sense its immediate surroundings and perform actions
that work towards a desired goal. If we fit the Smart Street Light to this def-
inition, we can describe the system as composed to things called Street Lights
equipped with sensors, illustrated in Fig. 1, to sense ambient light, motion, tem-
perature, humidity, faults and moving objects, acquire data through the sensors,
apply some pre-processing algorithms to compress data, and send it to the Cloud
via an IoT gateway. From the Cloud which consists of a better Infrastructure
and storage and Analytical capabilities, users can retrieve beneficial informa-
tion including visibility in foggy days, ambient light, temperature and humidity
in sunny days, traffic density, etc. and researchers can fetch useful data and
analyze those data for further research that includes temperature and humidity
characteristics of a particular location, season, etc.

2 Literature Survey
Many researchers have proposed a number of methodologies and ideas to reduce
the energy wastage of the street lights and also highlighted the ways to make
 Real Time Adaptive Street Lighting System 225

them smarter. Mullner et al. [9] has done an innovative literature review in the
field of saving energy with the help of electro-technology as well as Information
and Communication Technology (ICT) in Street Lights. Mahoor et al. [7] dis-
cussed about the state of smart street light which consists of monitoring and
executive programming. Lavric et al. [6] found that the information exchange
between lamp nodes and wireless sensor networks (WSN) can be a feasible since
it is composed of three parts: Control center, Remote concentrator and Street
lighting control terminals. Abhishek et al. [1] presented the idea of sending the
information of center station by ZigBee wireless communication. Gupta et al.
[4] proposed the solution for controlling the intensity of light with the help of
dimmer using PWM and Firing angles. With the use of dimmer we can conserve
15% to 20% of the energy. Escolar et al. [3] reviewed the street light approach
by using a simulator that combines wireless sensor network (WSN) and belief
desire intention (BDI) agents to provide information. It tried to exploit our app-
roach on more dynamic scenarios. Mohamed et al. [8] reviewed the state of smart
street light system based on the innovation technology named as Vehicular Ad-
Hoc Network (VANET). Smart Street Lighting system can be integrated with
VANET to reduce the cost and use the rich service and communication feature
of VANET. Pizzuti et al. [10] put forth an approach for adaptive street lighting
control based on energy demand idea and showed that the conventional street
lighting regulation system he tested showed that adaptive control provides on
an average almost double energy saving. Badgaiyan et al. [2] reviewed on using
ZigBee and wireless sensor network (WSN) – PIR (Proximity Infrared) sensor
and found that this was using low power and was highly efficient from con-
ventional street lighting system. A Smart Street Lighting (SSL) system [9] is
proposed for switching off the street lights based on the desired level of safety
and location of pedestrians. The proposed system consisted of Zigbee enabled
motes, a Base Station, a centralized SSL Server, server applications for system
configurations and Web-Application for monitoring and controlling the lighting
zones and lampposts and smartphones of pedestrians with GPS functionality.
A street lighting system based on ZigBee has also been proposed [7]. The
monitoring of the connected sensors for fault detection, power adjustment and
on/off control is done. The proposed system gathers the parameters of street
light via GPRS and Zigbee communication. It includes three main components:
a terminal for street monitoring, concentrator and monitoring center. The con-
centrator acts as a repeater and works as a bridge between monitoring center
and control terminals. Its main task is to forward command and data. The task
of the control terminal is to fetch the data and make the required adjustments
in the power.
An autonomous-distributed-controlled light system [11] was proposed in
which lights get turned on before a pedestrian actual arrives at that loca-
tion and then gets off after a pedestrian leaves a certain location. There are
access points which consist of the controller and the communication device. An
Automatic Intensity Varying Smart Street Lighting System [1] is presented in
which the initial step was to replace the existing lampposts by LED lights in
order to reduce the energy dissipation as LED lights are more energy efficient.
226 S. Garg et al.

Secondly, Light Dependent Resistor (LDR) has been deployed in order to control
the lights automatically without any human intervention. In order to control the
luminosity of the lights, dimmers are installed. A novel street lighting system [4]
is proposed which uses the renewable energy source i.e. solar energy. The lumi-
nosity of the street lights is controlled based on the intensity of sunlight which
is sensed by LDR. In case of low intensity of solar energy, the resistance value
of LDR is high and based on this value it is decided that whether the street
lights should be on/off. In case of moderate value of resistance, the dimming
function of lights is invoked. PIR sensor is also utilized to detect the motion
of the pedestrian. A novel fuzzy logic based controller [5] is presented whose
main objective is to minimize the wastage of power in street light systems. Solar
energy is utilized in order to save electric power. A fuzzy rule base is designed
which takes natural intensity and occurrence of vehicles as inputs based on the
rule set defined by fuzzy logic.
A hybrid street lighting system based on Vehicular Ad-Hoc Networks
(VANET) [8] and traditional street lighting system is proposed. In VANETs
Road-Side Units (RSUs) are deployed usually 300–400 m apart. The main task
of these RSUs is to report the data to the light controllers to switch on and
off a light. A reliable on-demand street lighting model [10] is proposed in which
the concept of traffic flow rate forecasting model is utilized. There are number
of methods based on 1-h prediction whose results are compared with the model
based on ANN which reports the best results. As per the above discussed litera-
ture, it is clearly concluded that most of the ideas are based on motion detection
of a person or a vehicle on a road.

Fig. 2. Architecture of the proposed solution depicting various units in the Smart
Street Light system
 Real Time Adaptive Street Lighting System 227

3 Proposed Methodology
Based on the previous works and current problems we defined the objectives of
our work. The broad objectives of this Smart Street Light Project are as follows:
– Design and development of low cost and energy efficient Smart Lighting Sys-
tem.
– Deployment of multiple sensors for motion detection, ambient light detection,
optical fog sensing, fault detection, fire detection etc.
– To control the intensity of light of deployed LED lights and their power
consumption based on weather conditions, diurnal timings, traffic levels and
motion detection.
– To record essential data and apply analytics to extract useful information
which further sends a feedback to the main network improving its performance
and adding “Smartness”.

4 Components of Smart Street Lighting System

The Smart Street Light as illustrated in Fig. 2 had broadly four units: (1) Field
devices which sense the environment and generate triggers and pulses and also
actuate based on the pulse received from the microcontroller (2) IoT gateway
which acted as a link between the microcontroller and the network (3) Cloud
which stored all the data received from the IoT network and performed Analytics
on that (4) End User which could fetch information from the cloud as well as
control the field actuators via the Cloud services. The Field devices that we
installed onto the Smart Street Light included a microwave based motion sensor,
Light Dependent Resistor for Ambient Light recording, LDR for Fault detection,
Flame sensor for fire detection, Digital Temperature and Humidity sensor to
record temperature and humidity readings at a particular location making it
a weather station as well. For surveillance and object recognition we have also
added a camera that is continuously streaming video clip to the control center
and also training the object recognition model using a CNN (Convolution Neural
Network) model capable of feature extraction and thus record the frequency of
various entities including vehicles and human beings. The entire Smart Street

Fig. 3. Comparison between open and closed system with GD as the transfer function
and C and r as output and input respectively. H is the feedback element and E represent
the error signal
228 S. Garg et al.

lighting system works using the concept of feedback in order to minimize error.
Illustrated in Fig. 3, Let GD denotes the transfer function of the measurement
system. Let C and r denote the output and input signal respectively. Also, let H
denote the feedback element in a closed loop system and e be the error or the
interference introduces in the system. Consider an open loop system as shown
in Fig. 3. We know that
C = GD .r (1)
Let the transfer function change from GD to GD + ΔGD which changes C to
C + ΔC therefore, in an open loop:

C + ΔC = (GD + ΔGD ).r (2)

Comparing LHS with RHS we get,

ΔC = ΔGD .r (3)

Now, consider the closed loop system with H as the feedback element and error
introduced is e,
C = GD .e (4)
e = r − CH (5)
C = GD .(r − CH) (6)
GD .r
C= (7)
(1 + GD H)
Let the transfer function change from GD to GD + ΔGD which changes C to
C + ΔC therefore, in a closed loop:

(GD + ΔGD ).r

C + ΔC = (8)
1 + (GD + ΔGD ).H

ΔGD .r
ΔC = (9)
1 + (GD + ΔGD ).H
If GD >> ΔGD so,
ΔGD .r
ΔC = (10)
1 + GD .H
It is clear that ΔC in closed loop is very less as compared to that in an open loop.
This method of high gain feedback therefore reduces error in a measurement sys-
tem with the help of feedback. The sensors are positioned in such a way so as to
reduce error from internal temperature and heat generation, hindrance in path of
light from ambience to the LDR, etc. All the field sensors are capable of working
at a voltage of 3.3 V DC or 5 V DC which is supplied from the voltage regulators
connected to the main DC power supply unit which is charged through Solar Panel
of rated 24 V output voltage as well as through the mains with 220 V 50 Hz stan-
dard power supply. The AC Dimmer is a device that is designed to control the
AC voltage, which can transfer current up to 220 V (5 A 10 A) (TRIAC BTA16
 Real Time Adaptive Street Lighting System 229

for 600 V/16 A). Discharge lamps do not support dimming. Power part of dimmer
is isolated from the control part, to exclude the possibility of high current disrup-
tion to a microcontroller. The logical level is tolerant to 5 V and 3.3 V, therefore
it can be connected to the microcontroller with 5 V and 3.3 V level logic. Dim-
ming can be achieved by Pulse Skip Modulation: Technique 1: One or more cycles
(sine wave signal) are transferred to the load while following one or several cycles
are blocked. Technique 2: Partial transference of each sine wave to the load. Tech-
nique 3: Generation of modulated full sine signal of different frequency up to few
hundred Hertz. This method requires specialized powerful AC generators with dif-
ferent modulation. Methods 1 and 2 are the easiest to execute with the help of a
Dimmer and program code: in both cases, there is a need of circuit that detects
the zero crossing and can control a TRIAC. Figure 4 shows the circuit diagram of
a basic dimmer circuit where the symbols have their usual meaning. The specifi-
cations of the dimmer used in the project are presented in Table 1. Alarm gener-
ation happens when the flame sensor that is based on YG1006 sensor which is a
high speed and high sensitivity NPN silicon photo-transistor detects a fire source
or other light source of wavelength in the range of 760 nm–1100 nm. Due to the
black epoxy coated on the diode, it is sensitive to Infrared Radiation. When the
sensor detects flame, the signal LED will light up and the digital pin goes low as
designed by the manufacturer using an internal Data Acquisition circuit. Table 2
describes the specifications of the flame sensor used in the street lights. Generic
procedure for Fault Detection:

– Set EXTLDR = External (isolated from lamp light) LDR reading

– Set INTLDR = Internal (Placed in Lamp box) LDR reading
– if EXTLDR is less than Dusk-light threshold and INTLDR is less than Dim-
light threshold then Generate fault notification;
– otherwise Read EXTLDR and INTLDR;

Table 1. Table depicting the specifications of the dimmer used in the project

Parameter Details
Power Up to 220 V (5 A 10 A)
AC frequency 50/60 Hz
TRIAC BTA16—600B
Isolation Optocoupler
Logic level 3.3 V/5 V
Zero point Logic level
Modulation (DIM/PSM) Logic level ON/OFF TRIAC
Signal current Greater than 10 mA
Environment For indoor and outdoor use Operating temperatures:
20 ◦ C to 80 ◦ C Operating humidity: Dry environment
only
ROHS3 Compliant
230 S. Garg et al.

Fig. 4. Circuit Diagram of a general purpose dimmer used to vary the intensity of AC
Street Light

Table 2. Table depicting the features of the Flame sensor used in the Smart Street
lights

Parameter Details
Working voltage 3.3 V–5 V
Detect range 60◦
Output type Digital/analog
On-board IC LM393 chip
Dimension 3.2 cm × 1.4 cm

Note : The value of Dusk-light threshold is generally taken as 50 on a scale

of [0–1023] while the value of Dimlight threshold is taken as 900 on a scale of
[0–1023]. The value of Dusk-light threshold is basically the value that the LDR
senses at the time of dusk i.e. when we want our Street light to glow while the
Dim-light threshold is the reading of the LDR when the street light is in Dim
state i.e. when it sees no motion. A gateway is a device that on behalf of the
local network interacts with other networks or the cloud. In our solution, the
gateway is inbuilt in one of the street lights of a local network of street lights
which we have labelled as ‘Master Street Light’ while other street lights that
are left are termed as ‘Slave Street Lights’. The master street light is the one
that communicates with the slave street lights using the ZigBee Protocol which
is power efficient for considerable ranges. There is a need for data pre-processing
at the gateway since we have only one node that interacts with other networks
on behalf of the entire local network. Data pre-processing at the gateway takes
place in the following parts:
– Receiving of data from the Slave Street Lights which includes temperature,
humidity, motion, ambient light intensity and fire/fault flags.
 Real Time Adaptive Street Lighting System 231

Table 3. Table highlighting the various characteristic parameters of various device to

device communication technologies

Wi-Fi IEEE Bluetooth IEEE Zigbee IEEE

802.11b 802.15.1 802.15.4
Radio Direct sequence Frequency Direct sequence
spread spectrum hopping spread spread spectrum
spectrum
Data rate 11 Mbps 1 Mbps 250 kbps
Nodes per master 32 7 64,000
Slave enumeration latency Upto 3 s Upto 10 s 30 ms
Data type Video, audio, Audio, graphics, Small data
graphics, pictures, files packet
pictures, files
Range [m] 100 10 10–100
Extendibility Roaming No Yes
possible
Complexity Complex Very complex Simple
Positioning technology CoO, CoO (tri)lateration,
(tri)lateration, fingerprinting
fingerprinting

– Calculation of Mean of the temperature and Humidity values from the street
lights in the small local area network and then send it to the thingspeak server
and Blynk server.
– Motion/fire/fault flags are interrupt driven and it is generated by each indi-
vidual street light with its Unique ID (UID). The notification, if generated is
sent to the Blynk server with the UID and a mail is generated for fire/fault
which is sent to the Admin with the UID of street light where the interrupt
was generated.
The next important thing is the IoT gateway which links the local microcon-
troller network with the Cloud. The protocols used in the device to device
communication is IEEE802.15.4 (Zigbee) for slave street lights and IEEE802.11
(Wi-Fi) for master street lights as shown in Table 3. REST APIs are used to
read/write data onto the Cloud server with the help of a unique authorization
key which is valid for a particular channel. The data is sent to the server every 5 s
from the gateway using a client and fetched information from the Cloud server
if required if some logic permits. This timing is set to match the flow between
sender and receiver and to reduce constant polling which affects the performance
of the communication and accounts to the lag in actuation. The microcontroller
used is ATMEGA328p of the Atmel series of microcontrollers and it has been
interfaced with ESP8266 WiFi module to add WiFi connectivity to data trans-
fer. The ATMEGA328p is a 8-bit high performance microcontroller which works
optimally with cost and energy effectiveness as shown in Table 4. Zigbee is a
232 S. Garg et al.

good alternative for Street Light to Street Light communication as it has a low
cost, low power utilization, and appreciable performance.
Master Street Light is the main street light with some important tasks. Tasks
of Master Street Light include:

– Uniquely identifying each street light (slave) in the network.

– Fetching data from these slave street lights and store them in the database.
Since transmission of data by each lamp will consume more power so one
master lamp is installed that is responsible for interacting with the external
server and the cloud.
– Control the intensity of each street light in case of emergency.
– Communicate commands from the cloud and communicate it to the slave
street light.
– On an abstract level it acts as a gateway to the local team of slave street
lights.

Table 4. Specification of the Atmel Microcontroller that is used as a Programmable

Logic Controller for the smart street light

Parameter Value
Microcontroller ATmega328p
Operating voltage 5V
Digital I/O Pins 14 (of which 6 provide PWM output)
Analog Input Pins 6
DC current per I/O Pins 40 mA
DC current for 3.3 V Pin 50 mA
Flash memory 32 KB
SRAM 2 KB
EEPROM 1 KB
Clock speed 16 MHz
ICSP header In System Programming, used when
you want to bootload, you’ll need an
AVR-ISP

The data received from the sensors via an IoT gateway is sent to the cloud
after pre-processing and compression to reduce un-necessary overheads. We have
used two different clouds in order to reduce burden. One of the clouds is respon-
sible for gathering data from the sensors and recording their data so that they
can be further analyzed using Machine Learning and Analytics. We can fetch
the spreadsheet of the data recorded with its timestamp and location parameters
from the Cloud directly. We have installed MATLAB onto the cloud for analysis
which is a great tool for analysis of data through different plotting techniques
 Real Time Adaptive Street Lighting System 233

available in MATLAB. Referring Fig. 8, The other cloud server is responsible for
handling requests from the users. The application which is running on a mobile
continuously show real-time status of the sensors and the data that is being sent
by them including ambient light, temperature and humidity, as illustrated in
Fig. 5, and generates an alarm if there is any fault or fire. There are libraries
available which enable the required information to be fetched or sent from mobile
application server to Cloud server with the help of APIs.

5 Idea of Fuzzy Logic

Wireless Sensor Networks have limited processing capability and limited battery
power. Due to large collection of input data, it is difficult to manage the data
along with different domains. Hence energy efficient data aggregation technique
is required for efficient data collection. Data aggregation is the method in which
data coming from different sensors is combined and provides useful aggregated
information. Keeping in view the above issue, a novel energy efficient fuzzy logic
based data aggregation technique is proposed. The proposed technique collects,
analyzes, classifies and aggregates the data of different domains automatically
which is reported by various sensors. Further, fuzzy logic technique is applied
as it has capability to deal with dynamic situations and to model the condi-
tions which are inherently imprecisely defined. Proposed data aggregation tech-
nique aggregates the incoming data in an effective manner by reducing energy

Fig. 5. Flowchart depicting the flow of temperature and humidity data from sensor to
the cloud
234 S. Garg et al.

consumption based on different fuzzy rules designed in knowledge base, which

further improves network lifetime.

5.1 Results

For Automation of Street Light based on the ambient light intensity, we found
that climatic conditions play a major role. Also, the latitude and the location

Fig. 6. Flowchart depicting the logic involved in fault detection of Smart Street Light
and flow of control

Fig. 7. Process of data aggregation used in fuzzy

 Real Time Adaptive Street Lighting System 235

Fig. 8. Glimpse of the user interface of MATLAB’s thingspeak platform showing plots
of data collected
Table 5. Table depicting the variation of voltage with changes in the intensity of light

Luminous flux [lm] Lamp wattage [W] Lamp voltage (AC) [V]
5000 50 220
3600 44 200
2520 37 180
1714 30 160
1114 24 140

Table 6. Table depicting the parameters used in fuzzy

Parameter Value
2
Network coverage/m 200 × 200 m2
Number of sensors 50 300
Initial energy/J 0.5
Data packet size/Bit 500
Control packet size/Bit 12
A 2.5
Maximum transmission range/m 30

Fig. 9. Bar plot showing the the variation of voltage with changes in the intensity of
light and lamp wattage
236 S. Garg et al.

Fig. 10. Fuzzy subsets of fuzzy inputs and fuzzy output

decide the angle of incidence of sunlight therefore the installation of Light Depen-
dent Resistors should be done accordingly. Only locations lying along one line of
latitude on the surface of the Earth can receive sunlight at a 90◦ angle on a given
day. All other places receive sunlight at lesser intensities. Regardless of latitude
or time of year, the sun’s angle reaches closest to 90◦ and is therefore at its most
intense at the midpoint of the day: noon. Therefore, we have to create a logic
based on this variation so we decided to fetch the time of operation and location
of a street light in addition to the actual LDR readings sent by the Street Light.
Based on the actual calibrated LDR data, we adjusted the lamp wattage and
lamp voltage with the help of a dimming module and a instruction set executed
by a microcontroller. The fault of the street light can be detected if the LDR at
the street light shows a low reading when it is supposed to be ON. The real time
clock attached to the server lets the microcontroller know when the lamp needs
to be in ON state and to detect whether it is ON or Off can be found out by the
LDR attached to it and a trigger is generated of failure or fault as illustrated
in Fig. 6. With reference to the works of Mahoor et al. [7], we have conducted
experiments in order to obtain data with the aim of building A smart street
lighting control system for optimization of energy consumption and lamp life.
The Table 5 highlights the variation of voltage, wattage and intensity of light
during the day thereby reducing power loss and increasing efficiency. Figure 9 is
a pictorial representation of the data in Table 5 and it was observed that The
electrical cost was reduced by 20% and the Lamp Life was enhanced by 100%
for a time span of 12 h. With the observations made it was recorded that the
flow of vehicles as well as human traffic is at its apex during 6PM to 10PM time
frame so, our operational strategy of street light was such that the intensity and
luminous flux will be at its peak during these hours. Past, 10PM with the usage
 Real Time Adaptive Street Lighting System 237

of AC Dimmer module the voltage takes downward leap to 160 V up till 12AM
after that the light operates on 140V till it get completely dimmed out with the
dawn. The path with minimum () is decided by fuzzy member functions and data
is transmitted. Referring Fig. 10, The evaluation of FLDA is done through MAT-
LAB and FuzzyJ Toolkit used for Java to implement the necessary fuzzy logic
functionality. Figure 7 is a block diagram depicting the process whereas Table 6
describes the simulation parameters used in this experiment work. i.e. denotes
the energy dissipated in the transmission of message from source to destination.
Based on the experiments the flowchart for control flow can be summarized in
figures with impetus to motion detection, fire detection as in Fig. 11, detection of
ambient light, detection of fault and temperature and humidity detection. With
these experiments and the logic proposed, we have successfully collaborated all
the essential features of a Smart Street Light and provided a generalized model
that would help in future research endeavours.

Fig. 11. Flowchart expressing the procedure involved in detection of fire and various
steps involved in it

6 Analysis of Cost and Energy Consumption

This project is an amalgamation of all the positive results of researchers in the

literature survey. The requirements of this project includes a street lamp that are
already existent i.e. we need not re-install street lights since our peripherals that
we will attach are compatible will both DC and AC systems with just a minor
difference for variation of brightness. In addition, we will require a Raspberry
238 S. Garg et al.

Pi or a NodeMCU board that can connect to the network using Wi-Fi which is
already available in smart cities these days. We will then install a Master Street
Light that can send the data to the server which cuts down the cost of gateways
and also decreases power consumption by these devices. Our novelty lies in the
fact that we make out street lights dim when there is no pedestrian or traffic
around thereby cutting down energy consumption. As depicted in Fig. 9, the
luminous intensity of the lamp is proportional to the energy consumption, we
are enabling the lights of the street light at dusk and disabling them at dawn
by defining a threshold as mentioned earlier and when the are enabled we are
further cutting down energy consuming by not running them at full intensity
by running them in a dim mode. We some traffic is there they run at their full
intensity thereby not hindering with the normal working.

7 Conclusion
From the discussion done in this paper, the characteristics of a smart street light
have been defined in detail. A Smart Street should be energy efficient, able to
sense the environment and act in an appropriate manner based on a certain
set of instructions. It should also have the ability to send data efficiently with
cost effectiveness to a standard server which is available all the time. Motion
detection, ambient light and fire detection adds to its integral features and are
essential to be smart in present scenarios. With all the methodology operated
and results achieved a significant optimized model of smart street lamp light is
presented and it was observed that model provides a vital energy saving model
and corresponding to the results achieved, the model provides sufficient energy
saving. The weather station further will integrate the data of forecast with the
cloud storage and accordingly will give the inputs to smart street light.

References
1. Abhishek, K., Srikanth, K.: Design of smart street lighting system. Int. J. Adv.
Eng. 1(1), 23–27 (2015)
2. Badgaiyan, C., Sehgal, P.: Smart street lighting system. Int. J. Sci. Res. 4(7),
271–274 (2015)
3. Escolar, S., Carretero, J., Marinescu, M.C., Chessa, S.: Estimating energy savings
in smart street lighting by using an adaptive control system. Int. J. Distrib. Sens.
Netw. 10(5), 971587 (2014)
4. Gupta, A., Gupta, S.: Design of automatic intensity varying smart street lighting
system. In: IOP Conference Series: Materials Science and Engineering. vol. 225, p.
012126. IOP Publishing (2017)
5. Jinila, Y.B.: Solar powered intelligent street lighting system based 1 on fuzzy logic
controller. Int. Rev. Electr. Eng. 10(3), 399–403 (2016)
6. Lavric, A., Popa, V., Sfichi, S.: Street lighting control system based on large-
scale WSN: a step towards a smart city. In: 2014 International Conference and
Exposition on Electrical and Power Engineering (EPE), pp. 673–676. IEEE (2014)
 Real Time Adaptive Street Lighting System 239

7. Mahoor, M., Najafaabadi, T.A., Salmasi, F.R.: A smart street lighting control
system for optimization of energy consumption and lamp life. In: 2014 22nd Iranian
Conference on Electrical Engineering (ICEE), pp. 1290–1294. IEEE (2014)
8. Mohamed, S.A.E.: Smart street lighting control and monitoring system for elec-
trical power saving by using vanet. Int. J. Commun. Netw. Syst. Sci. 6(08), 351
(2013)
9. Müllner, R., Riener, A.: An energy efficient pedestrian aware smart street lighting
system. Int. J. Pervasive Comput. Commun. 7(2), 147–161 (2011)
10. Pizzuti, S., Annunziato, M., Moretti, F.: Smart street lighting management. Energ.
Effi. 6(3), 607–616 (2013)
11. Popa, M., Cepişcă, C.: Energy consumption saving solutions based on intelligent
street lighting control system. UPB Sci. Bull. Ser. C 73(4), 297–308 (2011)
 A Review on Coverage and Connectivity
Through Random Deployment of Nodes
in Wireless Sensor Network

Rajib Chowdhuri(B) and Mrinal Kanti Deb Barma

National Institute of Technology Agartala, Agartala, India

[email protected], [email protected]
http://www.nita.ac.in/fac/cv mkdb cse.pdf

Abstract. Deployment of sensor nodes is a very critical issue in Wireless

Sensor Network (WSN) because proper deployment of nodes can decrease
the complexity problems in the network. Moreover, network lifespan can
also be extended though minimization of energy consumed by the nodes.
Sensor nodes can be randomly deployed over large and inaccessible areas
due to wireless communication between them. However, distance among
the nodes poses constraint in the accumulation of data from various
motes. This paper deals with coverage and connectivity of network since
these two are the most vital issues of WSN for ensuring that the entire
region of interest is covered by the sensor nodes deployed and effective
communication of data takes place between various sensor nodes and
sink.

Keywords: Random deployment · Sensing coverage · Network

connectivity

1 Introduction

WSN comprises of considerable size of sensors called motes which form an adhoc
network once they are deployed in the region of interest (ROI). They are used
for detecting environmental changes such as pressure, humidity, temperature,
level of pollution, etc. All nodes have a connection to Base Station (BS) through
wireless links and send the sensed data periodically to the BS. The BS is respon-
sible for aggregating and processing the data and the processed data is send to
the web server through internet [1].
The sensor node can be installed by simply dropping the node from
Unmanned Aerial Vehicle (UAV) in case of accessible region or simply by placing
the node in the field of interest. The network can be extended through addition
of more nodes in a similar way without any extra endeavor. Installation costs
are also lowered since WSN have the capacity to respond to change in topology.

Supported by NIT Agartala.

c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 240–250, 2020.
https://doi.org/10.1007/978-981-15-3666-3_20
 A Review on Coverage and Connectivity Through Random Deployment 241

WSN can be heterogeneous also as some nodes with higher energy are deployed
to extend the lifespan of the network (Fig. 1).

Fig. 1. A WSN where sensors detect application specific events and send sensing data
to BS (https://www.researchgate.net/figure/Wireless-Sensor-Network-Architecture-7
fig1 281459697).

The basic unit of WSN [2] is a sensor node which comprises of the following:
(1) sensing transducer which measures various environmental changes within its
vicinity but it has a limited sensing range. (2) Data Processor transforms the
measured or sensed data into an analog signal. (3) Radio Transceiver is used to
transmit the analog signal to the BS through a single hop or multi hop wireless
communication links. (4) Embedded batteries are used as a source of power for
sensing, to process the sensed data and for efficient communication. But the only
disadvantage is that these batteries are non – rechargeable. So extensive research
is going on to facilitate all these tasks using solar powered batteries (Fig. 2).

Fig. 2. Four major components of a sensor node (https://www.electronicshub.org/

wireless-sensor-networks-wsn/).
242 R. Chowdhuri and M. K. Deb Barma

In WSN, we need to ascertain that there is a connection of every node to

information collection node called sink. In case of multi hop communication, the
most important condition for communication is that the network should be fully
connected. In a fully connected network, communication occurs between each
and every node directly using single hop or through neighboring nodes. The cost
of the network rises due to large number of nodes and hence the main objective is
to curtail the number of nodes ensuring both coverage and connectivity. Routing
process is inefficient and complicated in WSN which results in large overhead
due to large number of possible paths. Moreover, batteries of sensors are non-
rechargeable, hence only the nodes which participate in routing are turned on and
remaining nodes are turned off. In order to avoid all the nodes from transmitting
the analog signal to the sink node, a cluster is formed comprising of a set of nodes,
In each cluster, there is a cluster head whose task is to accumulate the sensed
data from all nodes in its cluster and then send the accumulated data (analog
signal) to the sink node, thereby saving energy of the nodes. Since energy of one
node will be exploited, hence cluster head will be rotated among all the nodes
within a cluster [1].

2 Application of WSN

The applications of WSN are as follow [3]:

– Area Monitoring: Sensor nodes are installed in a region which is to be exam-

ined. For e.g., in military application, sensor nodes are used for detecting
intrusion of enemies.
– Air pollution monitoring: Sensor nodes are placed in various metropolitan
cities to measure the concentration of pollutants in different densely popu-
lated residing areas of the city.
– Forest fire detection: Sensor nodes deployed in the forest generates an alarm
signal when it detects fire in the forest, enabling the fire brigade to immedi-
ately reach the forest and stop it from further spreading.
– Landslide detection: Due to continuous rainfall, there is a chance of movement
of soil which can be detected by the sensor nodes placed on the mountains
or hilly regions, thereby enabling the authority to blockade the road passing
through the hilly terrains so that passengers doesn’t get stuck under the
landslide.
– Water quality monitoring: By using underground water sensors, water quality
can be monitored in rivers, dams, lakes and oceans to check whether the water
is arsenic free or not.
– Natural disaster prevention: Sensor nodes deployed along river bed to measure
the level of water and flow of water and based on these measured data an
alarming signal is generated before the occurrence of flood so that authority
gets enough time to move the residents from the flood prone areas.
– Greenhouse: Sensor nodes are used to monitor temperature and humidity
inside commercial greenhouses and when they go below the threshold level,
greenhouse manager is notified via message.
 A Review on Coverage and Connectivity Through Random Deployment 243

3 Random and Deterministic Deployment

In WSN, there is a finite sensing range for each node denoted by rs and a
finite communication range denoted by rc. The concatenation of sensing range
of all the sensor nodes reflects the area covered by the network [4]. For successful
communication, the network should be well connected so that not a single sensor
remains isolated and should have the potential to send the sensed signal to the
cluster head or the sink node for data aggregation [5]. Coverage and connectivity
of network can be better understood from network deployment models, sensing
model and communication model.
Depending on the accessibility of the region of interest, sensor deployment
is basically of two types: deterministic where sensors are deployed in human
friendly environment basically indoor areas and random where sensors are
deployed in large open inaccessible areas. Deterministic deployment of sensors
resulting in pattern-based grid offers better connectivity and sensing coverage
in contrast to random deployment [6]. However, sometimes random deployment
outperforms deterministic deployment when full coverage of the region is not
required such as intrusion detection [7].
Let us assume that the sensor nodes deployed along the river bed is a square
lattice of length y, then the area to be covered by the network is Area = y × y.
Say, N number of nodes are installed in region A and the location of each sensor
is represented by (pi , qi ). In case of random deployment, Probability Density
Function (PDF) for sensor at any location will be:

f (pi , qi ) = 1/Area (1)

Figures 3 and 4 represent 100 number of sensors deployed in a square based

lattice WSN and randomly deployed WSN respectively.

Fig. 3. Voronoi diagram of a square grid WSN (https://link.springer.com/chapter/10.

1007/978-81-322-2129-6 2).
244 R. Chowdhuri and M. K. Deb Barma

Fig. 4. Voronoi diagram of a random WSN (https://www.researchgate.net/figure/

Voronoi-diagram-in-localization-network fig5 275680777).

Voronoi diagram represents adjacency of deployed sensor nodes within the

region of interest. If there are N numbers of sensors, then there will be N Voronoi
polygons. There will be a Voronoi polygon for each sensor and each point within
that polygon will be close to that sensor compared to the points outside the
polygon [8]. In case of deterministic deployment, Voronoi polygon remains same
for all the sensors in square grid WSN.

4 Connectivity Model
The communication model of WSN can be expressed in terms of a graph compris-
ing of vertices and edges, denoted by G = (V, E) sensor node represent vertices
and wireless communication links are denoted by edges. Two sensor nodes will
be neighbor nodes if Euclidean distance is less than or equal to communication
range. Figure 5 illustrates that sensors A and B are neighbors since they are
inside the communication range of each sensor, but node C is isolated.

Fig. 5. Communication links in a WSN graph (https://link.springer.com/chapter/10.

1007/978-81-322-2129-6 2).
 A Review on Coverage and Connectivity Through Random Deployment 245

A WSN remains connected if there exist a single hop or multi hop path
between the sensor nodes. Connectivity in WSN depends upon the location where
the nodes are deployed and their communication range.

5 Coverage Model
Environmental changes are monitored by the sensors deployed within ROI. A
point ‘a’ is covered by the sensor if Euclidean distance between point and the
sensor si is less than or equal to the sensing range rs of the sensor [9].

d(a, si ) ≤ rs (2)
The region of interest is said to be fully covered if there exists at least one
sensor and there is no hole in that ROI. Disk sensing model doesn’t provide
full sensing capability of real time sensors because they are small and cheap
and are unable to provide same sensing capability in each and every direction.
Therefore, a probabilistic model called Elfes sensing model is utilized to model
sensing capability of realistic sensors. Using Elfes sensing model [10], a sensor
can detect an event at a distance within (rs – e) with probability 1, and at a
distance greater than (rs + e) is with probability 0, and within the distance
interval (rs – e, rs + e) with probability e−a(r−rs ) , where rs is average sensing
range and e measure uncertainty in sensing of sensors detection capability.

6 Coverage and Connectivity of WSN in Lattice

In deterministic deployment, sensor nodes are placed at desired locations in a
controlled manner following some patterns like square, triangle and hexagonal
shape. These patterns are repeated to wrap the entire ROI without any area
being overlapped. These patterns are widely used in practice due to their sim-
plicity and ease of deployment. Extensive research is going on rhombus type of
patterns with minimum number of sensors to wrap the full region along with
maintaining Quality of Service (QoS). Figure 6 illustrates Voronoi polygon area
of a sensor in a square, triangle and hexagonal lattice.

Fig. 6. Voronoi polygon of a sensor in a square, triangle and hexagonal lattice (https://
link.springer.com/chapter/10.1007/978-81-322-2129-6 2).
246 R. Chowdhuri and M. K. Deb Barma

Voronoi polygon determines the number of nodes required to wrap the full
area. Assume Area is the application field to be covered and Av is the Voronoi
Polygon Area, then number of sensors Ns required to complete the entire region
is given by:
Area
Ns = (3)
Av
The area of the Voronoi Polygon is computed based on the length of the
deployment lattice pattern, i.e., deployment distance lx . This parameter in turn
depends on sensing range and communication range of sensor.

7 Area Coverage Through Random Deployment

In random deployment, sensors are dropped from aero plane or Unmanned Aerial
Vehicle (UAV). This approach is extensively used now a day since it is scalable
and covers large and open hostile areas where humans cannot penetrate.
Node density in a WSN formed by random deployment of Ns number of
sensors in an application area, say Area is given by:
Ns
P = (4)
Area
Let us consider the disk sensing model where any point within an application
area can be monitored by a sensor whose Euclidean distance between point
and sensor is smaller than the sensing range of the sensor [11] and the sensing
coverage is given as [1]:
2
Pcoverage = 1 − e−pπrs (5)
Let us imagine a point T arbitrarily, then the probability of N number of
sensors residing in that circular area ST with T as Centre and radius rs is given
by [1]:
(P ST )n −P ST
P (n, T ) = e , where ST = πrs2 (6)
n!
The probability that not a single sensor is deployed within the region of
interest is given by [1]:
2
P (0, T ) = e−pπrs (7)
Henceforth, probability that there exists minimum one sensor node in the
application area is given by [1]:
2
Pcoverage (T ) = 1 − e−pπrs (8)

As N number of sensor nodes is independently deployed over the entire appli-

cation area, each point within that region has equal probability of being covered.
Therefore, sensing coverage of entire network is given as [1] (Fig. 7):
2
Pcoverage = Pcoverage (T ) = 1 − e−pπrs (9)
 A Review on Coverage and Connectivity Through Random Deployment 247

Fig. 7. Random WSN with 100 independently deployed sensors (https://www.

researchgate.net/figure/The-randomly-deployed-sensor-nodes-in-a-WSN fig1 300413
635).

8 Network Connectivity Through Random Deployment

Another fundamental issue in WSN is network connectivity to ensure data com-
munication takes place between sensor node and cluster head and from cluster
head to base station or sink node efficiently. If there exist a single hop or multi
hop communication path between sensor node and the BS, then only we can
state that the sensors are connected and connectivity among all nodes suggests
that the entire network is connected.

Fig. 8. Illustration of connected WSN [12].

From Fig. 8, it is absolutely clear that all sensors are connected and there is
no isolated sensor. If the degree of a node is 0, then it is called as isolated sensor.
Number of neighbors that a node possesses within its communication range is
defined as degree of a node.
248 R. Chowdhuri and M. K. Deb Barma

Let us consider a sensor node si arbitrarily and the probability of the node
being isolated is almost equivalent to the probability that the node has no neigh-
bors within its communication range and is as follow [1]:
2
Pisolated (si ) = e−pπrc (10)

Probability of the sensor not being isolated is given as [1]:

2
Pnonisolated (si ) = 1 − e−pπrc (11)

Since N number of sensor nodes are independently deployed over the appli-
cation area, all sensor node possesses equal probability of being non isolated.
Therefore, probability that there doesn’t prevail any isolated sensor in the net-
work is given as [1]:


N
2
Pnonisolated = Pnonisolated (si ) = (1 − e−pπrc )N (12)
i=1

Necessary condition for a network to be connected is that there should not

be any isolated sensors and the probability that WSN is connected with each
sensor having at least one neighbor is given as [1] (Fig. 9):
2
Pconnected = Pnonisolated = (1 − e−pπrc )N (13)

Fig. 9. Illustration of a un connected WSN (https://link.springer.com/chapter/10.

1007/978-81-322-2129-6 2).

9 Performance Metrics
9.1 K – Coverage
It refers to minimum K – Coverage. A WSN is presumed to have K – Cover-
age, if every point in the network is covered by minimum k number of sensors.
 A Review on Coverage and Connectivity Through Random Deployment 249

K – Coverage is useful for applications which require surveillance. Measuring k

– coverage is not at all suitable for performance comparison because it is solely
designed for network coverage. Hence only the relative frequency of k covered
points in the network is only counted [3].

9.2 K Coverage Map

This map is used to check whether the entire region in the network is covered or
not and it is also used to measure the number of k covered points. Using this map,
we can measure the performance of the coverage area through node deployment
scheme. K – Coverage Map is defined as the total area of the application area
covered using k number of sensors [3]. The following assumptions are used for
modelling K – Coverage Map:

– A disk-based sensing model comprising of homogeneous nodes are used with

maximum sensing range.
– Sensing range is of same length as that of a unit cell. Hence sensing range
varies for different type of deployment strategies.
– A point is said to be covered if it lies on the circumference of the disk or
inside the disk.

Boundary conditions for any type of deployment strategies are not considered
and hence suitable for large scale WSN applications.

10 Conclusion

In this paper, three important aspects of WSN have been investigated in detail:
Node deployment models which are of two types: deterministic and random.
Connectivity model is discussed which assures whether the network is fully con-
nected or not and whether reliable data communication is taking place or not
between sensor nodes and cluster head and also from cluster head to the sink
node or base station for further processing of data. Coverage model is also dis-
cussed to find whether each and every point in the application area is covered
or not without leaving any holes in the region of interest. Coverage and Connec-
tivity are the most essential requirements for maintaining Quality of service in
a network.

References
1. Wang, Y., Zhang, Y., Liu, J., Bhandari, R.: Coverage, connectivity, and deployment
in wireless sensor networks. In: Patnaik, S., Li, X., Yang, Y.-M. (eds.) Recent
Development in Wireless Sensor and Ad-hoc Networks. SCT, pp. 25–44. Springer,
New Delhi (2015). https://doi.org/10.1007/978-81-322-2129-6 2
2. Al-Karaki, J.N., Kamal, A.E.: Routing techniques in wireless sensor networks: a
survey. IEEE Wirel. Commun. 11(6), 6–28 (2004)
250 R. Chowdhuri and M. K. Deb Barma

3. Devi, G., Bal, R.S., Nayak, S.M.: Node deployment and coverage in wireless sensor
network. Int. J. Innov. Res. Adv. Eng. (IJIRAE) 1(2), 139–145 (2015)
4. Wang, Y., Wang, X., Agrawal, D.P., Minai, A.A.: Impact of heterogeneity on cov-
erage and broadcast reachability in wireless sensor networks. In: The Fifteenth
International Conference on Computer Communications and Networks (ICCCN)
(2006)
5. Wang, X., Xing, G., Zhang, Y., Lu, C., Pless, R., Gill, C.: Integrated coverage
and connectivity configuration in wireless sensor networks. In: ACM SenSys 2003
(2003)
6. Zhang, H., Hou, J.C.: Is deterministic deployment worse than random deployment
for wireless sensor networks? In: INFOCOM (2006)
7. Wang, Y., Kutta, A.: Joint and simultaneous K-sensing detection in determinis-
tic and random sensor networks. In: Proceedings of the 26th IEEE International
Parallel and Distributed Processing Symposium Workshop on Dependable Paral-
lel, Distributed and Network-Centric Systems, Shanghai, China, 21–25 May 2012
(2012)
8. Fortune, S.: Voronoi diagrams and Delaunay triangulations. Comput. Euclidean
Geom. 1, 193–233 (1992)
9. Liu, B., Towsley, D.: A study of the coverage of large-scale sensor networks. In:
IEEE International Conference on Mobile Ad-Hoc and Sensor Systems, pp. 475–
483. IEEE (2004)
10. Elfes, A.: Occupancy grids: a stochastic spatial representation for active robot
perception. In: Iyenger, S.S., Elfes, A. (eds.) Autonomous Mobile Robots: Percep-
tion, Mapping and Navigation, vol. 1, pp. 60–70. IEEE Computer Society Press,
Washington, D.C. (1991)
11. Liu, B., Towsley, D.: A study of the coverage of large-scale sensor networks (2004)
12. Wu, H., Zhu, L.: Coverage strategy for heterogeneous nodes in wireless sensor
network based on temporal variability of farmland. https://doi.org/10.17559/TV-
20160517052746. ISSN 1330–3651 (Print), ISSN 1848–6339 (Online)
 Role of Open Hardware and IoT in Home
Automation: A Case Study

Anurag De1(B) , Ashim Saha1 , and Praveen Kumar2

1
NIT Agartala, Agartala, Tripura, India
[email protected]
2
VNIT Nagpur, Nagpur, Maharashtra, India

Abstract. Nowadays Open Source platforms has become very popu-

lar to test and implement various home automation and IoT based
projects as it provides the designer with a physical programmable cir-
cuit board and one corresponding Integrated Development Environment
(IDE) which plays on your personal computer and is used to write, com-
pile and upload the code on to the circuit board, thus resulting as a
complete package and a great choice for beginners. To keep track with
the changing world, IoT and open source platforms is becoming a part
and parcel of our day to day lives at an increasingly fast pace. This paper
is mainly aimed at elaborating the role of Internet of Things and Open
Source platform in implementing home automation with the help of an
application that will control appliances based on the room temperature
and humidity readings. Here we have used Arduino Uno and ESP8266
NodeMCU open source platform for implementing the project.

Keywords: Home automation · Internet of Things · Open Source

Hardware

1 Introduction
Open Source hardware and Internet of Things are the technologies of today’s
world which is penetrating rapidly into our everyday lives with various smart
and innovative solutions which not only simplify our living but also allow us to
save power and make our life style energy efficient and smart. When things get
connected, we can feel more in our control and thus enhancing and empowering
our normal day to day living. IoT is having a great impact on improvising home
automation solutions and designing home automation with having open source
hardware and sensors as the pillars to implement where everything smart solution
will be taken into account by the devices interconnected on IoT. With these all
the personal devices connected to internet will enable us to author our lives.
For example, in medical science these technologies have played a major role in
enabling diagnostic devices and systems to sense upcoming ailments and act
accordingly on it.
Supported by NIT Agartala.
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 251–262, 2020.
https://doi.org/10.1007/978-981-15-3666-3_21
252 A. De et al.

The objective of this project is to make life easier through automation and
rooted on few major advances over the technology accessible in automation, such
as advancement in wireless automation technology as well as dropping of price
points as the market gets matured enough to accept home automation require-
ments in substantial amounts. And we have taken a step forward in remotely
observing the temperature and humidity of a room and actuating an actuator
based on the requirement to smartly control various equipment based on the
level of temperature and humidity.
According to the McKinsey report “Disruptive technologies: Advances that
will transform life, business, and the global economy”, the Internet of things
(IoT) is one of the top three technological advancements of the next decade
(together with the mobile internet and the automation of knowledge work). The
report goes on to say that “The Internet of Things is such a sweeping concept
that it is a challenge to even imagine all the possible ways in which it will affect
business, economies, and society” [1,11].
IoT comprises of all the web enabled devices which gather data, send data
and act on the data which they get from the surrounding with the help of various
sensors with embedded software, microprocessors and communication hardware.
These devices are smart and connected devices and can also communicate with
other devices a technique known as machine to machine communication. We can
also interact with the devices to setup them with instructions. Although the
devices do much of their job on their own without much human intervention.
Thus, we can say IoT (Internet of Things) is mostly about connecting various
daily life devices to internet and interact with them to gather information and
also control them when required remotely. One example is our room air condi-
tioner which we can suppose control based on the ambient room temperature
and humidity using sensors using our mobile phone or tablet from anywhere on
the planet.

Open Source Hardware: Open Hardware or Open Source Hardware are the
physical entities or objects whose design features and specifications are licensed
in a manner that the said entity can be studied upon, can be modified, recreated
and also distributed if required by anybody. The term Open Hardware does not
only specify for computers, it can be any type of entity as it is nothing but a
collection of design principles and legal practices which can be defined for any
entity or object to be termed as Open Hardware or Open Source Hardware. It
can thus specify objects such as Computers, Vehicles, Furniture, Buildings, and
so on [12].
This technology is best suited for implementing Home Automation as it gives
the engineer the liberty to choose from a variety of available hardware platforms
to upload their codes and interface sensors and if required the freedom to update,
build and distribute the technology.
 Role of Open Hardware and IoT in Home Automation: A Case Study 253

1.1 Home Automation and Its Necessity

Need of the hour is, our home should be smart and efficient both in terms of
energy efficiency and smart home appliances. But what if we can control our
appliances without physically moving up to them using some device or remote
control and also if the devices or appliances automatically convey their status
and even run based on some intelligence, our smart appliances would rather
become smarter and thus make our homes even smarter. This concept of embed-
ding intelligence into our household appliances like air conditioner, television,
refrigerator, lighting, heating appliances, etc. and controlling and automating
them leads to home automation solutions [2,3]. This can be achieved by con-
necting these appliances to internet via Wifi or over a personal area network via
Bluetooth technologies and controlling and tracking their real time status via a
mobile device like a smartphone, phablet, personal computer, etc. Sensors are
installed in the appliances to acquire real time environmental parameters and
thus making the devices capable to sensing various parameters in order to react
or operate as per the readings.
There are thus many real life scenarios which lead to the need of home
automation like, lights get switched on when you enter the room, the tea maker
alarms when the tea is ready, the refrigerator tell us what all vegetables are there
in it, the dustbin opens when you come near to it, etc. Likewise we can have lot
many such scenarios where home automation solutions can make our life easier
and this concept is making the base of what we know as Internet of Things or
Internet of Everything.

2 Related Work
There has been a lot of work done on Home Automation using Open Source
platform and Internet of Things and is still going on at full pace. As our world
is changing very fast in terms of technological advancement, lifestyle of human
beings and also climate, the need of automation and smart appliances are becom-
ing increasingly high as all these are required to complement our changing envi-
ronment in a smarter and more sustainable way which in turn will bring efficiency
in power consumption, reduction in time required to do a particular task, cost
cutting and above all the power to connect devices and appliances remotely from
anywhere in the world.
Few of the noted applications in this field are discussed here. Sipani et al. [6]
demonstrated temperature and humidity monitoring and control using Arduino
Open source board. Patnaikuni [4] presented a comparative study of various
IoT development boards such as Arduino, Raspberry Pi and ESP8266. Kunal
et al. [2] developed an end to end smart home automation system using Internet
of Things. Malathi et al. [5] developed a home automation system with the help
of ESP8266. Kale and Kulkarni [7] demonstrated a real time temperature and
humidity monitoring system using Arduino and Xbee S2. Saiteja et al. [3] devel-
oped a home automation system using Raspberry Pi board which can control
appliances remotely and also provide security for any mishap when the user is
254 A. De et al.

not present at home. Liu and Wang [8] elaborately presented the future scope
and research prospects of Internet of Things and its applications (Fig. 1).

Fig. 1. A representation of home automation using a mobile device controlling various

home appliances [9].

3 Implementation of Case Study

The case study being implemented using Open Source hardware and IoT is for
reading the temperature and humidity of a room and remotely observing the
values and also actuating an actuator based on the level of temperature and
humidity thereby analyzing the result based on the obtained values. Room tem-
perature and humidity level can be found out during regular intervals constantly.
And if the temperature is above or below a threshold, alerts can be generated or
can trigger an actuator for a specific purpose along with remotely observing the
readings. For this to happen an cloud based IoT application should be installed
on a mobile device and internet connection should also be present.
Figures 3 and 4 shows the connection diagram of the Temperature/Humidity
sensor with the Arduino Uno open source board and the ESP8266 wifi module
board respectively.

3.1 Tools Used for Implementation

The technologies and devices we used to implement the case study are:
– Arduino Uno board [4]/ESP8266 NodeMCU wifi module [5]
– DHT11 Temperature/Humidity sensor
 Role of Open Hardware and IoT in Home Automation: A Case Study 255

– Suitable Actuator (any appliance which can be controlled based on

temp/humid readings such as Air Conditioner, Fan, Heater, etc.)
– Blynk mobile application [10]

3.2 System Architecture

Figure 2 represents the way the IoT mobile application works to enable Internet
of Things. It is capable of controlling the Open Hardware devices remotely and
display, store and visualize sensor data. The three components of the system
works as follows.

Fig. 2. Communication between IoT mobile application and Open Hardware platform.
256 A. De et al.

IoT Mobile App: It helps us to create user friendly interfaces for various IoT
based projects using variety of widgets.
IoT Cloud Server: It controls all necessary communications between the
mobile device and open hardware.
IoT Libraries: It stores all types of required files compatible with different
Open Hardware platforms (like ESP8266 NodeMCU, Arduino Uno, etc.) and
also controls and process all the to and fro commands from the server.
The mobile application is connected to the Open Hardware platforms via the IoT
cloud server which maintains the communication between them with the help
of various library files required as per the compatibility of the Open Hardware,
thus enabling them to work in coordination. The internet source can be of user
choice like Ethernet, wireless connectivity or 4G connection (SIM Card based)
as available based on the environment.
Figures 3 and 4 shows the architecture of the system and the connection
diagram between the Temperature/Humidity sensor with the Arduino Uno open
source board and the ESP8266 wifi module board respectively [5–7].

Fig. 3. Connection diagram of Temperature/Humidity sensor and Arduino Uno.

Figure 3 shows the connection established between the Temperature/Humid

sensor and the Arduino Uno board and the actuator. The negative (−) pin of
the sensor is connected to the ground pin of the Arduino Uno, similarly the
positive (+) pin of the sensor gets connected to the 3 v power supply pin of
the Arduino Uno and the data pin of the sensor (OUT) gets connected to one
of the digital pin of Arduino Uno (here it is Digital Pin No. 2). Similarly the
respective actuator used for triggering an action depending on the scenario is
also connected to the Arduino Uno Board. Here the Arduino is standalone and
whatever code is uploaded in it will execute and depending on that the actuator
will be controlled based on the collected sensor data. There will be no remote
access of the system.
 Role of Open Hardware and IoT in Home Automation: A Case Study 257

Figure 4 depicts the connection established between the Temperature/Humid

sensor and the ESP8266 wifi module board. Here the negative (−) pin of the
sensor is connected similarly with the ground of the ESP8266, the positive (+)
of the sensor gets connected to the 3 V power supply of the ESP8266 and the
data pin of the sensor is attached with any of the digital pins of the ESP8266
(here it is Digital Pin No. 4). Further, the ESP8266 is connected to the mobile
device via the Blynk App through the Blynk cloud. Thus, here the ESP8266
NodeMCU board can be remotely accessed and controlled via the cloud network
from the mobile platform which in turn can control various actuators based on
sensor data.

Fig. 4. Connection diagram of Temperature/Humidity sensor, ESP8266 NodeMCU

and Mobile device.

Comparative Study of NodeMCU and Arduino Uno: As both ESP8266

NodeMCU and Arduino Uno are low cost and easily available Open Source
boards, we have chosen them for our case study. Both of them have some benefits
and flaws over each other. In Table no. 1, the feature and specification based
comparison between NodeMCU and Arduino Uno are listed [13].
Thus, it can be stated that, NodeMCU is having more flash memory than
Arduino Uno and can store more programs than Arduino Uno. The size of the
NodeMCU Wifi board is smaller compared to Arduino which again makes it more
compatible to various application. NodeMCU is low cost, has low energy con-
sumption and most importantly is Wifi integrated, making the Open Hardware
more advanced to switch over from Arduino Uno (Table 1).
258 A. De et al.

Table 1. NodeMCU and Arduino Uno specification and feature based comparison

Features NodeMCU Arduino Uno

Processor used ESP8266 Wifi Chip ATMega328P
Operating voltage 3.3 V 5V
Memory (System) 128 KB 2 KB
Memory (Flash) 4 MB 32 KB
EEPROM - 1 KB
IDE Arduino IDE, Lua Loader Arduino IDE
Supported Wiring, C, C++ Wiring
programming language
Clock speed 80 MHz to 160 MHz 16 MHz
Communication IEEE 802.11 b/g/n IEEE 802.11 b/g/n,
protocols IEEE 802.15.4 433RF
BLE 4.0 via Shield

3.3 Procedure to Implement the Case Study

First, the DHT library is to be added which is available on the Arduino official
website [14]. Once the library is included, the pin number is defined. Now, con-
nect the sensor to the respective pin thus creating a DHT object. At the time of
setup the serial communication is required to be initialized as the serial monitor
will be used to display the result. Read () function is then used to read the data
from the sensor and place the respective values of humidity and temperature
into corresponding humidity (h) and temperature (t) variables. Once the above
tasks are completed the serial monitor will display the humidity and temperature
readings.
Figure 5 shows the ESP8266 being connected with DHT11 Temp/Humid sen-
sor. This reading can also be extended to be viewed remotely via the Blynk App
and also used to trigger an actuator based on a threshold thus helping us to
remotely control appliances based on the threshold level of temperature and
humidity with the help of Open Source hardware and IoT.
Firstly connection to the blynk app is established via the ESP8266 board
which will be interfacing with the DHT11 temperature and humidity sensor.
Once the app is downloaded on our mobile device, we need to create a new
project on the app and select the device as ESP8266. Once project is created,
an authorization token is being issued which is then used as the key to connect
the mobile device to the ESP8266 board via the blynk cloud. For Arduino Uno
board, a separate Wifi shield is required to enable it to connect to the mobile
device via the internet.
 Role of Open Hardware and IoT in Home Automation: A Case Study 259

Fig. 5. Snapshot of ESP8266 NodeMCU connected with DHT11 Temp/Humid sensor.

Code Snippet of the Blynk App Authorization Token: char auth[ ] =

“9a3eaa706656431390ab7930323c623a”;

Code Snippet of Wifi Credentials: char ssid[ ] = “Redmi”; char pass[ ] =

“1234567a”;
These code snippets works for the Arduino IDE and are used to connect
the mobile platform (here, Blynk Application) to the Open Hardware board
(here, ESP8266 NodeMCU/Arduino Uno with shield) via the internet wirelessly
[10,14].

4 Results and Analysis

This section discusses about the results acquired from the case study and the
analysis done on the results. Figure 6 shows the obtained real time temperature
and humidity reading remotely on the mobile device. The red meter shows the
current temperature being sensed and the blue meter shows the present atmo-
spheric humidity level.
Based on the reading of the current humidity and temperature, the readings
over a span of time is being tracked and thus we can analyze the humidity and
temperature variation over a span of time and can configure and automate an
actuator which can be an Air Conditioner, Heater, Fan, etc. based on requirement
enabling home automation and will be useful in saving power resulting in a green
solution and energy efficiency.
260 A. De et al.

Fig. 6. Real time temperature and humidity reading on mobile device [10]. (Color
figure online)

Fig. 7. Temperature and humidity variation over a period of time. (Color figure online)
 Role of Open Hardware and IoT in Home Automation: A Case Study 261

Figure 7 shows the humidity and temperature over a span of time graphi-
cally. The blue and the red lines depicts the temperature and humidity variation
respectively over a span of time. The X-axis represents the span of time and the
Y-axis represents the range of temperature and humidity. Here the temperature
and humidity variation is manually controlled over a very short span of time
(reading taken every 5 s) to visualize the variation scenario. In the real field of
study, the data gathered will be spread over a larger and broader span of time.
Thus, based on the data gathered and tracking done, its possible to take a
decision to smartly control our home appliances, for example.
– If the Temperature and Humidity are above a threshold, Air Conditioning
will turn ON
– If Temperature is below a threshold, Heater will be working, etc.

5 Conclusion
In this paper, the case study experimented can measure temperature and humid-
ity of a room using a sensor with help of an Open Source hardware platform
and also read the real time humidity and temperature values remotely with the
help of a mobile application using IoT based cloud platform, thus helping us to
monitor the readings over a period of time and enable us to analyze the read-
ing and accordingly perform necessary statistical calculations and also trigger
various actuators to control house hold appliances in order to promote smart
home automation and energy efficiency. Thus we can conclude this paper as an
experimental work on a specific case study to visualize home automation from a
particular point of view using Open Source hardware and IoT thus emphasizing
their role in its implementation.

References
1. Lueth, K.L.: IOT basics: getting started with the Internet of Things. IOT Anal.
1–8 (2015)
2. Kunal, D., et al.: Smart home automation using IOT. Int. J. Adv. Res. Comput.
Commun. Eng. (IJARCCE) 6(2), 576–578 (2016)
3. Saiteja, K., et al.: Home automation using IOT. Int. J. Eng. Trends Technol.
(IJETT) (2017). ISSN 2231–5381, 229–233
4. Patnaikuni, D.R.P.: A comparative study of Arduino, Raspberry Pie, ESP8266 as
IOT development board. Int. J. Adv. Res. Comput. Sci. 8(5) (2017). ISSN 0976–
5697
5. Malathi, M., et al.: Home automation on ESP8266. SSRG Int. J. Comput. Sci.
Eng. (2017). ISSN 2348–8387
6. Sipani, J.P., et al.: Temperature and humidity monitoring and control system based
on Arduino and SIM900A GSM shield. Int. J. Electr. Electron. Data Commun.
5(11), 62–68 (2017)
7. Kale, V.S., Kulkarni, R.D.: Real time remote temperature and humidity monitoring
using Arduino and Xbee S2. Int. J. Innov. Res. Electr. Electron. Instrum. Control
Eng. 4(6), 175–179 (2016)
262 A. De et al.

8. Liu, R., Wang, J.: Internet of Things: application and prospect. In: MATEC
Web of Conferences, vol. 100, p. 02034 (2017). https://doi.org/10.1051/matecconf/
201710002034
9. Pyramid Technology. http://pyramidtec.com
10. Blynk IoT Platform. https://www.blynk.cc
11. Disruptive technologies: advances that will transform life, business, and the global
economy (2013). https://www.mckinsey.com/business-functions/digitalmckinsey/
our-insights/disruptive-technologies
12. https://opensource.com/resources/what-open-hardware
13. Ashwini, P., et al.: A comparative study of Arduino Uno, NodeMCU in IoT. Int.
J. Sci. Res. Sci. Eng. Technol. (IJSRSET) 4(7) (2018). ISSN 2394–4099
14. Arduino Official Website. www.arduino.cc
 Design and Simulation of Load Balancing
Algorithms in Wireless Mesh Network

Priyanka Sarkar(B)

National Institute of Technology, Agartala, Agartala, India

[email protected]

Abstract. This work deals with balancing the load which is caused due
to the congestion created in Wireless Mesh Network. It is known that
each Mesh Router is associated with a gateway for transmission and
reception of internet traffic. So, if a lot of Mesh Routers are served by
a single gateway from the gateway set, the gateway becomes congested.
This deteriorates the overall performance of the network. Many algo-
rithms for load balancing have been introduced to solve this problem. In
this paper, a hop count based load balancing algorithm and a Last in
first out based load balancing algorithm have been proposed to balance
the load in Wireless Mesh Networks (WMNs) having multiple gateways.
The algorithms in the proposed scheme are executed when the network
is found to be overloaded. The proposed schemes reroutes the flow of
the mesh router from the overloaded domain by taking care that the
flow of the rest of the mesh routers in the overloaded domain should not
be hampered. The proposed schemes are compared with each other and
have been simulated. The simulation results show that the hop count
based load balancing algorithm balances the load more efficiently than
Last in first out based load balancing algorithm.

Keywords: Wireless Mesh Network · Load balancing · Gateway ·

Mesh Routers · Mesh Clients

1 Introduction
Many wireless networks have been introduced into the next generation to have
better services. Now-a-days, wireless LAN technology is gaining lots of popu-
larity as it provides mobile access in remote areas and public spaces. When the
need of mobility is being realized, this technology has been introduced in the
mainstream technology. Recently, due to the rapid development of networks,
people always want to access high-bandwidth network anytime and anywhere
for enjoying ubiquitous network services. For this, new innovative wireless tech-
nologies are introduced. Wireless Mesh Networks (WMNs) are new kind of wire-
less multi-hop network architecture. The performance of WMNs is influenced
by radio techniques, scalability, mesh connectivity, security, compatibility and
Supported by NIT Agartala.
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 263–272, 2020.
https://doi.org/10.1007/978-981-15-3666-3_22
264 P. Sarkar

interoperability [3]. It provides various functionalities. WMNs are of three types:

Infrastructure or Backbone WMNs, Client WMNs and Hybrid WMNs. Wireless
Mesh Networks are formed using Mesh Routers (MRs) and Mesh Clients (MCs)
[1,2]. Mesh Routers are static or semi static in nature and are connected to each
other by wireless links. It provides the backbone for the WMNs. A small subset
of MRs, directly connected to the wired network, are called Internet Gateways
(IGWs). IGWs are the key components of WMN of the traffic of WMNs will
be directed through IGWs. The Mesh Routers route the packets to or from the
MCs. MRs have to route all the internet traffic from the MCs through the Inter-
net Gateways to provide internet connectivity. Mesh Routers (sink) forms the
domains under the gateway to get the access of the internet traffic directed to
the sinks. All the traffic destined to the sink under the domain is passed through
the gateway of that domain. For that reason the Interne become the bottleneck
very easily as all the internet traffic passes through them. Some of them become
overloaded and others are not utilized properly. The performance of the network
deteriorates for this reason. To overcome this problem, load balancing among
the IGWs of WMN is necessary.

Fig. 1. Wireless Mesh Network

 Design and Simulation of Load Balancing Algorithms in WMN 265

The remainder of this paper is organized as follows. Section 2 gives the

description of the related works. In Sect. 3 the load balancing algorithm have
been propose Simulation and performance analysis are done in Sect. 4. Section 5
describes conclusion and future work.

2 Related Work

Load balancing is necessary to avoid congestion, provide reliability and increase

network throughput. It can be classified net gateway and load balancing of paths.
Load balancing of path aims at improving network throughput. In Adaptive
Situation Aware (ASA) metric based approach [5] the ASA metric cost of the
paths are calculated. The path having minimum ASA metric cost is chosen. But
the selected path may have the uneven metric cost. To avoid this problem and
to achieve load balance, the concept of Max-flow Min-cut is used.
Another important load balancing type is Internet Gateway load balancing.
Many algorithms have already been proposed for solving this problem. In parti-
tion based load balancing approach [7] the whole network divide into domains.
Load balancing is performed within the domain first and then across the domain.
Another scheme proposed by Galvez et al. is a centralized approach [6]. Here
unlocked sinks are listed first. For each unlocked sink the least loaded gateway
is chosen as its current gateway. If there is the existence of more than one least
loaded gateway, the gateway having the lowest cost to reach is chosen as its
current gateway.
In Diffusion based Distributed Load Balancing approach, the load balancing
is done based on the fairness index (β) [8]. The value of β ranges from 0 to 1.
Higher values denote more fairness in load distribution in WMN. When a fairness
index gets value 1, it indicates a perfect balanced utilization. This approach
tries to improve fairness index β. The load-balancing process is invoked if β of
a domain reaches below some threshold.
In probabilistic approach [9], simulated annealing technique is used. Here a
fairness index (β) and load balance index (β’) is used. After adjusting the load,
the network goes to a stable state having the better fairness of the traffic load.
MR is considered as the bordering node. It is able to connect to the multiple
IGWs. The MR is selected for migration using two basic methods: random selec-
tion and the greedy selection. In random selection, any MR is selected whose
migration will increase the balance index. In the greedy selection it selects the
MR for migration that will improve the balance index the most. But in some
conditions, the intermediate migration is also taken which doesn’t increase the
global balance index.
The scheme proposed by Nandiraju et al. [4] monitors the average queue
length at the IGW. If the queue length exceeds a certain threshold value in that
time period, it indicates that congestion may happen at the Internet Gateway. It
then identifies the sources having high traffic and sends notification to them to
search an alternative gateway that is less congested. After that, the MR switches
to less congested gateway.
266 P. Sarkar

In WCETT-Load Balancing Metric approach, the load balancing metric has

been integrated to WCETT metric. An average queue length at each node is
considered to check the load of the path. In this way it can consider the traffic
congestion in addition the basic functionalities of WCETT metric.

3 Proposed Scheme

Hop Count Based Load Balance Scheme (HCLB): In this section the hop
count based algorithm for load balancing has been proposed. If the network is
found to be overloaded, load balancing algorithm will be executed. It balances
the load by changing the route of the traffic flow of the sink from the overloaded
domain to the under loaded domain.
At first, the sinks are waiting to be assigned to their primary gateways.
The gateways periodically send beacon signal to advertise their presence.
After receiving the advertisement, the MR registers itself to that gateway as
its primary gateway. When the MR needs to send a data packet to the destina-
tion, it sends the route request to the gateway. After receiving the route request,
the gateway checks whether the Mesh Router is sending route request for the
first time or not. If yes, the entry of the MR under the gateway is done in a
table. Each IGW checks its load every time it receives the request from a new
mesh router. The load of domain d is defined as:
load(d) = total number of sinks under the domain.
If the domain is found to be overloaded, the algorithm is executed to balance
the overload of the network. The overload of a domain d is given by:

0, if load (d) < C
Overload(d) = (1)
load(d) − C, otherwise
Here C is the capacity of the domain d, i.e, the number of MRs, a gateway
can serve without being overloaded.
The sinks under the overloaded domain are sorted in decreasing order of
distance from the overloaded gateway. The MR having the highest hop count
in this order is selected to migrate its flow to another domain. The idea behind
this is that the probability of most distanced node being the boundary node is
very high. The MR then starts selection of new gateway for routing its internet
traffic. The set of neighboring under loaded gateway are arranged in ascending
order of their hop distance from the MR. The gateway, then sends a route error
message to that MR. After receiving the route error message, the MR deletes
its route to the gateway. The gateway having the least hop count is selected as
serving gateway. After the gateway selection, the route to the new gateway is
established. Now the MR continues its flow through the new gateway.
Figure 2(a) shows the algorithm for HCLB. Here C is the capacity of each
gateway and G is the number of gateways in the network.
 Design and Simulation of Load Balancing Algorithms in WMN 267

Last in First Out Based Load Balancing (LBLB): This section pro-
poses the hop count based algorithm for load balancing. In this procedure, the
mesh routers are stored in a queue. Each gateway maintains a separate queue.
When a mesh router appears to the network, its existence is known to the gate-
way. Each gateway maintains a Gateway Queue to keep track of the mesh routers
which are associated to that gateway. The gateway, then, makes an entry of the
new mesh router in that queue. The entry will be in Last in first out method. If
the gateway becomes overloaded, it executes the LBLB algorithm. As the name
implies, here the mesh router which has entered last in the gateway queue is first
shifted to another domain for load balancing. This technique is applied because
the mesh router which has arrived lastly in the internet is least associated with
other mesh routers and the transmission and reception of packets.
Figure 2(b) shows the algorithm for LBLB. Here C is the capacity of each
gateway.

Fig. 2. (a) HCLB and (b) LBLB

268 P. Sarkar

4 Performance Analysis
This section presents extensive simulation results that illustrate the impact of
using proposed load balancing schemes. Network Simulator 2 (NS2) has been
used to simulate the proposed schemes. The performance before and after load
balancing has been compared.
Figure 3 shows the scenario used for simulation. The scenario has 11 CBR
flows generated randomly to the 3 gateways. The other simulation parameters
are shown in the Table 1. Finally, the throughput, average delay and calculated
packet delivery ratio has been compared so that the load-balancing ability of
the proposed scheme can be proved. The values of HCLB are compared to the
values before load balancing and LIFO based load balancing (LBLB).

Fig. 3. Scenario used for simulation

Table 1. Parameters used for simulation

Parameter Values
Scenario size 800 m × 500 m
Transmission range 250 m
Mesh router Generated 11 nodes
Gateway Generated 3 gateways
Traffic type CBR
Flows 11 CBR flows
Packet size 512 bytes
Simulation time 300 s
 Design and Simulation of Load Balancing Algorithms in WMN 269

Throughput is the total amount of data transferred from the source to des-
tination or processed in a specified amount of time. Here it is shown that the
throughput of the network is increased after the HCLB algorithm is executed.
The throughput in LBLB is less than HCLB because in LBLB, the MR to be
shifted to the other domain for load balancing may be nearer to the gateway. So,
other MRs may connect to the gateway through the selected MR. If the nearer
MR is shifted to another domain, it deteriorates the performance of other MRs
also. It degrades the throughput. But in HCLB, as the MR having maximum
distance from the gateway is selected to reroute the flow and it increases the
throughput. Figure 4 shows the comparison of throughput for both the schemes.

Fig. 4. Growth of throughput in kbps

The average delay of a network is the time taken by a bit of data to travel
across the network from one node or endpoint to another. From the observation,
it is shown that the average delay is decreased after the execution of HCLB.
Before the execution of the load balance scheme, congestion occurs at the gate-
ways and delay is increased. As the proposed scheme distributes the load uni-
formly among the IGWs, it reduces the average delay. In LBLB, the selected
MR to shift another domain may be the intermediate MR from the gateway.
In such situation, if the MR is shifted, other MRs which were connected to the
gateway through this MR, have to configure their routes to gateway again. This
causes delay in the network. This deteriorates the performance. Figure 5 shows
the decrease of average delay after execution of the proposed scheme.
270 P. Sarkar

Fig. 5. Average delay (in sec) v/s time

Packet delivery ratio is the ratio of the number of delivered data packet
from source to destination. The analysis shows that after HCLB is executed, the
packet delivery ratio of the network is increased. HCLB algorithm distributes
the load in such a way so that all the IGWs are utilized properly. In LBLB the
MR which is to reroute its flow may be the intermediate node. In such case,
the other MRs connected to the gateway through this MR has to reconfigure
their routes. For this reason, the packets already arrived to those MRs cannot
be delivered properly. So the delivery of packets gets affected. Figure 6 shows
increase of the packet delivery ratio after the execution of the load balancing
algorithm.
 Design and Simulation of Load Balancing Algorithms in WMN 271

Fig. 6. Packet delivery ratio (in sec) vs time

5 Conclusion and Future Works

This paper discusses degradation of performance of WMN due to imbalance of
load in IGWs. A new Hop count based load balancing scheme and Last in first out
based load balancing algorithm have been proposed and simulated using NS2.
Simulation results show that when the hop count based load balancing scheme
is used, WMN performs better with respect to throughput, average delay and
packet delivery ratio.
As a part of the future work, link capacity can be incorporated in the IGW
selection process to improve the performance.

References
1. Zhang, Y., Luo, J., Hu, H.: Wireless Mesh Networking: Architectures, Protocols and
Standards. Auerbach Publications, Boca Raton (2006)
2. Akyildiz, I.F., Wang, X., Wang, W.: Wireless mesh networks: a survey. Comput.
Netw. J. 47(4), 445–487 (2005)
3. Akyildiz, I.F., Wang, X.: A survey on wireless mesh networks. Comput. Netw. ISDN
Syst. (2005)
4. Nandiraju, D., Santhanam, L., Nandiraju, N., Agrawal, D.P.: Achieving load bal-
ancing in wireless mesh networks through multiple gateways. In: Mobile Adhoc and
Sensor Systems (MASS), pp. 807–812. IEEE International Conference (2006)
272 P. Sarkar

5. Liao, G.-L., Chen, C.-Y., Hsu, S.-W., Wu, T.-Y., Chao, H.-C.: Adaptive situation-
aware load balance scheme for mobile wireless mesh networks. In: Computer Com-
munications Workshops (INFOCOM WKSHPS), pp. 391–396. IEEE (2011)
6. Galvez, J.J., Ruiz, P.M., Skarmeta, A.F.: Responsive on-line gateway load-balancing
for wireless mesh networks. Ad Hoc Netw. J. 10, 46–61 (2012)
7. Choi, H.-G., Han, S.J.: Load balancing routing for wireless mesh networks: an adap-
tive partitioning approach. In: Consumer Communications and Networking Confer-
ence, CCNC 2008, pp. 1–5 (2008)
8. He, B., Sun, D., Agrawal, D.P.: Diffusion based distributed internet gateway load
balancing in a wireless mesh network. In: Global Telecommunications Conference
2009, pp. 1–6 (2009)
9. Ma, L., Denko, M.K.: A routing metric for load-balancing in wireless mesh net-
works. In: 21st International Conference on Advanced Information Networking and
Applications Workshops (AINAW 2007) (2007)
 6LoWSD: A Protocol Platform
for Interoperability Between SDN
and IoT

Wanbanker Khongbuh(B) and Goutam Saha

North Eastern Hill University, Shillong, Meghalaya, India

[email protected]
https://nehu.ac.in

Abstract. With advent of smart cities, the IoT infrastructure is expand-

ing in an exponential manner. This emerging infrastructure is going to
produce a huge amount of data to be processed. Besides the data process-
ing bottleneck, energy efficiency, security, interoperability and scalability
of the network standard poses another problem to the present Internet of
Things (IoT) architecture. So this become an interesting research issue
which aims to improve the present IoT architecture by incorporating
Software Defined Network (SDN) system in it. SDN, a relatively new
paradigm has the capability of resolving the mentioned complexities. If
SDN can be incorporated in traditional IoT architecture this will help
to resolve many IoT problems. Now the research issue becomes as how
to design an effective protocol which can bridge between the traditional
IoT and SDN system for effective interoperability. In SDN, control oper-
ation are designated in the middle-ware layer, where SDN controllers
are placed. Controllers maintains the operation of the whole network. In
this paper, an investigation has been made to design a protocol, termed
as 6LoWSD protocol, which has the ability to integrate the two sys-
tems. Required experimentation have been carried out on simulation to
ascertain the effectiveness of the proposed protocol. Here, 6LoWSD was
simulated on Contiki platform. 6LoWPAN has been used for the IoT
system and OpenFlow for the SDN system. The experimental results
displayed quite satisfactory outcomes.

Keywords: Internet of Things · Software Defined Network ·

OpenFlow · 6LoWPAN · 6LoWSD

1 Introduction
The Internet of Things (IoT) is refer to as the Internet of Everything or the
Industrial Internet. This is a new technology paradigm, envisioned as a global
network of machines and devices that are capable of interacting with each other.
The IoT is recognized as one of the most important emerging areas of the future
technology and it succeeds in getting attention from a wide range of indus-
tries. The real utility of IoT for enterprises can be fully realized when connected
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 273–287, 2020.
https://doi.org/10.1007/978-981-15-3666-3_23
274 W. Khongbuh and G. Saha

devices are able to communicate with each other and integrate with respective
vendor-managed inventory systems, customer support systems, business intelli-
gence applications, and business analytic [1].
The Three Layer Architecture of the IoT system is described in the bottom
to top hierarchy as defined below:

1. The Perception Layer is the physical layer consisting of devices, which

has sensors and actuators for sensing and gathering information about the
environment.
2. The Network Layer termed as the Transmission Layer is responsible for con-
necting to the network as well as to other devices.
3. The Application Layer is responsible for delivering application specific ser-
vices to the users.

Fig. 1. Three layer architecture of IoT [2]

Figure 1 represents a conventional IoT architecture [2,3]. Through IoT appli-

cation are very much beneficial for the society but its design implementation on
the real world still faces some challenges. Some of these challenges are energy
efficiency, scalability, interoperability, reliability and security [4,5]. These chal-
lenges cannot be solved by using the traditional IoT system because it is resource
as well as energy constrained in nature. So, significant work have been carried
out to try to integrate IoT with other technologies such as SDN and Fog Com-
puting etc. to meet the existing IoT challenges. Here, we have focused on the
interoperability issue of IoT with SDN. This paper emphasize on incorporating
SDN with IoT which can provide be a viable solution for IoT shortcomings.
IoT networks are less flexible, hardware dependent which are often propri-
etary item [6] or devices. SDN is an emerging open source network system which
decouples the control plane from its data plane. The sensor nodes are present
in the data plane which are devices responsible for forwarding the data to the
control plane for further processing. The controller on the other hand plays the
main device for controlling the entire network.
 6LoWSD: A Protocol Platform for Interoperability Between SDN and IoT 275

Fig. 2. Three layer architecture of SDN [7–9]

The SDN architecture is shown in Fig. 2. It consists of [7–9]:

1. Data Plane: Data Plane consists of networking elements (switch, router, etc).
The data plane switches maintains a flowtable for the traversal of the data.
This flowtable is configured by the Controller via SouthBound Interface (SBI).
One such SBI is the OpenFlow protocol.
2. Control Plane: Forwarding devices are scheduled by the Controller. The con-
troller is a decision making entity which governs the overall operation of the
network.
3. Application Plane: Network services can be provided to the user via this
plane. The communication between Control Plane and Application is through
NorthBound Interface (NBI).

The SDN based IoT will be able to manage the network more efficiently [6].
Absence of an intermediate protocol for bridging the two technologies came as
a disadvantage. In this light, a new research initiative has been introduced in
this paper, where SDN has been integrated with IoT infrastructure. Further to
encounter the interoperability issue between the two, a new protocol 6LoWSD
has been tried. Some of the notable protocols of IoT are ZigBee, 6LoWPAN,
etc and that of SDN includes OpenFlow, Border Gateway Protocol (BGP), etc.
In this context we have used the IoT protocol - 6LoWPAN with that of SDN
protocol - OpenFlow for experimental purpose. For this purpose, an integrated
communication protocol 6LoWSD has been proposed in this work. This proposed
protocol combines the features of 6LoWPAN and OpenFlow protocol to improve
the interoperability constraint.
The contents of the paper are organized as follows: Sect. 2 explains features
of IoT protocol - 6LoWPAN and SDN protocol - OpenFlow. The Challenging
interoperability issue is discussed in Sect. 3. The literature review is given in
Sect. 4. The proposed 6LoWSD protocol is presented in Sect. 5. Experimentation
and results are discussed in Sect. 6. Finally, Sect. 7 concludes the paper.
276 W. Khongbuh and G. Saha

2 Protocol Integration of IoT and SDN

The fundamental concept for integrating SDN and IoT operates on the principle
of combining OpenFlow and 6LoWPAN. OpenFlow provides a programmable
interface which helps the user to operate the network efficiently. 6LoWPAN
provides an end-to-end connectivity with mechanism to carry data over IEEE
802.15.4 and other networks. This section explains the basics of OpenFlow and
6LoWPAN.

2.1 IoT Protocol

6LoWPAN (IPv6 over Low-Power Wireless Personal Area Networks) is an open
standard IoT architecture. It is defined as per RFC 6282 formulated by the
Internet Engineering Task Force (IETF). 6LoWPAN is also defined in the IEEE
802.15.4 standard. It is a network technology based upon an adaptation layer
[10], where IPv6 packets are allowed to be executed efficiently in a small link
layer frames. 6LoWPAN provides a Low-power, IP-driven nodes connected in
star or mesh topology.

Fig. 3. The 6LoWPAN architecture [11]

A LoWPAN architecture consist of 6LoWPAN Nodes sharing a common IPv6

address prefix, implying that the address remains the same [11]. Low-powered
wireless area networks (LoWPANs) are the essential components of 6LoWPAN
IoT architecture. This architecture is presented in Fig. 3. Three different types
of LoWPAN format are addressed. Firstly, simple LoWPANs which is connected
through one LoWPAN Edge Router to another IP network. Secondly, extended
LoWPANs which consists of multiple LoWPANs along with a backbone link
(e.g. Ethernet) interconnecting them. Thirdly, ad-hoc LoWPANs which is an
infrastructure-less architecture and it does not require Internet connectivity.
The Fig. 4 shows the 6LoWPAN protocol stack that supports IPv6, for which
a small adaptation layer (called the LoWPAN adaptation layer) has been defined
to optimize IPv6 over IEEE 802.15.4. The transport protocol of 6LoWPAN uses
 6LoWSD: A Protocol Platform for Interoperability Between SDN and IoT 277

‘User Datagram Protocol (UDP)’, which can also be compressed using the LoW-
PAN format. The ‘Internet control message protocol version 6 (ICMPv6)’ is used
to control messaging process. Adaptation between IPv6 and LoWPAN format is
done by routers at the edge of the 6LoWPAN, referred to as edge routers. This
transformation is transparent, efficient and stateless in both directions [12].

Fig. 4. 6LoWPAN protocol stack [11]

2.2 SDN Protocol

OpenFlow is an open source protocol that helps in the implementation of the

SDN concept in both software and hardware. It is a flow-oriented protocol that
contains many switches and ports to control the data flow. In a SDN system,
the controller is responsible for managing the matter of traffic control arising
out from a collection of switches [9]. OpenFlow architecture is shown in Fig. 5.
An OpenFlow protocol can manipulate switch configuration and manage certain
switch events as well. It is possible to have multiple flow-table, a group table and
an OpenFlow channel using an OpenFlow switch. The flowtable consist of flow
entries which are rules for determining the actions needed to be initiated. The
OpenFlow switch use this flowtable to communicates with the controller. The
flow entries can be configured in the group table as well. OpenFlow switches are
connected together using OpenFlow ports [13].
OpenFlow Protocol stack has several stages. State machine supervises the
operations of the system for example delivery, flow control and capability dis-
covery. The core of the given OpenFlow protocol stack is the message layer.
It helps to maintain valid structure and semantics of messages. The system
interface guides the protocol to interact with the outside network. Most of the
protocol aspects will have configuration values associated with them [15]. Here,
the configuration stands for anything that can cover from default buffer sizes to
the intervals of reply as shown in Fig. 6. The OpenFlow-enable SDN devices is
shown in Fig. 7 depicting the OpenFlow concept in SDN.
278 W. Khongbuh and G. Saha

Fig. 5. OpenFlow architecture [14] Fig. 6. OpenFlow protocol [14]

Fig. 7. OpenFlow-enabled SDN devices [15]

3 Challenges Related to Interoperability Issue

Interoperability plays a significant role in designing and building an interac-
tion between multi platform devices. The interoperability of the IoT devices
can be solved with the concepts of SDN by overcoming the heterogeneity of the
devices. Interoperability issues in IoT arises when we have heterogeneous devices
exchanging data in different formats. The diversification of protocols used for
machine to machine (M2M) interactions adds to the interoperability issue. The
inter-connectivity of large number of different devices and their transmission
cooperation can also hinder the performance of the network as well. The two
paradigm - SDN and IoT, have different architecture complying to their struc-
ture. Their interoperability places a major challenging issue for the interaction
of the two systems. This therefore needs to be resolved efficiently. Thus, this
becomes a challenge to the researchers as how to make a common platform for
different devices to operate [4]. In our approach we consider the aid of Open-
Flow protocol to be incorporated along with the IoT 802.15.4 standard sensors,
thereby any new information it gets from the network, it is sent to the controller,
so that it can decide what to do with the new incoming application or device.
It is easier for the controller of the SDN to handle such situation as it is not
 6LoWSD: A Protocol Platform for Interoperability Between SDN and IoT 279

constrained by energy and resources [7]. With the SDN approach, flexibility can
be achieved as to allow different objects connected to heterogeneous networks to
allow communication with each other.

4 Literature Review
IoT and SDN are the emerging field in the networking domain. Many experts
proposed improved SDN based IoT architecture and with some impasses on pro-
tocol bridging between the two. Some of the most prominent works are described
in this section which can effectively solve the challenging issue between the two.
‘SD-WISE’ (Software-Defined WIreless SEnsor network) [16] focuses their
attention on the two main problems: network flexibility and security. SD-WISE
extends the SDN approach to WSN by introducing a more flexible way to define
flows as well as the possibility to control the duty cycles of the node radios to
increase energy efficiency.
‘SD6WSN’ (software-defined 6LoWPAN wireless sensor network) architec-
ture developed by Miguel et al. [17] aims to control the routing contention of the
data traffic in 6LoWPAN according to the SDN approach. It takes into account
the specific resource constraint WSN issues such as low data transfer rate, high
latency and low processing power. By incorporating the flow-based forwarding
flexibility provided by SDN, a development of specific networking applications
based on a unified view has be achieved.
de Oliveira et al. [18] defined ‘Spotled’ an architecture with distributed and
hierarchical controller. The local controllers use local information to reply to
nodes in its domain and a global controller maintain the whole network state.
This architecture is able to minimize the impact of the control traffic throughput
introduced by the SDN paradigm in WSN, thus reducing the data flow rate
installation on the network.
‘SDN-WISE’ [19] is another SDN solution for WIreless SEnsor networks. It is
a stateless based protocol. SDN-WISE aims to reduce the amount of information
exchanged between the SDN network controller and sensor nodes. SDN-WISE
offers APIs that allow software developers to implement the SDN Controller pro-
grammatically. This increases flexibility and simplicity in network programming.
‘SDIoT’ [20] is another comprehensive SDN based framework model. It aims
to simplifies the IoT management process found in traditional IoT. The archi-
tecture allows to forward, store, and secure the produced data from the IoT
objects. It integrates the SDN, software defined storage, and software defined
security into one software defined based control model. It is less efficient then
SDN-WISE as it merge every entities of the model into one single entity. The
failure of which can degrade the overall performance.
The proposed ‘TinySDN’ architecture [21] discusses about a multiple
controller TinyOS-based architecture and hardware independent systems for
software-defined wireless sensor network nodes. TinySDN provides flexibility in
communication. It introduced memory overhead that does not hinder other fea-
tures related to the WSN application. It suffers from the drawback of delay in
forwarding packets from the control plane level.
280 W. Khongbuh and G. Saha

Han and Ren [22] have used OpenFlow protocol in Wireless Sensor Network
(WSN). The centralized concept of SDN implies by having a master node (con-
troller) programmed by NOX as Network Operating System (NOS) and a centre
nodes (OpenFlow switches). In this work, some solution for WSNs as per with
the OpenFlow paradigm on a traditional network is presented.
The proposed ‘Sensor OpenFlow system’ [23] has two components: an archi-
tecture featuring a clear demarcation between the control and data plane. It
is a core component that standardizes the protocol pertaining to communica-
tion between the two planes. The authors address resource under-utilization and
counter-productivity by promoting sharing of hardware resources between dif-
ferent application and reuse of implemented functionalities.
A good minimal amount of work based on the above mentioned literature deal
with SDN-IoT architecture to address the IoT challenges. The proposed protocol,
6LoWSD intends to provide a probable solution as well for a systematic network
by integrating SDN feature (SBI) into IoT ecosystem. This new collaboration
of 6LoWPAN with OpenFlow architecture helps to resolve the interoperability
constraint of IoT devices. Other issues like power management, scalability and
reliability of the existing IoT devices can be mitigated to some extend using the
proposed 6LoWSD Protocol.

5 An Approach to 6LoWSD Protocol

Internet of things (IoT) uses a low-rate wireless personal area network with a
short communication range and within limited resource. Software Defined Net-
work (SDN) has a centralized controller managing the devices of its domain and
maintaining the whole network view. With time the wireless network is expand-
ing exponentially with its devices embedded with various heterogeneous compo-
nents. The IoT system in this respect is facing several challenges with respect to
management, interoperability and power management. In this context our cur-
rent network architecture approach we have integrated the SDN infrastructure
features with the IoT network system. In this regard our main purpose lies in
alleviating the interoperability challenges prevailing on the existing IoT system.
However, the challenge remains here, with the bridging of the different architec-
tures protocols: 6LoWPAN and OpenFlow. Our objective endeavour is therefore
directed as how to manage interoperability between 6LoWPAN and OpenFlow.
The Fig. 8 represents the abstract of the proposed 6LoWSD protocol in block
diagram.
The proposed 6LoWSD protocol operates on the propriety 802.15.4 standard
at the Physical and the Link Layer. The Network Layer interfaces 6LoWPAN
with the adaptation layer. The SDN concepts are used for decoupling the control
and the data plane of the devices. Initially, OpenFlow assigns each nodes as SDN
enabled sensor devices. Thereby some nodes acts as the data plane devices while
one or few others take the role of the controller. The data nodes are simply
nodes based on flowrules and meant for sending data to the controller. The
controller nodes maintains the functionality and the controlling management
 6LoWSD: A Protocol Platform for Interoperability Between SDN and IoT 281

Fig. 8. Proposed 6LoWSD protocol merging SDN and IoT system

Fig. 9. Proposed protocol flow-table flowchart

of the whole network-view. It provides a mean to interact with the network

data plane elements by sending beacons or acknowledgment on the flowrules.
The protocol operates from SBI (South Bound Interface) frames towards the
network configuration module. Figure 9 shows the basic concepts of 6LoWSD
using flowchart module.

5.1 Pseudo-code Representing the Proposed 6LoWSD Protocol

Basic Algorithm

Input: Sensor Data Packets

Output: Compatible SDN-IoT Packets
Initialization of the node’s address:
Deployment initialization: Broadcast or Unicast.
Loop: Sensor nodes extracts the data.
Nodes and Flow-Table rules Configuration setup.
if The received packet matches the Flow-Table rules then
Perform the actions
if Flow-Table rules confirms then
Forward the Packets according to the predefined rules.
282 W. Khongbuh and G. Saha

else
The Sink controller will -
Create the Entry.
Allocate an entry.
Create the new rules.
Add the new rules.
end if
else
If Packets don’t confirms and have no relevance
Drop the packet
end if
return SDN-IoT Compatible Packets

6 Experimental Setup

6.1 Experimentation

The proposed protocol has been experimented on simulation in Contiki using the
Cooja simulator. While performing experimentation for the proposed protocol
on Contiki platform, two object nodes were considered: an iotnode and a con-
troller. The object nodes comply with the sky mote configuration on Contiki.
The object nodes configuration were compiled according to the flow concept
of the OpenFlow architecture. The initial configuration of the node includes
Node Configuration, Routing tables, Packet transmission and Flow-Tables con-
figuration. The experimental attributes and parameters tends to change with
the mobility of the topology. In the Node Configuration mode, it initializes the
node with an address. It initializes the node with a packet header, a flow table
entry which defines the basic node standard configuration. Here in the Contiki
platform, the node is a sky mote, which is an MSP430-based board with an
IEEE 802.15.4-compatibility. This is equipped with a CC2420 radio chip. The
Routing Table contains the node addresses of the adjoining or the neighbour
nodes. It defines the necessary routing information for the nodes to traverse
along the topology of the network. Packet transmission contains the configura-
tion of the packets: its header contents and the necessary source and destination
configuration. In the Flow tables, rules, actions and statistics are configured.
The working principle of the proposed protocol involves routing configuration
for the low power mesh network topology. As given in the proposed algorithm, it
checks initially whether the beacon signal is broadcasted or unicasted. Then the
nodes are assigned with proper IP addresses. A packet header is created for main-
taining the packet structure of the header as “header; netId; addr; src addr; dest
addr;”. After receiving the beacon signal, the adjacent nodes will also broadcast
their Neighbour Tables, which shows their address and relative distance from
each other. The rules of the flowTable will direct further the transmission or
dropping of the packet as required.
 6LoWSD: A Protocol Platform for Interoperability Between SDN and IoT 283

6.2 Implementation of 6LoWSD on Contiki in COOJA Environment

The experiment was performed on a system with Virtual Machine whose con-
figuration is a Processor i7, Primary Memory of 6 GB, OS with Windows
(Main)/Contiki-3.0(VM) and a Secondary memory having 100 GB. The Contiki-
Cooja Configuration is specified in Table 1.

Table 1. Cooja specifications

Parameters Configuration values

Forwarding protocol UDP
Network protocol 6LoWSD (6LowPAN + Openflow)
MAC protocol 802.15.4
Encapsulation IEEE 802.15.4 Wireless PAN
Queue Tail drop
Network size 10 × 10 Sq. Cooja Simulator area
Antenna radio CC2420/Omni–Directional
Nodes and sink MSP430 (Sky Mote)
No. of Nodes 10
Deployment Random (Unicast/Broadcast)
Transmission range of nodes 10 to 300 m (Normal)/10 m (simulation)
Traffic type Burst
Size of each packets 45 bytes
Bit rate Exponential

6.3 Results
The results obtained from simulated experiments are presented. The modeling
behavior of the iotnodes and the controller have been executed in simulation.
The simulated network topology used for the experimentation is presented in
Fig. 10. Here, iotnodes are configured on the 802.15.4 standard and they interact
on the network layer with the other networks using the adaptation layer of the
LoWPAN. Accordingly, controller node 1 and 2 acts as the SDN controllers.
These controllers determine the policies and flow rules, which can be applied for
the further operation of the simulated iotnodes. The simulated controller can
be used to create an extended virtual network. As a result, the simulated and
real nodes can be fully integrated and they can interact with each other. This
will be useful for real-time test-bed networking scenario. In the experimentation
process, due to space constraints and memory limitation, the experimentation
was carried out in a limited virtual simulated environment. However, the initial
results produced during the experimentation was quite encouraging.
The results obtained are based on various test scenarios with the parameters
indicated in the Table 2.
284 W. Khongbuh and G. Saha

Fig. 10. Simulated network topology

Table 2. Results specifications: 802.15.4 -> 6LoWSD

Parameters Values
IoT nodes 8
Controller 2
Elapsed time 00:10:00
Avg. packets/sec 45
Avg. packets size 40–50 bytes
Burst rate 1.4
Total packets transmitted 55972

Fig. 11. Packets captured with 2 controller and 8 iotnodes

 6LoWSD: A Protocol Platform for Interoperability Between SDN and IoT 285

For comparison effectiveness of our proposed protocol, we consider 6LoW-

PAN as a test benchmark protocol. Our experimental results are shown in,
Figs. 11 and 12. The graphs from Fig. 11 represent the packet captured during
the simulation of our proposed protocol and Fig. 12 indicates the packets cap-
tured by 6LoWPAN as the network layer on an ad-hoc network configuration.
The experimentation was carried out on different setup conditions. Figures 11
and 12 indicates data transfer rate under different parameters as indicated in
Tables 2 and 3. From the obtained results, it can be safely inferred that our
proposed 6LoWSD Protocol has the capability of satisfactorily improving the
interoperability of SDN-OpenFlow and the IoT-6LoWPAN. It can be used as a
working protocol that functions satisfactorily as compared to normal 6LoWPAN
in the event of interoperability features.

Table 3. Results specifications: 802.15.4 -> 6LoWPAN

Parameters Values
IoT nodes 8
Elapsed time 00:10:00
Avg. packets/sec 55
Avg. packets size 56 bytes
Burst rate 0.05
Total packets transmitted 8863

Fig. 12. Packets captured with 8 Sensor nodes on 6LoWPAN

286 W. Khongbuh and G. Saha

7 Conclusion
The integration of SDN-IoT model is a very challenging task. The present
IoT network suffers from the resource constraint limitations thus compelling
researchers to develop a resource conscious system. This still remains illusive
despite many efforts by the past researchers. The introduction of the concept
of SDN in IoT system opens a path which helps in leveraging the challenges
of the resource constraint in IoT. But here the main constraint becomes inter-
operability between IoT and SDN platforms. In our proposed work, the SDN’s
OpenFlow concepts has been introduced into the IoT network protocol frame-
work. This has given birth to a new protocol 6LoWSD. The aim of this protocol
is to enhance the interoperability of these two heterogeneous platforms. The
same was experimented on simulation on Contiki using Cooja simulator. The
result obtained was much encouraging in the present limited hardware environ-
ment. The same experiment can be extrapolated in real time SDN based IoT
network also, which remains a future scope of work. The same can also be exper-
imented in dynamic mobile environment which also remain future scope of work.
The results obtained while comparing with the conventional 6LoWPAN under
the given condition, found that the proposed algorithm displayed comparable
performance with respect to interoperability.

References
1. Lee, I., Lee, K.: The Internet of Things (IoT): applications, investments, and chal-
lenges for enterprises. Bus. Horiz. 58(4), 431–440 (2015)
2. Zanella, A., Bui, N., Castellani, A., Vangelista, L., Zorzi, M.: Internet of Things
for smart cities. IEEE Internet Things J. 1(1), 22–32 (2014)
3. Silva, B.N., Khan, M., Han, K.: Internet of Things: a comprehensive review of
enabling technologies, architecture, and challenges. IETE Tech. Rev. 35(2), 205–
220 (2018)
4. Al-Fuqaha, A., Guizani, M., Mohammadi, M., Aledhari, M., Ayyash, M.: Internet
of Things: a survey on enabling technologies, protocols, and applications. IEEE
Commun. Surv. Tutor. 17(4), 2347–2376 (2015)
5. da Cruz, M.A., Rodrigues, J.J.P., Al-Muhtadi, J., Korotaev, V.V., de Albuquerque,
V.H.C.: A reference model for Internet of Things middleware. IEEE Internet Things
J. 5(2), 871–883 (2018)
6. Qin, Z., Denker, G., Giannelli, C., Bellavista, P., Venkatasubramanian, N.: A soft-
ware defined networking architecture for the Internet-of-Things. In: 2014 IEEE
Network Operations and Management Symposium (NOMS), pp. 1–9. IEEE (2014)
7. Jarraya, Y., Madi, T., Debbabi, M.: A survey and a layered taxonomy of software-
defined networking. IEEE Commun. Surv. Tutor. 16(4), 1955–1980 (2014)
8. Hu, F., Hao, Q., Bao, K.: A survey on software-defined network and openflow: from
concept to implementation. IEEE Commun. Surv. Tutor. 16(4), 2181–2206 (2014)
9. Kobo, H.I., Abu-Mahfouz, A.M., Hancke, G.P.: A survey on software-defined wire-
less sensor networks: challenges and design requirements. IEEE Access 5, 1872–
1899 (2017)
 6LoWSD: A Protocol Platform for Interoperability Between SDN and IoT 287

10. Thubert, P., Hui, J.W.: Compression format for IPv6 datagrams over IEEE
802.15.4 - based networks. https://tools.ietf.org/html/rfc6282. Accessed 18 May
2019
11. Shelby, Z., Bormann, C.: 6LoWPAN: The Wireless Embedded Internet, vol. 43.
Wiley, Hoboken (2011)
12. Palattella, M.R., et al.: Standardized protocol stack for the Internet of (important)
Things. IEEE Commun. Surv. Tutor. 15(3), 1389–1406 (2012)
13. Baktir, A.C., Ozgovde, A., Ersoy, C.: How can edge computing benefit from
software-defined networking: a survey, use cases, and future directions. IEEE Com-
mun. Surv. Tutor. 19(4), 2359–2391 (2017)
14. Flowgrammable: SDN Protocol Decomposition. http://flowgrammable.org/sdn/
openflow/. Accessed 10 March 2019
15. Kreutz, D., Ramos, F., Verissimo, P.E., Rothenberg, C.E., Azodolmolky, S., Uhlig,
S.: Software-defined networking: a comprehensive survey. Proc. IEEE 103(1), 14–
76 (2015)
16. Anadiotis, A.-C., Galluccio, L., Milardo, S., Morabito, G., Palazzo, S.: SD-WISE:
a software-defined wireless sensor network. Comput. Netw. 159, 84–95 (2019)
17. Miguel, M., Jamhour, E., Pellenz, M., Penna, M.: SDN architecture for 6LoWPAN
wireless sensor networks. Sensors 18(11), 3738 (2018)
18. de Oliveira, B.T., Margi, C.B.: Distributed control plane architecture for software-
defined wireless sensor networks. In: 2016 IEEE International Symposium on Con-
sumer Electronics (ISCE), pp. 85–86. IEEE (2016)
19. Galluccio, L., Milardo, S., Morabito, G., Palazzo, S.: SDN-wise: design, prototyping
and experimentation of a stateful SDN solution for wireless sensor networks. In:
2015 IEEE Conference on Computer Communications (INFOCOM), pp. 513–521.
IEEE (2015)
20. Jararweh, Y., Al-Ayyoub, M., Benkhelifa, E., Vouk, M., Rindos, A., et al.: SDIoT: a
software defined based Internet of Things framework. J. Ambient Intell. Humaniz.
Comput. 6(4), 453–461 (2015)
21. De Oliveira, B.T., Gabriel, L.B., Margi, C.B.: TinySDN: enabling multiple con-
trollers for software-defined wireless sensor networks. IEEE Lat. Am. Trans.
13(11), 3690–3696 (2015)
22. Han, Z., Ren, W.: A novel wireless sensor networks structure based on the SDN.
Int. J. Distrib. Sens. Netw. 10(3), 874047 (2014)
23. Luo, T., Tan, H.P., Quek, T.Q.: Sensor OpenFlow: enabling software-defined wire-
less sensor networks. IEEE Commun. Lett. 16(11), 1896–1899 (2012)
 Machine Learning Based Efficient
Multi-copy Routing for OppIoT Networks

N. N. Srinidhi(B) , C. S. Sagar, S. Deepak Chethan, J. Shreyas,

and S. M. Dilip Kumar

Department of Computer Science and Engineering,

University Visvesvaraya College of Engineering, Bangalore, India
[email protected]

Abstract. Opportunistic networks are one of the important categories

of ad hoc networks in Internet of Things (IoT), which exploits human
social characteristics like daily routines, similarities etc., to facilitate effi-
cient communication. In opportunistic networks, mobile nodes are used
to establish communication between nodes despite of non-availability of
a dedicated route between them. Furthermore, nodes don’t acquire any
knowledge in advance about the characteristics of the network such as the
network topology and the location of the other nodes. Hence, designing
a routing algorithm becomes a challenging task since traditional rout-
ing protocols used in the Internet are not feasible for the characteris-
tics inherent type of network. The proposed work propounds a multi-
copy routing algorithm based on machine learning named iProphet or
improved Prophet (Probability routing protocol using history of encoun-
ters and transitivity). iProphet, uses dynamically changing contextual
information of nodes and the delivery probability of Prophet to carry out
message transfer. The iProphet uses machine learning classifier known
as random forest to classify the node as a good forwarder or a bad for-
warder based on the supplied contextual information during each routing
decision. The classifier trained with large amount of data extracted using
simulation leads to precise classification of the nodes as reliable or unreli-
able nodes for carrying out the routing task. The simulation results show
that the proposed algorithm outperforms with respect to delivery prob-
ability, hop count, overhead ratio, latency but over costs with respect to
average buffer time in par with similar multi-copy routing algorithms.
The uniqueness of this paper lies in data extraction, categorization and
training the model to obtain reliable and unreliable nodes to facilitate
efficient multi-copy routing in IoT communication.

Keywords: Internet of Things · Machine learning · Mobility ·

Multi-copy · Random forest

c Springer Nature Singapore Pte Ltd. 2020

A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 288–302, 2020.
https://doi.org/10.1007/978-981-15-3666-3_24
 Machine Learning Based Efficient Multi-copy Routing for OppIoT Networks 289

1 Introduction
With the development in technogies, more and more devices are getting con-
nected to the Internet, which leads to the new criterion in computing and com-
munication known as the Internet of Things (IoT) [1]. As a result, more devices
are getting prevalently monitored and cannot be deployed statically. These IoT
devices while being transported by humans or different carriers will interact
either by statically or dynamically. For instance, in vehicular based IoT network
statically deployed devices may entail to recognize the mobile agents and employ
them in assembling and delivering their information to the intended target [2].
In recent times, opportunistic mobile networks rose as a novel mechanism for
transmissions in wireless systems. Unlike mobile ad hoc networks (MANETs)
[3] that needs end-to-end connected routes for information transfer, the com-
munication in opportunistic mobile networks takes place on the corporation of
opportunistic connections among mobile nodes without the availability of end-
to-end communication paths. These mobile devices establishes connection when
people comes in contact and these network type are considered as human social
networks. Hence, the opportunistic networks utilize the individual behaviors and
social relations to provide more effective and reliable message distribution sys-
tems. In OppIoT [4], data acquisition, propagation, and distribution of data
which typically necessitate opportunistic connections utilizing mobility, short-
range transmission, and technologies. During such operations, routing protocol
configuration poses challenge due to its inherent complexity in securing connec-
tivity among devices and recognizing a relevant forwarder to forward data packet
to its desired destination. Opportunistic networks [5], which are the derived class
of OppIoT and considered as evolved form of MANETs. In OppNets, nodes are
allowed to establish communication among several different nodes even though
they don’t have routing connection. Nodes are connected momentarily and the
routes keep dynamically changing due to nodes mobility or frequent nodes acti-
vation and deactivation. When the packets are en route between the source
node and the target nodes, any viable node in between can be opportunistically
employed as the next hop whenever it is plausible to deliver the packets to the
intended destination with minimum routing distance. Several protocols related to
OppNets that have been proposed such as epidemic, HBPR, HibOp etc., which
uses parameters like message propagation pattern, context erudition of nodes
(i.e. current position, the possibility of delivery, target node distance, prior and
existing history of the node), last synergy time and various other parameters in
order to predict which node is opted as next hop to deliver the message to the
intended destination. In this work, a machine learning based algorithm which
can be employed to OppNets is proposed. The proposed algorithm that differ
radically from the existing algorithm as this algorithm uses the random forest
to decide the neighboring node to route the data packet based on the present
context and location information. Random forest or otherwise referred to as ran-
dom decision forest are one of the most popular ensemble learning techniques
that are used in both classification and regression. Several decision trees are ran-
domly created with several subsets of data during the training period. The class
290 N. N. Srinidhi et al.

is assigned based on the mode. Random forest algorithm uses the technique of
bootstrap aggregation, which helps in improving the stability and accuracy of
the algorithm along with reducing the variance and overfitting of the proposed
iProphet.
The rest of the paper is organized as follows. Section 2 presents a summary of
various existing works with respect to multi-copy and oppurtunisitic networks.
Problem statement is presented in Sect. 3. Proposed algorithm has been pre-
sented in Sect. 4. The experimental evaluation and results has been explained in
Sect. 5. Conclusion along with future work has been presented in Sect. 6.

2 Related Work
Many protocols that are used in OppNets are related to the nodes context-
information and message distribution. Representative ones are summed as fol-
lows. Epidemic routing [6], is a flood-based routing algorithm whose foremost
objective is to deliver the packet with high probability to the desired destina-
tion with a minimum premise about the topology and the network. The work
is based on epidemic algorithm which manages two buffers one for the message
originating from the node itself and the other buffer is used to store messages
generated from different nodes. Each node contains a summary vector that car-
ries a short description of messages currently cached in the buffer. When nodes
come in contact of one another, they revise their summary vectors and each
node has an extra buffer to prevent verbose connection with other nodes. Nodes
exchange their summary vector with each other after a predefined time and after
exchanging their vectors the nodes rival their vectors to ascertain which message
is missing. Then, the nodes request for the copies of the messages which they
do not contain. But the proposed epidemic protocol uses higher bandwidth and
buffer space which will cause network congestion and it do not examine nodes
parameters to predict the possibility of message delivery. The HBPR (History
Based Prediction for Routing), protocol which was proposed by Dhurandher
et al. [7], uses election of the intermediate node as the message carrier is decided
based on specific parameters. These parameters are the duration for which a
pair of nodes meet, node’s history, time at which a couple of nodes meet, and
the path of the mobility of the node concerning the root and the target node.
The schema consists of three stages, i.e. identifying the home location, wherein
the nodes movement is utilized to recognize the subsequent mode location; the
message production stage, during which the intermediate node is determined,
and also the home location is renewed and lastly, the subsequent hop selection
phase, where metric is determined to help to judge on the assortment of the best
succeeding message forwarder. This protocol has achieved improved results when
compared with Epidemic concerning the message delivery number and overhead
ratio, but only after considering the individual mobility paradigm. Prophet rout-
ing protocol [8], uses non-arbitrary movement and association patterns in prac-
tical application situation to replicate the message to different nodes in order to
enhance the routeing performance. It is established on the basis that if a node
has communicated with a different node or visited a region regularly, then there
 Machine Learning Based Efficient Multi-copy Routing for OppIoT Networks 291

is a higher likelihood of revisiting the region or the node. In order to accomplish

this, a delivery probability is defined for every node which associates itself with
other nodes. Let assume that a node T wants to remit a packet to node U if this
node T comes in association with another node R which has a higher delivery
probability to node U compared to T only then a copy of the packet is copied
to the node R. The deliverability is established based on the buffer size aggre-
gate, position, popularity and predictability of delivery. If a source node comes
in association concurrently with various nodes the message is copied to the node
which has the highest value of deliverability. In distance-based Prophet, routing
algorithm distance is used as an added factor. Here each node containing the
packet checks the distance from all the bordering nodes and picks the nodes with
is closed as the packet forwarder. Thereby improving the probability of delivery
and lessening the delay probability. Prowait routing algorithm which was pro-
posed by Dhurandher et al. [9], can be considered as an amalgamation of the
Prophet and Spray and wait routing protocol. Prowait model utilities an uncom-
plicated forwarding schema, in which when two nodes engage, the transmission
of packet happens from the node with lower probability of delivery to the node
have higher probability concerning the desired destination. They mainly weigh
a couple of parameters during packet transmission (1) election of an adjacent
node (2) choosing the node for the next hop. For the election of the node, the
Prophet protocol is used to estimate delivery predictability and transferring of
the data is done using the spray and wait routing protocol. Choosing the node
for the next hop is done using the Prophet routing algorithm. Prowait routing
algorithm keeps the replicate data packets so that the packet is delivered to
the intended destination with minimum delay. It reduces the usage of resource
wastage by depreciating the amount of flooding compared to the epidemic pro-
tocol. It, however, does not predict forwarding of the message packet in order to
improve delivery delay in the network.

3 Problem Statement and Challenges of the Proposed

Work
In this section challenges and the problem statement of the proposed work is
explained.

3.1 Challenges
There are various challenges that opportunistic network in IoT faces. This is due
to the dynamic nature of nodes over time depending on the application, which
makes it difficult for routing the information and to identify the intermediate
nodes which are used for forwarding messages to destination.
– Data extraction: For training the machine learning based random forest
model, a huge data set has to be obtained. To obtain data that is close to the
real world was a challenge. ONE simulator [10], provides the closest solution
and to extract the data from the ONE simulator, the source code has to be
modified in order to train the model.
292 N. N. Srinidhi et al.

– Model Creation: After extracting the data from ONE simulator, a robust
random forest classifier had to be constructed. In order to do so, features and
algorithm related parameters has to be chosen wisely.
– Model Integration: After the model is constructed, it has to be properly inte-
grated with the ONE simulator in order to classify the nodes.

3.2 Problem Statement

The problem statement of the proposed work is as follows.
1. To classify the node as a reliable forwarder or a unreliable forwarder based
on the supplied contextual information during each routing decision in order
to optimize delivery probability, hop count, overhead ratio and latency of the
IoT communication.
2. To train the random forest classifier with large amount of data extracted from
the simulator to precise the classification of the nodes as reliable or unreliable
node for carrying out the routing information.

4 Proposed Algorithm
This section provides in detail explanation of the proposed iProphet algorithm.

4.1 Terminology
– S: Source node
– D: Destination node
– X: Selected forwarder node
– G: Set of all neighboring nodes
– Ni : Neighboring node i in the set S
– Mi : Message
– n: Current node, tasked with making a routing decision
– M : Set of all messages that are generated during data generation phase
– Ms : Set of messages that are delivered
– Md : Set of messages that are dropped or aborted
– E: Set of all the events recorded during data extraction phase of Prophet
– P (Ni ): Delivery probability as calculated by Prophet for the neighbor node
Ni using current node n
– f eature name(x): Value of the attribute feature name in the record x in any
set.

Consider N nodes forming an opportunistic network with a means to commu-

nicate and having sufficient energy. Every node has sufficient buffer to store the
state information like the nodes movement information, current location, deliv-
ery probability, direction, distance between source and destination, past history
and power status. All these attributes can be used to determine the route for
the message transfer. Prophet is one of the algorithm that uses previous contact
 Machine Learning Based Efficient Multi-copy Routing for OppIoT Networks 293

history between the nodes to calculate the delivery probability between them.
Even though this algorithm provides a sufficient performance, the method to
select the next node could be optimized by applying machine learning models.
Prophet does not completely utilize all the context information that are present
in the nodes. With the data driven approach, it is possible to select best pre-
dictors/features and use them to drive the routing decisions. In the proposed
Random Forest Based Optimized Multi-Copy Routing for Opportunistic IoT
Networks, random forest algorithm is used to choose the neighboring node to
route the data packet based on the present context and location information.
Random forest are one of the most popular ensemble learning techniques that are
used for both classification and regression. Several decision trees are randomly
constructed with varied subsets of data during the training period. The class
is assigned based on the mode. Random forest algorithm uses the technique of
bootstrap aggregation, which helps in improving the stability and accuracy of the
algorithm along with reducing the variance and overfitting. Formally, consider a
training set X = x1 , ..., xn along with their corresponding outputs Y = y1 , ..., yn
giving dataset Dn = (X1 , Y1 ), ..., (Xn , Yn ). Random forest is a collection of ran-
dom base regression trees {rn(x, θm , Dn ), m ≥ 1}, where θ1 , θ2 , ... are outputs of
a randomizing variable θ. The outputs of these random trees are aggregated to
get an estimate. Thus, random forest model is given by,

rn(X,
¯ Dn ) = Eθ [rn(X, θ, Dn )] (1)

Where Eθ is the expectation with respect to θ. The dataset is divided into

various subspaces based on this randomizing variable theta. It is also used to find
out on what basis the successive cuts are done when constructing the individual
trees. During the learning period, the algorithm will choose a random subset of
features, called as feature bagging. If a given feature is a strong predictor, it is
used repeatedly in several different trees. To find the optimal number of trees,
cross-validation techniques is applied. The main advantage of Random forest over
normal decision trees is that the predictions from single tree are susceptible to
noise. But the aggregation of forest will overcome from this drawback. Random
forest algorithm can also be viewed as weighted neighborhood schemes.

4.2 Usage of Random Forest in Proposed Algorithm

Random Forest is one of the most popular machine learning algorithms for
both classification and regression. The existing Prophet algorithm does not take
advantage of the available past data, it only uses the current context information
in its static model. Thus, it could be made more robust by integrating machine
learning capabilities to it. Random forest uses the ensemble learning technique.
By constructing several trees and taking an aggregate of them would only make
the model robust. Apart from this, it also provides better classification since it
uses boosting and bagging techniques ands helps in avoiding overfitting com-
pared to their counterparts. Random forest is better suited for this particular
application because it can work on multidimensional input data and the data
294 N. N. Srinidhi et al.

Decission Tree 1 Decission Tree 2 to n

Random forest

current node and

selected_neighbour
first node
quadrant fourth
second third quadrant
quadrant quadrant
destination destination destination destination
location location location location
first, second,
fourth quad rest

buffer size label = “dont Similar strategy

deliver” (0) as in the left
Similar to decission
full
optimum tree 1 but spllts
label = “dont on different set
deliver” (0) of attributes and
thresholds
movement speed > thresold

yes no final label for trees

2 to n = { 0|1, 0|1,
next time to move < others neighbours speed > .......0|1}
thresold movement speed

yes no no yes

PRoPHET prob PRoPHET prob PRoPHET prob PRoPHET prob

> thresold 1 > thresold 2 > thresold 3 > thresold 4
yes yes
yes
yes
no no

label =“deliver” (1) no label = “dont

deliver” (0) no

final label for tree Final label = majority label

1 = 0 | 1 across all decission tree

Fig. 1. Random forest architecture

used to train the model has a dimension. In an opportunistic network environ-

ment, each and every node contains the context and location information of its
neighbors. This contextual information is used with the proposed model to make
predictions.
Figure 1 shows the random forest architecture used to segregate reliable and
unreliable nodes while making each routing decision. The architecture consists of
n number of decision trees which are the natural choice of base models/learners
in the random forest technique. Bootstrap aggregation, also known as bagging
is then used to combine the results of these base models to produce the final
results. Majority vote is the combining technique used in classification tasks
which produces the output/label as the one that has been predicted by most
of the base learners. In this figure, the decision tree has been expanded for one
among the n decision trees used in the random forest architecture. Each base
decision tree is trained using random sample of inputs with replacements and
different set of features to provide overall effect of combining the results leads
to reduced variance in the final output.
 Machine Learning Based Efficient Multi-copy Routing for OppIoT Networks 295

In the decision tree 1, the first node considers the location co-ordinates of
the current node, the selected neighbor node and makes the decision based on
the quadrant of selected neighbor with respect to current node. Selected neighbor
node is the one that the current node has established the connection with and
the current node needs to make decision about the reliability while transferring
it to the neighbor node. For each quadrant, the next decision node considers the
location co-ordinates of the destination to proceed to the next decision node. If
the Selected neighbor node belong to the first quadrant, then the destinations
lying in the first, second, and fourth quadrant are considered as valid, the Selected
neighbor node could eventually move and therefore allowed to next level of the
decision tree. Similarly, for neighbors in other quadrants, i.e. quadrants of the
destination nodes surrounding the neighbor quadrants are considered as valid.
For e.g, for second quadrant, quadrant 1, 2, 3 are the surrounding quadrants
and quadrant 2, 3, 4 are for third quadrant. If the destination doesn’t lie in the
surrounding quadrant, then the message with that destination is not transferred
to the Selected neighbor node. The next decision node considers the buffer size
of the Selected neighbor node node. The nodes with optimum level of free buffer
size in order to accommodate the incoming message are proceeded down the
tree, where as the nodes with lesser space will be discarded (i.e. the message is
not forwarded). The next decision is based on the criteria of movement speed
of the Selected neighbor node node. The movement speed is compared against
the threshold as calculated by the decision tree and is allowed to proceed along
different paths. If the movement speed is less than the threshold, the nodes are
not immediately treated as unreliable as shown in the decision nodes. Instead,
the decision nodes will make comparison of the feature value across various
other neighbors feature values to make appropriate decision regarding reliability.
Finally, the Selected neighbor node PRoPHET values meeting the threshold set
by their respective groups is considered for final message transfer. This procedure
is repeated for other base learners with different set of features and random
sampling of the data.
For a given node, extracted data about one neighbor at a time will be fed into
trained model. Based on the features, the model classifies the neighbors as good
forwarder or bad forwarder. After going through all the neighbors, data packets
are forwarded to all of the good forwarders as proposed in the Algorithm 1.
Since it is an improvement over the Prophet algorithm, the Prophet probability
is also considered one of the feature for classification. In order to see how the
model performed in real world, the source code of the Prophet algorithm in ONE
simulator was modified to include the trained model.

5 Experimental Evaluation

In this section, environment considered for simulating proposed work along with
discussion about results obtained will be explained.
296 N. N. Srinidhi et al.

Begin;
// Original Prophet
foreach node Ni as i do
dp[i] <– random()
end
for every 5 seconds do
foreach node Ni as i do
dp[i] <– GetP rophetP robabilty(i);
end
end
// End of original Prophet
foreach message m in current node n buffer do
foreach node Ni as n in neighborhood set G do
P(n) <– dp[n];
if dp[n] ≥ lP (Ni ) then
if Ni == X then
update X;
update G;
Label = RandomF orest1(X, G, P (X), P (n));
end
if Label == “deliver” then
Relay the message to node X;
else
if Label == “don’t deliver” then
Don’t relay the message to the node X;
end
end
end
if dp[n] < lP (Ni ) then
if Ni == X then
update X;
update G;
Label = RandomF orest2(X, G, P (X), P (n));
end
if Label == “deliver” then
Relay the message to node X;
else
if Label == “don’t deliver” then
Don’t relay the message to the node X;
end
end
end
end
end
Algorithm 1. Improved Prophet algorithm
 Machine Learning Based Efficient Multi-copy Routing for OppIoT Networks 297

5.1 Simulation Settings

In this section, the performance of the proposed iProphet is evaluated using the
Opportunistic Network Environment (ONE) [10] simulator. The simulation is
done on a terrain size of 4,500 × 3,400 m2 with 51 to 198 nodes deployed ran-
domly. The transmission range of the node is set to 20 m. The various parameters
considered during simulation are shown in Table 1. Here the proposed iProphet
performance has been compared with Epidemic [6], Prophet [8] and Spray and
Wait [11] routing protocols, under varying TTL values from 100 to 300 s, number
of nodes from 51 to 198, and message creation interval from 25 to 35 s.

Table 1. Simulation parameters

Parameter Value
Simulation area (4,500 × 3,400) m2
Total number of nodes 198
Total nodes group 6
Pedestrains group 3
Nodes number in each pedestrian group 30
Pedestrian walking speed 0.5–1.5 km/h
Pedestrian buffer size 15 Mb
Tram groups 3
Nodes number in each tram group 2
Tram speed 6.5 Km/h
Tram buffer size 59 Mb
Transmission speed of bluetooth 250 K
Transmission range of bluetooth 20 m
High speed interface transmission speed 10 Mb
High speed interface transmission range 1500 m
Time-to-live for message in each group 100 min
Message generation intervals 25–35 s
Size of message 500 Kb–1 Mb
Mobility model Shortage path map based movement model
Simulation time 100000 s

5.2 Simulation Results and Performance Analysis

In this section, the performance of the proposed algorithm with respect to deliv-
ery probability, buffer time, hop count, overhead ratio and latency is evaluated
with similar existing algorithm.
Figure 2 compares the delivery probability of various algorithms when the
TTL is varied accordingly. It can be observed that delivery probability of algo-
rithms other than Epidemic and iPRoPHET decreases with the increase in TTL.
298 N. N. Srinidhi et al.

Fig. 2. Comparison of the delivery probability vs TTL.

This is due to the fact the lifetime of messages increases with the increase in
TTL and hence the messages remain in the node buffer for longer time than
usual which results in more messages getting dropped. However, in iPRoPHET
algorithm, the delivery probability increases because of the selection of the next
forwarder using random forest classifier which considers PRoPHET probability
and other contextual information of the forwarders. The next forwarders are
selected not only when their PRoPHET probability is greater but also when the
probability is lower which helps to clear the messages from the buffer as quickly
as possible.

Fig. 3. Comparison of the hop count vs number of nodes.

 Machine Learning Based Efficient Multi-copy Routing for OppIoT Networks 299

Figure 3 compares the Hop Count of various algorithms when the Number
of Nodes is varied accordingly. When the number of nodes is increased, it leads
to corresponding increase in number of forwarders available for routing the mes-
sages. Among these forwarders, the random forest classifier picks the fit nodes
accurately leading to lesser overall hop count of the proposed work.

Fig. 4. Comparison of the average latency vs number of nodes.

Figure 4 compares the average latency of various algorithms when the

Number of Nodes, TTL and Message Generation Interval is varied accord-
ingly. iPRoPHET algorithm has only slightly lesser average latency time than
PRoPHET because of the selection of the nodes with lesser PRoPHET delivery
probability which is required to maximize the overall delivery probability but
in doing that, the algorithm may select few outliers that might keep the mes-
sages for far longer time than usual and as number of nodes, the outlier count
increases which increases the message delivery latency. However, this is com-
pensated by the lesser hop count delivery of the non-outlier nodes that help to
keep the latency times in control for majority of the messages. But the major-
ity of non-outlier nodes overcome such impact with their timely delivery of the
messages to their destinations.
Figure 5 compares the overhead ratio of various algorithms when the message
generation interval is varied from 25–35 to 65–75. It is observed that the overhead
ratio is better in iProphet because of the increased delivery probability resulting
from better forwarders chosen as part of random forest classifier.
Figure 6 compares the average buffer time of various algorithms when the
number of nodes and TTL is varied. iPRoPHET algorithm has slightly greater
average buffer time because of the selection of the nodes with lesser PRoPHET
delivery probability which is required to maximize the overall delivery probability
but in doing that, the algorithm may select few outliers that might keep the
300 N. N. Srinidhi et al.

Fig. 5. Comparison of the overhead ratio vs message generation interval.

Fig. 6. Comparison of the average buffer time vs number of nodes.

messages for far longer time than usual and as number of nodes increases, the
outlier count increases keeping the average buffer time slightly larger and outlier
prone.

6 Conclusions and Future Work

In this paper multi-copy routing algorithm for opportunistic networks named
iProphet or improved Prophet is proposed. This method makes use of a machine
learning classifier called random forest and delivery probability of existing
Prophet to establish routing of messages. With the simulation results it can be
 Machine Learning Based Efficient Multi-copy Routing for OppIoT Networks 301

concluded that: (i) While making each routing decision, random forest classifies
the nodes as reliable or unreliable which led to better selection of intermediate
nodes to carry out efficient routing. (ii) Proposed method outperforms in terms
of delivery probability, hop count, overhead ratio and latency in comparison with
similar existing algorithms. (iii) The classifier trained with large amount of data
extracted using simulation leads to precise classification of the nodes as reliable
or unreliable nodes to provide efficient communication. However the proposed
method will have poor performance in terms of average buffer time in par with
similar multi-copy routing algorithm.
The average buffer time has experienced a performance issue due to the
presence of the outlier nodes, which needs to be addressed as part of future
enhancement. Furthermore, energy consumption of the nodes needs to be opti-
mized for working with compute-intensive machine learning algorithms in the
future work.

Acknowledgment. This research work has been funded by the Science and Engi-
neering Research Board (SERB-DST) Project File No: EEQ/2017/000681. Authors
sincerely thank SERB-DST for intellectual generosity and research support provided.

References
1. Srinidhi, N.N., Lakshmi, J., Dilip Kumar, S.M.: Hybrid energy efficient and QoS
aware algorithm to prolong IoT network lifetime. In: Kumar, N., Venkatesha
Prasad, R. (eds.) UBICNET 2019. LNICST, vol. 276, pp. 80–95. Springer, Cham
(2019). https://doi.org/10.1007/978-3-030-20615-4 6
2. Srinidhi, N.N., Dilip Kumar, S.M., Venugopal, K.R.: Network optimizations in the
internet of things: a review. Eng. Sci. Technol. Int. J. 22(1), 1–21 (2019)
3. Dilip Kumar, S.M., Vijaya Kumar, B.P.: Energy-aware multicast routing in manets:
a genetic algorithm approach. Int. J. (2009)
4. Guo, B., Zhang, D., Wang, Z., Zhou, X., Yu, Z.: Opportunistic IoT: exploring
the harmonious interaction between human and the internet of things. J. Netw.
Comput. Appl. 36, 1531–1539 (2013)
5. Pelusi, L., Passarella, A., Conti, M.: Opportunistic networking: data forwarding
in disconnected mobile ad hoc networks. IEEE Commun. Mag. 44(11), 134–141
(2006)
6. Vahdat, A., Becker, D.: Epidemic routing for partially connected ad hoc networks.
Technical report CS-2000-06 (2000)
7. Dhurandher, S.K., Sharma, D.K., Woungang, I., Bhati, S.: HBPR: history based
prediction for routing in infrastructure-less opportunistic networks. In: IEEE AINA
2013, Barcelona, Spain, 25–28 March 2013 (2013)
8. Lindgren, A., Doria, A., Schelen, O.: Probabilistic routing in intermittently con-
nected networks. ACM SIGMOBILE Mob. Comput. Commun. Rev. 7(3), 19–20
(2003)
9. Dhurandher, S.K., Borah, S.J., Obaidat, M.S., Sharma, D.K., Baruah, B., Gupta,
S.: Probability-based controlled flooding in opportunistic networks. In: Proceed-
ings of International Conference on Wireless Information Networks and System
(WINSYS), Colmar, France, 20–22 July 2015, pp. 3–8 (2015)
302 N. N. Srinidhi et al.

10. Keranen, A.: Opportunistic network environment simulator. Special Assignment

report, Helsinki University of Technology, Department of Communications and
Networking (2008)
11. Spyropoulos, T., Psounis, K., Raghavendra, C.S.: Spray and wait: an efficient rout-
ing scheme for intermittently connected mobile networks. In: Proceedings of the
2005 ACM SIGCOMM Workshop on Delay-Tolerant Networking, pp. 252–259.
ACM (2005)
 An Open-Hardware Approach for IoT
Enabled Smart Meter Monitoring
and Controlling System Using
MQTT Protocol

G. M. Madhu1(B) , C. Vyjayanthi1 , and Chirag Modi2

1
Department of Electrical and Electronics Engineering,
National Institute of Technology Goa, Ponda 403-401, Goa, India
{madhugm,c.vyjayanthi}@nitgoa.ac.in
2
Department of Computer Science Engineering,
National Institute of Technology Goa, Ponda 403-401, Goa, India
[email protected]

Abstract. Smart metering and direct load control at the individual

consumer level are essential for the proper operation of smart grids. Dif-
ferent approaches have been proposed in the literature to address these
problems. However, these approaches are not cost-effective, open-source
and lack direct load control. In this paper, we propose cost-effective,
open hardware-based smart meter monitoring and controlling system to
monitor and control individual consumer using Internet of Things (IoT).
We developed a smart meter based on a Message Queuing Telemetry
Transport (MQTT) protocol, which helps to continuously monitor the
individual consumer’s consumption and to log the use at server using
publish-subscribe architecture. The proposed smart meter is capable of
controlling the supply to the consumer based on the commands from the
server with an additional feature of meter tamper detection. To monitor
consumption and billing information of the consumer, an android appli-
cation was developed. The prototype of the proposed system is built
using open-source hardware components, and various test conditions are
applied to validate the proposed method of approach.

Keywords: Android application · Energy monitoring and controlling ·

Internet of Things (IoT) · Message Queuing Telemetry Transport
(MQTT) · Smart meter

1 Introduction
In the present energy scenario, the demand for electricity is continuously increas-
ing and the global energy consumption will be tripled by the end of 2035 [1].
This work is a part of research project “Development of Intelligent Multipurpose con-
troller for Nano Grid Operation”, funded by Ministry of New & Renewable Energy
Resources, Government of India. Grant No: 24/29/2016-SWES (R&D).
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 303–317, 2020.
https://doi.org/10.1007/978-981-15-3666-3_25
304 G. M. Madhu et al.

Among these, domestic loads correspond to an average of 20–40% of the total

demand [1,2]. In order to optimize the ever-increasing load demand and to pro-
vide continuous and reliable power supply, the concept of smart grid is evolved.
It uses an advanced ICT to operate the electrical network in an optimal and reli-
able manner. With the application of smart grid technologies, substantial energy
savings can be achieved by means of utilizing Distributed Energy Sources (DES)
and energy management algorithms. However, such optimizations require energy
consumption information continuously [3]. In addition, the electricity bill gen-
eration of individual consumer is still a manual process in many regions and
it requires a large number of manpower for billing. Hence, there is a scope for
improving the present system with the application of ICT. In order to address
these issues, various metering and data logging techniques are presented for con-
sumption monitoring and billing of individual consumers [4].
Internet of Things (IoT) is proven to be the most feasible solution for data
monitoring as it offers cloud centric approach for data processing and storing
[5]. IoT is a computing concept describing ubiquitous connection to the Internet,
turning common objects into web connected devices. It can help to collect data
from sensors, analyzing the data and controlling the devices remotely [7]. A
general IOT platform consists of four layers viz; sensor, aggregator, analysis
and decision-making [8]. IOT platform can be used for real time monitoring
and controlling services. Researchers put efforts towards developing secure and
reliable solution for monitoring and billing of individual consumer. However, very
few attempts have been reported yet to control the electricity supply through
smart meters [9,10]. The existing smart metering systems lack in offering high
accuracy of measurement, less cost and reliability.
To address above problems, we propose a cost effective and reliable IoT
enabled smart meter based energy monitoring, billing and controlling system.
The proposed system uses open hardware components which reduces the cost of
the system and makes it customizable. We use LEM sensors to measure the con-
sumption more precisely. The major contributions of this work are as follows: the
proposed system is low cost and open hardware-based system for monitoring and
controlling energy. It uses light weight MQTT based communication protocol for
data exchange. The developed android application helps in real time monitoring
and billing information. It is secure, robust and has reliable design. The pro-
totype of the proposed system is tested by applying various test conditions to
validate the proposed system.
The rest of the paper is organized as follows: Sect. 2 discusses the existing
efforts made towards smart meters, which monitor and control energy followed
by the discussion on different IoT platforms. Section 3 presents the architecture
and working of the proposed system, while the prototype implementation of the
proposed system, results and analysis is given in Sect. 4. Finally, Sect. 5 concludes
our research work with the future scope.
 An Open-Hardware Approach for IoT 305

2 Literature Review
2.1 Existing Smart Meters for Monitoring and Controlling
the Energy

Researchers have put efforts towards developing smart meters for monitoring
and controlling of individual consumers. Priyanka et al. [11] have presented an
IoT based smart meter for remotely monitoring of the renewable energy sources.
Here, FPGA based controller is used for computing and monitoring. However, the
overall system cost is very high and the controlling features are not implemented
yet. Aboelmaged et al. [12] have proposed an IoT based metering system which
uses Bluetooth technology to send the data over android application which in
turn sends the data to cloud. This approach has many flaws in terms of commu-
nication technologies and system design. Shibu et al. [13] have developed a smart
energy meter for monitoring and controlling of consumer loads. However, it lacks
in the detailed design and technologies used in the development. Lohokare et al.
[14] have presented an IoT ecosystem for wireless home automation systems.
The real-time implementation of this work to test its feasibility is not reported
yet. Barman et al. [15] have presented an IoT based smart meter which uses
open IoT platform. It is capable of uploading the consumption information at
IoT server. However, it has only one directional communication and the control-
ling part are not realized. Ali-Ali et al. [16] have presented an IoT based utility
meter which sends the monitored data to IoT server. Here, Raspberry Pi based
local server is created and the developed meter can monitor electricity, gas and
water consumption. Apart from these, Yaghmaee et al. [17] have developed an
IoT based smart meter which can monitor various electrical parameters of the
consumption, such as power factor, active power, apparent power and reactive
power. Here, an open IoT platform is used. However, it has high latency in the
data uploading. Giridhar et al. [18] have presented MQTT based industrial smart
grid monitoring system which monitors the individual load consumption. This
system lacks in controlling the loads. From the literature, we have observed that
there is a need of a system that can monitor as well as control the electricity
supplied to the consumers with the improved performance.

2.2 Review on IoT Platforms

In literature, various IOT platforms have been investigated to collect sensor val-
ues from hardware and can have less latency. ThingSpeak [19] is an IOT analytic
platform which allows a user to send, aggregate, visualize and analyse the data
from hardware. It offers MATLAB integration support to perform analysis and
processing. It offers both free tire and commercial use where free tire allows up to
8 devices to connect in single channel, with 8500 messages/day limit and has a
delay time of 15 s for every data updating. Thing Board [20] is an open source
IOT platform which supports devices to connect with cloud and to perform
processing, visualization and device management. It supports multiple IOT pro-
tocols such as MQTT, CoAP, HTTP, etc. Other similar platforms are Kaa [21],
306 G. M. Madhu et al.

Fig. 1. Architecture of the proposed smart meter based monitoring and controlling
system (three layered architecture).

Thinger [22] and openIOT [23]. Carriots [24] is an end to end IoT platform which
offers platform as a service (PaaS) to design machine to cloud and machine to
machine projects. It uses application programming interfaces (APIs) for infor-
mation exchange. In contrast to ThingSpeak, it allows only 2 devices to monitor
in free tire. Cloud MQTT [25] has very less latency and can communicate with
HTTP, HTTPS, TCP and UDP protocols. This makes the developer to choose
a protocol accordingly to their design. Here, an MQTT server is implemented in
the local PC as it provides very fast communication and there are no limits on
the storage and number of devices connected.

3 Architecture of the Proposed Smart Metering and

Controlling System
The objective of this paper is to design an IoT enabled smart meter for monitor-
ing and controlling energy of individual consumer. As shown in Fig. 1, the pro-
posed system consists of mainly three layers, where smart meters form a device
layer. The android application for consumers and authorities form a client layer
and the central cloud server forms an IoT server layer (MQTT Broker). These
layers interact with each other through Internet using HTTP requests and Appli-
cation Programming In-terfaces (APIs). The server uses four kinds of requests
for communicating, updating, retrieving and deleting the data. These requests
include GET request for retrieving the data from the server, PUT request for
updating a value in the database of a server, POST request for continuous feeding
real time value to server database and DELETE request for removing an entry
from the database. All the devices in device layer and client layer communicate
directly with the server. The real time energy information is sent from smart
meter to the server through publish API. At server, the information processing
 An Open-Hardware Approach for IoT 307

Fig. 2. Block diagram of the proposed IoT enabled smart meter.

is performed and values are stored in a database which can be accessed using
subscribe API. The design of the proposed smart meter is as follows:

3.1 Design of the Proposed Smart Meter for Monitoring and

Controlling

Smart meters are important in proper operation of the proposed system. A design
of IoT enabled smart meter with tamper detection circuit is shown in Fig. 2. This
smart meter has the capability to directly interact with the IoT server and has
the feature to turn OFF/ON the supply to a consumer based on the commands
from the Server/Authority. The readings measured at every interval are sent to
the server through HTTP requests. A consumer can access these readings along
with billing information using the developed android application. A tampering
detection circuit continuously monitors for meter tampering. On detection of
the meter tampering, the smart meter automatically turns off the supply and
an alert message is sent to the concerned authority.

3.2 Work Flow of the Proposed Smart Meter

A work flow of the proposed smart meter is given in Fig. 3. Initially, the proposed
smart meter at individual consumer premises connects to available local network
in turn connecting to the server using SSID (Service Set Identifier) and API keys.
Then, the smart meter reads the power flow parameters and sends them to the
server. At server, these values are processed and stored in the database. Then,
the meter continuously checks the supply flag whose value is controlled by the
power authorities for providing the supply. Based on the supply flag, the smart
meter continuous or interrupts the power supply. In addition, the proposed smart
meter continuously checks for the meter tampering using PIR sensor values. If
there is any tampering detected, then the supply is turned OFF and the smart
meter changes the tampering flag to 1 at the server.
308 G. M. Madhu et al.

4 Experimental Results and Analysis

4.1 Prototype Implementation of the Proposed System

The implementation of the proposed system can be considered in two parts viz;
software design and hardware design. In software development, Message Queuing
Telemetry Transport (MQTT) based server is created and an android application
is developed. In hardware development, a low cost IoT enabled smart meter with
tampering detection is developed.

Fig. 3. Work flow of the proposed smart meter.

 An Open-Hardware Approach for IoT 309

Fig. 4. Internal circuit diagram of LEM LV25P voltage transducer.

Table 1. Voltage transducer calibration results.

Reference voltage (V) Measured value (V) Percentage deviation (%)

300 299.95 0.016
250 249.99 0.004
200 199.87 0.065
100 99.91 0.09
50 49.81 0.38

Fig. 5. Internal circuit diagram of LEM LA25P current transducer.

Table 2. Current transducer calibration results.

Reference current (A) Measured value (A) Percentage deviation (%)

25 24.98 0.08
20 19.95 0.25
10 10.01 −.01
1 0.997 0.3
0.5 0.495 1
310 G. M. Madhu et al.

Fig. 6. Circuit diagram of zero crossing detector.

Hardware Design. As shown in Fig. 2, the proposed smart meter consists of a

voltage sensor, current sensor, zero crossing detector, signal conditioning circuit,
ATMega4809 microcontroller, and integrated ESP32. The High Voltage (HV)
and Low Voltage (LV) circuits are isolated by optocouplers. A protective circuit
is built for voltage and current sensors using OpAmps for protecting the sensors
from any damage in the event of fault. We have used LEM based voltage and
current sensors as these sensors will offer high precision readings. We have used
ATMega4809 based Arduino Uno Wi-Fi Rev v 2 micro-controller. It is a low
power, low cost multi-purpose board which has 14 digital input/output pins, 6
analog pins, 5 PWM pins, 48 kb of flash memory, 6144 bytes of SRAM, 256 bytes
of EEPROM with clock speed of 16 MHz. This board has an ADC sampling rate
of 9615 Hz which is sufficient for precise calculation. In addition, it has System
on Chip (SoC) ESP 32 module for Wi-Fi communication. Zero crossing detector
circuit is built using OpAmps and XOR gates. This circuit is used for calculating
the frequency and phase shift. An auxiliary supply of 5 V is built using 7405 IC
for providing regulated supply.
We use LEM LV 25P voltage transducer for sensing the voltage, as shown in
Fig. 4. This is a hall effect-based sensor having an error of ±0.5%. It requires a
high input resistance and an adjustable pot resistance at the output side. It is
capable of measuring 500 V AC/DC with the maximum input current of 15 mA.
Hence, the input resistance R1 , R2 design is 5 Watt, 20 Ω each. At the output
side, a 10 K Ω pot resistor is used such that the output voltage can be adjusted
between 0–5 V or 0–3 V. Fluke multi-meter is used to calibrate this transducer.
Table 1 shows the reading of voltage transducer and percentage deviation after
calibration.
To measure current, we have used LEM LA 25 P current transducer that is a
hall effect-based transducer, as shown in Fig. 5. It gives voltage corresponding to
input current and has an error of ±0.5%. It can measure current up to 25 A RMS.
 An Open-Hardware Approach for IoT 311

Fig. 7. Developed prototype of the proposed smart meter.

Table 3. Current transducer calibration results.

Parameter Value
Voltage range 0–300 V RMS
Current range 0–25 A RMS
Peak current 30 A
Sampling frequency 9615 Hz
Temperature 10–70 deg
Relay capacity 30 A RMS
Supply voltage 5V DC
Communication Wi-Fi 802.11 b/g/n

A pot resistance Rm is used to adjust the output voltage between 0–5 V or 0–

3 V. Fluke multi-meter is used to calibrate this transducer. Table 2 shows the
readings of current transducer and percentage deviation after calibration.
We have built an analog zero crossing detection circuit using OpAmp and
XOR gate, as shown in Fig. 6. It gives the square wave output for sinusoidal
voltage and current signals and calculate phase shift based on ON time of XOR
gate, which is used to compute the power factor.
For communication, we have used an ESP 32 with built in antenna, which
is integrated with the Arduino Uno Wi-Fi Rev V 2. This is an ultra-low power,
system on Chip (SoC) module with TCP/IP protocol. It supports 802.11 b/g/n
security features and has Wi-Fi direct feature. It communicates through SPI,
Universal Asynchronous Receiver Transmitter (UART) and I2C protocols. It is
used to connect to the router through Wi-Fi feature and sends data to the IoT
server.
As shown in Fig. 7, a prototype is built using the printed circuit boards
(PCB). The specifications of it is given in Table 3.
312 G. M. Madhu et al.

The developed prototype of the proposed smart meter consist of Arduino

Uno Wi-Fi Rev V 2 microcontroller. The board is self-powered as it takes power
from input line to turn ON all auxiliary units and the microcontroller. Arduino
IDE is used to code the smart metering programs.
Software Design. We have developed a MQTT protocol based server using
publish subscribe architecture for theft detection and smart meter data moni-
toring. Here, Eclipse and C language is used to create the publish commands and
subscribe commands. Different databases are used for storing the meter data of
consumers. Figure 8 shows the snapshot of the developed server home screen.

Fig. 8. Photograph of the developed MQTT based server.

Android Application: We have developed an android application using

android studio 2.3 IDE for consumers and authorities. API 23 (Marshmallow)
software development tool kit (SDK) is used in developing the app. The con-
sumer app consists of billing activity to check the billing information and real
time energy monitoring activity for seeing the present and past energy consump-
tion information. Authorities can use it to access all smart meter data and turn
ON/OFF feature in a control and monitor activity. The activities in the devel-
oped app are written in XML language and the back-end operation is coded
using the derived Java for android. Figure 9 shows the photograph of developed
activities. The activity life cycle of the developed apps is shown in Algorithms 1
and 2.
Figure 10 shows the snapshot of the experimental setup at NIT Goa. An
MQTT server is created using python and eclipse Paho in a local computer,
acting as an IoT server and smart meters are coded with API’s to access this
server. A separate API is created to access the metering information of a par-
ticular consumer for a particular date and time and plot the graph out of it. For
denying the supply, the relay is triggered to break the continuity of the circuit.
 An Open-Hardware Approach for IoT 313

4.2 Results and Discussion

We have considered various test cases to validate the proposed smart metering
and controlling system.
Monitoring and Billing Results: The proposed system monitors the individ-
ual consumer’s metering information and sends this information to client app
when requested. Figure 11 presents the client app reading, the real time energy

Algorithm 1. Activity life cycle for consumer app

1. Splash Activity → Delay
2. Main Activity,
a) Initialize onclick listener for Selection option between billing and
monitoring button
b) On click, → move to corresponding activity
c) Update memory
3. Billing Info Activity,
a) Connect to WiFi → return acknowledgement.
b) Connect to server using API.
c) Fetch Consumer billing info.
d) Display results.
e) On back button pressed →return to previous activity state from memory.
4. Monitor Activity,
a) Connect to WIFI → return acknowledgement.
b) Connect to server using API.
c) Fetch Consumer utilization info of corresponding date.
d) Display results in graph.
e) On back button pressed → return to previous activity state from memory.

Algorithm 2. Activity life cycle for authority app

1. Splash Activity → Delay
2. Main Activity,
a) Initialize onclick listener for Selection option between smart meters and
control button
b) On click, → move to corresponding activity
c) Update memory
3. Smart Meter Activity,
a) Initialize onclick listener for Selection option between smart meters.
b) Onclick, Connect to WIFI → Connect to server using API.
c) Fetch smart meter info.
d) Display results.
e) On back button pressed →return to previous activity state from memory.
4. Control Activity,
a) Connect to WIFI →Connect to server using API.
b) Send commands to either turn ON / OFF the smart meter return →
response.
c) On back button pressed → return to previous activity state from memory
314 G. M. Madhu et al.

Fig. 9. The activities developed in Android. (a), (b), (c) - consumer apps. (d), (e), (f)
- authority apps.

Fig. 10. Experimental setup of the proposed system at NIT Goa.

 An Open-Hardware Approach for IoT 315

Fig. 11. Monitoring of real time energy utilization information (a) from consumer app
(b) event log in smart meter.

consumption information with time stamp and the smart meter sending the data
to server.
Meter Tampering Detection Results: A scenario is created to tamper the
smart meter by making the PIR sensor to give motion detect output. The smart
meter reads the sensor value and sends the information to server detecting the
meter tampering event. Figure 12 shows the server-side data received from the
smart meter.

Fig. 12. Server-side event log during the meter tampering.

Results on Controlling Consumer Loads: During the event of meter tam-

pering, the server is coded to send command for stopping the supply to the con-
sumer. Hence, when the tampering is detected, server is sending a command to
316 G. M. Madhu et al.

Fig. 13. Server sending a stop command to smart meter in the event of meter
tampering.

stop the supply and acknowledgement is received from the smart meter. Figure 13
shows the event log happened during this condition.

5 Conclusion

In this paper, we have proposed a open hardware-based IoT enabled real time
smart meter monitoring and controlling system. We have used a new architecture
based on MQTT protocol for developing the monitoring and controlling system.
Android apps are developed for easy monitoring of energy utilization/billing
information and alerting the tampering information for authorities. A prototype
of the proposed system is implemented by considering various test scenarios to
validate the proposed system. The experimental results are very encouraging to
make the proposed system widely utilized in real time environment. In future,
the proposed smart meter can be extended to measure power quality parameters
of individual consumer and to detect the energy theft at each smart meter level.

References
1. World energy outlook 2018. www.webstore.iea.org/world-energy-outlook-2018.
Accessed 15 Oct 2019
2. Ouf, M.M., Issa, M.H.: Energy consumption analysis of school buildings in Mani-
toba, Canada. Int. J. Sustain. Built Environ. 6(3), 359–371 (2017)
3. IEEE Vision for Smart Grid Controls: 2030 and Beyond Reference Model. In IEEE
Vision for Smart Grid Control: 2030 and Beyond Reference Model, pp. 1–10 (2013)
4. Campbell, M., Watson, N.R., Miller, A.: Smart meters to monitor power quality
at consumer premises. In: EEA Conference & Exhibition (2015)
5. Bedi, G., Venayagamoorthy, G.K., Singh, R., Brooks, R.R., Wang, K.: Review of
internet of things (IoT) in electric power and energy systems. IEEE Internet Things
J. 5(2), 847–870 (2018)
 An Open-Hardware Approach for IoT 317

6. Abubakar, I., Khalid, S.N., Mustafa, M.W., Shareef, H., Mustapha, M.: Application
of load monitoring in appliances’ energy management-a review. Renew. Sustain.
Energy Rev. 67, 235–245 (2017)
7. Morello, R., De Capua, C., Fulco, G., Mukhopadhyay, S.C.: A smart power meter
to monitor energy flow in smart grids: the role of advanced sensing and IoT in the
electric grid of the future. IEEE Sens. J. 17(23), 7828–7837 (2017)
8. Gan, S., Li, K., Wang, Y., Cameron, C.: IoT based energy consumption monitoring
plat-form for industrial processes. In: 2018 UKACC 12th International Conference
on Control (CONTROL), pp. 236–240 (2018)
9. Gunturi, S.K.S., Reddy, M.S.K.: IoT based domestic energy monitoring device. In:
2018 3rd International Conference for Convergence in Technology (I2CT), pp. 1–4
(2018)
10. Tom, R.J., Sankaranarayanan, S., Rodrigues, J.J.P.C.: Smart energy management
and demand reduction by consumers and utilities in an IoT-fog based power dis-
tribution system. IEEE Internet Things J. 6(5), 7386–7394 (2019)
11. Priyanka, B.D., Joshi, V.M.: IOT based smart grid to remotely monitor and control
renewable energy sources using VLSI. Int. J. Innov. Res. Sci. Eng. Technol. 6(11),
21601–21607 (2017)
12. Aboelmaged, M., Abdelghani, Y., Ghany, M.A.A.E.: Wireless IoT based metering
system for energy efficient smart cites. In: 2017 29th International Conference on
Microelectronics (ICM), pp. 1–4 (2017)
13. Shibu, N.B.S., Hanumanthiah, A., Sai Rohith, S., Yaswanth, C., Hemanth Krishna,
P., Pavan, J.V.S.: Development of IoT enabled smart energy meter with remote
load management. In: 2018 IEEE International Conference on Computational Intel-
ligence and Computing Research (ICCIC), pp. 1–4 (2018)
14. Lohokare, J., Dani, R., Rajurkar, A., Apte, A.: An IoT ecosystem for the imple-
mentation of scalable wireless home automation systems at smart city level. In:
TENCON 2017-2017, pp. 1503–1508 (2017)
15. Barman, B.K., Yadav, S.N., Kumar, S., Gope, S.: IOT based smart energy meter
for efficient energy utilization in smart grid. In: 2018 2nd International Conference
on Power, Energy and Environment: Towards Smart Technology (ICEPE), pp. 1–5
(2018)
16. Al-Ali, A.R., Landolsi, T., Hassan, M.H., Ezzeddine, M., Abdelsalam, M., Baseet,
M.: An IoT-based smart utility meter. In: 2018 2nd International Conference on
Smart Grid and Smart Cities (ICSGSC), pp. 80–83 (2018)
17. Yaghmaee, M.H., Hejazi, H.: Design and implementation of an internet of things
based smart energy metering. In: 2018 IEEE International Conference on Smart
Energy Grid Engineering (SEGE), pp. 191–194 (2018)
18. Giridhar, M., Kayalvizhi, S.: IOT based industrial smart grid monitoring using
MQTT protocol. Int. J. Pure Appl. Math. 118(22), 41–49 (2018)
19. ThingSpeak IoT platform. https://thingspeak.com. Accessed 15 Oct 2019
20. ThingBoard IoT platform. https://thingsboard.io. Accessed 15 Oct 2019
21. Kaa IoT platform. https://www.kaaproject.org. Accessed 15 Oct 2019
22. Thinger IoT platform. https://thinger.io/. Accessed 15 Oct 2019
23. Open IoT. http://www.openiot.eu. Accessed 15 Oct 2019
24. Carriots. https://www.altairsmartworks.com/index.php. Accessed 15 Oct 2019
25. Cloud MQTT. https://www.cloudmqtt.com/docs/index.html. Accessed 15 Oct
2019
 Performance of NOMA-MUSA System
for 5G Using m-ZCZ Sequences

M. Jayasheela1(B) , E. Gomathi1(B) , M. Vijila2(B) ,

and Hepzibah A. Christinal2(B)
1
KIT-Kalaignarkarunanidhi Institute of Technology, Coimbatore, India
[email protected], [email protected]
2
Karunya Institute of Technology and Sciences, Coimbatore, India
[email protected], [email protected]

Abstract. In NOMA-MUSA system for 5G applications the multi

access interference (MAI) will degrade the Capacity BER of the sys-
tem because of the similarity between the received signals of different
users in the various channel conditions and near far effect. If suppres-
sion of MAI is not done, the probability with which the decoding of
undesired user becomes more than that of the desired user. Thus, the
interference due to the unwanted users affects the performance of the
NOMA-MUSA system. The capacity of system is also largely improved
by suppressing the MAI. In zero correlation zone sequences based on
maximal (m-ZCZ) sequence has very good cross correlation properties.
In this paper this concept is utilized in order to suppress the MAI and
increase the performance NOMA-MUSA system for 5G wireless appli-
cation. Also the performance of NOMA-MUSA has been compared with
other SCMA-NOMA and PDMA-NOMA system.

Keywords: Edge NOMA · MUSA · m-ZCZ · SIC · Multiple access

interference

1 Introduction
There are growing demands for reliable high data rate in mobile communication
system. The multi user interference occur between various users by simultaneous
sharing of a common channel. MAI provides a limitation on the capacity and
BER of the non-orthogonal multiple access (NOMA) system when more number
of users are active. MAI occurs when multiple users are simultaneously accessing
the same channel. The MAI in NOMA system is because of the larger similar-
ity between the obtained signals of different users in the receiver side under
Rayleigh channel and near far effect. The characteristics and feature of 5G are
high data rate coverage in the edge of the cell, increase in base station capacity,
and user’s quality of experience, reduction in latency, very good energy and spec-
trum efficiency. It is also utilizing spectrum from the available LTE frequency
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 318–328, 2020.
https://doi.org/10.1007/978-981-15-3666-3_26
 Performance of NOMA-MUSA System for 5G Using m-ZCZ Sequences 319

band and millimeter frequency bands which supports data rate up to 20 gigabits
per second [5]. Multiple access is an important issue in the 5G communication.
This happens due to the higher cross correlation between the transmitted sig-
nals from different users in 5G and near far effect. NOMA is promising access
technology in 5G to make efficient use of spectrum resources [10]. NOMA’s main
objective is to improve spectrum sharing between different users by exploiting
power domain multiplexing which differs from traditional multiple orthogonal
access OMA [3,6,7,13].
NOMA has capability of handling different users at the equal time, frequency
and code but with varying power levels, which achieves a substantial spectral
efficiency OMA. Performance of NOMA better than OMA and attains better
spectrum utilization [8]. Due to increase of interconnection of devices through
IoT there is a need transferring data from one device to another such as video
conferencing, broadcast services, lifeline communications and E-health applica-
tions. Successive interference cancellation (SIC) with NOMA is achieved these
requirements in 5Gwireless system [4]. Pattern division multiple access (PDMA),
sparse code multiple access (SCMA), and multiuser shared access (MUSA) are
three main schemes of NOMA technologies. Here the performance of MUSA
system is investigated for 5G communication. Wang et al. [11] maximal length
sequence is used due to low cross correlation. Hayashi et al. [2] proved that
zero correlation zone sequences are good candidate for giving best orthogonal-
ity since it has zero auto correlation and cross correlation properties. In this
paper the BER performance of NOMA-MUSA using m-ZCZ are analysed and
compared. This paper is explained as follows: Sect. 2 deals with MUSA, Sect. 3
briefs about other code domain NOMA schemes, Sect. 4 explains about m-ZCZ
sequences, Sect. 5 deals with the proposed NOMA-MUSA system model with m-
ZCZ sequences, result and discussion are summarized in Sect. 6 and conclusion
is narrated in Sect. 7.

2 MUSA
MUSA is the code domain scheme of NOMA [13]. MUSA is utilizing the low
correlation spreading sequences at transmitting end by considering overloading
factor into account [11]. In MUSA the transmitted signal is spreaded by specially
designed sequence. Then these signals are transmitted over the channel simulta-
neously with same frequency. A non-linear type of MUD scheme: SIC is utilized
at the receiver in which the users are successively decoded. In this method it
successively cancels the substantial users by re-encoding the decoded bits. SIC
improves the BER performance of all the users [4].

3 Other Code Domain NOMA Schemes

3.1 SCMA
This multiple access scheme is constructed by sparse codebooks. The codebooks
are made based on multidimensional constellations, and the optimizing the gain.
320 M. Jayasheela et al.

The same blocks are transmitted by multiple users in SCMA with different code-
books. With sparse codebooks, the collision between users is decreased, there-
fore SCMA able to recover the inter-user interference. Because of Sparsity the
receiver complexity is reduced and able to achieve better performance using
message passing algorithm (MPA).

3.2 PDMA

In this multiple access scheme non-orthogonal patterns are allotted to differ-

ent users to perform multiplexing. These patterns are carefully designed in the
multiple domains of code, power and space, to achieve successive interference
cancellation (SIC) property. Due to this property, the low-complexity SIC based
MPA multiuser detection method gives better performance. At the transmitter,
as mentioned in SCMA, the users in PDMA are also spread by a sparse signature
matrix. The main difference is that the number of resource blocks allocated for
each user are varied in PDMA.

4 m-ZCZ Sequences
The auto correlation and cross correlation value of m-ZCZ sequence is zero.
Hence it can able to eliminate MAI [7], m-ZCZ sequences are superior to Gold
sequences, Kasami sequences and m-sequences whose correlative properties are
good but not optimal. The auto correlation value must be zero except for time
shift of zero and cross correlation value of all the sequences is zero for every time
shift. Since the m-ZCZ sequences satisfy the above requirements, the number of
users supported by m-ZCZ sequences is larger than the conventional binary ZCZ
codes [5].
Construction of m-ZCZ Sequences
The m-ZCZ sequence is represented as

m − ZCZ(KL , J, Mmin ) = Zi1 , Zi2 (1)

KL represents the sequence length, J is the sequence size and Mmin is the min-
imum length of oneside ZCZ respectively.
The first sub-sequence of the ith sequence with the length of L is given by,

Zi1 = ai.Mmin where i = 0, 1, . . . S − 1 (2)

Where a0 refers m-sequence with its period L, and ai.Mmin refers the sequence
generated by cyclically leftward shifted a0 with i.Mmin chips. This is given as
follows,
ai.M
n
min
= a0 (n + i.Mmin )modL (3)
where n = 0, 1, . . . L − 1 and n is the index of the chip. The second sub-
sequence Zi2 = +1 contains only one “+1 ” chip. The congregated code length is
 Performance of NOMA-MUSA System for 5G Using m-ZCZ Sequences 321
 
L
KL = L + 1. The set size of the sequence is J = where x is the max-
Mmin
imum integer not larger than x.
  for KL = 16, Mmin = 4 and the
Ye et al. [12] m-ZCZ sequence have developed
15
set of m-ZCZ sequences containing J = = 3 sequences are defined as,
4

(Z01 , Z02 ) = (+1 + 1 + 1 − 1 − 1 − 1 − 1 + 1 − 1 + 1 − 1 − 1 + 1 + 1 − 1, +1)

(Z11 , Z12 ) = (−1 − 1 − 1 + 1 − 1 + 1 − 1 − 1 + 1 + 1 − 1 + 1 + 1 + 1 − 1, +1)
(Z21 , Z22 ) = (−1 + 1 − 1 − 1 + 1 + 1 − 1 + 1 + 1 + 1 − 1 − 1 − 1 − 1 + 1, +1)

where sub-sequences are separated by commas.  It is noted that for a given

L
sequence length KL when Mmin = 1 and J = = L, m-ZCZ sequence
Mmin
is then equal to a Walsh Hadamard code set excluding the all one code [9].

Fig. 1. Comparison of ACF between m-sequences and m-ZCZ sequences

Figures 1 and 2 show the ACF and CCF comparisons between m-sequences
and m-ZCZ sequences, respectively. ACF and CCF of ZCZ sequences are almost
zero when compared to m-sequences [1]. From Figs. 1 and 2, it is noted that
m-ZCZ sequences have improved cross correlation than that of PN-sequences.

5 Proposed NOMA-MUSA System Model with M-ZCZ

Sequences

Figure 3 shows the proposed NOMA-MUSA system model with m-ZCZ

sequences. The binary information stream is transmitted using a QPSK
modulation. The modulated information is spreaded using m-ZCZ sequence.
322 M. Jayasheela et al.

Fig. 2. Comparison of CCF between m-sequences and m-ZCZ sequences

The spreaded sequences are transmitted through Rayleigh fading channels. Then
it is processed by the SIC receiver. It is then fed into the decision making device
and they are converted into bit sequence. The sequence is despread and the
output data is obtained.
The transmitted signal in proposed NOMA-MUSA system is expressed as
follows,
Nu
 ∞

S k (t) =
(k)
ak q(t − jTs − jTf − Cj Tc ) (4)
k=1 j=−∞

Ts represents the symbol duration, Nu represents the number of users and ak is

the amplitude of the k th user. In order to allow all the users to access the same
channel and to eliminate the extremely harmful collision, each user is allotted
with a distinguished spreading sequence called m-ZCZ sequence which takes
values from 0 to Nu − 1. It is assumed that both the MUSA transmitter and
MUSA receiver are synchronized.
The spreading codes are usually designed with very near zero cross correlation
values. Hence, the effect of MAI on the required user is small. However, as the
count of users increase, MAI increases. If there is total count of users utilizing
the proposed NOMA-MUSA system, the received signal is then given by,
Nu

y(t) = S (k) (t − τk ) + n(t) (5)
k=1

where τk ∈ [0, Ts ] is the transmission delay of k th user and n(t) is the noise due to
Rayleigh fading. The proposed NOMA-MUSA system is implemented with SIC
receiver at the receiving end. To eliminate the error propagation, optimal order-
ing technique is used. While SIC is combined with optimal ordering technique,
 Performance of NOMA-MUSA System for 5G Using m-ZCZ Sequences 323

Fig. 3. Proposed NOMA-MUSA system model with m-ZCZ sequences

knowledge in choosing the signal of s1 first or s2 is high and then subtracts the
respective s1 or s2 from the obtained signal. In this scheme, the strongest signal
is eliminated first followed by the second strongest, and so forth.
⎧
Nu  (j+1)Tf (k) (k)
⎪
⎨ j=0 jTf y(t)hi (t − Cj Tc )dt,
(k) (k)
Yi = s + IM AI + n, for i = bi (6)
⎪
⎩ (k)
IM AI + n for i = bi
where s = ak and n is the Rayleigh component and IM AI denotes the MAI
component caused by Nu − 1 interfering users.

5.1 Pseudo Code for the Proposed NOMA-MUSA System

Step 1: Input binary sequence of +1s and −1s is generated randomly.
Step 2: Random input binary sequence is spreaded using m-ZCZ and is
grouped them into symbols.
Step 3: It is modulated using QPSK modulation.

Table 1. Parameters taken for the proposed system

Parameter Values used in simulation

Modulation order 4
Number of users 10–100
Modulation scheme QPSK
Multiple access MUSA/SCMA/PDMA
Orthogonal resources 4
Frame length 1000
Channel model AWGN channel
324 M. Jayasheela et al.

Step 4: The converted symbols are transmitted via Rayleigh channel.

Step 5: The Output sequences of transmitter are decoded using SIC receiver.
Step 6: Hard decision decoding is performed and calculated BER.
Step 7: BER performance of proposed NOMA-MUSA system using m-ZCZ
has been compared with other NOMA schemes.

6 Results and Discussion

Using Matlab R2018a, Monte Carlo simulations have been performed to simulate
the performances of m-sequences and m-ZCZ sequences for different values of
Eb
( ) and varying number of users under AWGN channel.
N0
BER has been obtained by evaluating the transmitted signal with the received
signal and computing the error count for the proposed system. Simulation param-
eters of the proposed system are given in Table 1.
Output Parameter
BER is the output parameter of the proposed system. It is the ratio of the
number of bits in error to the total number of bits transmitted. The BER is
calculated as follows,
N umber of bits in error
BER = (7)
T otal number of bits transmitted

6.1 BER Performance of the Proposed NOMA-MUSA System with

m-ZCZ in AWGN Channel with Constant Power Level

Figure 4 depicts the BER of proposed NOMA-MUSA system with m-ZCZ

Eb
sequences with respect to in AWGN channel. A NOMA-MUSA system with
No
m-ZCZ is proposed and BER performance of the system with m-ZCZ sequences
under AWGN channels are compared for PDMA and SCMA for 10 users. For
Eb
a single user good BER is achieved when has very low value. The pro-
No
posed NOMA-MUSA system using m-ZCZ sequences confirms the BER values
Eb Eb
are 1.1 × 10−3 at = 8 dB and 3.5 × 10−4 at = 12 dB for 10 users under
No No
constant power level that is each user has same power of 0.01 m Watts. For same
number of users SCMA and PDMA gives the BER of 2 × 10−3 and 4 × 10−2 at
Eb Eb
= 8 dB respectively. Also when = 12 dB SCMA and PDMA gives the
No No
BER of 1.9 × 10−3 and 2 × 10−2 From the graph, it is proved that the proposed
NOMA-MUSA system exhibits better BER improvement.
 Performance of NOMA-MUSA System for 5G Using m-ZCZ Sequences 325

Fig. 4. BER of the proposed NOMA-MUSA system with m-ZCZ sequences as a func-
tion of in AWGN channel (constant power level)

6.2 BER Performance of the Proposed NOMA-MUSA System with

m-ZCZ in AWGN Channel with Random Power Level

In random power level in which all the users’ power are random values. With
random power level for the users, the MAI of every user is not the same.
Figure 5 depicts the BER performance of NOMA-MUSA system using m-
Eb
ZCZ sequences with respect to in AWGN channel for random power level.
N0
The BER performance of a single user with a random power level is not as
good as the constant power level. The proposed NOMA-MUSA system using m-
Eb
ZCZ sequences shows the BER values of 2 × 10−2 at = 8 dB and 2 × 10−3 at
N0
Eb
= 10 dB for 10 users under random power level (from 0.1 to 1mW). For same
N0
number of users SCMA and PDMA gives the BER of 1.6 × 10−1 and 3 × 10−1
Eb
at = 8 dB, SCMA and PDMA gives the BER of 1 × 10−1 and 2.5 × 10−1 .
N0
From the results, it is proved that the proposed NOMA-MUSA system exhibits
good BER when compared with SCMA-NOMA and PDMA-NOMA system.
326 M. Jayasheela et al.

Fig. 5. BER performance of the proposed NOMA-MUSA system using m-ZCZ

sequences as a function of in AWGN channel (random power level)

Table 2. Capacity of proposed NOMA-MUSA system with m-ZCZ sequences in AWGN

Eb
channel for different for different number of users
N0
Eb Eb Eb
Number of users BER for = 4 dB BER for = 8 dB BER for = 12 dB
N0 N0 N0
5 0.046 0 0
10 0.08 0.0011 0.00035
20 0.125 0.043 0.008
30 0.184 0.087 0.01
40 0.24 0.124 0.039
50 0.281 0.137 0.08
60 0.302 0.158 0.12
100 0.33 0.259 0.18

6.3 Capacity of Proposed NOMA-MUSA System Using m-ZCZ

Eb
Sequences in AWGN Channel for Different with Varying
N0
Number of Users (Constant Power Level)
Table 2 shows the capacity of proposed NOMA-MUSA system using m-ZCZ
Eb
sequences in AWGN channel for different with varying number of users.
N0
 Performance of NOMA-MUSA System for 5G Using m-ZCZ Sequences 327

Eb
From Table 2 a BER of 0.08 is obtained for 12 dB at 50 users. At = 8 dB,
N0
Eb
BER is increased to 0.1370 at the similar number of users. At = 4 dB BER
N0
value of 0.2810 is obtained for the identical number of users. Hence it is inferred
Eb Eb
that the less BER is obtained for = 12 dB. Thus, as value increases,
N0 N0
BER value decreases.

7 Conclusion
A novel NOMA-MUSA system with m-ZCZ is proposed and BER performance
of the system m-ZCZ sequences under AWGN channel is compared with other
NOMA schemes. The proposed NOMA-MUSA system using m-ZCZ sequences
exhibits 3 times BER improvement than that of SCMA-NOMA system and 40
Eb
times PDMA-NOMA at = 8 dB for 10 users under AWGN channel. From
N0
the results, it is also observed that proposed NOMA-MUSA system using m-ZCZ
yields better BER performance than that of other NOMA schemes.

References
1. Bae, J.N., Choi, Y., Kim, J.Y., Kwon, J.W., Kim, D.I.: Efficient interference can-
cellation scheme for wireless body area networks. J. Commun. Netw. 13, 167–174
(2011)
2. Hayashi, T., et al.: A novel class of quadriphase zero-correlation zone sequence sets.
IEICE Trans. Fundam. Electron. Commun. Comput. Sci. 100(4), 953–960 (2017)
3. He, B., et al.: On the design of secure non-orthogonal multiple access systems.
IEEE J. Sel. Areas Commun. 35(10), 2196–2206 (2017)
4. Higuchi, K., Benjebbour, A.: Non-orthogonal multiple access (NOMA) with suc-
cessive interference cancellation for future radio access. IEICE Trans. Commun.
98(3), 403–414 (2015)
5. Li, S., Xu, L.D., Zhao, S.: 5G internet of things: a survey. J. Ind. Inf. Integr. 10,
1–9 (2018)
6. Liu, Y., et al.: Enhancing the physical layer security of non-orthogonal multiple
access in large-scale networks. IEEE Trans. Wirel. Commun. 16(3), 1656–1672
(2017)
7. Liu, Y., Qin, Z., Elkashlan, M., Nallanathan, A., McCann, A.: Non-orthogonal
multiple access in large-scale heterogenous networks. IEEE J. Sel. Areas Commun.
35, 2667–2680 (2017)
8. Lv, L., et al.: Cognitive non-orthogonal multiple access with cooperative relaying:
a new wireless frontier for 5G spectrum sharing. IEEE Commun. Mag. 56(4), 188–
195 (2018)
9. Ping, L.: Code-shift division multiple access for cellular mobile. In: 1997 IEEE 47th
Vehicular Technology Conference. Technology in Motion, vol. 1. IEEE (1997)
10. Qin, Z., et al.: Physical layer security for 5G non-orthogonal multiple access in
large-scale networks. In: 2016 IEEE International Conference on Communications
(ICC). IEEE (2016)
328 M. Jayasheela et al.

11. Wang, B., et al.: Comparison study of non-orthogonal multiple access schemes for
5G. In: 2015 IEEE International Symposium on Broadband Multimedia Systems
and Broadcasting. IEEE (2015)
12. Ye, G., et al.: A novel ZCZ code based on m-sequences and its applications in
CDMA systems. IEEE Commun. Lett. 11(6), 465–467 (2007)
13. Zhang, Z., Sun, H., Hu, R.Q.: Downlink and uplink non-orthogonal multiple access
in a dense wireless network. IEEE J. Sel. Areas Commun. 35(12), 2771–2784 (2017)
 IoT Based Multi-sensor Data Acquisition
System for the Application
of Smart Agriculture

Saurabh Kumar and Nagesh Ch(B)

Department of Electronics and Communication Engineering,

Indian Institute of Information Technology Senapati Manipur,
Imphal 795002, India
{saurabh,nagesh}@iiitmanipur.ac.in
http://www.iiitmanipur.ac.in

Abstract. In today’s world Internet-of-Things (IoT) is creating fas-

cinating applications by providing the common platform for different
domains. This paper proposes an IoT based Multi-Sensor Data Acquisi-
tion System to control, monitor and manage different devices together for
the application of smart agriculture. The system is designed to measure
some of the important parameters related to the crop yield such as soil
moisture, temperature, relative humidity, and light intensity. It provides
an automated irrigation to crop based on the data obtained from dif-
ferent sensors. In addition, the present system provides an added future
for monitoring data and controlling the irrigation system in real-time
without losing data during transmission. The irrigation system can be
controlled from any location through a mobile phone in two ways one is
automatic and the other one is manual control. Further, it is made to
provide ease of operation and doesn’t require the knowledge of modern
technology for the same. The developed system is very economical and it
makes use of solar energy to increase the autonomy of embedded and its
associated devices to provide perpetual operation. The aim of the system
is to improve the crop yield by the effective use of natural resources.

Keywords: IoT · Sensor · Irrigation system · Mobile phone

1 Introduction
The Internet of Things (IoT) is an enabling technology for the revolution in
health, agriculture, education, entertainment, and industry sector [1]. The tech-
nological developments are allowing every domain to employ new techniques
and methods to enhance their end results. One of them is the modern agri-
culture, employing new technologies and equipments to improve the yield. The
optimization of water usage with help of soil moisture sensor for the irrigation
process has been demonstrated [2]. The IoT and modern techniques transformed
conventional agriculture into smart agriculture. It helps in reducing the human
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 329–342, 2020.
https://doi.org/10.1007/978-981-15-3666-3_27
330 S. Kumar and N. Ch

work by monitoring the crop health and field parameters remotely. IoT utilizes
a remote sensor (mobile phone) that acts as the backbone for collecting infor-
mation for these monitoring and control applications. The system consists of
different sensors to monitor various parameters such as a temperature, humid-
ity, light intensity and soil moisture. The system is capable of communicating
this data with the other devices [3,4] for better control and monitoring. The
major challenges faced in the agricultural sector is data loss during the trans-
mission of data between multi-sensor system and mobile phone [5]. The data loss
will effect the controlling of irrigation system.
The limitation of the existing systems has motivated to propose an IoT based
multi-sensor data acquisition system for multiple activities including control,
monitor and manage different devices together for the application of smart agri-
culture. The system is built with an added feature to monitor the data and con-
trol the irrigation system in real-time without losing data during transmission.
The system is designed to measure different parameters such as a soil moisture,
temperature and relative humidity. Further, it helps to control irrigation process
remotely using a mobile phone. The developed system reduces the involvement
of human being as it makes cost-effective and time-efficient. In addition, the pro-
posed system uses solar energy to provide continuous operation which in turn
increases the autonomy of embedded system and its associated devices. Thus,
it helps farmers to contend with the enormous challenges faced in the agricul-
tural sector including overwatering in the field, unavailability of electric power
for irrigation, etc. The experimental results depicts that the accuracy of sending
control command for irrigation control is 100% and the accuracy of receiving
data from multiple sensor to mobile application is 98%.
This paper is organizes as follows. Section 2 briefly describe the data acquisi-
tion system. Section 3 discusses details about the Architecture of Data Acquisi-
tion System. Experimental results are shown in Sect. 4 and conclusions are given
in Sect. 5.

2 Data Acquisition System

Data acquisition system (DAQ) is designed to control and monitor the irrigation
system based on the real world data obtained from the sensors. Data acquisition
is the process of measuring and analyzing various electrical and physical param-
eters like voltage, current, temperature and pressure. Figure 1 shows a DAQ and
it consists of transducer, signal conditioning circuit, analog to digital converter,
and application software.
The important functions in data acquisition system is summarized as follows:
– Physical System - Quantity which we are measuring/sensing like tempera-
ture, pressure, sound etc.
– Transducer - Transducer senses the physical parameters and then converts
into the electrical signals.
– Signal Conditioning Circuit - The converted electrical signal are sent to
signal conditioning system which filters, amplify according to the requirement.
 IoT Based Multi-sensor DAQ for the Application of Smart Agriculture 331

Fig. 1. Data acquisition system

– Analog to Digital Converter - Converts the analog electrical signal to the

digital signal which can be further processed by application software.
– Application Software - Records the digital signal and display the required
information at the output, further, the data can be used for control and
monitoring of irrigation system.

Transducer sense the physical phenomenon such as a soil moisture and con-
vert them into an electrical signal like voltage or current. However, these signals
may not be in the appropriate form for the measurement. The signals needs to be
modified appropriately as per the requirement in terms of amplitude (low/high)
and frequency. Further, the noise will be removed so that the signal will be
suitable for other circuit blocks.
Signal conditioning is one of the most important components of any data
acquisition system [6]. It is the interface between real-world analog signals and
the rest of the system. Signal conditioning system enhances the quality of signal
coming from a sensor in terms of:

1. Right type of signal, to convert the output signal from a transducer into the
desired form i.e. voltage/current.
2. Right level of the signal, to amplify or attenuate the signals to a right/
acceptable level for the further processing.
3. Noise, to eliminate noise from a signal.
4. Protection, to protect the damage to the next block of a system such as a
micro-controller from the high current or voltage signals.

3 Architecture of Data Acquisition System

The proposed system architecture is shown in Fig. 2 and it is designed for the
application of smart agriculture. The system employees different sensors such as a
soil moisture, temperature, relative humidity and LDR. In addition, ATmega328
micro-controller, NodeMCU and firebase cloud are used as part of DAQ for
controlling the irrigation system. The micro-controller will read the data from
sensors in an appropriate form using application software (Arduino IDE) for
further processing. The sensors and micro-controller unit (MCU) are integrated
to collect the information about the changes in the environmental parameters.
332 S. Kumar and N. Ch

Fig. 2. System architecture

Depending on the obtained information, the irrigation process can be optimized

through the mobile application.
The NodeMCU is added to the system to facilitate the communication (data
transfer) between micro-controller and firebase cloud. It is a self-contained SOC
(system on a chip) with integrated TCP/IP protocol stack that permits any
micro-controller to access through Wi-Fi network. The micro-controller trans-
fer the information to firebase cloud through NodeMCU, thereafter it can be
accessed from anywhere using a mobile phone. In return, the user can control
the irrigation system through a mobile phone or same can be controlled by the
my controller if it is an auto mode.
In case of manual control, the user sends the control signal based on the
available information using a mobile phone. The control signal will be trans-
ferred to firebase cloud, and then it will be passed to micro-controller through
a NodeMCU. Finally, the irrigation system will get the control signal from the
micro-control in an either cases such as an auto mode or manual mode. Firebase
cloud is a cloud which is a NoSQL database that provides the facility to store,
synchronize, authenticate, and query data for mobile and web application. It
is a real-time database and provides an API that enables application data to
be synchronized across clients and stored on firebase cloud. Initially, the data
collected from the sensors are synchronized with the data in the application,
thereafter, the data is monitored in real-time through a mobile phone.

3.1 Temperature and Humidity Detection

The circuit diagram for measuring the temperature and humidity (DHT22) are
given in Fig. 3. Initially, micro-controller sends a start signal to the DHT22 to
change from standby status to running status. In return, the DHT22 send a
response signal of 40-bit data that contains temperature and relative humidity
back to the micro-controller unit (MCU). The DHT22 will not response to MCU
 IoT Based Multi-sensor DAQ for the Application of Smart Agriculture 333

unless it receives the start signal. However, DHT22 will change from running
status to standby status when transmission has completed or when the start
signal is not received from the MCU for the next operation [5].

Fig. 3. Circuit for DHT22

A resistor connected between pin1 and pin2 of DHT22 shown in Fig. 3 [5] is
used as a pull-up resistor. This resistor essentially pulls up the voltage at pin2
to 5V in times where the DHT sensor is not actually sending any data (i.e. not
HIGH or LOW). Therefore, we use pull-up resistor to prevent this condition
called floating.

3.2 Soil Moisture Detection

Soil moisture is one of the most significant factor in plant growth. Plants behaves
differently at different soil water content levels. When the soil is dry, plants feel
stressed and their growth reflects the adverse effects of stress. Stress also makes
them vulnerable to pests and diseases. By observing the level of soil moisture, the
irrigation process can be optimized to precisely meet the crop’s requirements [7].
The sensor for measuring soil moisture is given in Fig. 4 having two probes
and they are used to determine the volumetric water content. The two probes
allow the current to pass through the soil and in response the resistance value
is measured according to the moisture levels. If the water content is more in a
given soil then the resultant current will be high which implies low resistance. It
indicates that the moisture levels are high. If the measured current is low then
it indicates that the soil is dry and water content is low. In response, high value
of resistance and lower levels of moisture.

3.3 Solar Tracker Using LDR

The solar tracker is used for controlling solar panel in the direction where light
intensity is more. The system is integrated with a sun tracking solar device
334 S. Kumar and N. Ch

Fig. 4. Resistive soil moisture

to increase the autonomy of embedded system and its associated devices to

provide continuous operation. The LDR (Light Dependent Resistor) sensors are
interfaced with motors that are used for rotation through a micro-controller to
cause a maximum incident of solar radiation on the solar panel. An LDR is a
photoconductive sensor which works on the principle of photoconductivity. Five
LDR sensors (R1 = R2 = R3 = R4 = R5 = 10k) are placed in a hemisphere
shaped structure to measure the output voltage according to the light intensity.
The measured voltage will decides the movement of the solar panel.

Fig. 5. Solar tracker using LDR

The circuit for solar tracker using LDR is shown in Fig. 5. The minimum
output voltage out of all LDRs are used to rotate the solar panel through servo
 IoT Based Multi-sensor DAQ for the Application of Smart Agriculture 335

motor. The servo motor will move the solar panel in the direction where light
intensity is more. Hence, the solar panel effectively charges the battery connected
with it so as to operate the DAQ system. The curve shown in Fig. 6 tells that
the LDR voltage is inversely proportional to the light intensity.

4.5

3.5

3
LDR Voltage (V)

2.5

1.5

0.5

0
0 100 200 300 400 500 600 700 800 900 1000
Illumination (LUX)

Fig. 6. LDR voltage v/s illumination curve

3.4 Communication Architecture

This section describes about serial communication between NodeMCU and

Arduino. It also describes the transmission of data between NodeMCU and
Firebase.

Data Transmission Between Arduino and NodeMCU. The Arduino pro-

vides the facility for communicating with a computer or other microcontroller.
It provides UART (universal asynchronous receiver-transmitter) serial commu-
nication available on digital pins 0 (Rx) and 1 (Tx). The transmission of data
happens via serial communication between Arduino and NodeMCU microcon-
troller. To build up serial communication between two devices, the devices should
be connected as shown in Fig. 7. NodeMCU Rx connects to the Arduino Tx, and
the NodeMCU Tx connects to the Arduino Rx.
336 S. Kumar and N. Ch

Fig. 7. Serial communication

Data Transmission for Communication Within the Network. The

NodeMCU development board provides a solution for programming micro-
controllers and being part of IoT. It consists of a self-contained SOC (system on
a chip) with integrated TCP/IP protocol stack that permits any micro-controller
to access a Wi-Fi network [8]. NodeMCU acts like an intermediate between the
sensors and the cloud.
The firebase interface shown in Fig. 8 includes control signal to operate the
irrigation system and collects data in the form of string from different sensors
including soil moisture, temperature, relative humidity, and light intensity. It
acts as a cloud which is a NoSQL database that provides the facility to store,
synchronize, authenticate, and query data for mobile and web application. It is
a real-time database and provides an API that enables application data to be
synchronized across clients and stores it on firebase’s cloud.

Fig. 8. Firebase

The designed system utilizes a Wi-Fi connection to communicate between

NodeMCU and Cloud Firebase. The data is sent from the sensor and synchro-
nized with the data in the application. And afterward, the data is monitored as
well as controlled in real-time through a mobile phone.
 IoT Based Multi-sensor DAQ for the Application of Smart Agriculture 337

Data Synchronization in Android App. Android Studio is the official inte-

grated development environment (IDE) for Google’s Android operating system,
built on JetBrains’ IntelliJ IDEA software and designed specifically for Android
development. The collected information from different sensors has been processed
and transmitted to firebase, and same is forwarded to android application which
includes soil moisture (degree 0–99), humidity (RH degree 0–99), the tempera-
ture in degree Celsius and light intensity (in lux) as shown in Fig. 9.

Fig. 9. The reporting and control system via mobile application

ATmega328 Micro-Controller Flowchart. The flow chart shown in Fig. 10

explains the data read by the micro-controller through Arduino IDE software.
It reads the data including temperature, relative humidity, soil moisture, and
light intensity. The maximum light intensity out of five LDR sensor is used
for the application of solar tracker. Further, the data from all the sensors is
converted into a single string. This string is then transmitted to NodeMCU
micro-controller.

NodeMCU Micro-Controller Flowchart. The flowchart shown in Fig. 11

explains the data read by NodeMCU micro-controller and then transmits it to
firebase. The NodeMCU consist of ESP8266 Wi-Fi SoC (system on chip) with
integrated TCP/IP protocol stack that permits micro-controller to access Wi-
Fi network. The string sent from ATmega328 micro-controller is corresponds to
the sensors data. NodeMCU micro-controller will read the string of data and
it transmits to firebase at a particular node. At the same time, the data from
338 S. Kumar and N. Ch

Fig. 10. ATmega328 micro-controller flowchart

sensors are updated in mobile application which further helps to control the
irrigation system.
Whenever control command is given by the user for controlling irrigation
system through mobile application and the same has been gets updated in the
firebase cloud at a particular node in real-time. The firebase cloud communicates
with Wi-Fi NodeMCU and it follows up based on data received from firebase
 IoT Based Multi-sensor DAQ for the Application of Smart Agriculture 339

Fig. 11. NodeMCU microcontroller flowchart

cloud. If NodeMCU receives serial data 1, it sends the signal to the Arduino
micro-controller to turn ON the irrigation system through 5 volt relay, and if
NodeMCU receives serial input 0 then it command the Arduino micro-controller
to turn OFF the irrigation system.

4 Experimental Results
The proposed system has been designed and demonstrated for controlling and
monitoring the irrigation system through a mobile application for the optimum
usage of resources. The Fig. 12 shows an experimental setup for proposed DAQ
system. The optimum usage of water as per the moisture/humidity levels can be
controlled through a mobile application. The system is designed such way that
it can automatically turn on/off the irrigation system through a relay. Further,
the system can be controlled remotely via internet through a mobile application.
Besides that, the power supply to turn on/off the irrigation system is provided
by a battery which is charged by solar tracker.
340 S. Kumar and N. Ch

Fig. 12. Experimental setup

In this experiment, the soil and environmental information has been collected
from the sensors and it is forward to mobile application so as to control the
irrigation system. These experiments were carried out by sending and receiving
control commands via the internet from the mobile application. A unique control
code has been utilized at firebase cloud to receive the control command from the
mobile application and the same command is forwarded to NodeMCU. Further,
the NodeMCU forwards the command to ATmega328 to control the 5volt relay
which helps in switching the irrigation system.
The experimental results in Table 1 shows that the 49 times out of 50 times
the data transmitted has been received at the end node. It confirms the per-
centage of successful data transmission is 98%. In Table 2, the control command
sent from mobile phone to the end node has accuracy of 100%. It means that
the measuring environmental parameters and controlling the irrigation system
are successfully demonstrated without fail which fulfils the requirements of the
proposed system for the said application.

Table 1. Data transmission

Parameters Data sent Data received Data received Data received

(ATmega328) (NodeMCU) (firebase) (in app)
Sensor’s 50 49 49 49
data
 IoT Based Multi-sensor DAQ for the Application of Smart Agriculture 341

Table 2. Control signal transmission

Parameters Data sent Data received Data received Data received

(from app) (firebase) (NodeMCU) (ATmega328)
Control 30 30 30 30
command

5 Conclusion

IoT based Multi-Sensor Data Acquisition System was designed and demon-
strated successfully for the application of smart agriculture. As per the require-
ment of the user, it provides the flexibility of controlling irrigation system in
two ways, one is manual control and another is automatic control. The system
automatically updates the data collected from different sensors in real time and
at the same time makes it available to the user through a mobile application.
As a result, it allows the user to maintain soil moisture and other parameters
within an allowable range to yield better crop. This developed system reduces
the involvement of human which makes it cost-effective and time-efficient. Fur-
ther, the proposed system was designed to reduce the data loss close to zero,
and found that the data loss in case of control signal transmission is 0% and in
case of data transmission it was 2%. In addition, the system was integrated with
solar tracker to provide a solar energy for the continuous operation.

Acknowledgment. The design and implementation was carried out at IIIT Senapati
Manipur with the support of TEQIP-III, NPIU and Government of India. Finally,
authors would like to thank Dept. of ECE as well as others who have helped in the
demonstration of DAQ system for the said practical application.

References
1. Al-Fuqaha, A., Guizani, M., Mohammadi, M., Aledhari, M., Ayyash, M.: Inter-
net of things: a survey on enabling technologies, protocols, and applications. IEEE
Commun. Surv. Tutor. 17(4), 2347–2376 (2015)
2. Kim, Y., Evans, R.G., Iversen, W.M.: Remote sensing and control of an irrigation
system using a distributed wireless sensor network. IEEE Trans. Instrum. Meas.
57(7), 1379–1387 (2008)
3. Liu, Y., Zhang, C., Zhu, P.: The temperature humidity monitoring system of soil
based on wireless sensor networks. In: 2011 International Conference on Electric
Information and Control Engineering, pp. 1850–1853. IEEE (2011)
4. Ivanov, S., Bhargava, K., Donnelly, W.: Precision farming: sensor analytics. IEEE
Intell. Syst. 30(4), 76–80 (2015)
5. Kaewmard, N., Saiyod, S.: Sensor data collection and irrigation control on vegetable
crop using smart phone and wireless sensor networks for smart farm. In: 2014 IEEE
Conference on Wireless Sensors (ICWiSE), pp. 106–112. IEEE (2014)
342 S. Kumar and N. Ch

6. Preejith, S., Alex, A., Joseph, J., Sivaprakasam, M.: Design, development and clin-
ical validation of a wrist-based optical heart rate monitor. In: IEEE International
Symposium on Medical Measurements and Applications (MeMeA), pp. 1–6. IEEE
2016 (2016)
7. Hari Ram, V.V., Vishal, H., Dhanalakshmi, S., Vidya, P.M.: Regulation of water in
agriculture field using internet of things. In: IEEE Technological Innovation in ICT
for Agriculture and Rural Development (TIAR), pp. 112–115. IEEE (2015)
8. Zhao, J.-C., Zhang, J.-F., Feng, Y., Guo, J.-X.: The study and application of the
IoT technology in agriculture. In: 2010 3rd International Conference on Computer
Science and Information Technology, vol. 2, pp. 462–465. IEEE (2010)
 Context-Oriented User-Centric Search
System for the IoT Based
on Fuzzy Clustering

Santosh Pattar1 , Sandhya CR1(B) , Darshil Vala1 , Dharamendra Chouhan1 ,

Rajkumar Buyya2 , Venugopal KR3 , S. S. Iyengar4 , and L. M. Patnaik5
1
IoT Lab, University Visvesvaraya College of Engineering, Bengaluru, India
[email protected]
2
University of Melbourne, Melbourne, Australia
3
Bangalore University, Bengaluru, India
4
Florida International University, Miami, USA
5
National Institute of Advanced Studies, Bengaluru, India

Abstract. The Internet of Things (IoT) paradigm envisions to support

the creation of several applications that aids in the betterment of the soci-
ety from various sectors such as environment, finance, industry etc. These
applications are to be user-centric for their larger acceptance by the soci-
ety. With the increase in the number of sensors that should are getting
connected to the IoT infrastructure, there is an augmented increase in
the amount of data generated by these sensors. Therefore it becomes
a fundamental requirement to search for the sensors that produce the
most applicable data required by the application. In this regard, context
parameters of the sensors and the application users can be utilized to
effectively filter out sensors from a large group. This paper proposes a
sensor search scheme based on semantic-weights and fuzzy clustering.
We have modified the traditional fuzzy c-means clustering algorithm by
incorporating the semantic and context attributes of the sensors to obtain
fuzzy clusters. During the query resolution phase, the query is directed
to the most appropriate cluster. These clusters are formed through the
use of linguistic variables rather than quantitative attributes and thus
aid in effective user-centric search results. Experimental results indicate
that the proposed scheme achieves better performance when compared
to the existing techniques.

Keywords: Context-aware · Fuzzy clustering · Internet of Things ·

Resource discovery · Semantic similarity · Sensor search

1 Introduction
The infrastructure of Internet of Things (IoT) is undergoing an expeditious devel-
opment, wherein several physical components (such as sensors, actuators, etc.,)
inclusive of the user are being linked to the digital world. These components
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 343–356, 2020.
https://doi.org/10.1007/978-981-15-3666-3_28
344 S. Pattar et al.

collaborate to capture, analyze, communicate and exchange data among them-

selves, allowing the development of intelligent applications that are beneficial to
human life [18]. As the number of devices being connected to the IoT environ-
ment is increasing, the volume of data that is being continuously generated and
exchanged by them is also increasing. This makes it difficult for the applications
to fetch the relevant IoT resource in the search space and thereby extract the
required data to provide solutions. Therefore, there is a need for an efficient
search system in the IoT that aids in extracting appropriate IoT resources from
the huge amount of devices connected in the IoT environment [12].
In regard to an IoT scenario, the term context refers to information that
describes a situation, circumstance or characteristics of an entity. Contemplating
this information in the IoT search techniques, enables application users to specify
their requirement in terms of sensor attributes predefined by the system. These
attributes include location, accuracy, type, battery life, etc [16]. Although there
are search techniques that consider context-based parameters of both the user
and IoT devices to find the relevant IoT resources, they are prone to two main
challenges namely, i.e, (i) performance of the search system decreases with the
increase in the number of connected devices in the IoT network (scalability),
and (ii) time required to resolve a given query increases with increase in the size
of the search space (communication overhead).
In this paper, we focus on designing a search system for the IoT based on
fuzzy logic that restructures the search space. It takes into consideration the con-
text parameters of both the users and IoT devices. Fuzzy logic deals with linguis-
tic uncertainty which is very sensitive and plays a vital role in processing context
information. Therefore, the user need not specify the requirements through a
precise quantitative value, it can rather be imprecise and also expressed using
natural language [22].
The rest of our paper is structured as follows. In Sect. 2, we present the
literature survey on the topics, context-based search for the IoT and fuzzy-based
IoT search systems. Section 3 defines the problem statement and elaborates the
importance of fuzzy logic in the IoT search systems. In Sect. 4, we describe in
detail, our proposed search system. Section 5 discusses on the implementation
and performance evaluation of our proposed search model and we conclude our
work in Sect. 6.

2 Literature Survey

In this section, we present an overview of the recent works that address the
sensor search challenge in the IoT. In the first subsection, we discuss the previous
studies related to the context-based search systems for IoT and in the second
subsection, we illustrate the recent works that apply fuzzy theory to the IoT
search applications. We put forth how our work differs from the existing works
as our contributions in this field.
 Context-Oriented Fuzzy Based Search System for the IoT 345

2.1 Context-Based Search Systems for the IoT

Context-aware information is a core component of ubiquitous computing. A

bird’s eye view of the recent works on context-oriented search systems for locat-
ing the IoT resources are as follows. Search functionality becomes critical as
and when more sensors are deployed into the IoT environment. To address this
problem, Perera et al. [17] presented a model called CASSARAM, which could
effectively select a small group of relevant sensors from a large group of sensors
bearing similar functionality and capabilities. On receiving the input query from
the user, a score called Comparative-Priority based Weighted Index (CPWI) is
calculated by comparing the sensor’s context property and user’s priority values
for each sensor in the search space. Finally, the sensors are indexed using the
generated CPWI score and returned to the user. Even though this framework
made significant progress, the process is extremely complex as it is inefficient
when applied to a large number of sensors. To overcome this challenge, machine
learning techniques can be utilized. Ebrahimi et al. [9] proposed an adaptive
meta-heuristic cluster-based approach for sensor search that increases the scala-
bility and efficiency of context-aware search algorithms. An ant-based clustering
technique is used to create clusters containing sensors that are identical to each
other based on their context properties. On receiving the query from the user, it is
forwarded only to a suitable cluster that matches the query and thus returns the
result to the user. This process reduces the size of search space thereby increas-
ing the efficiency of the search technique as mentioned earlier. This method also
proved to have lower execution time in comparison with CARASSAM.

2.2 Fuzzy-Based Systems for the IoT

Few recent works on fuzzy-based IoT search systems are discussed in this sub-
section. Based on the data generated by the sensors, Truong et al. [19] put forth
a content-based search approach built through fuzzy theory. Although this work
achieves low communication overhead, it does not provide user-centric results.
Similarly, Truong et al. [20] in one of their other works have proposed an algo-
rithm for computing similarity index of the sensors deployed in Web of Things
(WoT) using fuzzy rules. This approach is highly accurate in computing sim-
ilarity parameters between sensors of the same region but failed to compute
similarity among sensors between two different regions. Jun et al. [21] presented
a fuzzy logic-oriented information retrieval mechanism in IoT integrated Intel-
ligent Transport System (ITS) application. However, this mechanism handles
only simple fuzzy relations that are to be defined beforehand in the ontology.
In Table 1, we compare the different search techniques proposed for the IoT
systems.
From the above works, it can be inferred that the context-based search tech-
niques developed by several authors could not handle user requirements specified
through natural language or linguistic variables. Also, the existing fuzzy-based
search algorithms lack the applicability of context parameter requirement of the
user and the sensor attributes. In this regards, we have devised a search system
based on context-oriented weighted fuzzy c-means clustering algorithm to reduce
346 S. Pattar et al.

the sensor search space in the IoT and thereby decrease the communication over-
head for efficient query resolution. Our approach is also capable of handling the
queries given by the user through linguistic variables.

Table 1. Comparison of Different Existing Search Techniques in the IoT

Authors Approach Advantages Drawbacks

Context-based search techniques
Perera CARRASAM Distributed search and Inefficient in the
et al. [17] framework rank model based on aggregation and
context information management of
context-properties for a
large number of sensors
Ebrahimi A meta-heuristic User queries are Offline cluster formation
et al. [9] Ant-Clustering algorithm forwarded to a particular process is time
for search space matching cluster only consuming
clustering
Fuzzy-based Search Techniques
Truong Content-based fuzzy Achieves low Fails to provide
et al. [19] logic technique communication overhead user-centric information
since their preferences
are not considered
Truong Computation of Highly accurate query Could not compute
et al. [20] similarity index between resolution similarity index of
the sensors using fuzzy sensors between two
logic different regions
Jun Information retrieval User’s preference was Applicable only to
et al. [21] system based on fuzzy taken into consideration pre-defined simple fuzzy
logic for ITS relations

3 Problem Definition and Background

In this section, we outline the sensor search problem considering user queries
and sensor properties. In the first subsection, we formulate the problem math-
ematically along with an example. We discuss the importance of fuzzy logic
applicability for IoT search systems in the second subsection.

3.1 Problem Definition

An IoT application typically comprises of a large number of sensors. Depending

on the service that the system needs to provide, the application has to choose
a subset of sensors from the available large set. Due to this, there arises a need
to search and select sensors in the most efficient and effective possible way.
The problem at hand can be illustrated as follows. Consider a set of sensors
S = {s1 , s2 , s3 , . . . , sn }, with dynamic size, i.e., where n is number of sensors
present in the search space (collection of sensors over which search is to be per-
formed). Let, A = {a1 , a2 , a3 , . . . , am } be the set of properties possessed by the
 Context-Oriented Fuzzy Based Search System for the IoT 347

sensors (such as sensor type, data output, geographical location, life-time, accu-
racy, error rate, bandwidth, reliability, etc). These properties are used as param-
eters in the search process to select an appropriate set of resultant sensors. They
are classified into two types namely, content-properties and context-properties.
The former set of properties consists of the parameters of the data emitted by
the sensors (represented by the set O), while the latter set describes the situ-
ation based properties (context) of the sensors (represented by the set C). It
is necessary for us to find out an optimum subset P ⊆ S such that it satisfies
the conditions specified by the user. Some of the attributes of set A change over
time. Thus, a sensor s ∈ S can be represented as,

s: T → F (1)

where, T represents the set of time intervals within which sensor reading are
extracted and F is the set of attribute instances of a sensor, where F ⊆ A i.e.,
the sensor readings can either belong to output-data set (O) or context-based
attribute set (C). An IoT application has to capture user requirements in the
form of a query in order to perform search and select operation on the sensor
set S. We use query Q to represent the user-requirements, that in-turn specifies
the sensor properties x ∈ A required by the user along with the relationships
among them. This query is used to discover the “optimal subset” of pn sensors
that match the user-requirements. Thus the query Q, can be formulated by the
function as follows,
Q : T × X × pn → F (2)
where, the query poses a request to locate pn sensors from S, such that they
have x ∈ A properties at time-stamp t ∈ T . F ⊆ S defines the subset of “optimal
sensors” obtained after the query has been executed.

Example. Let us consider an IoT application designed for an ITS system with
n sensors. The objective of this application is to offer search service to the user
and help him/her in locating appropriate sensors for their use. Thus, S repre-
sents the set of sensors deployed in the ITS environment. Let the attribute set
defined for the sensor set S be A = {reading − data, latitude, longitude, battery −
lif e, reliability, time − stamp}. For a given set of four temperature sensors, let us
assume that set O = {{(30deg, 1.00 pm), (36deg, 1.02 pm)}, {(40deg, 1.05 pm)},
{(42deg, 1.10 pm)}} consists of temperature measured by the sensors at a partic-
ular time interval. Similarly, C contains such readings for two context-properties,
battery-life and reliability. A user query on such a search space can take form
like, To retrieve sensors that measure temperature (sensor type context-property)
between the time interval 1.00 pm to 1.15 pm (content property). Thus, a query
can have constraints on both the set of content as well as context-properties.

3.2 Fuzzy Logic and Its Importance in IoT

Everything present in the world we live in is a matter of measure. We encounter
many problems in our real life that are challenging to understand, measure or
solve using the concept of crisp logic (binary: 2 values only). This complexity
348 S. Pattar et al.

arises due to the uncertainty associated with the problems, making it difficult
to arrive at an accurate solution. These challenges pave the way for fuzzy logic-
based approaches. Fuzzy logic is an effective technique that handles the ambi-
guity present in complex problems, thereby providing the best possible solution.
Unlike binary logic that accepts only either of the two values 0 or 1 (true or
false), fuzzy logic is multi-valued. It deals with approximation rather than pre-
cision. Fuzzy logic also provides a medium to represent linguistic variables (in
natural language) that in-turn express human’s reasoning capabilities [8].
Let us take an example from an IoT search requirement scenario to better
understand the importance of fuzzy logic and how its association with context
parameters uplifts the competence of the search system. Two users Ravi and
Ranjan intend to acquire road traffic congestion data captured by the sensors
in a certain city, say Bengaluru, for different purposes. Ravi prefers to make
use of data collected from highly accurate sensors (say, 95% of accuracy) while
Ranjan is interested in using the information aggregated by the moderately
reliable sensors (say, 70% of reliability). Here, the terms ‘accurate/accuracy’,
‘reliable/reliability’ are referred to as “linguistic descriptor” that captures the
user requirements.
The existing search systems designed for the IoT require queries to be in
a crisp format and thus fails to provide a user-centric query resolution mecha-
nism. Fuzzy logic can be used to handle this challenge. In traditional crisp logic
methodology, the linguistic descriptor is expressed either as ‘accurate’ (value:
1) or ‘not accurate’ (value: 0), while in fuzzy systems, the linguistic variable is
expressed as ‘less-accurate’, ‘accurate’, ‘more-accurate’ and so on, where each
variable is associated with a truth value between 0 to 1 operating on the basis
of membership functions. If a sensor has an accuracy value of 0.9, then 0.9 is
called as the “degree of membership” of the sensor in a fuzzy set that represents
accuracy context-property.
In this work, we design a fuzzy logic-based search mechanism for the IoT sys-
tem. Initially, properties of the sensors are used to create clusters that hold sim-
ilar sensors (through our proposed semantic weight-based fuzzy c-means search-
space clustering algorithm). Then, a user query that expresses search constraints
in linguistic terms is mapped to one of the appropriate cluster (through the
selection of cluster head). Finally, a subset of sensors that matches the given
user requirements are selected and returned. This approach offers user-centric
results by considering the fuzzy representation of both sensor properties and
user constraints. Also, the overall communication overhead for query resolution
is effectively reduced due to the use of clustering approach.

4 The Proposed Sensor Search Scheme

In this section, we describe in detail the execution flow and construction of the
proposed sensor search model delineating the techniques used for query process-
ing, search space structuring and query resolution.
 Context-Oriented Fuzzy Based Search System for the IoT 349

Extract Query Fuzzy-based Search-

Query
Constraints space Structuring

Phase - I Phase - II

Search
Result Search within Cluster Head
Cluster Selection

Phase - III

Fig. 1. Overall workflow of the proposed sensor search scheme

4.1 Execution Flow of the Search System

Our proposed sensor search scheme executes in three phases as depicted in Fig. 1.
The processes carried out in each phase is explained in the following.

– Phase - I: Query Processing. Initially, user submits a query specifying the

search constraints. A graphical user interface captures this query, where two
kinds of user requirements are to be input i.e., (i) indefinite - these are non-
negotiable constraints, and (ii) definite - these requirements are negotiable.
Further, these constraints are expressed in the form of values for linguistic
variables that are predefined by the search system.
– Phase - II: Search Space Structuring. This is a offline phase where the data
from the sensors are gathered and used for clustering. To group similar sensors
based on their context-data and functionality, we use the semantic weighted
fuzzy c-means clustering algorithm which is explained in the subsequent
subsection.
– Phase - III: Query Resolution. In this phase search results containing appro-
priate sensors that match the given query constraints are constructed in two
subsequent sub-phases. First, a relevant cluster (represented by the cluster
head) is chosen and then the query is forwarded to it. Here, the cluster head
is selected by calculating the semantic similarity score (discussed in Sect. 4.2)
between the query and cluster head. In the second sub-phase, we employ the
multi-dimensional Euclidean distance to select the appropriate sensors in the
chosen cluster.

To abet the search technique, it is essential to annotate the sensors with

semantic knowledge using ontology [15]. Hence, we have used Semantic Sensor
Network Ontology (SSN) to interpret the attributes of the sensors and estab-
lish a link between them [11,14]. It also enables us to achieve interoperability
among heterogeneous devices and applications in the IoT [7]. The proposed
search scheme allows the user to query using imprecise values that are mod-
eled as linguistic variables. Through the proposed fuzzy clustering mechanism
these linguistic terms aid to obtain user-centric results. Also, the communication
overhead of the proposed search system is drastically reduced by incorporating
350 S. Pattar et al.

the search space structuring and cluster selection process, thus resulting in an
efficient search performance.

4.2 Fuzzy-Based Search Space Cluster Formation

We have employed a semantic weight-based fuzzy c-means algorithm to form

clusters, wherein each cluster consists of sensors with similar functionality and
properties. Fuzzy C-Means (FCM) approach was first put forth by Bezdek et al.
[6]. The drawback of this standard algorithm is that it treats every sensor in the
search space equally i.e., every sensor is given the same importance and thus
increases the processing time. Hence, a Weight-based Fuzzy C-Means (WFCM)
clustering approach is introduced by Hung et al. [10]. In this approach, weight
is calculated for each sensor in the search space where the weight defines the
sensor’s importance.
The main objective of the proposed clustering technique is to create a mem-
bership matrix for the given search space (i.e., S) by considering the attributes
of the sensors (i.e., A). As it is impossible for a user to precisely define the
query constraints quantitatively which leads to imperfect similarity search. Being
motivated by the semantic techniques like ontologies that perfectly capture the
descriptive features of the sensor data, we utilize them to define a similarity score
[13]. Given two sensors, si and sj , the similarity between them is represented
by sim(si , sj ). We extract the ontological concepts for these sensors (p denotes
the number of properties that are similar between the sensors) and calculate the
score using the below equation.

p Csim (si ,sj )
k=1 p , p>0
sim(si , sj ) = (3)
0, p=0
Here, Csim is the semantic similarity between two given ontological concepts
(i.e., linguistic variables) and is calculated using the Eq. 4.
⎧
⎪
⎨1, if linguistic variables have same terms
Csim (ci , cj ) = 0, if linguistic variables have dif f erent terms
⎪
⎩ ζ
|height(ci )−height(cj )| , otherwise
(4)
where ζ is the balancing factor that controls the contribution of the number of
levels between the given concepts to the similarity score.
We modified the technique presented by Hung et al. [10] in order to make it
suitable for sensor search problem. Algorithm 1 outlines the steps of our proposed
clustering scheme. Our objective is to cluster the sensor set S into z clusters.
The proposed approach is an iterative algorithm where initially random values
between 0 and 1 are assigned to the membership matrix U (Step-1 in Algorithm
1), and then cluster heads, denoted by set H, are calculated. Here, the semantic
similarity score is used as a weight to calculate H. Generally, the semantic score
of sensors that possess similar properties is high and thus it is multiplied with
 Context-Oriented Fuzzy Based Search System for the IoT 351

Algorithm 1. Semantic Weight-based Fuzzy C-means Search-space Clustering

Input : Sensor Search Space, S = {s1 , s2 , . . . sn };
Number of Clusters, z;
Output: Cluster Heads, H;
1: k ← 0
2: repeat
3: k←k+1
# Step-1: Setup the membership matrix.
4: for i ← 1 to z do
5: for j ← 1 to n do
6: uij ← random value between 0 and 1.
7: end for
8: end for
# Step-2: Calculate Cluster Heads.
9: for i ← 1 to z do
10: hi ← calculate cluster head using Equation 5.
11: H ← H ∪ hi
12: end for
# Step-3: Update the Membership Values.
13: for i ← 1 to z do
14: for j ← 1 to n do
15: uij ← update membership value using Equation 6.
16: U ← U ∪ uij
17: end for
18: end for
19: until U k − U k−1  < β
20: return H

the membership value to obtain the updated weighted cluster heads. Equation 5
is used for this purpose, where hi is the cluster head for ith cluster, m is the
fuzzification parameter, and sj is a particular sensor among n sensors.


n
sim(j, k) ∗ um ij
hi = n sj , j = 1, 2, . . . n. (5)
t=1,k=j sim(j, t) ∗ um
tj
k=1,k=j

In the third step, the membership values are updated using the below
equation.
−1/(m−1)
sj − hj 
uij = z −1/(m−1)
(6)
i=1 sj − hi 
The above steps are repeated until the difference between the membership
matrix of current and previous iterations are below a pre-defined tolerance value
(i.e., β).
352 S. Pattar et al.

5 Experiments and Results

In this section, we describe the methods of experimentation, assumptions taken
into consideration and datasets used to evaluate our proposed technique.

5.1 Implementation
We have implemented our proposed prototype search technique in Java. The
SSN ontology required for sensor data modeling is handled by Apache Jena
Framwork (Version 3.12.0) [3] and SPARQL querying language [5]. We have also
made use of libraries like Commons Math from Apache [2] to calculate semantic
weights and euclidean distance. For our experimental purposes, we have used
environmental datasets MesoWest [4] and Air Quality Sensor Dataset [1] that
capture the air quality measurement reading recorded from sensors. As at present
there does not exists public dataset that houses context-properties of the sensors,
we have randomly generated ten context-properties for each of the sensors in the
above datasets. These properties include location, accuracy, type, battery life,
life-time, error rate, bandwidth, reliability, energy, and availability. To evaluate
and measure the performance of the proposed search system, we used a computer
with Intel(R) Core i5-4200M 2.50 GHz processor with 8 GB RAM. The dataset
constructed consists of a total 1,00,000 readings.
After, conducting several empirical tests we have set the optimal values of
the parameters as ζ = 0.7, z = 10, m = 1.75 and β = 0.5. In the following, we
discuss them: (i) The balancing factor (ζ) in semantic score calculation controls
the contribution of sensor attributes to the cluster formation, thus it decides
the quality of the clusters. In this work as we used ten context-properties which
mostly have three linguistic terms, the ontology thus has a tree with a maximum
depth of 10 and at most 30 nodes (concepts). Here, a factor of 0.7 covers all the
context-properties with these values of the ontology’s depth and the number of
nodes. (ii) As the number of context-properties in the dataset is limited to 10, the
number of clusters (z) at this value achieves better search results. (iii) The fuzzi-
fication parameter (m) controls the magnitude of membership value. A higher
value will magnify the addition of context-property, while a lower value decreases
the role of the context-property to clustering process. It is observed that a 1.75
value balances the membership matrix with all the context-properties. (iv) The
tolerance value (β) controls the number of iterations required for cluster forma-
tion. We noted that at 0.5 value of β, the difference in membership updation is
no longer decreasing and thus effectively stops the clustering process.

5.2 Performance Evaluation and Analysis

In this subsection, we compare the proposed fuzzy based search technique with
one of the recent search technique in the area of sensor search which is based
on ant-colony optimization (AntClust). Among many works, this is the only
technique that uses a meta-heuristic approach for clustering of sensors along
with considering many sensor context-properties for implementation [9]. We have
 Context-Oriented Fuzzy Based Search System for the IoT 353

constructed a total of five queries by combining different number of sensor-

properties along with several combinations of their linguistic terms and report
the mean of 10 executions. For comparison we have considered some common
parameters that are the characteristic features of the search system which are
detailed as follows:

Processing Time. It is the time taken by the search system to resolve a given
query (i.e., search space structuring, query processing, and resolution). As seen
in Fig. 2, the proposed technique takes comparatively less time than AntClust.
This is attributed to the fact that AntClust involves a two-phase cluster for-
mation process where at first level sensors are grouped based on their type and
further at second level they are clustered based on their attributes. However,
in our approach, only one phase is involved where semantic-based clustering
technique groups the sensors with similar descriptive (linguistic) properties, we
thus eliminate the two-way cluster formation and search. Due to this, the overall
communication overhead in the search system is drastically reduced.

Fig. 2. Performance comparison of proposed work with AntClust.

Accuracy of Cluster Formation. In this experiment, the quality of clusters

formed by the search technique is measured. The obtained results indicate that
our approach creates high-quality clusters than AntClust technique (Fig. 3). As
the search space involves multi-modal and multi-dimensional attributes of the
sensors, it is a challenging task to define a perfect boundary that separates
the sensors that belong to more than one cluster. AntClust technique makes
use of most similar sensor (MSS), a memory-based score to assign a sensor
to a particular cluster. With a high feature search space (i.e., a large number
of context-properties) this approach fails to identify the belongingness of the
sensor to a particular group. However, owing to the semantic similarity score
354 S. Pattar et al.

our clustering approach can easily identify the sensor’s affinity to a particular
cluster as all the matching context-properties of that cluster are considered.

Fig. 3. Comparison of accuracy (quality) of cluster.

Search Accuracy. We conducted another experiment to measure the effective-

ness of the proposed approach with respect to the search results. As seen in Fig. 4
search accuracy of the proposed work is better than the existing AntClust algo-
rithm. As we earlier mentioned, our proposed algorithm aids in forming precise
clusters and also the use query is forwarded only to the most acceptable clus-
ter due to which the accuracy of search results are significantly higher. Thus,
user-centric results are provided the proposed search scheme due to the use of
semantic weights and fuzzy clustering technique.

Fig. 4. Comparison of search accuracy.

 Context-Oriented Fuzzy Based Search System for the IoT 355

6 Conclusions
Searching for relevant sensors among a large pool of connected devices in the IoT
has become a significant task with the creation of several prominent applications
across various sectors of the society. Thus, numerous techniques are proposed in
the past to address this problem, but they do not consider the user’s context
and also take query in rudimentary quantity values. In this paper, we have
devised a fuzzy-based context-oriented search algorithm that aids in forming
clusters through semantic weights. The submitted user query is directed to the
cluster that is most appropriate to the context of the query. Experimental results
demonstrate that the proposed search space structuring technique drastically
decrease the search processing time owing to the effective semantic-based cluster
head selection in comparison to the meta-heuristic approach. It also achieves
lower communication overhead through the one-way inter-cluster search. Also,
when compared with the existing work our method yields effective results and
thus provide a superior user-centric search system.

References
1. Air Quality Sensor Dataset. https://github.com/BathHacked/documentation/
wiki/Bath-Air-Quality-Sensor-Data. Accessed 20 Aug 2019
2. Apache Commons Math. https://commons.apache.org/proper/commons-math/.
Accessed 20 Aug 2019
3. Apache Jena. https://jena.apache.org/. Accessed 20 Aug 2019
4. MesoWest Dataset. http://mesowest.utah.edu. Accessed 20 Aug 2019
5. SPARQL Query Language. https://www.w3.org/TR/sparql11-query/. Accessed
20 Aug 2019
6. Bezdek, J.C., Ehrlich, R., Full, W.: FCM: the fuzzy c-means clustering algorithm.
Comput. Geosci. 10(2–3), 191–203 (1984)
7. Desai, P., Sheth, A., Anantharam, P.: Semantic gateway as a service architecture
for IoT interoperability. In: IEEE International Conference on Mobile Services, pp.
313–319. IEEE (2015)
8. Dilli, R., Argou, A., Pilla, M., Pernas, A.M., Reiser, R., Yamin, A.: Fuzzy logic and
MCDA in IoT resources classification. In: Proceedings of the 33rd Annual ACM
Symposium on Applied Computing, pp. 761–766. ACM (2018)
9. Ebrahimi, M., ShafieiBavani, E., Wong, R.K., Fong, S., Fiaidhi, J.: An adaptive
meta-heuristic search for the Internet of Things. Future Gener. Comput. Syst. 76,
486–494 (2017)
10. Hung, C.C., Kulkarni, S., Kuo, B.C.: A new weighted fuzzy c-means clustering
algorithm for remotely sensed image classification. IEEE J. Sel. Top. Sign. Proces.
5(3), 543–553 (2011)
11. Lefort, L., et al.: Semantic sensor network. XG final report (2011)
12. Pattar, S., Buyya, R., Venugopal, K.R., Iyengar, S.S., Patnaik, L.M.: Searching for
the IoT resources: fundamentals, requirements, comprehensive review, and future
directions. IEEE Commun. Surv. Tutorials 20(3), 2101–2132 (2018)
356 S. Pattar et al.

13. Pattar, S., et al.: Progressive search algorithm for service discovery in an IoT
ecosystem. In: 2019 International Conference on Internet of Things (iThings)
and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber,
Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData),
pp. 1041–1048 (2019)
14. Pattar, S., et al.: Ontology based service discovery for intelligent transport systems
using Internet of Things. In: Fourteenth International Conference on Information
Processing (ICInPro), pp. 223–225 (2018)
15. Perera, C., Vasilakos, A.V.: A knowledge-based resource discovery for Internet of
Things. Knowl. Based Syst. 109, 122–136 (2016)
16. Perera, C., Zaslavsky, A., Christen, P., Georgakopoulos, D.: Context aware com-
puting for the Internet of Things: a survey. IEEE Commun. Surv. Tutorials 16(1),
414–454 (2014)
17. Perera, C., Zaslavsky, A., Liu, C.H., Compton, M., Christen, P., Georgakopoulos,
D.: Sensor search techniques for sensing as a service architecture for the Internet
of Things. IEEE Sens. J. 14(2), 406–420 (2014)
18. Roopa, M., Pattar, S., Buyya, R., Venugopal, K.R., Iyengar, S., Patnaik, L.: Social
Internet of Things (SIoT): foundations, thrust areas, systematic review and future
directions. Comput. Commun. 139, 32–57 (2019)
19. Truong, C., Römer, K.: Content-based sensor search for the Web of Things. In:
Proceedings of the GLOBECOM - IEEE Global Communications Conference, pp.
2654–2660 (2013)
20. Truong, C., Römer, K., Chen, K.: Fuzzy-based sensor search in the Web of Things.
In: Proceedings of International Conference on the Internet of Things, (IoT), pp.
127–134 (2012)
21. Zhai, J., Liang, Y., Yu, Y., Jiang, J.: Semantic information retrieval based on fuzzy
ontology for electronic commerce. JSW 3(9), 20–27 (2008)
22. Zhang, C.: FuzWare: a fuzzy-based middleware for context-aware service. In: IEEE
2nd Advanced Information Technology Conference (IAEAC) on Electronic and
Automation Control, pp. 1181–1185 (2017)
 Implementation of a Smart Embedded
System for Passenger Vessel Safety

F. M. Javed Mehedi Shamrat, Naimul Islam Nobel, Zarrin Tasnim(B) ,

and Razu Ahmed

Department of Software Engineering, Daffodil International University,

Dhaka, Bangladesh
[email protected], [email protected],
[email protected], [email protected]

Abstract. An automated embedded system with overweight and loca-

tion detection of a vessel has been planned, developed and executed. To
eliminate overweight issues and able to locate the vessel for stopping acci-
dents, an embedded system has been developed. Based on “Archimedes
Principle Formula” an algorithm has been formed, it works on data that
have been collected by the “water floating switch sensor”. A sensor called
“Water floating switch” used to detect the exceeding water level and send
the data to the server purpose of monitoring an overweight issue. Pro-
gramming language (python) with its default library has been utilized
to code and some hardware components (Like raspberry pi B, microcon-
troller, water floating switch, GPS antenna, monitor, etc) are used for
the significant execution of the algorithm. This embedded system can be
accomplished in any type of vessel, utilize this system above the vessel
as well as a passenger vessel, general cargo vessel, etc.

Keywords: Embedded system · Safety · Microcontroller · Floating

switch · Automation · IoT · Vessel · Raspberry pi zero

1 Introduction

Even though one of the most commonly used means of transport in Bangladesh
is the passenger’s vessel, the rate of fatal accidents has increased at an alarming
rate according to the department of shipping. In 38 years (1977 to 2013), the
number of death totaled to 4420, 520 people were injured and 400 people are still
missing. Overloading being the leading cause accounts for the maximum share
of 56% of the accidents in water routes. In order to earn maximum profit, oper-
ators overload their vessels with passengers and cargos. Very common scenery is
a passenger vessel loaded with cargos in the upper deck as well as letting passen-
gers in the upper deck. As a result, the top weight increases making the vessel
unstable. In most cases, vessels met accidents due to being 4/5 times overloaded

Supported by Daffodil International University, Dhaka, Bangladesh.

c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 357–370, 2020.
https://doi.org/10.1007/978-981-15-3666-3_29
358 F. M. J. M. Shamrat et al.

than its capacity. Sometimes passenger vessel operators load passengers from
riverbanks and waterways in order to earn some extra money making the vessel
overweighted. To detect and prevent accidents due to overweight, an IoT based
embedded system has been developed called Safety measuring model for a pas-
senger’s vessel. The prototype of this embedded system sends the weight status
and current location of the vessel to the server via the internet. If the vessel at
any point detects overweighting, the signal is sent to the server which allows the
automated embedded system to let the coast guards know of the vessel’s condi-
tion and take action accordingly. In this system, two types of sensors are used.
First is the float switch sensor to get the reading of the weight of the vessel in
the water. Another is a GPS tracker to trace the current location of the vessel. A
microcontroller receives the raw data and sends it to a Raspberry pi through the
internet from further processing. The Raspberry pi analyzes the received data
to make a decision by identifying the weight status of the vessel. Based on that
decision the coast guard is notified if any action is to be taken. With this embed-
ded system, vessel overweight incident can be prevented and countless lives can
be spared. The rest of the paper discusses the literature review in Sect. 2, the
proposed system methodology, the mathematical exploration consists of Sect. 3,
in Sect. 4 discussed each component, in Sect. 5 broadly discussed the embedded
system with graphical visualization and implementation set.

2 Literature Review
In recent years, a number of research papers were published showing the types
of vessels, vessel maintenance, and architecture and production costs. Yet very
few kinds of research are done to ensure safety, measuring the weight of vessels
to avoid accidents caused by overweight.
The primary reason for each accident study is to give decision-makers sub-
stantial and solid data so as to settle on educated and ideally better choices. In
this paper, mishap data for passenger vessels gathered from Bangladesh Inland
Water Transport Authority (BIWTA) is introduced and an investigation is com-
pleted to decide the normal reasons for accidents on passenger vessels [1].
The purpose of this paper is to structure the applied model concerning an
advancement overloaded sensor that can be utilized to distinguish and reduce the
overburden issue of passenger ships. To beat this issue, an inventive innovation
joins the lift ideas utilizing the load sensor (HCC-High Capacity Compression)
and utilize the batching controller to set the base and most extreme limits are
prescribed to be installed at the entrance dot of the passenger’s vessel [2].
In this paper, it has been seen that the regular reasons for accidents are
impacts and contacts, fire and blasts, overburdening, harsh climate, and so on.
At long last, in light of this investigation, various proposals have been proposed
so as to guarantee more prominent security of passengers [3].
The exploration disclosed that the significant reasons for waterway accidents
are impact, Nor’wester or tempest, overloading, dependability disappointment,
 Implementation of a Smart Embedded System for Passenger Vessel Safety 359

unnecessary current, base harm. The investigation demonstrates that cargo ves-
sels and passenger’s vessels are confronting more accidents contrasted with dif-
ferent kinds of vessels. It was likewise seen that numerous mishaps and appli-
cable data frequently stay unreported. Besides, the recorded waterway accident
information is likewise very short data for leading standard research works. At
last, a few suggestions are advanced with a vision to develop a more secure
waterway transportation system for Bangladesh and for further research and
investigations [4].
The general point is to acquire an expanded comprehension of measurements
in maritime security culture, its impact on wellbeing the executives and its
connection to the International Safety Management Code. This paper presents
results from two security culture studies directed on two sorts of vessels. The
outcomes indicated contrasts among officials and team in how wellbeing society
measurements were reported [5].
Addressing the research gap in the researches done previously on passenger’s
vessels, an embedded system is proposed with the approach to make sure the
wellbeing of the passenger’s vessel. The embedded system is developed in such a
way that it will provide the overweight status and location of the vessel to help
prevent any accidents before it happens. The developed embedded system will
help prevent vessels from loading more than its capacity thus avoiding accidents
due to overweight and save lives every day.

3 Proposed System

In the proposed embedded system, the first concern is to detect or take the read-
ing of the value of the current water level. This collected data will be analyzed.
A breakpoint needs to be determined to measure if the water level or the weight
of the vessel is exceeding beyond the expected level. To determine the breaking
point, the weight of the vessel is to be calculated. It all comes down to a rule
called “buoyancy” and can be clarified by what is known as “Archimedes’ Prin-
ciple.” What this rule fundamentally states is that a floating object uproots a
measure of fluid equivalent in weight to the heaviness of the object. This implies
the measure of water uprooted by a boat, regardless of how full that vessel is,
weighs as much as the boat. The weight is identified with the volume of water
displaced by,

Weight = (Gravity) ∗ (Density of Water) ∗ (Volume of water displaced).

This is the place the dimensions of the boat become significant. The volume
of water uprooted by the boat is equivalent to the volume involved by the piece
of the boat that is underneath the surfaces of the water. When the volume of
dislodged water is more prominent than the all total out volume of the boat, it
implies that the highest point of the boat will never again be above the water
and the boat will sink [6].
360 F. M. J. M. Shamrat et al.

We can locate the absolute upward force on the can be applied by water
pressure (which we call the buoyant force Fbuoyant ) by basically taking the dis-
tinction between the magnitudes of the upward force Fup and downward force
Fdown ,
Fbuoyant = Fup − Fdown (1)
“Archimedes Principle Formula” is recognized as [7],
F = ρgV (2)
Where, Buoyant force of a given body = F,
Volume of the displaced fluid = v,
acceleration due to gravity = g.
As we see that density ρρ = ρf − ρg. Here ρf is the density of fluid and ρg
is the density of the body. Accordingly, the principle can also be expressed as,
F = (ρf − ρg)gV (3)
Archimedes theory equation is useful in finding the buoyant force, the volume
of the dislodged body, the density of fluid or density of body if a portion of these
numeric values are known or firstly need to find out these numeric values to
implement Archimedes formula.
In Figs. 1 and 2 we have shown the flowchart diagram and process of the
proposed model of the embedded system that we have actually done in the
embedded system.

Fig. 1. Flowchart illustration of the proposed embedded system.

Fig. 2. The procedure of the proposed embedded model.

 Implementation of a Smart Embedded System for Passenger Vessel Safety 361

3.1 Mathematical Exploration

To build up the mentioned embedded system, water level raw data are gathered
by water float switching sensors. The sensors do the job of unmistakable sorts of
critical merit. The sensor collects the data by applying the algorithm and sends
it to the Raspberry Pi server (Node-Red), to make the decisions. The sensor will
detect the current water level like max, min and normal.

Algorithm 1. Mathematical Exploration

1: Displaceed Fluid (DsF) = V
2: Acceleration due to Gravity (AG) = g
3: Density of Fluid (DF) = pf
4: Density of the Body (DB) = pg
5: Density (pp) = pf – pg
6: So, BF = pp x gv
7: Sensor = s
8: Water Level = w
9: Water Level Exceed = Wmax
10: Water Level Minimum = Wmin
11: Water Level Normal = Wn
12: Read the value from s
13: S.value = w1,w2,w3,. . . . . . ,wn;
14: If w value is equal or more than Wmax (w >= Wmax)
15: W == Exceed
16: If w value is equal or less than Wmin (w <= Wmin)
17: W == Defect
18: If w value is equal or more than Wmin (w == Wn)
19: W == Normal
20: Later than a needed time.
21: Restart at 1 [Auto Start Loop process ()]

4 Explanation of Applied Components

Building up the proposed implanted vessel safety system, the following hard-
ware component is used. Here, the hardware with their figures and portrayal are
introduced. Component required raspberry pi B, Microcontroller Node MCU
(version 2), water level floating sensor, GPS antenna, Accessible Wi-Fi, output
monitor.

4.1 Raspberry pi B

The Raspberry Pi is ease, MasterCard assessed PC that associates with a PC

screen or TV. It is a capable little contraption that enables individuals of all
ages to examine the preparing and to make sense of how to program in dialects
362 F. M. J. M. Shamrat et al.

like Scratch and Python. It contains 1 GHz single-focus CPU, 512 MB RAM,
Mini HDMI port, Micro USB OTG port, Micro USB control, HAT-impeccable
40-stick header, Composite video and reset headers, CSI camera connector [8].
In Fig. 3 we showed the image of Raspberry pi zero.

Fig. 3. Raspberry pi B.

4.2 Microcontroller Node MCU

The NodeMCU is an open-source firmware and improvement pack that makes

us Prototype our IOT thing inside two or three Lua substance lines. Open-
source, Interactive, Programmable, Low cost, Simple Smart, and WI-FI engaged.
The Development Kit subject to ESP8266, organizes GPIO, PWM, IIC, 1-Wire,
and ADC in all cases. Power your improvement in the fastest way mixes with
NodeMCU Firmware! USB-TTL included, attachment and play, 10 GPIO, each
GPIO can be PWM, I2C, 1-wire, FCC CERTIFIED WI-FI module, PCB gath-
ering mechanical assembly [9]. In Fig. 4 we demonstrated the image of the micro-
controller.
 Implementation of a Smart Embedded System for Passenger Vessel Safety 363

Fig. 4. Microcontroller Node MCU.

4.3 Water Floating Switch Sensor

The switch can measure or monitor the water level by moving up and down in
its row. It’s basically made only for counting water level data. In Fig. 5 we have
shown the floating switch [10].

Fig. 5. Water floating switch.

364 F. M. J. M. Shamrat et al.

4.4 GPS Antenna

Give Ultimate GPS V3 a boost with this outer dynamic antenna. This GPS
radio wire draws about 10mA and will give an extra 28 dB of increase. It has a
5-m long cable so it will effortlessly reach any place that needs it to. The antenna
is magnetic so it will adhere to the highest point of a vehicle or truck. Used to
detect the location of the specific object [11]. In Fig. 6 we have shown the GPS
antenna.

Fig. 6. GPS antenna.

By assembling all of these elements, the mentioned embedded system has

been planned and developed. Figure 7 illustrates the design of the exhaustive
system.
 Implementation of a Smart Embedded System for Passenger Vessel Safety 365

Fig. 7. Design of the exhaustive system.

5 Algorithm Approach for the Proposed Embedded

System

To plan the programming fragment of the proposed model, every one of the
duties is summarized and put in a progression of procedures with the goal that
one can interpret them in programming code. An algorithmic methodology for
the arrangement of activities is offered below (Fig. 8):

Fig. 8. The algorithmic procedure of the proposed embedded system.

Based on the over symposium, the organization of the purposed algorithm

for the programming viewpoint following to Algorithm 2:
366 F. M. J. M. Shamrat et al.

Algorithm 2. Proposed System Execution Methodology

1: Import library;
2: Call function(){← Calling Function;
3: Water level detection();
4: Detect location();
5: }
6: Sensor1 Read Water Level ← Detect water level;
7: Sensor2 Read Water Level ← Detect water level;
8: Sensor3 Read Water Level ← Detect water level;
9: SensorN Read Water Level ← Detect water level;
10: Function Water level detection() {
11: While(pass Read Value By sensors)
12: If (water level exceed = = True)
13: Send SMS to the coast guard;
14: Detect location();
15: Break;
16: ELSE
17: Detect water level ← Read Water Current Value;
18: Detect location();
19: Continue;
20: end while)
21: }
22: Function Detect location(){
23: GPS antenna detect current location;
24: Display location ();
25: }
26: Restart at 1 [Auto Start process ()]

6 Implementation
The proposed procedure has been designed and developed in default IDE and
furthermore can develop Anaconda application. A passenger’s vessel is selected
for the implementation and test this embedded system, where max capacity
is defined 1500 kg by the help of the “Archimedes” formula. There 35 passen-
gers were available in the vessel. They all are of different weights, collected the
weight data in a variety of ways since the passenger arrives. Data is collected
by the water floating switch sensor and location share is implying with the GPS
antenna. In excess of a couple of experiments have been observed to test the
algorithm of the proposed system. Evaluate these outcomes of the experiments
with each other. The outcome of the introduced system is bringing into being in
a predictable range.

6.1 Experiment
An overall 22 experiments are done for the measurement of the performance
accuracy of the embedded system. By installing the embedded system, read the
 Implementation of a Smart Embedded System for Passenger Vessel Safety 367

weight of the passenger’s vessel that is displayed in Table 1 and the relating
graphical portrayals are exhibited as graphical visualize (1, 2) and are shown in
Figs. 9 and 10.

Table 1. Ship weight with over-weight measurement by implemented sensors.

No of experiment Ship weight (kg) Over-weight

1 750 No
2 1980 Yes
3 792 No
4 1324 No
5 1765 Yes
6 1592 No
7 803 No
8 1689 Yes
9 1422 No
10 906 No
11 947 No
12 1894 Yes
13 1009 No
14 1712 Yes
15 1932 Yes
16 1209 No
17 648 No
18 1501 Yes
19 1619 Yes
20 1109 No
21 1200 No
22 761 No

Using the estimation of the table recognizes by the sensors and the output
gets by using our calculation. In Figs. 9 and 10 graphical visualize of the infor-
mation that is collected by sensors and output has been exhibited.
In several experiments of the embedded system, water floating sensors switch
are installed at the breakpoint of the vessel (see Fig. 7). It reads the water level
if the vessel is overweighed than the water level of the vessel will increase. We
get weight raw data by utilizing sensors from a vessel and save it in Table 1
and deploy our proposed Algorithms 1 and 2 and on it. We get the result of
overweight issues when the weight overcomes the max capacity (weight ≥ 1500)
of the vessel, the embedded system shows the overweight positive, either negative
(Table 1). It only passes data overweight “yes” when vessel weight exceeds the
368 F. M. J. M. Shamrat et al.

Fig. 9. Graphical Visualize (1) of Table 1 data

Fig. 10. Graphical Visualize (2) of Table 1 data.

max weight. We have shown the graphical visualize of Table 1 data in Figs. 9 and
10. Sensors will send the data request and decision of the system to the server
(Node-RED) through the internet, raspberry pi B will read the data from the
server and visualize the data and location of the vessel to the output monitor.
 Implementation of a Smart Embedded System for Passenger Vessel Safety 369

The monitoring man will notify the coast guard to stop the vessel from safe
thousands of lives.

7 Conclusion
The automated embedded systems in the passenger’s vessels are very urgent
at the present time for the safety of both the passengers and the vessel due
to the overloading issue. With the help of some methodology, the issue can
be dealt with. The framework that is implemented in the system is created,
structured and actualized using the python programming language. To examine
the methodology and to determine an agreeable level, a number of investigations
were performed. Implementation of this framework can help thousands of lives,
families, and dreams every year from meeting a fatal end.

References
1. Rahman, S.: An analysis of passenger vessel accidents in Bangladesh. In: 10th
International Conference on Marine Technology, MARTEC (2016)
2. Abdul Rahman, N.S.F., Rosli, H.Z.: An innovation approach for improving pas-
senger vessels safety level: overload problem. Int. J. Bus. Tourism Appl. Sci. 2(2),
61–74 (2014)
3. Awal, Z.I., Rafiqul Islam, M., Hoque, M.: Analysis of passenger vessel accidents in
the inland waterways of Bangladesh. In: Session I-B-1 Analysis of Passenger Vessel
Accidents in the Inland Waterways of Bangladesh (2006)
4. Rashid, K., Islam, M.R.: Reasons and remedies of inland passenger vessels accidents
in Bangladesh. In: AIP Conference Proceedings, vol. 1919, p. 020036 (2018)
5. Ek, Å.: A study of safety culture in passenger shipping. In: The 3rd Safety and
Reliability International Conference, vol. 3, pp. 99–106 (2003). The Publishing and
Printing House of Air Force Institute of Technology
6. Archimedes law and buoyant force information. https://www.khanacademy.org/
science/physics/fluids/buoyant-force-and-archimedes-principle/a/buoyant-force-
and-archimedes-principle-article. Accessed 26 Apr 2019
7. Vessel floating and displaced water information. http://scienceline.ucsb.edu/
getkey.php?key=620. Accessed 26 Apr 2019
8. Raspberry pi zero. https://learn.adafruit.com/introducing-the-raspberry-pi-zero?
view=all. Accessed 26 Apr 2019
9. Microcontroller. https://learn.adafruit.com/programming-microcontrollers-using-
openocdon-raspberry-pi?view=all. Accessed 26 Apr 2019
10. Water floating switch sensor. https://www.quora.com/How-does-a-ship-float-on-
water. Accessed 26 Apr 2019
11. GPS antenna. https://learn.adafruit.com/adafruit-ultimategps/external-antenna.
Accessed 26 Apr 2019
12. Gypa, I., Boulougouris, E., Vassalos, D.: Decision support tool for modelling secu-
rity scenarios onboard passenger ships. Taylor & Francis Group, London (2018).
ISBN 978-0-8153-7993-5
13. Reddy, M., Rao, K.: Fire accident detection and prevention monitoring system
using wireless sensor network enabled Android application. Indian J. Sci. Technol.
9(17), 1–5 (2016)
370 F. M. J. M. Shamrat et al.

14. Shams, R., Hossain, S., Priyom, S., Fatema, N., Shakil, S.R., Rhaman, M.K.: An
automated firefighting system. In: 2015 12th International Conference on Fuzzy
Systems and Knowledge Discovery, FSKD 2015, pp. 2327–2331 (2015)
15. Chen, J.J., Lu, M.X., Wu, T.H., Hwang, Y.S.: Sub-1-V FastResponse hysteresis-
controlled CMOS buck converter using adaptive ramp techniques. IEEE Trans.
Very Large Scale Integr. (VLSI) Syst. 9(21), 1608–1618 (2013)
16. Cheng, Y.F., Hirdaris, S.E.: Improvement of ship safety through stability research
and innovations. In: 11th International Conference on the Stability of Ships and
Ocean Vehicles (2012)
17. Ahola, M., Romanoff, J., Kujala, P., Remes, H., Varsta, P.: Cruise and Ferry Expe-
rience program for future passenger ship designers. In: RINA Conference on Edu-
cation and Professional Development of Engineers, Newcastle, UK (2011)
18. Bouguila, N., Daoudi, K.: A statistical approach for binary vectors modeling and
clustering. In: Theeramunkong, T., Kijsirikul, B., Cercone, N., Ho, T.-B. (eds.)
PAKDD 2009. LNCS (LNAI), vol. 5476, pp. 184–195. Springer, Heidelberg (2009).
https://doi.org/10.1007/978-3-642-01307-2 19
Extended Track
 Empirical Analysis on the Effect of Image
Compression and Denoising Using
Different Wavelets on Iris Recognition

Pranita Baro1(B) , Malaya Dutta Borah1(B) , and Sushanta Mukhopadhyay2

1
National Institute of Technology Silchar, Silchar, Assam, India
[email protected], [email protected]
2
Indian Institute of Technology (ISM) Dhanbad, Dhanbad, Jharkhand, India
[email protected]

Abstract. The Iris recognition is commonly used as a security system

due to its robustness against imposters. Iris datasets are huge and hence
those datasets occupy more space. Iris image compression has become
an important part of better performance like speed and data storage.
The portable Iris system is in huge demand. That portable systems need
to transmit the iris images through a very small bandwidth channel. To
reduce the time for transferring a huge number of data over small band-
width channel, iris file can be compressed to some extent to minimize
the size. Another problem is that when an image is captured, it cap-
tures some noise that disturbs the recognition performance, so, denois-
ing is required for noise-free images. This paper separately analyzes the
impact of wavelet compression along with denoising on iris images. The
compression analysis is done using Embedded Zero Tree Wavelet, the
other technique used is Set Partitioning in Hierarchical Tree and the
third technique used is Spatial-Orientation Tree Wavelet. Denoising is
done using different wavelets Daubechies, Haar, Biorthogonal and Fejer-
Korovkin. The impact of the wavelet compression and denoising tech-
niques on recognition performance are compared with False Rejection
Rate and False Acceptance Rate. The quality of a compressed image
is calculated with different quality metrics. This work establishes that
compression and denoising of the images minimally affect the recognition
performance.

Keywords: Iris recognition · Quality metric · Image compression ·

Image denoising

1 Introduction

Iris is an organ whose structure remains stable throughout life. It serves as an

appreciable biometric for the authorizing identity of an individual. Biometric
is a skill that can analyze both the physical and behavioural characteristics of
every individual for the authentication of identity. Iris recognition is considered
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 373–386, 2020.
https://doi.org/10.1007/978-981-15-3666-3_30
374 P. Baro et al.

to be the most promising way of confirming the identity of a person because of

its minimal margin of the fault and faster speed. Since Iris is considered to be
the most reliable and unique part of the human, nowadays it is used in many
official purposes and this thing needs to be secured. The biometric systems need
to function at a very faster rate, if we use an original size of iris images to match
across the database, the computation time will be increased. From the existing
research it is observed that if we minimize the size of the image, the computation
time is reduced without any change in the recognition. And for this, different
wavelet compression techniques are used (Fig. 1).

Fig. 1. Structure of an eye.

When a Lossy compression technique is used at a lower bit rate, it creates

compression artifacts. These techniques are likely appropriate for natural images,
which can be like, photos in applications where a slight loss for accuracy is
accepted in achieving a significant reduction in bit rate. During the acquisition
of images, it is inevitable that noise will be captured and will spread across the
images. The development of noise can be during digitization and transmission of
the images. Corruption of Impulse noise occurs during the transmission process
or digital image acquisition procedure, resulting in channel bit errors or photo-
electronic sensor faults.
Denoising is a process where noises in the image will be suppressed preserv-
ing the features of the images. While denoising the images, wavelets are of main
interest as the noise will be spread out uniformly across the coefficients and the
image information will be concentrated at some larger coefficients. The thresh-
olds of these coefficients will help in eliminating unwanted signal components
such as noise. Thus the wavelet-based threshold will be used in achieving the
goal. Hard Thresholding will eliminate the values of coefficients based on the
fixed threshold level.
This research performs the analysis of different wavelet techniques to visualize
which wavelets works best for compression and denoising of images to overcome
the storage problem of large data sets and also for easier transmission. To get the
better accuracy we have also applied wavelet techniques so that the disturbance
during recognition is removed. The remainder of the paper is organized along
these lines Sect. 2 presents Background, Sect. 3 presents Proposed Work, Sect. 4
presents Results and Sect. 5 presents Conclusion.
 Empirical Analysis on the Effect of Image Compression and Denoising 375

2 Background
2.1 Related Work

Iris Biometric is considered to be one of the most efficient biometric systems.

Various research work has been done to identify a person with the help of an
iris recognition system because iris of a person is different from the other per-
son, even the two eyes of the same person differ from each other. Trokielewicz
et al. have discussed the Iris Recognition that is done after a person’s death
[1]. Gupta, Richa, and Priti Sehgal has discussed an Allay replay attack on
iris biometric using a Non-Deterministic approach [2]. Hamd et al. has designed
Intelligent Algorithms on biometric system specially designed for Iris recognition
[3]. In the recent literature survey, it is noticed that wavelet-based compression
and denoising techniques have become very popular. Several attempts have been
made to perform compression on iris images. Image compression helps in using
less data than an original image and also displays image content in a way that
highly reduces storage requirements and network transfer time [4]. In the most
recent era, iris biometrics is considered to be the most accurate way of identify-
ing humans, which is increasing day by day [5]. Many researchers are involved in
designing iris recognition algorithms and the datasets for research purposes are
also available. The algorithms designed by the researchers work effectively for a
large range of applied conditions. So, it requires huge databases of images stored
in various conditions. Robert et al. had studied the compression effects of an
iris images on iris recognition using a publicly available recognition algorithm
using JPEG (Joint Photographic Expert Group)-2000 compression technique,
Iris Challenge Evaluation (ICE) iris database was used in the research and also
the use of Quality metric was observed in the study [6]. JPEG-2000 is a dis-
crete wavelet transform (DWT) which is a compression standard that could be
adjusted for movement imaging video compression with the motion JPEG-2000
extension [6]. JPEG-2000 innovation was chosen as the video coding standard
for the computerized film in 2004 [6]. Paul et al. have studied the image quality
using different techniques. However, there was not an extensive analysis of the
recognition performance and the normalized iris images presented a very signif-
icant multiple Thresholding for detecting the noisy pixels. The computation of
threshold values is dependent on the intensities distributed over the normalized
image. The pupil pixels and eyelash are removed by a median filter, whereas,
the filling method is used to remove the eyelids and light reflections pixels. The
system was successful in achieving a 99.62% recognition rate and false rejection
rate of 0.04. Compression techniques are applied to iris images for minimizing
the storage size and the time of transmission. The compression methods are of
two types: Lossy and Lossless. From the past few years different types of image
compression techniques like JPEG-2000, JPEG-XR have been used for generat-
ing compressed data [7–9]. Daugman proposed the JPEG compression technique
by separating the area of interest. Ives has applied the JPEG-2000 technique in
iris images to observe the results of image compression and recognition perfor-
mances. Funk et al. has studied and elaborated the effect of JPEG-2000, JPEG,
376 P. Baro et al.

Predictive Residual Vector Quantizer (PRVQ) and compression on fractal

image on cross over the accuracy of the recognition system [10]. Paul et al. pro-
posed very convenient compression techniques for iris using different wavelets
that are applied in the iris recognition system. Wavelet transform is the most
robust way to express an image and also the selection of decompression level
plays an important role in wavelet transformation because it depends on com-
putational complexity [6]. Pankaj et al. have performed a wavelet Thresholding
approach for denoising [11]. Dehkordi, and Abu-Bakar have designed a method
for detecting and removing unwanted pixels of the lower and upper eyelids, upper
eyelashes, pupil regions and also removing reflections of light in the iris region
which are treated as noisy pixels [12]. From the above research some gaps are
identified as listed below: In the earlier research, it is observed that the Peak
Signal to Noise Ratio (PSNR) value, Equal Error Rate (EER) value and also
the quality metrics were not calculated with the different compression ratios and
different Thresholding values. The Peak Signal to noise ratio between the two
images is calculated in decibels. It measures the quality of an image between the
original and compressed one. The EER is a technique in the biometric system
to find the threshold point of False Acceptance Rate and False Rejection Rate.
Lesser the EER value, higher accuracy of the system.
From the above research some gaps are identified as listed below:
– In the earlier research it is observed that the Peak Signal to Noise Ratio
(PSNR) value, Equal Error Rate (EER) value and also the quality met-
rics were not calculated with the different compression ratios and different
Thresholding values. The Peak Signal to noise ratio between two images is
calculated in decibels. It measures the quality of an image between the orig-
inal and compressed one. The EER is a technique in biometric system to
find the threshold point of False Acceptance Rate and False Rejection Rate.
Lesser the EER value, higher accuracy of the system.
– In the earlier research it has been observed that denoising was done by adding
additional noises to the image.
This work focuses on
1. The quality metrics with every compression ratios and different level of
thresholds.
2. Removal of noises without adding any extra noises using wavelet denoising
techniques.

2.2 Preliminaries
Wavelet Compression. Wavelet Compression The images are compressed and
compared among three wavelet-based techniques, namely Embedded Zero Tree
Wavelets (EZW), Set Partitioning in Hierarchical Tree (SPIHT) and Spatial-
Orientation Tree Wavelet (STW). For all of the three compression techniques,
the Haar wavelet has been used for decomposition in which the image is filtered
into several sub-bands. The image being of dimension 280 × 320, it was zero-
padded to form 512 × 512 images so that decomposition can be made possible.
 Empirical Analysis on the Effect of Image Compression and Denoising 377

– Embedded Zero Tree Wavelet. EZW is a lossy picture compression tech-

nique. The EZW forms a tree which is shown in Fig. 2. And the wavelet
coefficients are scanned in EZW as shown in the Fig. 3. And zeros trees are
eliminated. Zeros tress are those coefficients whose subsequent coefficients
in the scanning order are zero. By eliminating such coefficients, redundant
information is discarded and compression is achieved.

Fig. 2. EZW tree Fig. 3. Scanning in EZW

– Set Partitioning in Hierarchical Trees. SPIHT depends on embedded

wavelet coding algorithms within the order of lowering energy tiers. SPIHT
is an extended version of EZW. In SPIHT decomposed image is partitioned
into two ways: significant and non-significant regions using the given formula
below.

1, M ax(i, j) ∈ T (|Ci,j |) ≥ 2n
Sn (t) = (1)
0, otherwise

Where Sn (t) denotes set of coordinates. i, j are coordinates and Cij is the
coefficient at that coordinate. The list of significant and insignificant pixels
can be used to eliminate redundant data and achieve compression.
– Spatial-Orientation Tree Wavelet. The STW is alike to EZW, only the
difference is how a tree is formed. STW follows the state transition model. For
one threshold value to the following, the position of transform values keeps
on changing from one state to another. The approach used in STW differs in
encrypting the information related to Zero Tree. The model makes easier in
reducing the number of bits required in encryption. It is observed that the
tree structure has more spatial scanning and does not follow a zigzag scanning
as seen in EZW. In Fig. 4 we can observe the STW tree and in Fig. 5 a STW
scanning is shown which is different from EZW.
378 P. Baro et al.

Fig. 4. STW tree Fig. 5. Scanning in STW

Wavelet Denoising. To analyze the impact of wavelet denoising on Iris recog-

nition, Thresholding based wavelet denoising is performed. In this paper, four
wavelets namely Daubechies1 (db1), Haar, Biorthogonal 1.3 (bior 1.3) and Fejer-
Korovkin 4 (FK 4) are used.

– Daubechies wavelet. The Daubechies wavelet is additionally known as db

has a higher variety of disappearing moments for a given channel width. They
are both orthogonal and biorthogonal. It is orthogonal as well as preserves
energy and compactly-supported. The family of this wavelet is identified by
dbN where N is the order number. This work is done using db1 wavelet. The
db1 looks similar like Haar wavelet.
– Haar Wavelet. Haar is known as the most common and simple form of
wavelet. It is the only wavelet that is orthogonal, biorthogonal, and supports
discrete wavelet transform as well as continuous wavelet transform. It is non-
continuous and looks similar to a step function. This wavelet transform can
be taken to pair up initial data values, keep the differences and move the
summation. The process is rerun over and over, pairing up the sums to offer
the following scale, subsequently consequences in differences and one very
last sum. This wavelet includes average and difference factors, storing the
important coefficients, removing redundant data, and reorganize the matrix
so that the final matrix is identical to the preliminary matrix.
– Biorthogonal wavelet. The extended family of wavelet is the Biorthogo-
nal wavelet. The Bior wavelet supports symmetry and actual reconstruction
by the usage of Finite Impulse Response (FIR) filters. It is a compressed
supported wavelets. Nowadays biorthogonal wavelet is mostly used in the
compression system. Different types of Biorthogonal wavelets are Bior 1.1,
Bior 1.3 Bior 3.1, Bior 4.4, Bior 5.5, etc. In this work we have used the
Biorthogonal 1.3 wavelet which is also called Bior 1.3.
– Fejer-Korovkinwavelet. (FK) wavelet filter separates the raw time series
into a high frequency (HF) component and a low frequency (LF) compo-
nent [13]. It is a two-dimensional Fourier transform over time and space.
 Empirical Analysis on the Effect of Image Compression and Denoising 379

By using the FK filters the quality metrics value of the water-marked image,
i.e., NC (normalized cross-correlation) Q (quality of the image) is computed
like the other wavelet filter. The idea behind having several wavelets is that
the decomposition will be more accurate when the signal (image) looks sim-
ilar to the wavelet. Thus the image and noise are well decomposed for some
wavelets compared to others. The image is decomposed using a 2D stationary
wavelet (SWT) decomposition technique as depicted below in Fig. 6.

In this, the image is filtered row-wise using the wavelet into the low pass and
high pass signals each of which requires further filtration into high pass and
low pass in column-wise. This decomposes the image into four sub-images each
specializing in: approximation, horizontal edges, vertical edges, and diagonal
edges. Additionally, downsampling is performed following each filtration without
loss of information according to NY Quist criteria.

Fig. 6. SWT

Hard Thresholding. Hard Thresholding Here we have applied Hard Thresh-

olding because it gives us better edge preservation in compared to Soft Thresh-
olding. The wavelet coefficients where values lie lesser than the set threshold are
eliminated (assigned to zero) and images are reconstructed using the remain-
ing coefficients. This is due to the assumption that the noise has lower signal
strength and its corresponding wavelet coefficient will also be of low value.

Dataset. In this work, the CASIA v1.0 dataset has been used. It is publicly
Dataset. This dataset is available at http://biometrics.idealtest.org/. The images
have been captured using a proprietary Iris camera by the National Labora-
tory of Pattern Recognition (NLPR), Institute of Automation (IA) and Chinese
Academy of Sciences (CAS). There were 108 eyes which consisted of 756 iris
images. For an eye 7 number of images were captured in two sessions, in the first
session 3 images were captured and the remaining 4 were captured in the second
session.
380 P. Baro et al.

3 Proposed Work
The flow diagram of the procedure adopted is shown in Fig. 7. These two flows
can be observed in the given diagram. Both flows have in common: the acquir-
ing of the dataset, application of compression and denoising techniques on the
dataset, recognition of iris and performance metric computation. However, one
flow concentrates on the impact of wavelet compression and others concentrate
on wavelet denoising. The images generally consist of redundant pixels and also
some noises, so, to remove redundancy and disturbance in an image, compression
and denoising using various methods has been used. The used dataset is CASIA
v1.0. The detailed explanation of the flow diagram is as follows:
At first we will read the gray scale images and convert it into discrete pixel
values. For the lossy image compression we have applied EZW, SPIHT and
STW techniques. We have categorized the output into four different ratios. It
comprises of 10:1, 15:1, 20:1, 30:1 compression ratios. For the recognition purpose
we have applied the Libor Masek which is an open-source code. And next we have
computed the values for different parameters like EER, FAR, FRR and Quality
measurement. For Denoising the gray scale images are read and converted into
discrete pixel values. The images are decomposed by 2-D stationary wavelet
using Daubichies, Haar, Bior 1.3 and Fk4 wavelets. Here, we have taken four
different thresholds 6, 8, 10, 12 and calculated the performance metrics as done
in the compression.

3.1 Quality and Performance Metric Computation

The quality metrics help to analyze the impact of wavelet-based compression
on the recognition described above. The four quality metrics namely, Feature
Information Measure (FIM), Occlusion Measure (OM), Dilation Measure (DM)
are measured. Additionally, different performance metrics, namely, false accept
rate (FAR), false reject rate (FRR) and Equal Error Rate (EER) have been
extracted.

Fig. 7. Flow diagram.

 Empirical Analysis on the Effect of Image Compression and Denoising 381

Feature Information Measure. Feature Information Measure The FIM is

calculated by adding the information distance between subsequent rows of the
iris template. The information is given by
    
J(p, q ) = D( p || q ) + D( q || p ) (2)

Where, p and q are neighbouring rows.

     
D( p || q ) = pl log2 pl /ql (3)

Finally, FIM is measured as


F IM = 1
N Ji (4)
i

Where N is the number of rows.

Occlusion Measure. The OM is measured by dividing number of noise pixels

by the total number of pixels in the template
pn
OM = (5)
pt
Where Pn denotes number of pixels in the noisy regions and Pt denotes total
number of pixels.

Dilation Measure. The DM is the ratio of radius of the Hough circle drawn
around the pupil and that of the iris.
Rp
DM = (6)
Ri
Where Rp denotes the radius of pupil and Ri denotes radius of iris in pixels.

Overall Quality Measure. The Overall Quality Measure (QM) is obtained as

a product of the three measures namely, FIM, OM and DM.

QM = F IM ∗ OM ∗ DM (7)

4 Results

The result of Compression and Denoising is given below:

382 P. Baro et al.

4.1 Compression

The following are representative images of compression. The compression ratio

can be achieved by changing the decomposition level. Although the exact com-
pression ratio (CR) cannot be achieved, target CR was fixed to be 10:1, 15:1,
20:1 and 30:1. The levels were chosen such that the obtained CRs were as close as
possible to the target CR. It is seen that when the compression ratio is reduced,
the image is smoother (Tables 1, 2 and 3).
The False acceptance and Rejection ratios are calculated as
FRR is defined as
(N o. of Incidents of F alse Rejections)
(%)F RR = ∗ 100% (8)
(T otal N o. of Samples)

And FAR is defined as

(N o. of Incidents of f alse Acceptance)
(%)F AR = ∗ 100% (9)
(T otal N o. of Samples)

Table 1. Best Accuracy using different techniques

Techniques Original Compression ratio

10 15 20 30
EZW 99.696228 99.694861 99.692594 99.690543 99.683612
SPIHT 99.693729 99.693466 99.696099 99.689977
STW 99.693824 99.694849 99.692087 99.690157

Table 2. FRR at FAR = 0.001

Techniques Original Compression ratio

10 15 20 30
EZW 0.033333 0.059524 0.041667 0.042857 0.041667
SPIHT 0.048810 0.047619 0.051190 0.051190
STW 0.053571 0.055952 0.039286 0.040476

Table 3. FRR at FAR = 0.0001

Techniques Original Compression ratio

10 15 20 30
EZW 0.064286 0.059524 0.053571 0.086905 0.090476
SPIHT 0.048810 0.069048 0.078571 0.065476
STW 0.067857 0.083333 0.073810 0.050000
 Empirical Analysis on the Effect of Image Compression and Denoising 383

Performance Curves
It is observed clearly that accuracy is lightly degraded up to the targeted com-
pression ratio. Higher Compression Ratios will degrade the image qualities and
recognition in relation of PSNR and Quality Metrics. The EER varies with dif-
ferent techniques. The quality of some images may reduce but some may not be.
For the set FAR there is a noticeable change in FRR. The STW with CR of 30:1
performs as good as the original image which can be observed in Figs. 8, 9 and
10. STW performs well for PSNR as well as Quality Metrics and also the EER
value is lower in STW. EZW gives good result at increasing compression ratio.
SPIHT performs as close as that of original image at a lower compression ratio.

Fig. 8. CR vs PSNR Fig. 9. CR vs QM Fig. 10. CR vs EER

4.2 Denoising
The results of denoising applying the different wavelets with variable thresholds
(6, 8, 10 and 12) are shown in Figs. 11, 12 and 13. Due to increase in thresh-
old, smoother images are obtained and fk4 wavelets perform higher smoothing
(Tables 4, 5 and 6).

Table 4. Best Accuracy using different Techniques

Techniques Original Thresholds

6 8 10 12
Db1 99.582767 99.584400 99.585805 99.586031 99.587348
Haar 99.584406 99.585041 99.586031 99.587348
Bior 1.3 99.584355 99.585729 99.585711 99.585965
FK4 99.585726 99.585805 99.586369 99.586646

It is seen that the accuracy slightly changes due to denoising. The FAR
remains unchanged but the FRR values changes with different denoising thresh-
olds. With an increase in threshold, the curves of EER move towards right ini-
tially and after the threshold of 8 towards the left. This indicates the improve-
ment of recognition at higher thresholds. This could be due to more elimination
384 P. Baro et al.

Table 5. FRR at FAR = 0.001

Techniques Original Thresholds

6 8 10 12
Db1 0.033333 0.036905 0.046429 0.041667 0.029762
Haar 0.036905 0.046429 0.041667 0.029762
Bior 1.3 0.035714 0.036905 0.042857 0.042857
FK4 0.034524 0.041667 0.041667 0.038095

Table 6. FRR at FAR = 0.0001

Techniques Original Thresholds

6 8 10 12
Db1 0.064286 0.046429 0.070238 0.05119 0.039286
Haar 0.046429 0.070238 0.05119 0.039286
Bior 1.3 0.047619 0.044048 0.054762 0.052381
FK4 0.058333 0.064286 0.063095 0.064286

Fig. 11. Threshold vs Fig. 12. Threshold vs QM Fig. 13. Threshold vs EER
PSNR

of noise at higher thresholds. The graph in Fig. 13 shows that bior1.3 gives the
minimum EER value. The metrics show that an increase in threshold leads to
a fall in PSNR indicating loss of signal strength. In Fig. 11 we can see that the
PSNR value is higher at lower threshold value. Haar, db1 and bior1.3 acts sim-
ilarly with an increase in threshold value. Different techniques show different
trends in quality. For db1 and Haar wavelets, it gives the similar curve. bior1.3
gives the best result in terms of achieving the better quality at thresholds of 12
respectively.

5 Conclusion

The impact of wavelet compression and denoising on iris images were studied
separately. The analysis is done on the basis of different compression ratios and
performance is observed on the set compression ratios, thresholds. The wavelet
 Empirical Analysis on the Effect of Image Compression and Denoising 385

based techniques could be useful for compressing the iris images without much
loss of recognition performance. STW performed well at higher compression rate.
It is observed that the wavelet compression techniques and wavelet denoising
techniques has not deteriorated the recognition performance. As an outcome,
huge dataset can be compressed to a small size and the matching is affected
minimally. The denoising using various wavelets and hard thresholds indicate
that both haar and db1 wavelets perform similarly. In terms of PSNR, QM and
EER we have seen that bior 1.3 gives the best output compared with other
wavelets. And in future we are trying to extend our work on colored image
compression along with denoising and see the impact on Iris Recognition.

Acknowledgement. The authors would like to acknowledge the Department of Com-

puter Science and Engineering and TEQIP-III cell, National Institute of Technology
Silchar for financial and infrastructural support to complete this research work.

References
1. Trokielewicz, M., Czajka, A., Maciejewicz, P.: Iris recognition after death. IEEE
Trans. Inf. Forensics Secur. 14(6), 1501–1514 (2019)
2. Gupta, R., Sehgal, P.: Non-deterministic approach to allay replay attack on iris
biometric. Pattern Anal. Appl. 22(2), 717–729 (2019)
3. Hamd, M.H., Ahmed, S.K.: Biometric system design for iris recognition using intel-
ligent algorithms. Int. J. Mod. Educ. Comput. Sci. 11(3), 9 (2018)
4. Shen, J.J., Yeh, C.H., Jan, J.K.: A new approach of lossy image compression based
on hybrid image resizing techniques. Int. Arab J. Inf. Technol. 16(2), 226–235
(2019)
5. Strela, V., Heller, P.N., Strang, G., Topiwala, P., Heil, C.: The application of
multiwavelet filterbanks to image processing. IEEE Trans. Image Process. 8(4),
548–563 (1999)
6. Paul, A., Khan, T.Z., Podder, P., Ahmed, R., Rahman, M.M., Khan, M.H.: Iris
image compression using wavelet transform coding. In: 2nd International Confer-
ence on Signal Processing and Integrated Networks (SPIN), pp. 544–548. IEEE
(2015)
7. Rakshit, S., Monro, D.M.: An evaluation of image sampling and compression for
human iris recognition. IEEE Trans. Inf. Forensics Secur. 2(3), 605–612 (2017)
8. Zemliachenko, A., Kozhemiakin, R., Vozel, B., Lukin, V.: Prediction of compres-
sion ratio in lossy compression of noisy images. In: 13th International Conference
on Modern Problems of Radio Engineering, Telecommunications and Computer
Science (TCSET), pp. 693–697. IEEE (2016)
9. Goyal, B., Dogra, A., Agrawal, S., Sohi, B.S.: Two-dimensional gray scale image
denoising via morphological operations in NSST domain & bitonic filtering. Future
Gener. Comput. Syst. 82, 158–175 (2018)
10. Funk, W., Arnold, M., Busch, C., Munde, A.: Evaluation of image compression
algorithms for fingerprint and face recognition systems. In: Proceedings from the
Sixth Annual IEEE SMC Information Assurance Workshop, pp. 72–78. IEEE
(2005)
11. Hedaoo, P., Godbole, S.S.: Wavelet thresholding approach for image denoising. Int.
J. Netw. Secur. Appl. (IJNSA) 3(4), 16–21 (2011)
386 P. Baro et al.

12. Dehkordi, A.B., Abu-Bakar, S.A.: Noise reduction in iris recognition using mul-
tiple thresholding. In: 2013 IEEE International Conference on Signal and Image
Processing Applications, pp. 140–144. IEEE (2013)
13. Rodriguez, N., Barba, L.: Fejer-Korovkin wavelet based MIMO model for multi-
step-ahead forecasting of monthly fishes catches. Polibits 56, 71–76 (2017)
14. Daugman, J., Downing, C.: Effect of severe image compression on iris recognition
performance. IEEE Trans. Inf. Forensics Secur. 3(1), 52–61 (2008)
15. Shapiro, J.M.: Embedded image coding using zerotrees of wavelet coefficients. IEEE
Trans. Signal Process. 41(12), 3445–3462 (1993)
16. Ives, R.W., Bishop, D.A., Du, Y., Belcher, C.: Iris recognition: the consequences
of image compression. EURASIP J. Adv. Signal Process. 2010(1), 680845 (2010)
17. Ives, R.W., Broussard, R.P., Kennell, L.R., Soldan, D.L.: Effects of image compres-
sion on iris recognition system performance. J. Electron. Imaging 17(1), 011015
(2008)
18. Varanis, M., Pederiva, R.: The influence of the wavelet filter in the parameters
extraction for signal classification: an experimental study. Proc. Ser. Braz. Soc.
Comput. Appl. Math. 5(1) (2017)
19. Ives, R.W., Bishop, D.A., Du, Y., Belcher, C.: Effects of image compression
on iris recognition performance and image quality. In: 2009 IEEE Workshop on
Computational Intelligence in Biometrics: Theory, Algorithms, and Applications,
pp. 16–21. IEEE (2009)
20. Mishra, K.N.: An efficient technique for online iris image compression and per-
sonal identification. In: Tiwari, B., Tiwari, V., Das, K.C., Mishra, D.K., Bansal,
J.C. (eds.) Proceedings of International Conference on Recent Advancement on
Computer and Communication. LNNS, vol. 34, pp. 335–343. Springer, Singapore
(2018). https://doi.org/10.1007/978-981-10-8198-9 35
21. Rai, H.M., Chatterjee, K.: Hybrid adaptive algorithm based on wavelet transform
and independent component analysis for denoising of MRI images. Measurement
144, 72–82 (2019)
22. Dua, M., Gupta, R., Khari, M., Crespo, R.G.: Biometric iris recognition using
radial basis function neural network. Soft Comput. 23(22), 11801–11815 (2019)
 Segmentation of Chart Images Using
H-Means Algorithm

Prerna Mishra1(B) , Santosh Kumar1(B) , and Mithilesh Kumar Chaube2

1
Department of Computer Science and Engineering, DSPM IIITNR, Raipur, India
{prerna,santosh}@iiitnr.edu.in
2
Department of Mathematics, DSPM IIITNR, Raipur, India
[email protected]

Abstract. For chart analysis it is necessary to segment region of interest

from background for better understanding of distinctive pieces of charts.
Many different methods such as thresholding, clustering, compression-
based methods etc. are proposed for image segmentation. But one of the
simplest and unsupervised algorithm is k-means clustering algorithm. K
means algorithm cannot generate value of K (number of clusters) pertain-
ing to any input image at real time. It has specified cluster numbers for
all types and pattern of images which results into improper segmentation
of images. Thus a novel method H-Means algorithm is suggested in this
paper which decides the number of clusters at real time depending upon
the input image. H-means algorithm uses histogram to decide number
of clusters by extracting peaks for each image and segments foreground
from background. An image database of info charts over 5280 images
covering 10 categories of chart images is introduced to the research
community.

Keywords: Histogram · Color segmentation · Information charts

1 Introduction
With the growing advancements in computer vision, clustering techniques
applied on datasets depending on their characteristics and features have
improved even more and gained importance too [9]. Clustering algorithms have
been a vital means scaled to various application areas apart from computer
vision, i.e., data mining, statistical analysis, image processing, quantization,
and video processing [8]. Image segmentation has to be efficient in categoriz-
ing objects in images from the background. Therefore, many methods for image
segmentation have already been proposed. Clustering is one of the common and
broadly used segmentation approach because of its simplicity and efficiency.
There are different clustering algorithms like K-means, fuzzy c-means, spectral
clustering, expectation, and maximization [9]. Variants of clustering algorithms
were proposed having different perspectives which were designed for different
purposes [10]. However, existing clustering algorithms requires user-specified
parameters as input.
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 387–397, 2020.
https://doi.org/10.1007/978-981-15-3666-3_31
388 P. Mishra et al.

In order to interpret charts properly, it is necessary to understand the role

of each of its pieces globally. Thus for a machine to understand charts, it must
be able to comprehend each piece for which piece-wise segmentation is required.
Moreover, every chart has different outlook and color styles; thus by segmenting
the image by its color styles generates accurate results. But in order to color
segment the chart images, it was again necessary to consider every color intensity
for which histogram was employed with K-means clustering for piecewise image
segmentation.
In this paper, a new method for image segmentation is introduced, which is
based on predefined correctly predicted K value, and inbuilt K means clustering
algorithm functions. The kernel function is applied in the algorithm by means
of peak values of the histogram of the RBG image which was converted into
greyscale, to acquire the cluster centroids which are further utilized in the K-
means clustering algorithm. Based on the classic method, the algorithm proposed
in the paper improves not only the accuracy of image segmentation and but also
algorithmic structure. A new method obtains the value of K, i.e., the number of
centroids in the K-means clustering algorithm.
The remainder of this paper is organized as follows: the previous work done in
the relevant area is discussed in Sect. 2, and the proposed algorithm is described
in Sect. 3. The performance analysis of our algorithm is given in Sect. 4. Finally,
the paper is concluded in Sect. 5.

2 Literature Survey

Till now many researchers have contributed in field of image segmentation by

proposing different methods. For chart analysis it is always important to seg-
ment region of interest from background. Every chart type has particular and
different structure to segment. [8] used intensity and morphological operations
for segmentation of bar charts. [9] processed pie chart to detect boundary and
slice pieces for pie to extract relevant details. [10] used split and merge color
clustering algorithm for segmenting regions of interest from charts.
K-means algorithm is amongst the simplest clustering algorithm and many
improved variants of this algorithms were previously given focusing on initializ-
ing the cluster centroids by various ways. Work was also done to segment images
using this clustering algorithm. As per the literature, customary image segmen-
tation algorithm depends on the segmentation method based on the threshold
value [2], edges [3] and the region of interest [10]. As image segmentation is scaled
over many disciplines like computer vision, pattern recognition, data mining, lot
of improved version pertaining the areas came into existence and importance of
same [4]. Majorly work was focused on selecting centroids in some manner but
the selections were more of static than dynamic selection.
[5] introduced an efficient approach of K-means algorithm to generate cen-
troid by diminishing the root mean square error (RMSE) of the clusters without
affecting the execution time. As per the results, the accuracy level is high for
dense dataset over sparse dataset. [7] proposed enhanced K-means clustering
 Segmentation of Chart Images Using H-Means Algorithm 389

algorithm which improved the way of initializing centre value. Proposed method
gives a novel way of determining the centroids without increasing the complexity
of algorithm. Although no other parameter like threshold value is used, but ini-
tializes the k i.e. the number of clusters. Similar work was suggested by [11] who
proposed an enhanced algorithm for better accuracy and efficiency by taking dif-
ferent centroid values and determining its data points. [1] used histogram before
segmentation to enhance the region of interest by equalizing RGB channel sep-
arately. [6] suggested a mechanism for image segmentation using K means and
morphological processing resulting into efficient segmentation accuracy. While
[12] gave segmentation technique by merging similar regions. As per the avail-
able literature it was notices that k means is considered to be the simplest
algorithm giving efficiently accurate results of segmentation.

3 Proposed Work
We proposed a method for finding the correct and dynamic value of K using peak
histogram values and segment the image using MATLAB functions. Input chart
image is converted into greyscale, of which histogram is obtained. The number
of highest peaks from the histogram is taken as K, i.e., number of centroids.
Image is segmented by using K means clustering algorithm, details of which are
shown in Fig. 1.

Fig. 1. System model of proposed method (Color figure online)

Every color image has tri band i.e., red, green and blue channel based his-
togram. Thus for a chart images having different color styles, it is difficult to
determine and discriminate the data from the histogram which can be used for K
clusters. Thus, color images are converted into greyscale. We proposed a method
for finding the correct and dynamic value of K using peak histogram values and
segment the image using MATLAB functions. Input chart image is converted
into greyscale, of which histogram is obtained. The number of highest peaks
from the histogram is taken as K, i.e., number of centroids. Image is segmented
by using K means clustering algorithm, details of which are shown in Fig. 1.
390 P. Mishra et al.

Fig. 2. (a) Original image, (b) Histogram of RGB Channel, (c) Histogram of Input
Image

Every color image has tri band i.e., red, green and blue channel based his-
togram. Thus for a chart images having different color styles, it is difficult to
determine and discriminate the data from the histogram which can be used for
K clusters. Thus, color images are converted into greyscale. After that, we can
easily find that there are some “peaks” on the histogram, which are the local
maxima of the greylevel intensity frequency. The number of peaks will be used
to decide how many K-values will be used in the image segmentation process.
Because the peaks mean that the pixel count with those tone values was high,
which means there are many pixels that have the same RGB values refer to the
original color image (Fig. 2).

Algorithm 1. Image segmentation using proposed H-Means

Clustering
Input: RGB image
Output: Segmented Image
Steps:
[1] Convert input RGB image into grayscale image.
[2] Determine the total count of peaks from its histogram.
[3] Assign K i.e. number of clusters = highest X (25,50,75,100)% of total
number of peaks.
[4] Segment image using K means Algorithm using latest value of K.
[5] Calculate the RMSE and PSNR values of the segmented image.
[6] Use a Gaussian filter with a standard deviation of 0.2 for generating the
referenced image.

4 Experimental Results and Analysis

The proposed method is implemented in Matlab 2019 (licensed version), and
for the experimental purpose, we have used random images. In the implemented
K-means clustering algorithm, the parameter K is needed to give a prior value.
Also, the paper mainly focuses on pixel distribution rather than the clarity of
the segmented image.
 Segmentation of Chart Images Using H-Means Algorithm 391

4.1 Database Preparation and Description

Due to unavailability of dataset online, multi-class chart images were handcrafted

to quantitatively asses the proposed model. While crafting dataset, care was
taken to generate chart different from another, so randomly data was formed
on which each chart was generated. Each chart image has x-axis values, y-axis
values, labels ticks, legends, chart titles, axis marks, and it was tried to retain
all types of chart properties in the images. An image database of info charts over
5280 images covering ten categories (pie chart, scatter chart including bubble
chart, line chart, bar chart, stacked bar chart, area chart including stacked area
chart, histogram, and node graph) is introduced to the research community.
To the best of our knowledge, this is the largest dataset publicly available for
research on chart interpretation and classification in graph research communities.
Samples of dataset is shown in Table 1.

Table 1. Chart image datasets

4.2 Evaluation Parameters

Performance parameters of the systems are measured by,

1. Root Mean Squared Error: It is considered as a standard performance

measurement of the output image. It defines how much the output image has
deviated from the input image. A smaller value of RMSE defined that the
image is of good quality

 nx −1 ny −1
 1 [(r(x, y))]2
RM SE =  0
nx −1 ny −1
0
(1)
nx ny 0 0 [(r(x, y)) − (t(x, y))]2

2. Peak Signal-to-Noise Ratio: It can be defined as the ratio is the proportion

between maximum attainable powers and the corrupting noise that influence
392 P. Mishra et al.

likeness of the images. It measures the quality of the output image. A smaller
value of PSNR means that the output image has poor quality
⎡ ⎤
⎢ max(r(x, y))2 ⎥
P SN R = 10. log1 0 ⎢
⎣ nx −1 ny −1
⎥
⎦ (2)
1 0 0 [(r(x,y))]2
nx ny nx −1 ny −1 [(r(x,y))−(t(x,y))]2
0 0

4.3 Result Analysis

The number of K-value can be determined correctly but not randomly general-
ized. The number of clusters will be suitable for any images for the segmentation.
From Table 2, it can be observed that the proposed method has the small values
of MSE and high values of PSNR.
The values of the RMSE and PSNR measures indicate that the higher value
of RMSE provides good significant about visual data of chart images. On the
other hand, the smaller value of PSNR measure depicted that the output images
are not up to significant enough to describe the visual content of complex images
of chart classes. Therefore, it can be concluded that the output image resulted
from the proposed method are of good quality.

Fig. 3. Comparison of RMSE values

Figures 3 and 4 shows the comparison of RMSE and PSNR values generated
for tested images. From these figures, it can be seen that resulted images are of
good quality in comparison to that of standard K means algorithm. Figure 5
shows the sample of tested images on the proposed method. The proposed
method is tested on a bar chart, area chart, stacked donut chart, pie chart,
line chart, scatter chart, and donut chart as shown in Fig. 6. Figure 6A is an
 Segmentation of Chart Images Using H-Means Algorithm 393

Fig. 4. Comparison of PSNR values

Table 2. Comparison of the proposed method with implemented method

394 P. Mishra et al.

original image, while Fig. 6B shows the segmented area of the targeted image
and Fig. 6C depicts the binary representation of segmented images. It can be
seen that the proposed method has finely segmented the required regions of
interest. Every small sub-region of the target image is precisely segmented from
the background. It is visible from area chart, pie chart and stacked donut chart
that every sub-area is properly segmented from other corresponding areas. How-
ever, the proposed systems generated desired results, but in the case of the pie
chart, it lacked in segmenting low-intensity areas from other sub-areas of chart
images.

Fig. 5. Comparison of obtained results with standard K means algorithm

Figure 5 illustrates a comparison of proposed k means algorithm with a stan-

dard k-means algorithm. It is visible that the standard k means algorithm fails to
segment the regions of interest from the background, whereas the proposed model
was able to segment the chart pieces correctly. For the stacked area chart, stacked
bar chart and pie chart, its small regions are correctly segmented. Whereas stan-
dard k means algorithm was not able to segment sub-regions of charts, also it
failed to segment the whole foreground regions from the background. Thus it
can be said that using histogram with k means algorithm has generated desired
results.
Table 3 depicts the number of K clusters taken for segmentation by selecting
the highest peaks out of total histogram peaks of images shown in Fig. 7. Figure 7
and Table 3 shows the segmentation results after setting a threshold on histogram
 Segmentation of Chart Images Using H-Means Algorithm 395

peaks. Peak threshold has been taken as 25%–50%–75%–100% of the highest

histogram peaks from total peaks for image segmentation. On taking 25% of
total histogram peaks (as shown in (b)) as K-clusters it was seen that only dark
regions of images were segmented correctly while other low intensities and light-
colored regions were not segmented. On taking 50% and 75% of histogram peaks
(as shown in (c)) images were segmented more correctly as compared to (b).
In the case of 75% of a threshold, even those areas were segmented which has
light background color as seen is treemap and pie chart. On consideration of all
peaks as K-centroids, even light intensity and low background color area were
segmented correctly. Thus color images were correctly segmented on taking all
the peak intensity values as centroids (as shown in (e)).

Fig. 6. Results obtained of chart images. (A) Original input (B) Segmentation result
(C) Binary form of segmentation result

Fig. 7. Image segmentation by threshold on histogram peaks

396 P. Mishra et al.

Table 3. Selection of Histogram peaks as K clusters

Image Total peaks Highest 25% Highest 50% Highest 75% Highest 50%
peaks selected peaks selected peaks selected peaks selected
Image 1 54 14 27 41 54
Image 2 101 26 51 76 101
Image 3 130 33 65 98 130
Image 4 254 64 127 191 254

5 Conclusion
A new image segmentation method is proposed based on the histogram and k-
means. Method uses histogram to determine peak value which is used as value
of K for segmentation. Apart from this, method proves to be robust towards
various types of chart images, separating the required RoI from complete chart
image. In this way every piece of chart is segmented proving to be effective
for better chart interpretation and analysis. Evaluation parameters RMSE and
PSNR shows that the method shows the better result accuracy in comparison
to standard algorithms.

References
1. Dhanachandra, N., Manglem, K., Chanu, Y.J.: Image segmentation using K-means
clustering algorithm and subtractive clustering algorithm. Procedia Comput. Sci.
54, 764–771 (2015)
2. Ishak, A.B.: A two-dimensional multilevel thresholding method for image segmen-
tation. Appl. Soft Comput. 52, 306–322 (2017)
3. Yuheng, S., Hao, Y.: Image segmentation algorithms overview. arXiv preprint
arXiv:1707.02051 (2017)
4. Bora, D.J.: Importance of image enhancement techniques in color image segmen-
tation: a comprehensive and comparative study. arXiv preprint arXiv:1708.05081
(2017)
5. Purohit, P., Joshi, R.: An efficient approach towards k-means clustering algorithm.
Int. J. Comput. Sci. Commun. Netw. 4(3), 12 (2013)
6. Zheng, X., Lei, Q., Yao, R., Gong, Y., Yin, Q.: Image segmentation based on
adaptive K-means algorithm. EURASIP J. Image Video Process. 1, 68 (2018)
7. Yedla, M., Pathakota, S.R., Srinivasa, T.M.: Enhancing K-means clustering algo-
rithm with improved initial center. Int. J. Comput. Sci. Inf. Technol. 1(2), 121–125
(2010)
8. Gao, J., Zhou, Y., Barner, K.E.: View: visual information extraction widget for
improving chart images accessibility. In: 2012 19th IEEE International Conference
on Image Processing, Florida, USA, pp. 2865–2868. IEEE (2012)
9. De, P.: Automatic data extraction from 2D and 3D pie chart images. In: 2018 8th
IEEE International Advance Computing Conference (IACC), India, pp. 20–25.
IEEE (2018)
 Segmentation of Chart Images Using H-Means Algorithm 397

10. Prasad, V.S.N., Siddiquie, B., Golbeck, J., Davis, L.S.: Classifying computer gener-
ated charts. In: 2007 IEEE International Workshop on Content-Based Multimedia
Indexing, France, pp. 85–92. IEEE (2007)
11. Nazeer, K.A., Sebastian, M.P.: Improving the accuracy and efficiency of the k-
means clustering algorithm. In: Proceedings of the World Congress on Engineering,
pp. 1–3. Association of Engineers, London (2009)
12. Sharma, A., Sharma, V.K., Srivastava, D.K.: Region merging based segmentation
approach for color image using maximal similarity index. In: 2nd International
Conference on Trends in Electronics and Informatics (ICOEI), India, pp. 403–408.
IEEE (2018)
 Hand Vein Biometric Recognition Using
Local Binary Pattern

Koushlendra Kumar Singh(B) , Arun Mishra, Anirudh Deep, Ashav Kumar,

and Bindesh Kumar

Department of Computer Science and Engineering, National Institute

of Technology Jamshedpur, Jamshedpur 831014, Jharkhand, India
[email protected]

Abstract. The study of physical or behavioral characteristics utilized

for the identification of a human being is called biometric. Dorsal hand
vein biometric traits have been used for biometric identification in
present manuscript. First of all, Input dorsal hand vein image is prepro-
cessed in proposed approach. The feature is extracted from preprocessed
image using Orientation of Local Binary Pattern (OLBP) in binary for-
mat. Hamming distance is used for matching of features in proposed
approach. Proposed approach has been tested on dataset of 600 vein
images collected from 50 people.

Keywords: Dorsal hand vein · Feature extraction · Hamming

distance · Local Binary Pattern (LBP)

1 Introduction

Biometric becomes popular among researcher since last two decades due to its
different applications in common life of people. Biometrics help to improve the
applications of different technology with reliability, identification and secure
access of users. Human biometric traits includes palmprints, facial images, hand
vein images, signature, fingerprint, retinal information etc. The distinctive infor-
mation in biometric images uses as a feature vector to identification verification.
Dorsal Hand vein identification has come-up with important biometrics traits.
It becomes a very important and effective biometrics trait. Vein structure is an
example of intrinsic biometric. It is more accurate than extrinsic as the vein
pattern is a bank of blood vessels situated directly beneath the person’s hand.
Study of this biometric feature has been started since the past two decades [1,2].
Many researchers had been contributed towards an accurate recognition system
or optimizing the matching time.
Eigen veins have been used to represents vein patterns and reduces its dimen-
sion of vein pattern through principal component analysis (PCA) [1]. FRR and
FAR have been reduced and matching of features is done by Euclidean distance.
Finger webs have been used to find the region of interest (ROI) and feature points
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 398–406, 2020.
https://doi.org/10.1007/978-981-15-3666-3_32
 Hand Vein Biometric Recognition Using Local Binary Pattern 399

extracted through watershed transformation. The multi resolution representa-

tion of input data used for matching purpose [2]. The application of directional
filter bank in dorsal vein recognition system has been explored by Lee et al. [3].
The vein patterns have acquire through different directions through directional
Gabor filter. The binary coded feature has been matched by Hamming distance.
Li et al. [4] suggested a coarse-to-fine segmentation method to explicitly
detect the boundaries of the vein areas. The width of vessel has been used for
identification and vein network topology has been consider with help of graph
model width skeleton model (WSM). Palm vein recognition has been proposed
by Elnasir et al. Two dimensional discrete wavelet transform (2D-DWT) and
Linear Discrimination Analysis (LDA) has been used in proposed approach.
The palm vein image has been subjected to 2D-DWT decomposition and used
initially to reduce features of input images. Cosine distance nearest neighbour
has been used for matching procedure. Convolutional Neural Network (CNN)
based approach has been used by Nadiya and Gopi [5]. Authors also compared
the different depth CNN models and also analysed effect of size of input images
dataset on result. The identification process has been done through logistic recog-
nition method. The knuckle tips have been used for image normalization and
finding of Region of Interest from input images [6]. In case of two matching
score the weighted score level has been calculated. The calculated weight score
has been used to authenticate the individuals. These weight level have been
calculated by two different approach as (i) in case of binarized vein structures
hierarchical matching score from the four topologies of triangulation and Knuckle
point perimeter distances of acquired images consists the its geometrical features.
Knuckle method has been applied as Key points for normalization of image and it
is also used for extraction of region of interest of images [7]. Two matching scores
have been used to calculate weighted score level of images. It has been calculated
in two stages to authenticate the individuals. Firstly, The hierarchical matching
score has been calculated in binarized vein structures using four topologies of
triangulation and secondly the geometric features of acquired images have been
consists of knuckle points distance.
Wang [8] suggested a new method to identify persons based on multiple key
points of sets of the dorsal hand vein. Local features are extracted on basis
of Scale-Invariant Feature Transform (SIFT) to give a high recognition per-
formance. The multiple key point sets are extracted and arranged into three
classes-exclusion, union and intersection that is on the basis of their inter as well
as intra class relationships.
Khan [9] had developed an algorithm that uses Independent Component
Analysis (ICA), in this approach vein feature matrix can be represented in
reduced size. ICA can remove the problem of increased matching time. Matching
of the features is done by Euclidean distance. Biometric graph matching (BGM)
based dorsal hand vein recognition system in uncontrolled environment has been
proposed by Zhong et al. [10]. Dorsal hand vein is used with hand geometry
to accurate personal authenticate system in multi model biometric [11]. Hand
dorsal vein recognition system has been developed by matching different local
features on multisource keypoints [12]. Noise removal can also be done through
DFOF [13].
400 K. K. Singh et al.

The present work has been encompasses the use of dorsal hand vein in person
indentification. The proposed approach has been based on Local binary pattern
of input image. The orientation of local binary pattern has been used as a feature.
Hamming distance has been used for calculation of difference of the distances
between the images.

2 Proposed Approach

The process of the dorsal hand vein biometric system consists of three major
steps, namely preprocessing of vein images, feature extraction and verification of
features. These steps of proposed approach is discussed in detail in the following
paragraphs.

2.1 Preprocessing

To identify the unique feature of vein extraction, hand vein images are applied
to preprocessing steps for changing image into acceptable formats. Input image
is taken as shown in Fig. 1.

Fig. 1. Input image Fig. 2. Preprocessed image

Image is resized to a common resolution of 1024 * 768. Histogram Equal-

ization along with median filtering is performed on input data set. Histogram
equalization has been used to uniform distribution of intensity values of input
images. The output image after performing histogram equalization contains the
uniform intensities. The noise present in the input image has been removed by
applying the median filter. It’s a nonlinear digital filter technique. Roberts edge
detector operator used for detection of edge of preprocessed image. The Roberts
operator is an approximation to the continuous gradient at the interpolated point
(Fig. 2).
 Hand Vein Biometric Recognition Using Local Binary Pattern 401

2.2 Feature Extraction

Local binary pattern has been used to enhance the detection rate on different
types of data sets. It works as visual descriptor which used for classification
purpose. The LBP feature vector has been created by applying the proposed
approach.
The selected window has been divided into different cells. The every pixel
of cell is compared to each of its 8 neighbours. The neighbours of each pixels
of a cell is pixels of its left-top, left-bottom, left-middle, left top, right-middle,
right-top etc. The each pixels followed along circle. It may be clock wise or anti-
clockwise. The numeric value of each pixels has been replaced by “0” or “1”. If
the value of center pixel is greater than the neighbour’s pixel value then write
“0” otherwise “1”. 8-digit binary number comes as a results of this comparison.
It can be expressed as follow (Fig. 3).

Fig. 3. Neighbourhoods of LBP

In this proposed method, features of vein is shown using the orientation of

Local Binary Patterns (LBP) [9]. A method has been developed that can obtain
directional information for dorsal vein feature representation. An orientation of
LBP feature extraction has been applied for implementing the LBP features on
the extracted veins, which give estimation about the vein orientation. Extracted
vein structure has been depicted in Fig. 6.

2.3 Feature Matching

Bitwise matching is performed using hamming distance between the image given
by the user and image present in the dataset. It creates a large gap between these
two. Normalized hamming distance can be found as follows: If the calculated
distance is below a given hamming distance threshold, it is considered as correct
matching. And if the calculated distance is above a given hamming distance
threshold, it is considered as incorrect matching (Figs. 4 and 5).
402 K. K. Singh et al.

Fig. 4. Generation of binary pattern

Fig. 5. Orientation configuration Fig. 6. Extracted vein structure

 Hand Vein Biometric Recognition Using Local Binary Pattern 403

3 Experiment
The proposed method has been validated on a standard dataset taken from the
Department of Biomedical and Systems Engineering, Cairo University. The data
set contains 600 images collected from 50 people. All simulations were done using
MATLAB 8.1.0.604 (R2017a).

4 Results and Discussion

The quantitative analysis of proposed approach has been done. Outcome of pro-
posed approach is calculated in terms of Correct Recognition Rate (CRR) and
False Acceptance Rate (FAR). False Rejection Rate (FRR) and Equal Error Rate
(EER) has been also calculated to validate the results of proposed approach.
These four different parameters are defined as follows:
False Acceptance Rate (FAR): FAR is describes as ratio of false acceptance
to total attempts imposter. FAR is simple a ratio of two numbers. Impostor
attempts affects inversely proposal to FAR.

F alseAcceptanceRate = N umberof f alseacceptance/N umberof totalimposterattempts (1)

False Rejection Rate (FRR): FAR is defined as a ratio of number of false

rejection and number of total authentic attempts. Authentic attempts affects
inversely proportional to FRR.

F alseRejectionRate = N umberof f alserejectance/N umberof totalauthenticattempts (2)

Equal Error Rate (ERR): The acceptance and rejection errors are of equal
error rate is ERR. ROC curve is used to find the value of the EER. Correct
Recognition Rate (CRR): It is defined as the proportion of the number of sam-
ples that were correctly classified to the whole number of test samples. Results
of proposed approach have been verified in two different ways. The identification
and verification both have been conducted for different test samples. The effec-
tiveness of identification part of part of proposed approach has been calculated
in terms of correct recognition rate.
If it is obtained from the same person, then recognition has been considered
correct. The false acceptance rate (FAR) and false rejection rate (FRR) has
been used to calculate successness in verification mode of proposed approach.
FAR and FRR values have been used to plot the curve of Receiver Operating
Characteristics (ROC). ROC states graph between FAR and FRR for all feasi-
ble hamming distance thresholds and are generally used to calculate the Equal
Error Rate. The proposed method have been used 600 vein images from the
database for calculating the CRR. Out of 600 vein images 586 images have been
classified correctly and obtained 97.66% of the accuracy. Different hamming dis-
tance threshold values have been selected for matching of vein pattern to get
the false rejection rate and false acceptance rate. The corresponding threshold
vs. False Acceptance Rate (FAR) and threshold vs. False Rejection Rate (FRR)
404 K. K. Singh et al.

Fig. 7. Threshold vs false acceptance rate Fig. 8. Threshold vs FRR

Fig. 9. ROC curve for equal error rate

graphs has been plotted in Figs. 7 and 8 respectively. The resulting Equal Error
Rate plot has been given in Fig. 9. Figure 7 shows that when we increases the
threshold value the false acceptance rate is reducing. It becomes minimum when
values of threshold is 99. From Figs. 7 and 8, it has been concluded that at a
Hamming distance threshold of 0.017 the best FAR and FRR values are 0.045
and 0.023 respectively. In Fig. 9, ERR obtained is 0.011. From the results we can
say that the proposed approach gives very less Equal Error Rate compared with
other algorithms for detection of vein. Table 1 shows the comparative study of
different algorithms for detection of hand vein. The values in the table clearly
shows that results obtained by using Principal Component Analysis (PCA), EP
Tree algorithm, Convolution Neural Network method and independent compo-
nent analysis. The quantitative parameters have been calculated for all these
approaches. The different number of users data set has been used for calculation
of ERR, FRR, FAR and CRR.
 Hand Vein Biometric Recognition Using Local Binary Pattern 405

Table 1. Results with similar methods on hand vein (back surface).

Approach Database Performance

PCA Algorithm 25 Users FAR-0%, FRR-0%
EP Tree Algorithm 214 Users CRR-99.64%, ERR-0.74%
Convolutional Neural Network 102 Users CRR-99.52%
Independent Component Analysis 48 Users FAR-0.63%, FRR-0.02%
Proposed Approach 50 Users FAR-0.045%, FRR-0.023%

5 Conclusions
The proposed algorithm developed a biometric technique for dorsal vein recog-
nition. The results clearly shown that the proposed approach can adequately
represent dorsal vein features and a high accuracy of 97.66 matching then some
classification of dorsal vein patterns can be added.

References
1. Subramanian, R.K.: Low dimensional representation of dorsal hand vein features
using principle component analysis (PCA). World Acad. Sci. Eng. Technol. Int. J.
Comput. Electr. Autom. Control Inf. Eng. 3, 1 (2009)
2. Lin, C.-L., Fan, K.-C.: Biometric verification using thermal images of palm dorsal
vein patterns. IEEE Trans. Circ. Syst. Video Technol. 14(2), 199–213 (2004)
3. Lee, J.-C., Lee, C.-H., Hsu, C.-B., Kuei, P.-Y., Chang, K.-C.: Dorsal hand vein
recognition based on 2D Gabor filters. Imag. Sci. J. 62(3), 127–138 (2013)
4. Li, X., Huang, D., Zhang, R., Wang, Y., Xie, X.: Dorsal hand vein recognition
based on Width Skeleton Models, State Key Laboratory of Software Development
Environment, Beihang University, Beijing (2016)
5. Nadiya, K., Gopi, V.P.: Dorsal hand vein recognition based on Orientation of
Local Binary Pattern, 1077 - Computational Intelligence for Multimodal Biomed-
ical Imaging (2015)
6. Wan, H., Song, H., Chen, L., Yang, J.: Dorsal hand vein recognition based on
Convolutional Neural Network (CNN). In: 2017 IEEE International Conference on
Bioinformatics and Biomedicine (BIBM) (2017)
7. Ajay Kumar, K., Prathyusha, V.: Personal authentication using hand vein Trian-
gulation and Knuckle Shape extraction. IEEE Trans. Image Process. 18(9), 2127–
2136 (2009)
8. Wang, Y., Zhang, K., Shark, L.-K.: Personal identification based on multiple key
point sets of dorsal hand vein images. IET Biometrics 3(4), 234–245 (2014)
9. Khan, M.H.-M., Khan, N.M.: Dorsal hand vein biometric using independent com-
ponent analysis. In: 6th International Conference on Internet Technology and
Secured Transactions, pp. 191–195 (2011)
10. Zhon, D., Shao, H., Liu, S.: Towards application of dorsal hand vein reconition
under uncontrolled environment based on biometric graph matching. IET Biomet-
rics 8(2), 159–187 (2019)
11. Gupta, P., Gupta, P.: Multibiometric authentication system using slap fingerprints,
palm Dorsal vein and Hand geometry. IEEE Trans. Ind. Electron. 66(2), 9777–9784
(2018)
406 K. K. Singh et al.

12. Huang, D., Tang, Y., Wang, Y., Chen, L., Wang, Y.: Hand-Dorse vein recognition
by matching local features of multisource keypoints. IEEE Trans. Cybern. 45(9),
1823–1837 (2015)
13. Singh, K.K., Bajpai, M.K., Pandey, R.K.: Reconstruction of original signal from
contaminated signal using fractional order differentiator. In: IEEE International
Symposium on Signal Processing and Information Technology (ISSPIT), pp. 274–
278 (2015)
 Distance Calculation of an Object
in a Stereo Vision System

Ningthoujam Johny Singh1(B) , Wanghengbam Kanan Kumar2 ,

and Kishorjit Nongmeikapam1
1
Indian Institute of Information Technology Manipur, Imphal, India
{johny,kishorjit}@iiitmanipur.ac.in
2
North Eastern Regional Institute of Science and Technology, Nirjuli, India
[email protected]

Abstract. The ability of a machine to capture 3D information from the

real world duplicating the ability of human vision is one of the great
interest in science and industry. Works on stereo vision duplicating the
real world environment for machines to adapt the ability of human vision
by many researchers.
Stereo camera based distance calculation of an object from an observer
taken in an image is performed and height of the object is also calculated.
Stereo parameters of both the cameras are estimated using Camera cal-
ibration. Rectification of the images from both the camera is performed
so that they are in the same plane. Disparity mapping between the left
and the right images is estimated. Using the disparity map and stereo
parameters of both the cameras then the 3-Dimensional (3D) view of
the scene is constructed. The distance of the object is calculated using
Euclidean distance by determining the centroid of the bounding box and
the camera and finally height of the object is calculated. Results shows
the output of the object being placed at the distance between 10 to 3 m.
The average error in the distance calculation is 1.74%.

Keywords: Camera calibration · Rectification · Disparity · Stereo

camera

1 Introduction

The advancement of technology in the fields of navigation, robotics, autonomous

driving car, security, etc calculation of distance for an object from an observer is
required [1]. With the rapid development in society, a new algorithm for finding
the distance of the object is required as many new smart vehicles are developing
such that the distance of the object from the vehicle can prevent road collision
and traffic congestion. Two ways to calculate the distance of an object from an
observer have been identified namely active and passive. Active measurement
is the use of signals, in this approach the distance to an object is measured by
sending signals to the object such as radar, infrared, ultrasonic sound, radio
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 407–416, 2020.
https://doi.org/10.1007/978-981-15-3666-3_33
408 N. J. Singh et al.

signal, laser, etc. [2]. Passive measurement is the approach where the distance
of the object from the observer is calculated without sending the signals. In
this experiment, passive measurement is used by gathering information without
the use of a signal [3]. This paper highlights the use of the visual information
obtained from the stereo based system to estimate the distance of an object from
an observer.
Stereo vision builds a three-dimensional description from the captured scene
from multiple points, the stereo vision technique is considered passive as addi-
tional lightning such as a laser beam in the scene is used. Passive stereo vision
is mainly used in many applications such as robotics, 3D object recognition and
navigation of mobile robots [3]. This paper uses a passive stereo vision technique
using a binocular vision formula using two cameras that capture the same scene
from a different viewpoint.

2 Related Works
Stereo vision has a wide range of applications in many fields. Many approaches
using stereo vision are presented in different areas [4]. Distance measurement
using a single camera is widely used in many existing systems but a constraint is
imposed in the result obtained [5]. In [6], it is proposed that the 3D reconstruc-
tion of the scene can be only used to a certain scale factor if a single camera is
used.
In [4], the author proposed that calibrated camera becomes useful for appli-
cations that require spatial data in Euclidean space. In [7], author presented the
2D mapping to 3D which enables to describes the point in 2D corresponds to
the point in the 3D scene. In [8], author demonstrated the calculation of the
distance of an object along with the size measurements using multiple vision
sensors. A new approach for creating a 3D scene for mobile robotics by com-
bining a predictive maintenance (PdM) camera and a stereo camera is proposed
in [9].
In [10], a new method is proposed by using real-time motion estimation
such that depth errors that occurred to camera shift is reduced. This helps in
rectifying the errors in alignment between the cameras. The motion estimation
framework is incorporated with the disparity between the stereo camera.

3 Theory
The stereo vision-based system to reconstruct the three-dimensional description
of the target object is based on triangle theory. Triangle is formed by the two
cameras and the target object. Measuring the distance of an object using stereo
vision is shown in Fig. 1.
Here, A is the target point representing the object, OL and OR represents
the optical centers for the stereo camera with the same focal length, AL and
AR gives the points for projection of the target point A of the stereo camera
respectively, the length of the left and right abscissa corresponding to the AL
 Distance Calculation of an Object in a Stereo Vision System 409

Fig. 1. Binocular vision measuring system

and AR in the camera are given by xl and xr respectively. So, the coordinates
representing the object is xl and xr , b denotes the actual distance between the
two cameras, f represents the focal length of the camera as shown in Fig. 1. The
disparity between the two sets of images is given by d = xl − xr and L represents
the actual distance of the target object from the camera, Eq. 1 gives the formula
for calculating the value of L for the basic stereo binocular vision.
b∗f
L= (1)
xl − xr
The coordinate system required to be considered for designing a stereo vision
for calculating the distance for the target object from the camera viewpoints are
given below [11]:
(1) World coordinate system (XW , YW , ZW ), user-defined coordinate system in
3 dimension.
(2) Camera coordinate system (XC , YC , ZC ), the coordinate system with the
camera as the center and z-axis is taken along the optical axis.
(3) Image plane coordinate system (x, y), this plane is on the intersection of
the optical center of the camera axis and the imaging point.
(4) Image coordinate system (u, v), which is in accordance with the pixel coor-
dinate system.
Understanding the relationship between different coordinate systems is
required to build a stereo based distance measurement system for calculating
the distance of the target object in the scene. Relationship between camera
coordinates and world coordinates is described by the translation vector t and
rotation matrix R which is given by:
⎡ ⎤ ⎡ ⎤
Xc  Xw
⎢ Yc ⎥ ⎢ ⎥
⎢ ⎥ = RT t ⎢ Yw ⎥ (2)
⎣ Zc ⎦ 0 0 ⎣ Zw ⎦
1 1
410 N. J. Singh et al.

Fig. 2. Geometric representation of the object from the observer in lateral view.

Equation 3 provides the relationship between the image plane coordinate sys-
tem and world coordinate system, where the focal length of the camera is denoted
by f and s is the skew coefficient [12].
⎡ ⎤
⎡ ⎤ ⎡ ⎤
x f 000  Xw
R t ⎢ ⎥
⎢ Yw ⎥
s ⎣y ⎦ = ⎣ 0 f 0 0⎦ T (3)
⎣
0 0 Zw ⎦
1 0 010
1
The relationship between image coordinate and world coordinate is presented
in [12]: ⎡ ⎤
⎡ ⎤ ⎡ ⎤ Xw
u ax 0 u0 0 ⎢ ⎥
Yw ⎥
s ⎣v ⎦ = ⎣ 0 ay v0 0⎦ ⎢
⎣ Zw ⎦ (4)
1 0 0 1 0
1
where, (u0 , v0 ) is the coordinate when the camera axis is projected to image
coordinate and, ax and ay is u, v axis scale factors in image coordinate.
Figure 2 shows the geometric representation of the object from the observer.
Point A represents the observer and BC represents the height of the object. h1
is the distance of the bottom of the object from the observer, h2 is the distance
of the center of the object from the observer, h3 is the distance of the top of the
object from the observer and h is the height of the object.
A new equation is formulated to calculate the height of the object is given
in Eq. 2.
h2 h2
h= + (5)
tan(sin−1 hh23 ) tan(sin−1 hh21 )

4 Proposed Method
The proposed method has five processes, calibration of the camera is the first
step followed by rectifying the image, next process is disparity mapping followed
by 3D reconstruction followed by detecting and estimating the distance of the
target object from the camera. The last step is the measuring of the height of
the target object. The parameters of the cameras are given in Table 1.
 Distance Calculation of an Object in a Stereo Vision System 411

4.1 Camera Calibration

Camera calibration is the process of estimating the parameters for the cameras as
shown in Fig. 3. Calibrating the camera provides the extrinsic as well as intrinsic
parameters of both the cameras. Extrinsic parameters explain the translation,
rotation, and orientation of the right camera with respect to the left camera. The
extrinsic parameter for the stereo vision is given in Fig. 4. The intrinsic parameter
gives the mapping transformation of the points in the camera coordinates to pixel
coordinates. The process of image rectification along with 3D reconstruction of
the scene is done using these two parameters [13]. Figure 3 represents the camera
calibration of the stereo vision system. The camera intrinsic matrix is denoted
by K, and it is given by:
⎡ ⎤
fx 0 0
⎣ s fy 0⎦
cx cy 0
where,
[cx , cy ] is the optical center of the camera in pixels,
(fx , fy ) is the focal length in pixels,
fx = pFx
fy = pFy
F = focal length in world units (in millimetres)
(px , py ) size of pixels in world unit.
s is the skew coefficient

Fig. 3. Camera calibration for the two cameras

4.2 Image Rectification

Image rectification is the process of projecting two or more images into a common
image plane. One of the difficult tasks in image rectification is to identify a point
captured by one camera to the corresponding point in the other camera. Image
rectification is performed as an alternative to perfect camera alignment.
In the stereo vision system, the relative positions of the two cameras are
already known. It takes a pair of image scenes from two viewpoints that pro-
vide the scene from both the cameras. The corresponding depth of the pixels
412 N. J. Singh et al.

is obtained by locating the matching pixels for every pixel in the pair of the
images, finally, to determine the distance of the target object, triangulation is
applied to the matches found. Figure 5 gives the rectified image.

4.3 Disparity Mapping

The disparity is the difference between the right rectified image and the left
rectified image in their corresponding points due to the combination of the two
rectified images from two different viewpoints. In a stereo vision system, the
disparity map gives the depth of the image in the given scene. The disparity can
provide the depth of an object in the scene as it is inversely proportional to the
depth in the image. The disparity is high when the object in consideration is
closer to the camera. Inversely, the disparity is low when the object is kept far
from the camera. Figure 6 gives the disparity mapping.

4.4 3D Reconstruction
This subsection explains the construction of a 3D representation of the image
scene. 3D reconstruction of the image scene is achieved by using disparity map-
ping and camera calibration parameters. The input of the disparity map is given
by the pixels which match with the coordinates in 3D representation. The coor-
dinates in the 3D coordinate system relative to the optical center of the left
camera are estimated using the fundamental and essential matrices. Figure 7
gives the reconstructed 3D representation of the scene.

Fig. 4. Stereo vision extrinsic parameter

4.5 Distance Calculation

3D representation of the scene is created, then the target object is detected. The
target object for this work is an upright person. Detection is done by using a
pre-trained detector that uses an aggregate channel feature. The detected object
is then enclosed inside a bounding box. To estimate the distance of the target
object, the distance between the centroid of the detected object and the camera,
 Distance Calculation of an Object in a Stereo Vision System 413

the distance used for the estimation is the euclidean distance. Let the coordinates
of the camera representing the viewpoint be (x, y, z) and the coordinates for the
centroid of the bounding box representing the target object be (xc , yc , zc ), then
the distance is calculated by using the following formula:

D = (xc − x)2 + (yc − y)2 + (zc − z)2 (6)

Figure 8 shows the measured distance of the detected object.

4.6 Height Estimation

The height of the object is estimated using Eq. 5. The find the value of h1 , h2
and h3 , the stereo vision system is used. h1 represents the distance of the top
point of the detected object from the viewpoint, h2 represents the distance of
the centroid of the detected object from the viewpoint and h3 represents the

Table 1. Camera parameters

Parameter Left camera Right camera

Resolution 1280 × 720 1280 × 720
Duration 4s 4s
Total frame 159 159
Frame rate 15 fps 15 fps
Distance between cameras 15 cm 15 cm

Fig. 5. Rectified image

Fig. 6. Disparity mapping

414 N. J. Singh et al.

Fig. 7. Reconstructed 3D

distance of the bottom point of the detected object from the viewpoint. Figure 9
gives the height of the detected object.

5 Result
Stereo based system for estimating the distance of a target object from the
viewpoint is successfully implemented. The distance is measured for the person
which moves towards the camera. The distance measurement of the object is
shown in Fig. 8. The error percentage is estimated by the following formula:

|D − M |
error% = ∗ 100 (7)
D
where,

D = actual distance.
M = measured distance.

Table 2 gives the percentage of error in estimating the distance of the object
from the camera along with the measured distance and the actual distance of
the object. The distance of an object measured using the stereo vision system

Fig. 8. Object detection with calculated distance.

 Distance Calculation of an Object in a Stereo Vision System 415

gives a minor error. As the analysis of the image is done frame by frame which
leads to slow processing which in turn provides the error. And most importantly
the timing for starting and stopping for the acquisition of the cameras does
not coincide at the same time, which results in the mismatched in the frame
matching between the pair of images.

Fig. 9. Height of the detected object.

Table 2. Distance measured

Actual distance (m) Measured distance (m) Error (%)

5 4.82 3.6
3.6 3.5 3.3
3.4 3.35 2.4
3.3 3.28 0.6
3 2.86 0.5

6 Conclusion
In this experiment, a stereo vision system to calculate the depth of an object
in an image has been implemented using binocular triangle theory, it is shown
that distance can be calculated from a stereo vision system with minimal error.
Camera calibration is applied to estimate the intrinsic and extrinsic parameters
of the camera. This method can be improved by minimizing flaws in matching
video frames. This method can be applied in finding a safe distance in driving
cars as well as in robotics and autonomous car etc.
416 N. J. Singh et al.

References
1. Hou, A.L., Cui, X., Geng, Y., Yuan, J., Hou, J.: Measurement of safe driving
distance based on stereo vision. In: Sixth International Conference on Image and
Graphics (ICIG), Hefei, Anhui, China, pp. 902–907 (2011)
2. Sukhla, N.: A review on image based target distance and height estimation tech-
nique using laser pointer and single video camera for robot vision. Int. J. Eng. Res.
Rev. 3(1), 128–135 (2015)
3. Nagar, S., Verma, J.X.: Distance measurement using stereo vision. Int. J. Electr.
Electron. Eng. 07(01), 50–53 (2015)
4. Appiah, N., Bandaru, N.: Obstacle detection using stereo vision for self-driving
cars. In: IEEE Intelligent Vehicles Symposium (IV) (2011)
5. Fathi, H., Brilakis, I.: Multistep explicit stereo camera calibration approach to
improve Euclidean accuracy of large-scale 3D reconstruction. J. Comput. Civ. Eng.
30, 04104120 (2014)
6. Pollefeys, M., et al.: Detailed real-time urban 3D reconstruction from video. Int.
J. Comput. Vision 78(2–3), 143–167 (2008)
7. Hsu, T.-S., Wang, T.-C.: An improvement stereo vision images processing for object
distance measurement. Int. J. Autom. Smart Technol. 5, 85–90 (2015)
8. Kuhnert, K.-D., Stommel, M.: Fusion of stereo-camera and PMD-camera data for
real-time suited precise 3D environment reconstruction. In: IEEE/RSJ Interna-
tional Conference on Intelligent Robots and Systems (2006)
9. Santoro, M., AlRegib, G., Altunbasak, Y.: Misalignment correction for depth esti-
mation using stereoscopic 3-D cameras. In: IEEE 14th International Workshop on
Multimedia Signal Processing (MMSP) (2012)
10. Mahammed, M.A., Melhum, A.I., Kochery, F.A.: Object distance measurement by
stereo vision. Int. J. Sci. Appl. Inf. Technol. (IJSAIT) 2, 05–08 (2013)
11. Hold, S., Nunn, C., Kummert, A., Muller-Schneiders, S.: Efficient and robust
extrinsic camera calibration procedure for lane departure warning. In: IEEE Intelli-
gent Vehicles Symposium, pp. 382–387 (2009). https://doi.org/10.1109/IVS.2009.
5164308
12. Steger, C., Ulrich, M., Wiedemann, C., Shaorong, Y.: Machine Vision Algorithms
and Applications, pp. 257–261. Tsinghua University Press, Beijing (2008)
13. Adi, K., Widodo, C.E.: Distance measurement with stereo vision. Int. J. Innov.
Res. Adv. Eng. (IJIRAE) 4(11), 24–27 (2017)
 Comparative Assessment of Consumer
Perception on Sports Brands Using Data
Mined from Online Communities

Vishu Singhvi(B) and Prateek Srivastava

Sir Padampat Singhania University, Udaipur, Rajasthan, India

[email protected], [email protected]

Abstract. Sportswear industry in recent times has become pretty much

prominent and the sports wears have become common in all age group
people as it helps them in a comfortable stylish living and greater adapt-
ability. Due to the increase in the demand of Sport wears, bigger man-
ufacturing companies like adidas, Puma, Nike, Fila and Reebok have
expanded their business and online sales specifically in India. The cur-
rent research study collects and does assessment of user experiences,
reviews or comments from e-commerce sites, blogs and various social
media platforms. Also, the study compares four brand analysis parame-
ters (BAP) namely Brand Name, Product Quality, Service Quality and
Mobile App/Website with respect to five sports brands which includes
adidas, Nike, Puma, Reebok and Fila. On the basis of consumer’s reviews
and comments, data accumulation was done and further the mined data
from various platforms was divided into positive and negative comments.
Using statistical approach the data was distributed among postive and
negative comments with respect to four mentioned brand analysis param-
eters of each sports brand. Once segregation was done, research was car-
ried forward and final comparison was done on all five sports brands
with respect to each BAP. The research did the comparative assessment
of consumer perception regarding selecting sports brands using Netnog-
raphy and Text Mining from data being mined from online communities.

Keywords: Netnography · Text Mining · Brand Analysis Parameters

(BAP) · Brand Name · Product Quality · Service Quality · Mobile
App/Website

1 Introduction

Sportswear industry and Sports Apparels have become popular among people
and especially in the youth as it makes them more comfortable and relaxed
in sportswear outfits. Due to the increase in the demand of sportswear, big
Sports Brands companies like adidas, Nike, Fila, Reebok and Puma tried to
increase their business with focus of sales of their products of various categories
and marketing in India. Sports Brands have become an important part in the
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 417–429, 2020.
https://doi.org/10.1007/978-981-15-3666-3_34
418 V. Singhvi and P. Srivastava

life of an athlete or any Sports Oriented person or physically fit people. The

basic 7 factors that contribute to brand loyalty with customer’s satisfaction are
Brand Name, Price, Product Quality, Promotion, Service Quality, Style and
Store Environment. Brand Name has shown strong bonding and interrelation
with the term Brand Loyalty.
The bigger manufacturing companies try to increase their Brands Loyalty
and the factor of trust among their existing and new customers to increase their
sales. Brand Loyalty has been considered as one of the basic goals which is to
be achieved by the company’s marketing team through which a bigger group of
loyal customers tend to purchase the product of same brand. A bigger section
of people buying the Sports-wear apparels including shoes, t-shirts, shorts, wrist
bands, caps etc. purchase or re-purchase a product after reading the reviews or
comments or feedback about the product. Huge number of e-commerce platforms
are available from where the customers buy the product online and accordingly
put their comments and reviews for that particular product’s brand, price, qual-
ity, service quality, the platform from which they bought the product i.e. mobile
app or its website and lot more. This shows that the decision of purchase or
repurchase of a product depends on its quality of performance, trust and its ser-
vice [3]. This means a bigger majority of customers check regarding the quality,
service quality or any other BAP by checking the reviews online or using the
positive or negative behavior from oral messages.
In the past 1 and a half decade there has been great attention made on the
consumer’s behavior. The main reason behind changes in customer or consumer’s
behavior is because consumer is being treated as one among the factors which
decides company’s fortune. The analysis of consumer perception in sports brands
industry helps in the understanding of consumer’s requirements. More the better
understanding of consumer’s behavior, more the designing and marketing team
can strategize to attract customers and accordingly greater quality and greater
service quality of a product and with lot more other things. The current paper
focus on 4 Brand Analysis Parameters (BAP) of Brand Name, Product Quality,
Service Quality and Mobile App/Website out of all the parameters and does a
comparative analysis on 5 sports brands namely adidas, Nike, Puma, Fila and
Reebok with respect to each BAP.

2 Review of Literature
In order to introduce or popularize a particular product quickly in the market,
Advertisement is one of the best and the most effective ways of branding a
particular product. For more popularity of a product more advertisement is
needed which directly impacts and affects the costing. Since, advertising needs
big amount of investment in the market to have a healthy competition in the
market. Researchers gave the suggestion of using Brand Extension technique
to achieve efficiency of advertising [1]. In addition to this scenario, extension
of brand can also become an important method to showcase new products in
market without advertising them which can save huge amount of money and
lower the cost of new products.
 Comparative Assessment of Consumer Perception on Sports Brands 419

Loyal consumers purchase and repurchase a product of the same brand and
accordingly brand loyalty can be measured but the commitment to repurchase a
particular product in addition to its services [2]. It is noted that up to 65% of even
the satisfied customers tend to leave or leave their current service provider with
respect to sports brands because of one of the pleasure lived during the immer-
sion into the service platform [4,5]. This gap leaves researchers with insufficient
information about what actually drives consumers towards Brand Loyalty.
In the year 2005, Langer and Beckman used Netnography to study cross-
consumer online communication about cosmetic surgery on Danish internet mes-
sage board. They further observed that Netnography was very much useful since
it is very difficult to get the information offline. On further observation, Netnog-
raphy provided a deeper insight into consumer’s concerns, opinion and motive
with an opportunity to express opinions freely. Cengiz and Yayala, 2007 com-
pleted their research on effect of marketing mix on positive word of mouth com-
munication. There were sub variables under each p. The project worked and iden-
tified 7 factors which contribute towards the word of mouth. The study focus and
emphasize on what the companies should do to have a positive oral communica-
tion out of which the key factor came out to be excellent service. Another study
reviewed for the study is “Factors influencing purchase of Apparels and Footwear
from organized retail outlet”. Through this study the major factors which affect
consumer perception were style, quality, variety, price, appeal, brand name and
referral group [6].
Some other group of researchers Ko, Kim and Hoon in 2008 tried to compare
consumer behaviors for sports brands in China and Korea. The perceived price
was founded to be negatively influencing the purchase intention in the country
China. One more study in similar field which was reviewed was “The external
factors that influences Taiwanese students shoe purchase intention”. This par-
ticular study was based on the 6 important external factors including brand,
promotion, price, style durability and comfort [8]. Another study in the similar
field which was reviewed for the research is “The external factors that influences
Taiwanese students shoe purchase intention”. This study was based on 6 external
factors which include brand, price, promotion, style, durability and comfort. The
motive of this study was to investigate various factors that influence the Tai-
wanese college students to buy a particular product. The key factors and results
came out of the study were the factors comfort and style as the most important
driving factors for Taiwanese college students shoe purchase intention [7].
Youn, Song and MacLachlan in the year 2008 examined how consumer’s
brand preferences and price sensitivities evolve as their levels of experience
increase in outdoor sports gear categories [9]. Another article reviewed for the
research is “Service quality as a mediator of the relationship between marketing
mix and consumer loyalty”. The 3 factors studied under this are Marketing mix,
Service quality and customer loyalty. The inferences showed that there exists a
strong and important relation between marketing and customer loyalty and a
positive relationship between customer loyalty and service quality. In addition
to above results, there is no intermediate effect of service quality between the
420 V. Singhvi and P. Srivastava

perception of marketing mix and consumer loyalty [10]. Saha, Dey, and Bhat-
tacharya in 2010 in their study “Factors affecting consumer buying behavior of
shoes in Kolkata” tried to identify various aspects in the market that drives the
purchase behavior of customers in city of Kolkata. Four major factors came out
from the study were price, product, place and promotion [12].
Krishna, 2012 focused on the behavior of youth’s buying sports items which
showed different consumers have got different decision making processes. He
also showed that the target goal of the buyer is in getting a qualitative and
quantitative product with low or minimal cost associated with it [13]. Yoh, Mohr
and Gordon in 2012 did an investigation on factors that influence Korean teens
athletic footwear purchase and the research was carried out on 7 basic factors of
style, price, color, brand name, comfort, quality, and celebrity endorsement. The
“Text mining” technique is the data mining technique which is used to automate
the discovery of any kind of knowledge from unstructured dataset or text [14].

3 Main Objective of Research

The major objective of the research study is to do a Comparative Assessment
of four BAP forming consumer perceptions for selecting sports brands through
Netnography and Text Mining using the data mined from online communities.
On concrete basis of literature review and post analysis, there were several fac-
tors which formed consumer perception but out of all the factors, the current
research compares and focus on the four factors or Brands Analysis Parameters
(BAP) which are Brand Name, Product Quality, Service Quality and Mobile
App/Website for 5 Sports Brands companies namely adidas, Nike, Puma, Reebok
and Fila.

4 Methodology
The research study takes key element from Text Mining and Netnography tech-
niques and accordingly proceed further to achieve the objective of the study.
Text Mining technique in software applications is used in the research study for
extracting the comments and important information from online communities for
the research. Netnography will be used by adaptation of the technique of observ-
ing an online community through the data they generate, research sprints and
java coding methods to divide the brand associations and comments based on
BAP. The research study follows 6 steps of Netnography [11] includes Research
Planning, Entrée, Data Collection, Interpretation, Ensuring ethical standards
and Research representation. The research study was carried forward in the fol-
lowing sequential steps:

4.1 Identification of Data Source

The Identification of data source was carried for selecting sports brands based on
Brand Name, Product Quality, Service Quality and popularity of Mobile Appli-
cation or website. Based on literature review and further analysis, there were
 Comparative Assessment of Consumer Perception on Sports Brands 421

several Brand Analysis Parameters i.e. Brand Name, Service Quality, Product
Quality, Price, Style, Mobile App/Website etc but only 4 of them which are
Brand Name, Service Quality, Product Quality, Mobile App/Website are con-
sidered in the current research study. Also, 5 popular sports brands in India
were taken into consideration namely adidas, Puma, Nike, Reebok and Fila.
E-commerce platforms of www.flipkart.com, www.amazon.in, www.mouthshut.
com, public tweets from Twitter social media platform, blogs and commenting
forums were taken as data sources from where data collection phase was carried
out [17].

4.2 Data Collection Phase

The data collection phase proceeded further after the Identification of data
source was completed and data got accumulated as raw data in this particu-
lar phase. The collected data was divided into primary and secondary data in
which the primary data constituted data from social media platforms and meta
data pertaining to the select a particular sports brands. The next phase would
be data collection using tools like : Data Minor, Web Scraper and using Twitter
API’s. The secondary data constituted the data from published articles from
magazines, online journals, print journals, e-books and newspapers. The data
was collected for the following five Sports Brands: adidas, Puma, Nike, Reebok
and Fila. For the research data collection was done from online e-commerce
platforms, social media sites, blogs and reviews websites. The major e-commerce
platforms used for data collection are: www.flipkart.com, www.amazon.in, www.
mouthshut.com and www.snapdeal.com. Also, the public tweets using twitter
APIs were extracted using JAVA [18] program for the extraction of tweets from
twitter and segregation of comments or reviews was done 4 BAP being used
in the research i.e. Brand Name, Product Quality, Service Quality and Mobile
App/Website. Online comments and Reviews were collected from the mentioned
data sources for each sports brands and finally the data model in terms of
Tables 1, 2, 3, 4 and 5 was created after the data processing step. Tools which
used to collect the data during the research are includes Data scrapper - Data
Minor, Web scrapper and Twitter APIs. Data collection has been done from vari-
ous e-commerce websites, blogs, using twitter APIs etc. and accordingly based on
the Data processing approach data has been divided into positive and negative
comments after reading them. Thus, before reading the comments data has to
be flattened out for which the JAVA programs has been written and accordingly
comments were segregated. Few samples of data collected from various sites are
shown in Figs. 1, 2, 3 and 4.
422 V. Singhvi and P. Srivastava

Fig. 1. A positive comment for Puma sports brand on product quality brand analysis
parameter on an e-commerce site.

Fig. 2. A positive comment for Reebok sports brand on product quality brand analysis
parameter on an e-commerce site.

4.3 Data Processing and Representation

Once the data was collected which was in a raw format, the data was further
segregated and was further processed through a sequence of steps to convert the
raw data into readable format. Using basic JAVA programs, Extra spaces, lines,
non-readable data and special characters, redundant data were removed from
the files containing the basic data in raw format. The first JAVA programs tries
 Comparative Assessment of Consumer Perception on Sports Brands 423

Fig. 3. A positive comment for Nike sports brand on product quality brand analysis
parameter on an e-commerce site.

Fig. 4. A negative comment for Nike sports brand on brand name and price brand
analysis parameter on MouthShut.com.

to convert the raw data into readable format and the second JAVA coded pro-
gram is being used to count the segregate the sentences containing the comments
and reviews of positive and negative nature. Further using statistical approach
and reading the reviews and comments, each and each meaningful comment and
review is divided into positive and negative comment with respect to the 4 BAP
of Brand Name, Product Quality, Service Quality and Mobile App/Website.
424 V. Singhvi and P. Srivastava

The above processing steps were carried for all the 5 Sports Brands of adidas,
Reebok, Nike, Puma and Fila w.r.t. the 4 mentioned BAP. In the end, the pro-
cessed finished data is collected and stored in MySQL [16] database with the
column as positive (+) and negative (−) for each consumer perception BAP.
Tableau Tool is being used as an Analytic tool for the representation of the pro-
cessed finished data, review and comments for all 5 sports brands with respect
to the 4 BAPs [15]. Tableau public provides strong computational research and
analytic visualization in terms of graphs, charts, diagrams, tables etc. helps in
assessment and doing comparative analysis on the data being mined from online
communities.
In the data representation phase, the data which got collected in Sect. 4.2
and further processed during Sect. 4.3, the same processed data is being used
for Data Representation using the research and analysis tool called Tableau.
The Data which got collected during data collection phase was created into
MySQL database and further using Data processing technique and SQL queries,
the data was flattened out to be in tabular format as shown in the Tables 1,
2, 3, 4 and 5 for Brand analysis parameters (BAP) of Brand Name, Product
Quality, Service Quality and Mobile App/Website. Tableau is a well known BI
Data Analytic tool prominently used in the IT industry for graphical, tabular,
representations and a lot more. The representation of the data is done in the
percentage format after the summation of the each BAP data for all positive and
negative comments total. The data is being transformed into percentage format
to make the base as common for all the parameters for all the sports brands
finally a comparative assessment can made of consumer percentage for selecting
a sports brand. Graphs from 1 to 4 represents the tabular data are shown for
each Sports Brands with each of their BAP in percentage format on the X-Axis.

Tables and Graphs. The Tables 1, 2, 3, 4 and 5 contains the processed data
comments from each of the sample collected from various e-commerce sites, twit-
ter APIs, blogs and forums. Graph 1 to 5 represents the graphical representation
of the tabular data contained in Tables 1, 2, 3, 4 and 5 in percentage format for
each brand.
The 5 tables (Tables 1, 2, 3, 4 and 5) shows the data collection and processed
results w.r.t. 5 sports brands adidas, Puma, Nike, Reebok and Fila with on four
Brand Analyses Parameters/factors i.e. Brand Name, Product Quality, Service
Quality and Mobile App/Website. Data has been collected from various social
networking, e-commerce sites, blogs, reviews post and using search engine tech-
nique results. The ‘+’ symbol represents Positive Reviews/Comments and ‘−’
symbol represents Negative Reviews/Comments.

5 Data Conversion to Percentage

Assuming any Brand Analysis Parameter (BAP) who has a positive and a neg-
ative variable as Param Positive and Param Negative respectively. The generic
% calculation formula for Brand Analysis Positive Parameter would be:
 Comparative Assessment of Consumer Perception on Sports Brands 425

n
 n

P aram P ositive% = ( (P aram P ositivek ) ∗ 100)/( (P aram P ositivek )
k=1 k=1
n

+ (P aram N egativek )) (1)
k=1

Table 1. Adidas data collection results (w.r.t. 4 brand analyses parameters i.e. Brand
Name, Product Quality, Service Quality and Mobile App/Website).

Brand Brand Product Product Service Service Mobile Mobile

Name Name Quality Quality Quality Quality App/Website App/Website
+ − + − + − + −
Sample 1 67 11 111 16 24 4 16 1
Sample 2 4 1 10 13 3 9 3 5
Sample 3 206 43 270 72 43 15 64 10
Sample 4 340 159 719 274 465 138 105 64
Sample 5 32 18 71 34 31 9 10 3

Table 2. Puma data collection results (w.r.t. 4 brand analyses parameters i.e. Brand
Name, Product Quality, Service Quality and Mobile App/Website).

Brand Brand Product Product Service Service Mobile Mobile

Name Name Quality Quality Quality Quality App/Website App/Website
+ − + − + − + −
Sample 1 66 14 171 56 46 11 18 5
Sample 2 29 8 65 17 17 3 9 4
Sample 3 34 7 82 16 22 4 7 5
Sample 4 12 5 15 5 5 2 2 1
Sample 5 49 21 419 103 33 13 37 13
Sample 6 6 2 11 11 4 9 2 3
Sample 7 3 0 8 2 5 0 1 0
Sample 8 413 124 948 373 107 49 24 15

Table 3. Nike data collection results (w.r.t. 4 brand analyses parameters i.e. Brand
Name, Product Quality, Service Quality and Mobile App/Website).

Brand Brand Product Product Service Service Mobile Mobile

Name Name Quality Quality Quality Quality App/Website App/Website
+ − + − + − + −
Sample 1 28 10 36 10 6 2 3 2
Sample 2 36 7 229 59 36 8 15 3
Sample 3 54 12 127 29 85 23 21 9
Sample 4 289 47 718 170 45 21 16 4
Sample 5 346 67 781 235 84 20 25 9
426 V. Singhvi and P. Srivastava

Similarly, the generic % calculation formula for Brand Analysis Negative Param-
eter would be:
n
 n

P aram N egative% = ( (P aram N egativek ) ∗ 100)/( (P aram P ositivek )
k=1 k=1
n

+ (P aram N egativek )) (2)
k=1

where n = Number of data samples for the specific parameter either +ve or
−ve In the same way, calculation is performed for the positive and negative
% values for the 6 brand analysis parameters i.e. brand name, product quality,
service quality and mobile App/Website. The 4 graphs (Figs. 5, 6, 7 and 8) shows
averaging percentage from Tables 1, 2, 3, 4 and 5 for Brand Name w.r.t. 5 Sports
Brands. The X-Axis percentage value has been calculated on the basis of formulas
(Param Positive% and Param Negative%) shown in the previous section of the
paper. The 4 graphs include Brand Name Parameter Graph (Fig. 5), Product
Quality Parameter Graph (Fig. 6), Service Quality Parameter Graph (Fig. 7)
and Mobile App or Website Parameter Graph (Fig. 8). All the 4 graphs shows
the comparative assessment of respective graph w.r.t. five Sports Brands which
are adidas, Nike, Puma, Reebok and Fila.

Table 4. Reebok data collection results (w.r.t. 4 brand analyses parameters i.e. Brand
Name, Product Quality, Service Quality and Mobile App/Website).

Brand Brand Product Product Service Service Mobile Mobile

Name Name Quality Quality Quality Quality App/Website App/Website
+ − + − + − + −
Sample 1 3 0 4 2 1 0 1 0
Sample 2 5 3 13 9 6 3 1 1
Sample 3 3 1 4 3 4 2 0 0
Sample 4 189 59 564 299 36 26 9 4
Sample 5 6 1 9 1 5 1 1 0
Sample 6 73 17 171 14 52 19 7 3

Table 5. Fila data collection results (w.r.t. 4 brand analyses parameters i.e. Brand
Name, Product Quality, Service Quality and Mobile App/Website).

Brand Brand Product Product Service Service Mobile Mobile

Name Name Quality Quality Quality Quality App/Website App/Website
+ − + − + − + −
Sample 1 21 4 75 18 12 2 1 0
Sample 2 54 0 91 36 16 9 6 2
Sample 3 12 3 41 16 2 2 1 2
Sample 4 2 1 4 3 3 1 1 0
 Comparative Assessment of Consumer Perception on Sports Brands 427

Fig. 5. Bar graph showing brand name BAP w.r.t. positive and negative percentages
of five sports brands which are adidas, Nike, Puma, Reebok and Fila.

Fig. 6. Bar graph showing Product Quality BAP w.r.t. positive and negative percent-
ages of five sports brands which are adidas, Nike, Puma, Reebok and Fila.

Fig. 7. Bar graph showing Service Quality BAP w.r.t. positive and negative percent-
ages of five sports brands of adidas, Nike, Puma, Reebok and Fila.

Fig. 8. Bar graph showing Mobile App/Website BAP w.r.t. positive and negative per-
centages of five sports brands of adidas, Nike, Puma, Reebok and Fila.
428 V. Singhvi and P. Srivastava

6 Conclusion
The Research study compares 4 Brand Analysis Parameters (BAP) which
includes Brand Name, Product Quality, Service Quality and Mobile
App/Website for driving the consumer perception w.r.t. 5 Sports Brands which
are adidas, Puma, Nike, Reebok and Fila. The paper focusses on the four BAP
factors for measuring the consumer perception in which consumer reviews and
comments from various social media platforms, e-commerce websites, twitter
accounts were taken into consideration. The consumer comments once divided
into positive and negative comments depending on the meaning of the sentence
and using data processing steps. Finally, a comparative assessment was done
for the 4 BAP i.e. Brand Name, Product Quality, Service Quality and Mobile
App/Website and all five sports brands were compared with respect to each
other on every BAP. The data collected and processed was represented in tabu-
lar format which was further converted into percentage values for baseline and
was represented in form of graphs with the help of Tableau business intelligence
and analytic tool. The research study used the steps of Netnography and Text
Mining methodology to collect the data, process it and in the end represented
the processed data using BI & A Tableau tool. Overall the research study, did
the comparative assessment of consumer perception for selecting Sports Brands
using Netnography and Text Mining with the help of data mined from Online
Communities.

References
1. Smith, D.C., Park, C.W.: The effects of brand extensions on market share and
advertising efficiency. J. Mark. Res. 29, 296–313 (1992)
2. Ryan, M.J., Rayner, R., Morrison, A.: Diagnosing customer loyalty drivers. Mark.
Res. 11(2), 18–26 (1999)
3. Chaudhuri, A., Holbrook, M.B.: The chain of effects from brand trust and brand
affect to brand performance: the role of brand loyalty. J. Mark. 65(4), 81 (2001)
4. Arvidsson, A.: Brands: a critical perspective. J. Consum. Cult. 5(2), 235–259 (2005)
5. Arvidsson, A.: Brands: Meaning and Value in Media Culture, p. 168. Routledge,
London (2006)
6. Cengiz, E., Yayala, H.E.: The effect of marketing mix on positive word of mouth
communication: evidence from accounting offices in Turkey. Innov. Mark. 3(7), 486
(2007)
7. Chen, Y.T.: External factors influencing Taiwanese students shoe purchase inten-
tion (2008)
8. Ko, K.H.: A cross cultural study of antecedents of purchase intention for sports
shoes in Korea and China. J. Glob. Acad. Mark. Sci. 18(1), 157–177 (2008)
9. Youn, N., Song, I., MacLachlan, D.: A multi-category approach to modeling con-
sumer preference evolution: the case of sporting goods, June 2008
10. Hu, Y.J.: Service quality as mediator of the relationship between marketing mix
and customer loyalty. Int. J. Organ. Innov. 2, 282 (2009)
11. Kozinets, R.V.: Netnography: Doing Ethnographic Research Online. SAGE Pub-
lications Inc., London (2010)
 Comparative Assessment of Consumer Perception on Sports Brands 429

12. Saha, S., Dey, M., Bhattacharya, S.: Factors Affecting consumer buying behavior
of shoes in Kolkata: a case study. IUP J. Manag. Res. 9(4), 39–60 (2010)
13. Krishna: Assessing youth’s buying behaviour towards sports shoes (a case study
of Nike). Int. J. Res. Commer. Manag. 3(9) (2012)
14. Yoh, T., Mohr, M., Gordon, B.: The effect of gender on Korean Teens’ athletic
footwear purchasing. Sport J. 15 (2012)
15. Tableau. https://www.tableau.com. Accessed 9 Oct 2019
16. MySQL. https://www.mysql.com/. Accessed 15 Aug 2019
17. Twitter Developer APIs. https://developer.twitter.com/en/docs. Accessed 10 Oct
2019
18. Oracle JAVA 8. https://www.java.com/en/. Accessed 16 Sept 2019
 Analytical Modelling of Internet
of Vehicles (IoV) by IoT Cloud
for Dipping Traffic Congestion and Safety
Alert: A Review

Sanjib Debnath(B) and Ashim Saha

Computer Science and Engineering Department, National Institute of Technology

Agartala, Agartala, India
[email protected], [email protected]

Abstract. In this era of technology, the main causes of accidents among

motorized vehicles are the sudden slowdown of traffic in highways and
hilly areas are characterized by failure of signals, poor law enforcement,
and bad traffic management. As a result, it’s lead to more traffic conges-
tion which becomes a major issue for governments, scientists and Vehicle
manufacturers. Road safety through vehicle control is the primary inter-
est of the Intelligent Transport System (ITS) studies. Such problems
can typically be mitigated by installing on highways that associate with
defined traffic sensors the mobile app of drivers through latest commu-
nication network, but needless to say, however, not all streets are fit-
ted with such types of equipment in hilly regions. Unfortunately, such
accidents occur usually on the road, causing serious injury and death,
infrastructure damage. In this regard, vehicular communication is being
altered into a new thought named as Internet of Vehicles (IoV). There-
fore, Internet of Things (IoT) concept is used to reduce human loss, to
avoid or prevent traffic accidents at an extreme level and rather for road
safety for an IoV system.

Keywords: Internet of Vehicles · IoT Cloud · Intelligent

Transportation System

1 Introduction

The amount of vehicles and the population in developing nations is growing

day by day, leading to issues with road traffic, particularly in urban towns. The
vehicles spend more time in traffic jams in metropolitan city’s leads to health
issues and environmental hazards. According to the World Health Organization,
highway crashes are a significant government health issue as more than 1.25
million have died and 50 million have been wounded with 90% [1]. The most
prevalent factors leading to the crashes are based on the behaviors of road users,
such as people’s driving style or road cycling, as well as other variables such as
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 430–440, 2020.
https://doi.org/10.1007/978-981-15-3666-3_35
 Analytical Modelling of Internet of Vehicles (IoV) by IoT Cloud 431

driver stress, traffic breaking regulations, etc. Road safety and vehicle conges-
tion are the main difficulties facing the transport industry. Due to more traffic
congestion, the efficiency of transport scheme has been reduced and as a result
it increase fuel consumption and travelling time also.
So few methods are require to decrease vehicle traffic congestion is needed to
alleviate the negative environmental and economic impacts imposed by transport
inefficiency [2]. Most vehicle accidents are caused by human errors that can be
avoided and inappropriate driving methods. By using direct or indirect Vehicle
to Vehicle (V2V), Vehicle to Infrastructure (V2I) a distributed system sharing
sensor information from vehicles can decrease accidents. Vehicle-connected sen-
sor technology enables drivers to enhance their driving experience. Self-driving,
linked vehicles and autonomous vehicles are becoming increasingly practicable
with latest developments in sensing techniques. The utilization of area based cau-
tions, which give drivers traffic related data alarms by means of different versatile
and online stages, for example, mapping administrations are utilized by traffic
the board frameworks trying to diminish vehicular blockage [3,4]. These frame-
works are restricted by the compelling inclusion territory just as the expenses of
executing the framework utilizing over the top cell systems, especially in remote
regions. In this way, a devoted traffic location and the executives framework that
improves the use and throughput of street framework is required for improving
the traffic stream by giving the drivers with ongoing area based alarms identi-
fying with suggested elective courses as well as driving paces.
The abrupt traffic stoppage particularly in quick looking over streets and
roadways portrayed by a rare imperceptibility is one of the significant reasons for
mishaps among vehicles. It tends to be brought about by different mishaps, work-
in-advance on streets, tops of traffic, etc. Some IoT Cloud framework is gone for
traffic observing and ready warning for drivers. Portable sensors are spoken to
by GSM/GPRS/GPS TK103 tracker based frameworks introduced in vehicles
that gather geo-confinement and speed information that are sent to an OpenGTS
server that stores them in a SQL database, giving a continuous Open- StreetMap
representation of the traffic situation. Furthermore approaching unstructured
geolocation information are sent to a GeoJSON [6] parsing microservice so as to
be embedded in a MongoDB database.
The utilization of vehicle mounted sensors are actualized by current street
security frameworks for helping drivers by giving vulnerable side help, keeping
up safe after separations what’s more, path help however are unequipped for
recognizing non viewable pathway components of impacts. The Internet of vehi-
cles (IoV) is an Internet of Things (IoT) based arrangement focused at meeting
the street wellbeing and traffic the board objectives of Intelligent transportation
System (ITS) for vehicular customers which has picked up progress because of
the expanded dependability of remote correspondence and sensor technologies.
The IoV arrangement comprises of a WSN containing a focal server which pro-
cedures the got information continuously to create relevant measures, vehicular
customer hubs and Roadside Units (RSU) which capacity as waypoints for gath-
ering the required tactile information and spread of wellbeing and traffic related
cautions [5].
432 S. Debnath and A. Saha

2 Related Works
In this technology era, for developing any IoT infrastructure, its very needful to
know about all connected things with limited computation power and efficiency.
There are numerous methodologies towards a progressively compelling street use
and an increasingly wise method for expanding the limit of the street organize,
since structure new framework is never again a fitting choice. Few examples of
advanced traffic control frameworks are, e.g., intelligent speed limits, versatile
incline metering or dynamic steering. These models depend on an incorporated
traffic the executives, which controls the activity and the reaction as per a given
traffic circumstance.
Nowadays the transportation systems get influenced only for the development
of society, economy and innovation, and also for the family unit qualities. The
adjustment in vehicle framework causes an adjustment in vehicle conduct and
locational example of the framework. The adjustment in family unit attributes,
transport conduct, locational design, and other development impacts bring about
the development of traffic. Traffic congestions included the change or improve-
ment in street limit is just as the after effect of progress in the transportation
framework and subsequently at long last a circumstance emerges where the traffic
request is more prominent than the limit of the roadway. Few autonomous vehicle
are equipped with some controls, so that they can calculate their neighbor vehi-
cle distance using distance algorithm which may reduce few traffic congestion.
To prevent accident distance ranging principal are one of the most important
technique.
As IoT Cloud [6] are used to vehicular traffic monitoring and vigilant notifica-
tion continuously, so its generate a large amount of data. So data storing, data
processing and data recovery are the major issues as the exponential growth
of vehicular traffic data. During accident, its need to generate data immedi-
ately and send immediately that correspondent node where security authorities
resides. Moreover in hilly areas such traffic congestion are high due to narrow
zig zag road, landslides during rainy season, flood effect etc.
An IoT Cloud [6] techniques were used to processing of big vehicular traffic
data which generated using MongoDB and OpenGTS. Considering the constant
time delay for any vehicle to Cloud Server and that to Client, it shows data
retrieval is so fast. So it may reduce traffic congestion and give a quick alert
notification to correspondent node. The same way the system can alert to all
neighbor node so that they can aware of the incident and may reduce traffic
congestion. All the mobile sensor node are characterized by few tracker based
system like GPRS, GSM, TK 103 etc. Vehicular Traffic vehicular data are col-
lected through unstructured way and send it to GeoJSON for data parsing and
thereafter insert into MongoDB database storage. Due high speeds and limited
degrees of independence in vehicle node, sometimes its hard to locate their posi-
tion. The data communication between different nodes are performing by 4G
network. Therefore micro-services and OpenGTS server are employed through
Docker containers to take benefit of the virtualization of resources.
 Analytical Modelling of Internet of Vehicles (IoV) by IoT Cloud 433

Using internet of things (IoT) and vehicular Cloud Computing in Connected

vehicles are the main motivative issues on recent technology era. Cloud Com-
puting technique and few IoT technologies are described only to reducing traffic
congestion. Few factors like high fuel prices, gas secretion which lead to increase
traffic congestion are discussed [7]. Few Cloud servers are responsible for heavy
load of gateway registration, path discovery, path selection and about handoff
scenario which generated from client side [7]. [8] shows performance analysis of
vehicular networks which changing vehicular concentrations and required simu-
lation time. In order to increase road safety and prevent road accident, a feature
selection technique are discussed using big data analysis [9].
Wireless Communication and Wireless Sensor network has a foremost role for
reliability of Intelligent Transportation System, as increased vehicle are causing
more traffic congestion. So Internet of Vehicles (IoV) is designed for the solution
to meet road safety and vehicular traffic managing outcome. To develop contex-
tual events, IoV follows a master node (server) which processes the received data
in real-time. Road Side Unit (RSU) and client node are accountable for taking
all sensory data and all data related to safety alerts. Here few correspondence
channel is explored and an IoV based framework for vehicular road safety has
been studied [10]. An Internet of Vehicles (IoV) [10] frameworks comprises of a
remote sensor system containing vehicular customer hubs, stationary waypoints
and an incorporated server. Stationary waypoints work as roadside units (RSU)
which are utilized for accumulation of tactile information from the vehicular hubs
and scattering of alarms to customer drivers produced on the server. Appara-
tus to gather tangible information from vehicular customer hubs alongside the
correspondence framework is contained in the vehicle onboard unit (OBU) [11].
IoV based WSN use between vehicle correspondence channel which termed as
vehicle to vehicle (V2V) communication and correspondence among RSUs and
vehicle OBUs termed as vehicle to infrastructure (V2I) communication. Road
safety related admonitions and cautions important to peer drivers are traded
utilizing V2V correspondence and V2I correspondence is utilized for gathering
of tangible information and dispersal of area based relevant cautions to cus-
tomer drivers [12]. Few technologies like DSRC, WAVE, Wi-Fi and ZigBee are
encompasses for cellular network using wireless communication protocol [13].
To identify traffic congestion, IoV uses traffic status data using several tech-
nique like optimized signal control and adaptive speed limit. The procedures are
planned for improving the proficiency and use of the framework by improving
traffic stream and street inhabitance rates. Tangible information gathered from
vehicular hubs incorporate recurrence of braking occasions, normal increasing
vehicle speed and its location [14]. The traffic related data is notable by utiliz-
ing tactile information gathered from vehicular hubs which is handled by some
control unit by utilizing measurable methods, vehicle reference point recurrence
and bunching algorithms. Some sensors are utilized to recognize occasions which
can reduced road security. Distributed information are distributing to stay away
434 S. Debnath and A. Saha

from accident that can be brought about by careless driving, inappropriate driv-
ing practices and absence of keeping up of safe driving separation. Moreover area
based alarm may installed for road safety related track.

3 An IoT Cloud Model

Several reasons are there for vehicular accidents caused such as lack of preserva-
tion of safe driving distance, blind spots misconception of road signs due to lack
of consciousness and negligent driving due to exhaustion, drowsy etc. Another
few causes are due to failure of signals, poor law administration and bad traffic
management which also lead to more traffic congestion. In this regard so many
technologies are designed to traffic monitoring including UHF and VHF wireless
communication [6]. The protocol used for that are IEEE 802.11 for short dis-
tance communication and IEEE 802.16, GSM, 3G, 4G, 5G. Several sensors are
used to detect vehicular node detection and to maintain the data transmission
between client node to server node. Sound Detection devices used to accumulate
sounds which are coming from vehicle engine, tire and air turbulence etc. So
many fixed installed sensor devices are essential to collect data including traffic
related information. IoT Cloud system [6] are act as storing for all traffic related
data coming from all the sensor devices which are installed in any vehicle and
from all RSU. By the help of using fastest communication network so there is
no need to develop a private network for sensor data. It does not require fixed
installed sensor on road side, as vehicle node will have all sensory data.
The IoT Cloud model implementation [6] displays a new scope in Intelligent
transportation system in addition to bring service discovery and road safety.
The term IoT Cloud means a lot, which assimilates all the conveniences on data
storage, data processing, data retrieval and all type of networking capabilities.
As it provides all types of cloud services, hence it may termed as IoT as a Service
(IoTaaS) [15] in different level. The system design shows in Fig. 1 for implemen-
tation of cloud model in IoT where sensor devices are installed in all mobile
nodes. The node now act as a sensor node individually and can provide all data
related to its speed, movement, acceleration etc. Traffic related location based
data are collected real time and send it to database for storage. All unstructured
data are collected from different nodes are forwarded to parsing through GeoJ-
SON. MongoDB database are help in this regard to store that data for parsing
and processing. While the vehicle node change its location from one base station
to another, its neighbor node should know about its updated location. Therefore
another micro services is needful to query about their location and other ser-
vices. Due to that movement sudden traffic slowdown may happen which needs
to notify to all neighbor node. Figure 1 shows the system model where it depicted
that sensor are installed in vehicle so that it can measure its geo location with
latitude, longitude etc. In addition another one (TK 103) provides message ser-
vice to client nodes and as well as data transmission through GPRS [6]. The
method use for web based tracking of any vehicle is OpenGTS which follows
Apache software. RESTful method use for communication between all tracker
 Analytical Modelling of Internet of Vehicles (IoV) by IoT Cloud 435

devices for vehicles and OpenGTS. Such communication features enable vehicles
to send those information via a fastest connection to an OpenGTS server [6].
The server supports the visualization of all location based data through a col-
laborative road map. All unstructured dta now store through SQL database in a
structured manner. GeoJSON is require for alert notification. Assuming latency
of communication network is constant for transferring data from Vehicle node
to infrastructure and vice versa. Here the analytical model require data pars-
ing and insertion and query time processing for that. As number of subsequent
request increasing so more data needs to store and retrieve that same. Vehicle
node speed are not beyond control so sometime few request may loss for that
speed. But considering a maximum speed, if the alert message can send, it will
help to forward the message to drivers app and prevent accident and as a result
traffic congestion also reduced.

Fig. 1. System design for IoT Cloud system for traffic monitoring and alert
notification [6]

4 Internet of Vehicle (IoV) System

The transformation of vehicular ad hoc networks obsessed by Internet of Things
which termed as Internet of Vehicle (IoV) [10]. The system consists of sev-
eral vehicular client nodes along with centralized server, road side unit (RSU)
through wireless sensor network. To gain more real-time traffic information accu-
rately, protect travel convenience and enhance travel comfort are the main aim
436 S. Debnath and A. Saha

of IoV. It provides not only vehicle communication assuredly but as well as vehi-
cle intelligent also. IoV is an integrated network system with a high managerial
level, usability, operationalization and credibility. It consisting of several clients,
several vehicles, several nodes and several networks. The bellow figure shows
heterogenous network structure of Internet of Vehicle. Internet of Vehicles (IoV)
architecture shows seven different layer [17] named as user vehicle interface layer,
Data Acquisition Layer, Data Filtering and Preprocessing Layer, Communica-
tion Layer, Control and Management Layer, Processing Layer, Security Layer.
First one shows the interaction between vehicle node with other correspondent
node. Data acquisition layer are responsible for collecting data from various inter-
connected vehicle nodes. The same phenomenon seen in IoT Cloud system with
different way where Open GTS server are responsible for collecting all unstruc-
tured data. Third layer are accountable to avoid the transmission of unrelated
content and to reduce network traffic Congestion. Communication Layer pro-
vides the appropriate network to send the information by using congestion and
some other parameter (Fig. 2).

Fig. 2. A Heterogeneous network structure in IoV [16]

Traffic management, traffic engineering, and packet transmission are the few
policies which maintained by Control and management layer. Huge number of
data are generating on such scenario which need to maintained properly by
database software. Data processing are the major issues which done processing
layer. Query data processing also helps to send alert message and notification to
all correspondent node with an useful time. Data authentication, data integrity,
data nonrepudiation, confidentiality and several access control among different
nodes are the functions of security layer. All seven layers for IoV architecture
are shown in bellow Fig. 3.
 Analytical Modelling of Internet of Vehicles (IoV) by IoT Cloud 437

Fig. 3. Layer of IoV architecture [16, 17].

4.1 IoV Simulations Model

Several open source discrete simulation platform use for IoV simulation for eval-
uating performance [10]. Few wireless communication technologies like ZigBee,
WiFi, DSRC are studied here for short range communication. IoV simulations
are affecting through model measurement, time delay, packet size, traffic model,
different communication channel. For short range communications using ZigBee,
WiFi, DSRC technologies needs a proper model dimension. During simulation
the sender node moves at particular speed to the receiving node. Therefore a
deterministic movement approach used in a straight line for Vehicle movement.
Simulation time can be calculated at fixed speed by the node to cover the chan-
nel’s range of wireless communication technology. There is no propagation delay
as packet covers a total distance which is less than 1 km. The time required for
transmission delay is 100 ms. Transmission delay can be computed by the ratio of
packet length and data rate of the channel. Therefore to find maximum amount
of packet which can be switched for ZigBee, WiFi and DSRC can be calculated
by the following formula [10].

P acketSize(l)
T ransmissionDelay(T ) = (1)
DataRate(r)
438 S. Debnath and A. Saha

In an IoV system the timing of data packet exchange surveys the memo-
ryless exponential Poisson distribution. The time of generating packets follows
an exponential distribution. During Simulation the mean frequency range varies
from 1 Hz to 1 kHz. Another comparison can be made for long range wireless
communication for Vehicle to vehicle communication and vehicle to infrastruc-
ture communication [18,19]. The ranges which support such communication is
of at least 1 km (Table 1).

Table 1. Comparison of wireless communication technologies [20]

Technologies Carrier fre- Series (R) Association Through Power con- Maximum Topology
quency (f) time (At ) put (Th ) sumption number of
(P) nodes (n)
WiFi 2.4/5 GHz 100 m–1 Km 4s 54 Mb/s >400 mA No limit Star and
Mesh
ZigBee 2.4 GHz 100 m 30 ms 250 Kb/s 30 mA >65000 Mesh
Bluetooth 2.4 GHz 10 m 600 ms 1 Mb/s 40 mA 8 Star and
Mesh

4.2 Role of Sender Node in IoV System

As vehicle speed is not controlled by any other node, so packets are sent from
moving nodes. Considering Packet size is constant for wireless communication
technologies WiFi, Zigbee and DSRC [10]. Packets are coming from moving
vehicle so few packet may loss during transmission. The highest packet loss
by DSRC technologies [10]. Due to lower bandwidth leads to lower congestion,
ZigBee can exchanged low frequency of message. The reason behind packet loss
not only traffic congestion, it might be due to gesture or sometimes for Doppler
Effect. When both the wave’s source and the wave’s receiver are in movement,
then only effect found. As a result it produce alteration of the received frequency.
The received frequency can be calculated using carrier frequency, differences
in speed and with the speed of light. The equation for variations in received
frequency for Doppler effect are as follows [10]

Δv
F requency = f c(1 ± ) (2)
c
Most of the packets sent are still effectively obtained as the frequency loss for
speeds in a particular value. Whereas the value is lower than the channel band-
width for WiFi, Zigbee and DSRC wireless communication techniques. Higher
data rates confirm that transmission delay with Wi-Fi technologies is lower than
DSRC technology. Thus congestion is minimized for increasing throughput of
the link.
 Analytical Modelling of Internet of Vehicles (IoV) by IoT Cloud 439

5 Conclusion and Future Work

In this paper it has been reviewed on some relevant technologies which integrates
IoV for the development of Intelligent Transport System. IoV offers countless
advantages including vibrant data services, smart car control and insurance rate
reduction and enhanced productivity owing to decreased traffic congestion. IoV
will encourage the integration of automotive and information technology. Such
system will contribute to growth of energy efficiency, connected devices, safety
and security applications in vehicular communication. Also help to reduce the
lack of coordination and communication between vehicles and improve products,
services and experiences through wireless communication technology. More secu-
rity feature is needed as connected wireless devices are increasing. Data process-
ing and performance may improve by machine learning algorithm. IoV system
with IoT Cloud model will increase efficiency in traffic monitoring and road
safety alert notification. As IoT Cloud model consider, so security analysis also
needs to be studied for increase efficiency. In order to take more advantage of
a ascendable Cloud-based platform both OpenGTS server and several micro-
services were organized to take more efficient on IoV.

References
1. Singh, S.K.: Road traffic accidents in India: issues and challenges. Transp. Res.
Procedia 25, 4708–4719 (2017). Science direct
2. Darwish, T., Abu Bakar, K.: Traffic density estimation in vehicular ad hoc net-
works: a review. Ad Hoc Netw. 4, 337–351 (2016). https://doi.org/10.10007/
1234567890
3. Zhu, Z., Loo, J., Chen, Y., Chai, K.K., Zhang, T.: Recent advances in connected
vehicles via information-centric networking. In: IET International Conference on
Intelligent and Connected Vehicles (ICV 2016), Chongqing, pp. 1–8 (2016)
4. Hirose, K., Ishibashi, K., Yamao, Y., Hirayama, Y., Sawada, M.: Low-power V2M
communication system with fast network association capability. In: 2015 IEEE 2nd
World Forum on Internet of Things (WF-IoT), Milan, pp. 204–209 (2015)
5. Djajadi, A., Putra, R.J.: Inter-cars safety communication system based on Android
smartphone. In: 2014 IEEE Conference on Open Systems (ICOS), Subang, pp. 12–
17 (2014)
6. Celesti, A., Galletta, A., Carnevale, L.: An IoT cloud system for traffic monitoring
and vehicular accidents prevention based on mobile sensor data processing. IEEE
Sens. J. 18(12), 4795–4802 (2018)
7. Guerrero-ibanez, J.A., Zeadally, S., Contreras-Castillo, J.: Integration challenges
of intelligent transportation systems with connected vehicle, cloud computing, and
internet of things technologies. IEEE Wirel. Commun. 22, 122–128 (2015)
8. Lin, F., Huang, Q., Tan, Q.: Distributed storage video cloud of intelligent trans-
portation system based on the backward cloud generator. Adv. Transp. Stud. 2,
15–22 (2015)
9. Abhale, A., Khandelwal, S.: Intelligence transportation service using vehicular
cloud network. In: 2016 IEEE International Conference on Advances in Electronics,
Communication and Computer Technology, ICAECCT 2016, pp. 277–282 (2017)
440 S. Debnath and A. Saha

10. Thakur, A., Malekian, R.: Internet of vehicles communication technologies for
traffic management and road safety applications. Wirel. Pers. Commun. 109(1),
31–49 (2019). https://doi.org/10.1007/s11277-019-06548-y. LLC, part of Springer
Nature, Springer Science Business Media
11. Barrachina, J., Sanguesa, J., Fogue, M., Garrido, P., Martinez, F., Cano, J.: V2Xd:
a vehicular density estimation system that combines V2V and V2I communications.
In: 2013 IFIP Wireless Days (WD) (2013)
12. Eze, E.C., Zhang, S.J., Liu, E.J., Eze, J.C.: Advances in vehicular ad-hoc net-
works (VANETs): challenges and road-map for future development. Int. J. Autom.
Comput. 13(1), 1–18 (2016)
13. Thakur, A., Malekian, R., Bogatinoska, D.C.: Internet of Things based solutions
for road safety and traffic management in intelligent transportation systems. In:
Trajanov, D., Bakeva, V. (eds.) ICT Innovations 2017. CCIS, vol. 778, pp. 47–56.
Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67597-8 5
14. Du, Z., HomChaudhuri, B., Pisu, P.: Coordination strategy for vehicles passing
multiple signalized intersections: a connected vehicle penetration rate study. In:
2017 American control conference (ACC) (2017)
15. Celesti, A., Mulfari, D., Fazio, M., Villari, M., Puliafito, A.: Exploring container
virtualization in iot clouds. In: 2016 IEEE International Conference on Smart Com-
puting (SMARTCOMP), pp. 1–6, May 2016
16. Cheng, J.J., Cheng, J.L., Zhou, M.C., Liu, F.Q., Gao, S.C., Liu, C.: Routing in
internet of vehicles: a review. IEEE Trans. Intell. Transp. Syst. 16(5), 2339–2352
(2015)
17. Golestan, K., Soua, R., Karray, F., Kamel, M.: Situation awareness within the
context of connected cars: a comprehensive review and recent trends. Inf. Fusion
29, 6–83 (2016)
18. Hirose, K., Ishibashi, K., Yamao, Y., Hirayama, Y., Sawada, M.: Low-power V2M
communication system with fast network association capability. In: 2015 IEEE 2nd
World Forum on Internet of Things (WF-IoT), Milan, pp. 204–209 (2015)
19. Hannan, M., Habib, S., Javadi, M., Samad, S., Muad, A., Hussain, A.: Inter-vehicle
wireless communications technologies, issues and challenges. Inf. Technol. J. 12(4),
558–568 (2013)
20. Thakur, A., Malekian, R., Bogatinoska, D.C.: Internet of things based solutions
for road safety and traffic management in intelligent transportation systems. In:
Trajanov, D., Bakeva, V. (eds.) ICT Innovations 2017. CCIS, vol. 778, pp. 47–56.
Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67597-8 5
 Object Detection Using Point Feature
Matching Based on SURF Algorithm

Monika Dey(B) , Ashim Saha, and Anurag De

National Institute of Technology, Agartala, India

[email protected], [email protected], [email protected]

Abstract. Object detection addresses detecting instances of objects of a

particular category in digitized images and videos. Every class containing
the object is associated with special features that assist in classifying the
class to which it belongs. The algorithm used can detect local features
and extract them for the concerned object, known as descriptor points [1].
Then compares those extracted features or descriptors with the presumed
features of the original image. Matching process among the original image
and the target image yields similar features, based on which a decision
is made. The mentioned algorithm is called Speeded up Robust Features
(SURF) algorithm. The algorithm is based on Scale-invariant feature
transform (SIFT) algorithm. The implementation procedure of SURF
algorithm along with experiment and its results are stated in the paper.
Also, the accuracy of detection of objects using point feature matching
methodology has been calculated by means of sensitivity and specificity
parameters.

Keywords: Object detection · Descriptor points · Point feature

matching · SURF algorithm

1 Introduction

Object detection plays a crucial part in image processing field. For implementa-
tion of object detection, an image is acquired called test image. Some descriptor
points are detected on the test image and a comparison is carried out with the ref-
erence image. Point feature detection is used to detect a specified target i.e. one
specific object instead of that kind of objects in a cluttered scene. The algorithm
in simple words is based on comparing and analyzing feature correspondences
between the reference target image and the given cluttered image.
The suggested approach is associated with trending and competent matching
methods. Specifically, to objects, which consider only images and their gradients
to detect objects. This algorithm gives more accuracy for objects that show
texture patterns that are non-repeating, which results in generation of matches
for unique features. Whereas for homogeneously colored objects and for objects

Supported by National Institute of Technology, Agartala.

c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 441–452, 2020.
https://doi.org/10.1007/978-981-15-3666-3_36
442 M. Dey et al.

bearing repeating patterns, this method is not likely to work [2]. The Speeded Up
Robust Features (SURF) algorithm is used for classifying objects of a specific cat-
egory. Speeded Up Robust Features (SURF) algorithm can be adopted for many
applications such as object recognition since it is a local feature and descriptor
algorithm. “SURF” considers quite abundant number of feature descriptors from
original image. This, in turn, results in a reduction of the errors caused by local
variation in the average of all feature matching. As SURF has feature descrip-
tor which is consistent to scale, partially consistent in illumination changes and
orientation, so it can identify and recognize objects robustly even in case of
clutter and partial occlusion. Speeded UP Robust Features “SURF” algorithm
incorporates the following three steps:
Detection Step: This is the first step in which distinguishing locations such as
blobs, corners, and other interest points are selected in the original image.
Description Step: It is the second step in which interest points selected in the
previous step should be allocated unique identifiers. These identifiers should be
in-dependent of features scale and rotations [3]. These identifiers thus achieved
are called descriptors. The descriptors are vectors that contain information about
the points itself and the surroundings.
Matching Step: The third step in SURF algorithm, where there is a comparison
of descriptor vectors between the object image and the new input or original
image. The matching score is calculated concerning the distance between vectors
e.g. Euclidian distance and the sign of Laplacian.
Object detection systems are extensively used in the fields of video surveil-
lance, and image retrieval. It has been observed that real-time object detection
and learning are demanding and high priority tasks. Domains of object detection
that have been quite exposed, comprises of face detection and pedestrian detec-
tion. Object detection has relevance in several progressive fields of computer
vision, such as robotics and industrial automation.

2 Literature Review
The paper by the authors Ms. Vimal Sudhakar Bodke, Prof. Omkar S Vaidya
mainly focus on repeatability of same features in two different images. The algo-
rithm proposed is SURF algorithm as feature detection [2].
The proposed paper by Faishal Badsha, Rafiqul Islam, Mohammad Farhad
Bulbul [5] has stated the comparison between SIFT algorithm and SURF algo-
rithm. While for the later, Hessian matrix is used for feature detection and Haar
wavelet is explained for assignment of descriptors. An architecture of the working
system has also been diagrammatically stated by the paper.
Authors Vishweshwar Todkari, Akash Pawar, Yogeshwar Shanbhag, Gan-
garam Rajput, T. S. Bhoye have done a survey on various object detection tech-
niques [7]. The algorithm discussed is SIFT (Scale Invariant Feature Transform)
algorithm which is the basis for SURF algorithm, for evaluating color descriptors
using photometric changes.
 Object Detection Using Point Feature Matching 443

Other methods such as dual background elimination has also been discussed.
In another paper proposed by Rasool Reddy, Krishna, Ravi Kumar [8] SURF
has been described to be robust for rotations up to certain degree.
Hemalatha and Santhosh Kumar [9] in the proposed paper, have explored
the usage of SURF algorithm in the field of security. The paper is focused on
copy-move forgery based on application of image segmentation. By comparing
the key features in an image using SURF algorithm the forgery is detected.

Comparison of Other Object Detection Algorithms with SURF Algo-

rithm: It was stated experimentally that SURF algorithm works better when
noise is considered by Saleem et al. and supported by graph [10]. In a paper
by Karami et al. a comparative study gave results as SURF algorithm to be
faster than SIFT algorithm and oriented FAST, rotated BRIEF (ORB) algorithm
when various parameters such as intensity, scaling, rotation, etc. were consid-
ered. These findings are supported by tabular representation of the experiment
performed [11]. Further in paper by Athani et al., parameters like repeatabil-
ity and time was calculated and compared. As, per the implementation, SURF
algorithm worked well considering both parameters together [12].

3 SURF Algorithm

The SURF Algorithm stands for Speeded up Robust Features. It has been devel-
oped by Bay et al. The SURF algorithm uses a geometric transform to distinguish
the locale of the object which is to be detected in the given image. As mentioned
above, it consists of 3 basic steps:

• Extraction,
• Description, and
• Matching

SURF algorithm uses integral images for extracting the features of an

image [3,4] i.e. using a summed-area table for the generation of the sum of
values quickly and efficiently in a rectangular subset of a grid, and box filters
i.e. an average of the surrounding pixel. A box filter is a convolution filter, often
used mathematical operation for multiplying two arrays to produce a third one.
Initially, as a part of preprocessing the original image is converted into inte-
gral image and is converted to a grey scale image. The importance of this step is
that by using the right pixel values, integral images can be smoothly computed
in near about constant time. In an integral image, pixel and the origin form a
rectangular window with the origin as the extreme top-left pixel. Addition of all
pixels located in the formed sub-grid is the represented by each pixel. Box filters
of second-order Gaussian partial derivatives is then used for extraction of filter
masks. The algorithm used for feature detection is based on Hessian matrix. It
is a second-order partial derivatives square matrix of a scalar-valued function.
Mathematically, the Hessian matrix describes the local curvature of a function
444 M. Dey et al.

of many variables. In an image, I, X = (x, y) is the given point, the Hessian

matrix H (x, σ) in X at scale σ, can be defined as
 
Lxx (x, σ) Lxy (x, σ)
H(x, σ) = (1)
Lyx (x, σ) Lyy (x, σ)

Where, Lxx (x, σ) is the convolution result of the second order derivative of
Gaussian filter (δ 2 g(σ))/(δx2 ) with the image I in point X and similarly for
Lxy (x, σ) and Lyy (x, σ).
Equation 1 Shows the Gaussian second orders partial derivatives in y-
direction and xy-direction [2,5] and can be given as under:

Lxx (x, σ) = I(x) ∗ (δ 2 g(σ))/(δx2 ) (2)

Lxy (x, σ) = I(x) ∗ (δ 2 g(σ))/(δxy ) (3)

The circular region that lies around the interest point provides information.
Based on this set of information a reproducible orientation is fixed. This is the
foundation of SURF descriptor. The SURF descriptor is obtained by constructing
a square region aligned to the chosen position. A unique and efficient feature
description can be achieved by extraction and definition of descriptors. The
SURF descriptor is formed on Haar wavelet responses in order to be invariant
to rotation in x and y direction [3,5]. Haar wavelet’s mother wavelet Ψ (t) can
be mathematically stated as:
⎧
⎪
⎨1, if 0  t < 1/2
Ψ (t) = −1, if 1/2  t < 1 (4)
⎪
⎩
0, otherwise

Its scaling function ψ(t) can be explained as:


1, if 0  t < 1
ψ(t) = (5)
0, otherwise

It can further be diagrammatically shown as below (Fig. 1):

Fig. 1. Haar wavelet types used for SURF [2, 5]

Generally, hundreds of interest points are diagnosed in a digital image of

one megapixel. For each of these interest points descriptors are generated. The
matching is then carried out among the descriptors of the primitive image and
the obtained integral image.
 Object Detection Using Point Feature Matching 445

4 Flow of SURF Algorithm for Point Feature Matching

As shown in the flow diagram below (see Fig. 2), there are several steps to imple-
ment point feature matching using SURF algorithm. The said algorithm has been
implemented and results are analysed based on it. The implementation has been
shown.

Fig. 2. Flow diagram for point feature matching using SURF algorithm
446 M. Dey et al.

5 Experimentation and Results of Implementation

of Point Feature Matching

5.1 Experiment

Initially, the algorithm reads the reference image that consists of the object of
interest (see Fig. 3). The next step is reading the target image containing more
than one object or the scene in which the object is to be detected (see Fig. 4).
Then, the feature points in both the images are detected and the strongest fea-
ture points are visualized in the image considered for reference (see Fig. 5) as well
as target image (see Fig. 6). Extraction of feature descriptors and their match-
ing is done using the attained descriptors at the interest points in both images.
Resulting in displaying of putatively matched features (see Fig. 7). Subsequently
displaying localized Object in the scene, done using presumed matches and elim-
inating outliers (see Fig. 8). Next figure (see Fig. 9) displays the detected object
by getting the bounding polygon of the reference image.
Next is the detection of other image, which can be done using the previously
mentioned steps (see Figs. 10 and 11). Finally, displaying the detected object by
point feature matching algorithm (see Fig. 12) [6,8].

Fig. 3. Reading the reference image that contains the object of interest

Fig. 4. Reading the target image containing more than one object
 Object Detection Using Point Feature Matching 447

Fig. 5. Visualization of feature points of reference image and extraction of descriptors

Fig. 6. Visualization of strongest feature points of target image and extraction of

descriptors

Fig. 7. Matching the features using descriptors achieved previously

Fig. 8. Displaying presumed matched features without outliners

448 M. Dey et al.

Fig. 9. Displaying the detected objects

Fig. 10. (a) Reading an image consisting of a second object of interest (b) Detecting
and visualizing feature points

Fig. 11. Displaying presumed matched features

Fig. 12. Displaying the detected object in a polygon

5.2 Result and Analysis of Implementation

Through the implementation of point feature matching, its observed that one of
the objects has been successfully identified using feature extraction and descrip-
tor assignment to the objects. A polygon is defined around it to depict the object
 Object Detection Using Point Feature Matching 449

recognized (see Fig. 12). It can be observed that this algorithm is robust to small
amount of out-of-plane rotation as in the above figures. The constraints that are
to be applied to this algorithm is illumination, orientation and presence of other
similar objects. This also doesn’t work for uniformly colored objects (see Fig. 7)
where, two objects were detected to have similar features [2]. Hence, the accuracy
of this system can be enhanced in the matching section of the algorithm.

6 Measurement of Accuracy
6.1 Table Depicting the Objects and Their Detection Using SURF
Algorithm
See Fig. 13.

Fig. 13. Table for measurement of accuracy.

450 M. Dey et al.

Fig. 13. (continued)

 Object Detection Using Point Feature Matching 451

6.2 Parameters Derived from Above Table:

– Sensitivity:
Sensitivity = TP/(TP + FN) = (Number of true positive assessments)/
(Number of all positive assessments) [13]
Here, Sensitivity = 7/8 = 0.875
– Specificity:
Specificity = TN/(TN + FP) = (Number of true negative assessments)/
(Number of all negative assessments) [13]
Here, Specificity = 1/2 = 0.5
– Accuracy:
Accuracy = (TN + TP)/(TN + TP + FN + FP) = (Number of correct
assessments)/Number of all assessments) [13]
Here, Specificity = 8/10 = 0.8

Thus, by the above assessment, the SURF algorithm gives an accuracy of 80%
when implemented on a set of input images. These images were captured from
different angles and distances.

7 Conclusion
The SURF algorithm is built on comparing and analyzing point correspondences
between the reference image and the target image. Use of integral images helps in
mathematically determining summations over image sub grids rapidly. Hessian
matrix used for feature detection is known for accuracy and performance. To
implement method of smoothing i.e. reducing the amount of intensity variation
between one pixel and the next, of images box filters are used. The proposed
system can be implemented with certain constraints in it. Where, the accuracy
of matching can be increased, although the system results in good overall per-
formance when the conditions of illumination, coloring, texture are looked after.
SURF Algorithm is somewhat influenced by the scale-invariant feature trans-
form (SIFT) algorithm. The difference being that SURF is most of the instances
has proved to be faster than SIFT and is said to be less fragile against diverse
image transformations than SIFT [5].

References
1. Yang, Z.L., Guo, B.L.: Image mosaic based on SIFT. In: International Conference
on Intelligent Information Hiding and Multimedia Signal Processing, pp. 1422–1425
(2008)
2. Bodke, V.S., Vaidya, O.S.: Object recognition in a cluttered scene using point
feature matching. IJRASET 5(IX), 286–290 (2017)
3. Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In:
Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–
417. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023 32
452 M. Dey et al.

4. S. U. R. F. (SURF): Herbet bay, andreas ess, tinne tuytelaars, luc van gool. Elsevier
pre-print (2008)
5. Viola, P.A., Jones, M.J.: Rapid object detection using a boosted cascade of simple
features. In: IEEE CVPR (2001)
6. Todkari, V., Pawar, A., Shanbhag, Y., Rajput, G., Bhoye, T.S.: A survey on sus-
picious object detection. IJARIIE 4(1), 628–631 (2018). ISSN(O)-2395-4396
7. Lowe, D.: Distinctive image features from scale-invariant key point. Int. J. Comput.
Vis. 60, 1–28 (2004)
8. Rasool Reddy, K., Krishna, K.V.S., Ravi Kumar, V.: Detection of objects in clut-
tered scenes using matching technique. IJECT 5(spl – 3), 42–44 (2014)
9. Hemalatha, S., Santhosh Kumar, S.: Image forgery detection using key-point
extraction and segmentation. IJPT 8, 13219–13229 (2016)
10. Saleem, S., Bais, A., Sablatnig, R.: A performance evaluation of SIFT and SURF
for multispectral image matching. In: Campilho, A., Kamel, M. (eds.) ICIAR 2012.
LNCS, vol. 7324, pp. 166–173. Springer, Heidelberg (2012). https://doi.org/10.
1007/978-3-642-31295-3 20
11. Karami, E., Prasad, S., Shehata, M.: Image matching using SIFT, SURF, BRIEF
and ORB: performance comparison for distorted images. In: Proceedings of the
2015 Newfoundland Electrical and Computer Engineering Conference (2015)
12. Athani, S., Tejeshwar, C.H.: Performance analysis of key frame extraction using
SIFT and SURF algorithms. IJCSIT Int. J. Comput. Sci. Inf. Technol 7(4), 2136–
2139 (2016)
13. Zhu, W., Zeng, N., Wang, N.: Sensitivity, Specificity, Accuracy, Associated Con-
fidence Interval and ROC Analysis with Practical SAS. R Health Care and Life
Sciences, NESUG (2010)
 Diabetes Mellitus Prediction Using
Ensemble Machine Learning Techniques

Md. Tanvir Islam1 , M. Raihan1(B) , Sheikh Raihan Islam Akash2 ,

Fahmida Farzana1 , and Nasrin Aktar1
1
North Western University, Khulna, Bangladesh
[email protected], {tanvirislamnwu,mraihan}@ieee.org,
[email protected], [email protected], [email protected],
[email protected]
2
Jashore University of Science and Technology, Khulna, Bangladesh
[email protected]

Abstract. Diabetes is a non-communicable disease and currently it is

increasing at an alarming rate. It may cause different serious damage
in particular; blur vision, myopia, burning extremities, kidney and heart
failure. At this moment it is becoming one of the major diseases. Diabetes
occurs when the level of sugar crosses a certain level or the human body
can not produce sufficient insulin to balance the level. Therefore, dia-
betes affected patients need to be informed about it so that they can get
proper treatments to control diabetes. For this reason, it is important
to predict and classify diabetes at an early stage. So, in this analysis,
two Machine Learning algorithms have been used to classify diabetes
and compared the performances of the algorithms. The collected dataset
has 340 instances and each instance has 26 features. In this study, two
Ensemble Machine Learning algorithms have been used, namely Bagging
and Decorate. Bagging classified the types of diabetes 95.59% accurately,
whereas Decorate classified 98.53% accurately.

Keywords: Machine Learning · Bagging · Decorate · Diabetes

Mellitus · Classification · Ensemble learning · Algorithms · Prediction

1 Introduction

At Present, probably the most common, complex, chronic illness named as dia-
betes. It is such a syndrome that affects the capability of a human body to pro-
duce or use insulin. Diabetes Mellitus (DM) is a clinical syndrome characterized
by persistent hyperglycemia due to an absolute deficiency of insulin [1]. There
are many types of DM but the most common types are Type-1 Diabetes (T1D)
and Type-2 Diabetes (T2D). T1D happens due to idiopathic, whereas T2D hap-
pens when the human body can not produce adequate insulin, or when the body
cells are not able to use insulin accurately, which is called insulin resistance [2].
Diabetes immune system attacks and destroys the beta cells in the pancreas of
c Springer Nature Singapore Pte Ltd. 2020
A. Saha et al. (Eds.): ICCISIoT 2019, CCIS 1192, pp. 453–467, 2020.
https://doi.org/10.1007/978-981-15-3666-3_37
454 Md. T. Islam et al.

the human body which produce insulin. Diabetes represents a critical budgetary
weight to people and society [3]. According to the statistics of International Dia-
betes Federation (IDF), in every seven seconds, someone is estimated to collapse
from diabetes or its entanglements, with half of those deaths (per year in total
4 million) are occurring under the age of 60. This is totally unusual and never
witnessed before in the background of a worldwide diabetes prevalence of 8.8%
(95% confidence interval 7.2% to 11.3%) of the world population in 2017 [4]. In
Bangladesh, the percentage of diabetes affected patients has increased substan-
tially to 10% or 8.4 million population of 14.8 million total population in 2011
while it was only 4% in 1995 to 2000, 5% in 2001 to 2005 and 6% in 2006 to 2010
[5]. It indicates that the current percentage of diabetic patients in Bangladesh is
the highest among all the neighboring countries. A report of Daily Science stated
that, if it was possible to prevent diabetes in an earlier stage, the possibility of
minimizing the destructive effects of diabetes would be more. There are several
Machine Learning (ML) techniques such as classifications, clustering and associ-
ation which are developed and broadly used in the medicinal forecast. Different
models of algorithms provide different accuracy, specificity, sensitivity. So, it’s
easy to find out which algorithm is the best among the existing algorithms.
We have analyzed to find out the accuracy of two ML algorithms, Bagging and
Decorate by analyzing our collected dataset and compared their performances.
The other parts of the manuscript are arranged as follows: in Sect. 2, Sect. 3
the related works and methodology have been elaborated with a distinguish-
ing destination to the justness of the classier algorithms respectively. In Sect. 4
the experimented results of this analysis has been clarified with the impulsion
to justify the novelty of this exploration work. Finally, this research paper is
terminated with Sect. 5.

2 Related Works
An analysis was conducted by a research team on big data of health-care to pre-
dict diabetes accurately. The dataset they used includes total 9 features which
have numerical and nominal attributes. They targeted to make a classifier sys-
tem by using several ML techniques. In that analysis, they found 79.13% accu-
racy using Support Vector Machine (SVM) [6]. Similarly, a group of researchers
applied ML techniques to classify diabetes. They applied a few ML algorithms, in
particular; Naı̈ve Bayes, SVM, Random Forest (RF) and Simple CART. Among
these algorithms, they found 76.5% accuracy using RF in terms of classification
of diabetes [7]. Dutta et al. have worked on critical features to predict diabetes.
They worked with 9 variables and RF algorithm where they found RF to give
84% accuracy [8]. Sreekanth Rallapalli et al. have proposed a predictive model
using CART model & Scalable RF to classify diabetes based on various fac-
tors. They used a dataset contains 1500 instances with having 6 attributes. In
that research, they found that the Scalable RF algorithm gave 87.5% accuracy
whereas the normal RF algorithm gave 75% accuracy [9]. Soumayadeep Manna
and his cooperators have proposed a system to predict important factors that
 Diabetes Mellitus Prediction Using Ensemble Machine Learning Techniques 455

cause diabetes. They used a dataset which has 3075 instances and each instance
has 8 features. They have used Logistic Regression (LR) and RF, whereas RF
gave 86.70% accuracy and LR gave 89.17% accuracy [10]. In the same man-
ner, research has been conducted based on ML algorithms where researchers
used SVM, AdaBoost, Bagging, K-NN and RF algorithms with a dataset of 506
instances with having 30 features for each instance. They got 75.49% accuracy
for AdaBoost, 76.28% for Bagging, 72.33% for K-NN, 75.30% for RF, 72.72% for
SVM [11]. Likewise, another research team proposed a predictive model using RF
algorithm based on some variables like age, weight, hip, waist, height, etc. In the
study, they performed the analysis based on 4 groups of datasets where RF gave
84.19% for the group 4 dataset [12]. Kinge et al. analyzed to determine the per-
formances of several algorithms named DT(J48), Naive Bayes, RF, AdaBoost,
Bagging, Multilayer Perceptron (MLP), Simple Logistic to predict diseases using
data mining and ML techniques. A dataset of heart disease that has total 303
instances with 74 raw attributes was taken and only one 14 significant features
were used among them. Accuracy for DT(J48), Naı̈ve Bayes, RF, AdaBoost,
Bagging, Multilayer Perceptron (MLP), Simple Logistic algorithms are 78.15%,
82.59%, 83.15%, 81.59%, 81.59%, 79.41% and 83.1% respectively [13]. Deepika
Verma and Nidhi Mishra conducted a study to identify Diabetes by using a
dataset on Naı̈ve Bayes, J48, Sequential Minimal Optimization (SMO), MLP,
and decreases Error Pruning Tree (REP-tree) algorithms. In their study, they
found the best accuracy, 76.80% from SMO on diabetes dataset [14].

3 Methodology

The total workflow of our study showed in Fig. 1. We have categorized our study
into four different sections. They are,

– Collection of data
– Preprocessing of data
– Training of data
– Application of Machine Learning Algorithms

3.1 Collection of Data

For the analysis, we have visited Khulna Diabetes Center in Khulna, Bangladesh
to collect data of diabetes patients’. This dataset has total 340 instances includ-
ing 26 unique features for each instance. Our dataset contains the fundamental
information of patients and several symptoms which can be divided into two
parts namely, Typical symptoms and Non-Typical symptoms. Table 1 shows the
name and type of symptoms. Besides, the list of features and data distribution
have been shown in Tables 2 and 3, respectively.
456 Md. T. Islam et al.

Start

Import dataset with 340

instances and 26 features

Data Preprocessing

Replace missing data with

Mean, Median and Mode

Dataset training

Apply
Classification algorithms

Bagging Decorate

Determine
Statistical Matrics

Compare Performance

End

Fig. 1. Work-flow of our study

3.2 Preprocessing of Data

In the beginning, the dataset had several missing information that was one of the
main drawbacks. To handle the issue, two well-liked and convenient functions
have been used in WEKA 3.8 (Waikato Environment for Knowledge).
At first, a function ReplaceMissingValue has been used to replace the missing
data from the dataset. This function mixes every missing information for each
nominal and numeric attribute with the modes and means [15]. Another function
has been used named Randomize which fills up the missing field not doing much
hamper the overall performance [16].

3.3 Training of Data

To train the model, we have applied 10-Fold Cross-Validation technique. It
is a resampling technique which can evaluate divining models by separating the
main sample into a new set which is called training set. Later, it trains the model
and set a test to assess it [16]. In this technique, K-1 fold is used to train the
dataset and it uses the rest to test the model. Here, K = 10 and it breaks the
dataset into 10 different groups and each fold’s instances change randomly [17].
 Diabetes Mellitus Prediction Using Ensemble Machine Learning Techniques 457

Table 1. Symptom names and types

Symptom types Symptom names

Typical Thirst
Hunger
Weight loss
Sexual weakness
Non-typical Headache for high blood pressure
Burning extremities
Physical weakness

3.4 Applications of Machine Learning Algorithms

After getting the preprocessed and trained dataset, two popular Machine Learn-
ing algorithms, Bagging and Decorate have been applied.

Bagging (BAG): Bagging or Bootstrap Aggregation is a simple and power-

ful ensemble technique. Bagging is an application of Bootstrap technique to a
high-variance ML algorithm which works based on decision tree [18]. It runs as
a method for broadening accuracy. It is connected to training plans for a spe-
cific numeric forecast and resamples the preparation data to build new models
[19]. Besides, it builds a troupe of arrangement models for training plan where
every model provides a similar weighted forecast [20]. Moreover, it modifies the
stability of a system by updating the accuracy and reducing variance, therefore
again decreasing the problem of overfitting [21]. If there is P almost independent
dataset of size S that can be denoted by,
 1 1     
z1 , z2 , ..., zS1 , z12 , z22 , ..., zS2 , ..., z1L , z2L , ..., zSL (1)

zs1 ≡ sth observation of the lth bootstrap sample

We can fit only P almost free weak learners,

w1 (.), w2 (.), ..., wP (.) (2)

After that, combined these into two balancing process to get an ensemble
system,
1 
P
sP (.) = wl (.) (3)
P
l=1

(simple average, for regression problem)

This is for simple majority vote and classification problem,

sP (.) = argk max [card(l | wl (.) = k)] (4)

458 Md. T. Islam et al.

Table 2. Features list (Part-1)

Attributes name Sub category Data distribution

Sex Male 58.529%
Female 41.471%
Age Minimum: 22 yrs M ean ± StandardDeviation
Maximum: 75 yrs 48.944 ± 16.047
Profession Retired 13.529%
Farmer 8.529%
Housewife 20.588%
Business 17.059%
Engineer 9.706%
Driver 0.883%
Police 23.235%
Day labor 2.353%
Clerk 3.235%
Executive officer 0.882%
Height Minimum: 140 cm 159.318 ± 7.005
Maximum: 176 cm
Weight Minimum: 43 kg 63.162 ± 8.972
Maximum: 88 kg
BMI Minimum: 20.8 kg/m−2 24.845 ± 2.763
Maximum: 32.7 kg/m−2
Heart rate Minimum: 60 bpm 75.686 ± 4.697
Maximum: 80 bpm
Systolic BP Minimum: 100 mmHg 123.323 ± 10.8
Maximum: 150 mmHg
Diastolic BP Minimum: 66 mmHg 80.528 ± 3.679
Maximum: 100 mmHg
Blood sugar before meal Minimum: 5.1 mmol/L 12.238 ± 3.979
Maximum: 20.8 mmol/L
Blood sugar after meal Minimum: 6.7 mmol/L 16.968 ± 4.512
Maximum: 28.8 mmol/L
Urine color before meal Green 72.647%
Yellow 3.529%
Blue 22.647%
Red 0.294%
Orange 0.882%
Urine color after meal Green 85.294%
Yellow 0.588%
Orange 14.118%
Drug history Yes 99.706%
No 0.294%
Weight loss Yes 85.882%
No 14.118%
Thirst Yes 97.647%
No 2.353%
Hunger Yes 86.765%
No 13.235%
Relatives Yes 75.882%
No 24.118%
 Diabetes Mellitus Prediction Using Ensemble Machine Learning Techniques 459

Table 3. Features list (Part-2)

Attributes name Sub category Data distribution

Headache for high blood pressure Yes 50.882%
No 49.118%
Physical activity Yes 99.412%
No 0.588%
Smoking Yes 4.412%
No 95.588%
Tabaco chewing Yes 14.412%
No 85.588%
Burning extremities Yes 50.882%
No 49.118%
Weakness Yes 99.118%
No 0.882%
Symptom duration Minimum: 1 Day 156.685 ± 161.12
Maximum: 1460 Days
Diabetes Mellitus Yes 100%
No 0%
Outcome Typical 58.824%
Non-typical 13.529%
Both 27.647%

There are many ways to aggregate the dynamic models fitted in parallel. The
working procedure of Bagging algorithm is given below:

1. Firstly, it randomly takes a sample if there are N observation and M features.

2. Secondly, a model is created with the sample observations and selected subset
of features.
3. Next, selects features from the subset for giving the best split in the training.
4. Then, iterate this process to create several models and then these models are
trained parallel.
5. Finally, on the basis of the aggregation, a prediction is given from all the
trained models.

Decorate: It is a meta-learner to build various ensembles of different classifiers,

by using uniquely assembled artificial training instances. Many experiments have
proved that Decorate technique always gives more accurate result than the base
classifiers such as Bagging, Random Forest. Decorate produces higher accuracy
than Boosting methods especially even the dataset is not large and it also gains
close performance in larger training sets [22].
Decorate works generating an ensemble iteratively, in each iteration it learns a
classifier and adds it to the current iteration. For containing the classifier which is
trained on the provided training data. In every successive iteration, the classifiers
460 Md. T. Islam et al.

are trained on actual training data as well as some artificial data. Artificial
training instances in every iteration are generated from data distribution; here
the number of instances to be generated is specified as a fraction, resize, of the
given training set size. The marks for these generated training instances are
chosen to vary maximally from the present ensemble’s predictions. The making
procedure of the artificial data is explained broadly in the following section. We
refer the marked artificially initiated training as to the diversity data. A new
classifier has been trained on the composition of the actual training data and
the diversity data. If after attaching this new classifier to the present ensembles
increase more ensemble training error, we immediately reject this classifier, else
it is added to the present ensemble. This process is continuously repeated until
getting our expected committee size or surpass the highest number of iterations.
For classifying an unmarked instance x, we use the following procedure. Every
base classifier, Ci , in the ensemble C provides probabilities for class membership
of x. If PCi,y (x) becomes the probability of instance x belongs to class y as
stated in the classifier Ci , then we calculate the class membership for the whole
ensemble as [23]:

∗ Pci,y (x)
Py (x) = ci c ∗ (5)
|C |

Where, Py (x) = Probability of x belonging to class y

Then we select the most probable as the mark for class x i.e.,

C ∗ (x) = argmaxyY Py (x) (6)

4 Experimented Results and Discussions

Age Year

Height cm

Weight kg

BMI kg/m2

Heart rate BPM

Systolic BP mmHg

Diastolic BP mmHg
Blood sugar mmol/L
before meal
Blood sugar mmol/L
after meal
Symptom Year
duration
X
0 50 100 150 200

Mean Standard Deviation

Fig. 2. Mean and Standard Deviation of some major attributes

 Diabetes Mellitus Prediction Using Ensemble Machine Learning Techniques 461

Figure 2 demonstrates the mean and standard deviation of several major numer-
ical features for instance, mean age of the patients in our dataset is 48.944 while
the standard deviation is 16.047. Again, the mean height in our dataset is 158.31
and the standard deviation is 7.005.

Age Year

Height cm

Weight kg

BMI kg/m2

Heart rate BPM

Systolic BP mmHg

Diastolic BP mmHg
Blood sugar mmol/L
before meal
Blood sugar mmol/L
after meal
Symptom Year
duration
X
0 50 100 150 200
Minimum Maximum

Fig. 3. Maximum and Minimum values of some major attributes

Figure 3 represents the maximum and minimum values of some major

attributes of diabetes patients’ from our dataset. For example, the maximum
aged patient in the dataset is 75 years and the minimum aged patient is 22
years. Similarly, the maximum height of the patients is 176 cm and the mini-
mum height is 140 cm.
The percentages of some nominal features in positive and negative aspects
have been shown in Fig. 4. For example, 99.706% of patients use drug and the
rest 0.294% don’t. Again, 85.882% of patients’ have lost their weight.
Table 4 represents that, Bagging algorithm has classified 95.5882% instances
correctly and 4.4118% instances incorrectly. Again, Kappa Statistic is 0.9214,
Mean Absolute Error is 0.0482 and Root Mean Squared Error is 0.1546. Relative
Absolute Error and Root Relative Squared Error are 12.9101% and 35.7936%,
respectively. Moreover, the TP rate is 0.956 while FP rate is 0.032. The Precision,
Recall and F-Measure are same, 0.956. Furthermore, the value of MCC is 0.921
and the value of ROC Area is 0.989 while PRC Area is 0.975. Apart from these,
we have got 0.949 or 94.9% Specificity for Bagging algorithm.
Table 5 reveals that, Decorate algorithm has given 98.3% Specificity and
98.53% Accuracy which are quite impressive. The value of KS is 0.9737 while
Mean Absolute Error and Root Mean Absolute Error are 0.03 and 0.0869, respec-
tively. Also, it gave 8.0203% Relative Absolute Error and 20.1163% Root Rela-
tive Square Error. Besides, TP and FP rates are 0.985 and 0.010, respectively.
462 Md. T. Islam et al.

Y Male Female Yes No

100

80

60
Percentages

40

20

0 X

high BP
Hunger

Headache for

extremities
Sex

Drug history

Weight loss

Thirst

Relatives

Physical activity

Smoking

Tabaco chewing

Burning

Weakness

Diabetes
Fig. 4. Percentages of several nominal features in terms of positive and negative

Table 4. Outcomes of Bagging algorithm

Evaluation metrics Bagging

Total number of instances 340
Correctly classified instances 95.5882%
Incorrectly classified instances 4.4118%
Kappa statistic 0.9214
Mean absolute error 0.0482
Root mean squared error 0.1546
Relative absolute error 12.9101%
Root relative squared error 35.7936%
TP rate 0.956
FP rate 0.032
Precision (weighted average) 0.956
Recall (weighted average) 0.956
F-Measure (weighted average) 0.956
MCC (weighted average) 0.921
ROC area (weighted average) 0.989
PRC area (weighted average) 0.975
Specificity (weighted average) 0.949

The values of Precision, Recall and F-measure are same at 0.985 while MCC is
0.975. For Decorate algorithm the outcomes of ROC and PRC Area are same,
0.999.
Diabetes Mellitus Prediction Using Ensemble Machine Learning Techniques 463

Table 5. Outcomes of Decorate algorithm

Evaluation metrics Decorate

Total number of instances 340
Correctly classified instances 98.5294%
Incorrectly classified instances 1.4706%
Kappa statistic 0.9737
Mean absolute error 0.03
Root mean squared error 0.0869
Relative absolute error 8.0203%
Root relative squared error 20.1163%
TP rate 0.985
FP rate 0.010
Precision (weighted average) 0.985
Recall (weighted average) 0.985
F-Measure (weighted average) 0.985
MCC (weighted average) 0.975
ROC area (weighted average) 0.999
PRC area (weighted average) 0.999
Specificity (weighted average) 0.983

Table 6. Comparison of performance between Bagging and Decorate algorithms

Evaluation metrics Algorithms

Bagging Decorate
Total number of instances 340 340
Correctly classified instances 95.5882% 98.5294%
Incorrectly classified instances 4.4118% 1.4706%
Kappa statistic 0.9214 0.9737
Mean absolute error 0.0482 0.03
Root mean squared error 0.1546 0.0869
Relative absolute error 12.9101% 8.0203%
Root relative squared error 35.7936% 20.1163%
TP rate 0.956 0.985
FP rate 0.032 0.010
Precision (weighted average) 0.956 0.985
Recall (weighted average) 0.956 0.985
F-Measure (weighted average) 0.956 0.985
MCC (weighted average) 0.921 0.975
ROC area (weighted average) 0.989 0.999
PRC area (weighted average) 0.975 0.999
Specificity (weighted average) 0.949 0.983
464 Md. T. Islam et al.

Table 7. Comparison between our proposed system and several existing systems

Reference number No. of features Sample size Algorithms Accuracy

[7] 8 768 Naı̈ve Bayes 77%
Simple CART 76.5%
SVM 79.13%
Random Forest 76.5%
[8] 9 768 Random Forest 84%
[9] 6 1500 Scalable Random Forest 87.5%
[10] 8 3075 Logistic Regression 89.17%
Random Forest 86.7%
[11] 30 506 AdaBoost 75.49%
Bagging 76.28%
K-NN 72.33%
SVM 72.72%
Random Forest 75.30%
[12] 10 373 Random Forest 84.19%
[13] 14 303 AdaBoost 81.59%
Bagging 81.59%
Decision Tree 78.15%
MLP 79.14%
Naı̈ve Bayes 82.59%
Simple Logistic 83.1%
Random Forest 83.15%
[14] 9 768 J48 84.11%
MLP 80.59%
Naı̈ve Bayes 76.30%
REP Tree 83.07%
SMO 77.47%
Our proposed model 26 340 Bagging 95.59%
Decorate 98.53%

Comparisons between the two algorithms we used in this study have been
shown in Table 6. After comparing the outcomes of the performance parameters
for both algorithms it is clear that Decorate performed better than Bagging. In
this analysis, Bagging has given 95.59% accuracy. On the other hand, Decorate
algorithm has given 98.53% accuracy. The Specificity of Bagging is 94.9% while
it is 98.3% for Decorate.
Figure 5 illustrates the accuracy of Bagging and Decorate algorithm in per-
centage according to our experiments.
We have compared our proposed model with some other existing systems
based on their accuracy and the dataset they used. The comparison has been
shown in Table 7 which clarifies that our proposed system has given the best
performance among the compared models.
 Diabetes Mellitus Prediction Using Ensemble Machine Learning Techniques 465

(a) (b)

Green(85.294%) Yellow (0.588%) Green (72.647%) Yellow (3.529%)

Orange (14.706%) Blue (22.647%) Orange (0.882%)
Red (0.294%)

Fig. 5. Percentages of different urine colors (a) after meal, and (b) before meal

5 Conclusion
In this contemporary world, a notable portion of people are suffering from dia-
betes all over the world and the most shocking matter is that most of the patients
are unaware of it. Even they don’t know which type of diabetes they have. If
the types of diabetes can be determined at an early stage and can take proper
treatment, diabetes can be controlled and can’t go to an alarming stage. There
are many effective studies have been done to find the easiest way to predict
human diseases using ML which have been very helpful for patients. So, This
is very convenient to use ML to perform this analysis. Our aim was to build
a more accurate model than the existing models using Bagging and Decorate.
During the study, there were many limitations, for example, limited data of the
patients, limitation of instances, features and the most important one is miss-
ing information which have been handled using ML techniques in WEKA tool.
Despite having these limitations, the analysis has been performed properly. How-
ever, the study can also be conducted using other advanced ML algorithms such
as ANFIS and more advanced ensemble techniques which combine Neural Net-
work, Fuzzy System, etc. with more instances and a wider range of attributes. By
comparing these models, an expert system can be developed to predict diabetes
and its types more accurately and effectively.

References
1. Insulin and diabetes, Diabetes UK (2019). https://www.diabetes.org.uk/guide-to-
diabetes/managing-your-diabetes/treating-your-diabetes/insulin. Accessed 02 Jan
2019
2. Diabetes definition causes and symptoms. Davita.com (2019). https://www.davita.
com/education/kidney-disease/risk-factors/diabetes. Accessed 05 Jan 2019
466 Md. T. Islam et al.

3. Introduction: Standards of Medical Care in Diabetes-2019. Diabetes Care 42(1),

S1–S2 (2018). https://doi.org/10.2337/dc19-sint01. Accessed 12 Jan 2019
4. Global statistics on diabetes, Escardio.org (2019). https://www.escardio.org/Sub-
specialty-communities/European-Association-of-Preventive-Cardiology-(EAPC)/
News/global-statistics-on-diabetes. Accessed 17 Jan 2019
5. Akter, S., Rahman, M., Abe, S., Sultana, P.: Prevalence of diabetes and prediabetes
and their risk factors among Bangladeshi adults: a nationwide survey. World Health
Organ. 92, 204–213A (2014)
6. Alahmar, A., Mohammed, E., Benlamri, R.: Application of data mining techniques
to predict the length of stay of hospitalized patients with diabetes. In: 2018 4th
International Conference on Big Data Innovations and Applications (Innovate-
Data), Barcelona, Spain (2018)
7. Mir, A., Dhage, S.: Diabetes disease prediction using machine learning on big data
of healthcare. In: 2018 Fourth International Conference on Computing Communi-
cation Control and Automation (ICCUBEA), Pune, India (2018)
8. Dutta, D., Paul, D., Ghosh, P.: Analysing feature importances for diabetes predic-
tion using machine learning. In: 2018 IEEE 9th Annual Information Technology,
Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC,
Canada (2018)
9. Rallapalli, S., Suryakanthi, T.: Predicting the risk of diabetes in big data electronic
health Records by using scalable random forest classification algorithm. In: 2016
International Conference on Advances in Computing and Communication Engi-
neering (ICACCE), Durban, South Africa (2016)
10. Manna, S., Maity, S., Munshi, S., Adhikari, M.: Diabetes prediction model using
cloud analytics. In: 2018 International Conference on Advances in Computing,
Communications and Informatics (ICACCI), Bangalore, India (2018)
11. Raihan, M., et al.: A comprehensive analysis on risk prediction of acute coronary
syndrome using machine learning approaches. In: 21st International Conference
of Computer and Information Technology (ICCIT), Dhaka, Bangladesh, pp. 1–6
(2018)
12. Xu, W., Zhang, J., Zhang, Q., Wei, X.: Risk prediction of type II diabetes based on
random forest model. In: Third International Conference on Advances in Electrical,
p. 2017. Electronics, Information, Communication and Bio-Informatics (AEEICB),
Chennai, India (2017)
13. Kinge, D., Gaikwad, S.: Survey on data mining techniques for disease prediction.
Int. Res. J. Eng. Technol. (IRJET) 05(01), 630–636 (2018). Accessed 11 May 2018
14. Verma, D., Mishra, N.: Analysis and prediction of breast cancer and diabetes dis-
ease datasets using data mining classification techniques. In: 2017 International
Conference on Intelligent Sustainable Systems (ICISS), Palladam, India (2017)
15. Witten, I., Frank, E., Hall, M.: Data Mining Practical Machine Learning Tools and
Techniques, 3rd edn, pp. 166–580. Morgan Kaufmann, Burlington (2011)
16. Han, J., Kamber, M., Pei, J.: Data Mining Concepts and Techniques, 3rd edn, pp.
370–382. Morgan Kaufmann, Burlington (2011)
17. Brownlee, J.: A Gentle Introduction to k-fold Cross-Validation, Machine Learning
Mastery (2018)
18. Shubham, J.: Ensemble Learning — Bagging and Boosting, Medium (2018).
https://becominghuman.ai/ensemble-learning-bagging-and-boosting-d20f38be
9b1e. Accessed 27 Jun 2018
19. Dean, J.: Big data, Data Mining, and Machine Learning, pp. 124–125. Wiley, Hobo-
ken (2014)
 Diabetes Mellitus Prediction Using Ensemble Machine Learning Techniques 467

20. Han, J., Kamber, M., Pei, J.: Data Mining, 3rd edn, pp. 370–382. Elsevier, Ams-
terdam (2011)
21. Kandan, H.: Bagging the skill of Bagging(Bootstrap aggregating). Medium (2018).
https://medium.com/@harishkandan95/bagging-the-skill-of-bagging-bootstrap-
aggregating-83c18dcabdf1. Accessed 29 Jun 2019
22. Decorate, Weka.sourceforge.net (2019). http://weka.sourceforge.net/doc.stable/
weka/classifiers/meta/Decorate.html. Accessed 31 Dec 2018
23. Melville, P., Mooney, R.: Constructing diverse classifier ensembles using artificial
training examples. In: Proceedings of the 18th International Joint Conference on
Artificial Intelligence, IJCAI 2003, Mexico, Acapulco, pp. 505–510 (2003)
 Author Index

Agnihotri, Shailja 195 Farzana, Fahmida 453

Ahmad Mir, Waseem 87
Ahmed, Razu 357 Gandhi, M. P. Indra 64
Ahuja, Sahil 223 Garg, Sahil 223
Akash, Sheikh Raihan Islam 453 Gomathi, E. 318
Aktar, Nasrin 453
Arunnehru, J. 16
Hanbar, Hrushikesh 132
Baro, Pranita 373
Bhajantri, Nagappa U. 26 Islam, Md. Tanvir 453
Bhattacharya, Debayan 3 Iyengar, S. S. 343
Biswas, Ankur 3 Izharuddin 87
Bobita, S. 53
Borah, Malaya Dutta 373 Jayasheela, M. 318
Buyya, Rajkumar 343
kala, D. Shashi 64
Ch, Nagesh 329 Kant, Umang 160
Chanu, Asem Debala 123 Kar, Nirmalya 170, 182, 207
Chaube, Mithilesh Kumar 387 Khongbuh, Wanbanker 273
Chaudhary, Krista 160 KR, Venugopal 343
Choudhury, Anwayee Roy 3 Kumar, Ajay 16
Choudhury, Bidisha 39 Kumar, Ashav 398
Chouhan, Dharamendra 343 Kumar, Bindesh 398
Chowdhuri, Rajib 240 Kumar, Pramod 160
Chowdhury, S. M. Abrar Kabir 146 Kumar, Praveen 251
Christinal, Hepzibah A. 318 Kumar, Santosh 387
CR, Sandhya 343 Kumar, Saurabh 329
Kumar, Wanghengbam Kanan 407
Das, Abhilash Kumar 170, 207 Kumari, Pooja 74, 98
Das, Abhrajyoti 170, 207
Das, Anupam 39 Lubna, Jahanara Islam 146
De, Anurag 251, 441
Deb, Suman 74, 98 Madhu, G. M. 303
Deb, Tapajyoti 111 Mishra, Arun 398
Deb Barma, Mrinal Kanti 240 Mishra, Prerna 387
Debnath, Abir 3 Modi, Chirag 132, 303
Debnath, Barnita 74, 98 Mukhopadhyay, Sushanta 373
Debnath, Sanjib 430
Deep, Anirudh 398 Nagavi, Trisiladevi C. 26
Deepak Chethan, S. 288 Nandini, M. S. 26
Devi, H. Kenny 53 Nissar, Iqra 87
Dey, Monika 441 Nobel, Naimul Islam 357
Dilip Kumar, S. M. 288 Nongmeikapam, Kishorjit 407
470 Author Index

Patnaik, L. M. 343 Shreyas, J. 288

Pattar, Santosh 343 Shukla, Varan 132
Singh, Koushlendra Kumar 398
Raghav, Gaurav 53 Singh, L. Sarbajit 53
Raihan, M. 453 Singh, Ningthoujam Johny 407
Ramkumar, K. R. 195 Singhvi, Vishu 417
Randhawa, Sukhchandan 223 Srinidhi, N. N. 288
Srivastava, Prateek 417
Sagar, C. S. 288
Saha, Ashim 111, 251, 430, 441 Tasnim, Zarrin 357
Saha, Goutam 273
Sarkar, Priyanka 263 Vala, Darshil 343
Satija, Tanu 182 Verma, J. P. 16
Shamrat, F. M. Javed Mehedi 357 Vijila, M. 318
Sharma, Bobby 123 Vishwas, Niti 111
Sharmila 160 Vyjayanthi, C. 132, 303