Project on

“Smart Attendance System

Using Facial Recognition”

Submitted By

A kaliprasad sarangi
(1701110111)

Guided By:
Mrs. Soumya Das
Assistant Professor

DEPT. OF COMPUTER SCIENCE AND ENGINEERING,

GOVERNMENT COLLEGE OF ENGINEERING
KALAHANDI, BHAWANIPATNA

i
 DECLARATION

We declare that this written submission represents our ideas in our own words and where others’
ideas have been included. we have adequately cited and referenced the original sources. We also
declare that we have adhered to all principles of academic honesty and integrity and have not
misrepresented or fabricated or falsified any idea/data/act source in our submission. We understand
that any violation of the above will be cause for disciplinary action by the institute and can also
evoke penal action from the sources which have thus not been properly citied or from whom proper
permission has been taken when needed.

A Kaliprasad Sarangi

ii
 CERTIFICATE

This is to certify that the project report entitled “Smart Attendance System Using Facial
Recognition” submitted by Sarvesh Dasgupta (Regd. No. 1701110053), Jatindra nath Mishra
(Regd. No. 1701110112), Shaheel Sahoo (Regd. No. 1701110158) and A Kaliprasad Sarangi
(Regd No. 1701110111) to Government College of Engineering, Kalahandi, Bhawanipatna, in
partial fulfillment for the award of the degree of B. Tech in Computer Science and Engineering
is a bona-fide work carried out by them under our supervision. The contents of this project, in
full or in parts, have not been submitted to any other institution or University for the award of
any degree or diploma.

Mr. Dilip Ranjan Nayak Mrs. Soumya Das

Head of Department External Examiner Asst. Professor
(Project Guide)

iii
 ACKNOWLEDGEMENT

The satisfaction that successful completion of this successful completion of this project would
be incomplete without the mention of the people who made it possible, without whose
constant guidance and encouragement would have made effort go in vain. We consider
ourselves privileged to express gratitude and respect towards all those who guided us through
the completion of this seminar.

We convey thanks to my guide Mrs. Soumya Das (Asst. Prof, CSE) for
providing encouragement, constant support and guidance which was a great help to complete
this minor project successfully.

We are very grateful to Mr. Dillip Ranjan Nayak, Head of the Department of
Computer Science and Engineering for giving support and encouragement that was necessary
for the completion of this minor project.

We would also like to express my gratitude to Prof. (Dr.) Dulu Patnaik,

Principal, Government College of Engineering Kalahandi, Bhawanipatna for providing us
congenial environment to work in.

A Kaliprasad Sarangi

iv
 ABSTRACT

Uniqueness or individuality of an individual is his face. In this project face of an individual is

used for the purpose of attendance making automatically. Attendance of the student is very
important for every college, universities and school. Conventional methodology for taking
attendance is by calling the name or roll number of the student and the attendance is recorded.
Time consumption for this purpose is an important point of concern. Assume that the duration
for one subject is around 60 minutes or 1 hour & to record attendance takes 5 to 10 minutes.
For every tutor this is consumption of time. To stay away from these losses, an automatic
process is used in this project which is based on image processing. In this project face
detection and face recognition is used. Face detection is used to locate the position of face
region and face recognition is used for marking the understudy’s attendance. The database of
all the students in the class is stored and when the face of the individual student matches with
one of the faces stored in the database then the attendance is recorded.

v
 List of Figures
Fig 1.1: Working Process 2
Fig 2.1: Types of Face detection 3
Fig 4.1: Gradient representation 8
Fig 4.2: RGB to gray scale conversion 9
Fig 4.3: Gradient Calculation 9
Fig 4.4: Vector Representation 10
Fig 4.5: Hog Features 10
Fig 4.6: Removal of insignificant Features 11
Fig 4.7: Sample picture 11
Fig 5.1: N*N training Image 12
Fig 5.2: Average image 13
Fig 5.3: Linear Combination of K 15
Fig 5.4: Coefficient Of Eigen-faces 16
Fig 5.5: Test image- Average image 17
Fig 6.1: Sample image 21
Fig 7.1: Haar features 22
Fig 7.2: Adaboost 23
Fig 7.3: Cascading 24
Fig 7.4: Conversing RGB Image to Gray sacale 24
Fig 7.5: Using Haar features 25
Fig 7.6: Haar like features For Face Detection 26

vi
Declaration ii
Certification iii
Acknowledgement iv
Abstract v
List of Figure vi

CONTENTS
1 Introduction

2 Face Detection 2
2.1 Knowledge- Based 3
2.2 Feature - Based 3
2.3 Template Matching: 3
2.3.1 Appearance -Based 4

3 Face Recognition
3.1 History of Face Recognition 5

4 HOG Algorithm for Face Detection

4.1 How the H.O.G works 6
5 PCA
5.1 Training algorithm 9
6 Fisher Faces Method 15
6.1 The Algorithm 16
6.1.1 Fisher Faces Process 16

7 Face Recognition using Haar Cascade

7.1 Haar features 18
7.2 Ada boosting 19
7.3 Cascading 19
7.4 How face detection works 20
7.4.1 How Facial Recognition System Works 21

8 Conclusion 23
9 Reference 24
 Chapter 1
Introduction

Attendance is prime important for both the teacher and student of an educational
organization. So it is very important to keep record of the attendance. The problem arises
when we think about the traditional process of taking attendance in classroom. Calling name
or roll number of the student for attendance is not only a problem of time consumption but
also it needs energy. So an automatic attendance system can solve all above problems.
There are some automatic attendances making system which are currently used by much
institution. One of such system is biometric technique. Although it is automatic and a step
ahead of traditional method it fails to meet the time constraint. The student has to wait in
queue for giving attendance, which is time taking.
This project introduces an involuntary attendance marking system, devoid of any kind of
interference with the normal teaching procedure. The system can be also implemented during
exam sessions or in other teaching activities where attendance is highly essential. This system
eliminates classical student identification such as calling name of the student, or checking
respective identification cards of the student, which can not only interfere with the ongoing
teaching process, but also can be stressful for students during examination sessions.[1]
It consists of two main parts i.e.
 Face Detection
 Face Recognition

Fig 1.1: Working Process

1
 Chapter 2
Face Detection
Face detection is defined as finding the position of the face of an individual. In other word it can be defined
as locating the face region in an image. After detecting the face of human its facial features is extracted and
has wide range of application like facial expression recognition, face recognition, observation systems,
human PC interface and so forth…Detecting face in an image of single person is easy but when we consider
a group image of an image containing multiple faces, the task becomes difficult. For the application of face
recognition, detection of face is very important and the first step.

After detecting face, the face recognition algorithm can only be functional. Face detection itself involves
some complexities for example surroundings, postures, enlightenment etc. There are some existing
methodologies for detection of face. Some of them are skin color based, characteristic or feature based
(feature like mouth, nose and eyes) and neural network based. Among the above techniques, the skin-based
procedure is well thought-out as simplest one. The approach premeditated and applied in this thesis is the
skin color-based face detection method. The algorithm is pretty dynamic as numerous people face can be
detected at one time from an image containing many people.

Fig: 2.1 Types of face detection

2
2.1 Knowledge- Based: -
The knowledge-based method depends on the set of rules, and it is based on human
knowledge to detect the faces. Ex- A face must have a nose, eyes, and mouth within certain
distances and positions with each other. The big problem with these methods is the difficulty
in building an appropriate set of rules. There could be many false positive if the rules were too
general or too detailed. This approach alone is insufficient and unable to find many faces in
multiple images.

2.2 Feature - Based: -

The feature-based method is to locate faces by extracting structural features of the face. It is
first trained as a classifier and then used to differentiate between facial and non-facial regions.
The idea is to overcome the limits of our instinctive knowledge of faces. This approach
divided into several steps and even photos with many faces they report a success rate of 94%.

2.3 Template Matching: -

Template matching method uses pre-defined or parameterized face templates to locate or
detect the faces by the correlation between the templates and input images. Ex- a human face
can be divided into eyes, face contour, nose, and mouth. Also, a face model can be built by
edges just by using edge detection method. This approach is simple to implement, but it is
inadequate for face detection. However, deformable templates have been proposed to deal
with these problems.

Fig:2.2: TEMPLATE MATCHING

3
 2.3.1 Appearance -Based: -
The appearance-based method depends on a set of delegate training face images to
find out face models. The appearance-based approach is better than other ways of
performance. In general appearance-based method rely on techniques from statistical analysis
and machine learning to find the relevant characteristics of face images. This method also
used in feature extraction for face recognition.

4
 Chapter 3

Face Recognition
Face recognition technology is the least intrusive and fastest biometric technology. It works
with the most obvious individual identifier – the human face. Instead of requiring people to
place their hand on a reader(a process not acceptable in some cultures as well as being a
source of illness transfer) or precisely position their eye in front of a scanner, face
recognition systems unobtrusively take pictures of people's faces as they enter a defined
area. There is no intrusion or delay, and in most cases the subjects are entirely unaware of
the process. They do not feel "under surveillance" or that their privacy has been invaded.[2]

3.1 History of Face Recognition

The subject of face recognition is as old as computer vision, both because of the practical
importance of the topic and theoretical interest from cognitive scientists. Despite the fact
that other methods of identification (such as fingerprints, or iris scans) can be more
accurate, face recognition has always remained a major focus of research because of its non-
invasive nature and because it is people's primary method of person identification.

Perhaps the most famous early example of a face recognition system is due to Kohonen,
who demonstrated that a simple neural net could perform face recognition for aligned and
normalized face images. The type of network he employed computed a face description by
approximating the eigenvectors of the face image's autocorrelation matrix; these
eigenvectors are now known as
`eigenfaces.'

Kohonen's system was not a practical success, however, because of the need for precise
alignment and normalization. In following years many researchers tried face recognition
schemes based on edges, inter-feature distances, and other neural net approaches. While
several were successful on small databases of aligned images, none successfully addressed
the more realistic problem of large databases where the location and scale of the face is
unknown.

Kirby and Sirovich (1989) later introduced an algebraic manipulation which made it easy to
directly calculate the eigenfaces and showed that fewer than 100 were required to accurately
code carefully aligned and normalized face images. Turk and Pentland (1991) then
demonstrated that the residual error when coding using the eigenfaces could be used both to
detect faces in cluttered natural imagery, and to determine the precise location and scale of
faces in an image.

They then demonstrated that by coupling this method for detecting and localizing faces with
the eigenface recognition method, one could achieve reliable, real-time recognition of faces
in a minimally constrained environment. This demonstration that simple, real-time pattern
recognition techniques could be combined to create a useful system sparked an explosion of
interest in the topic of face recognition

5
 Chapter 4

HOG Algorithm for Face Detection

The H.O.G(Histogram of Oriented Gradients) is a feature descriptor used in computer vision
for image processing for the purpose of object detection. This got traction after Navneet Dalal
and Bill Triggs published a paper called Histograms of Oriented Gradients for Human
Detection in 2005. This was powerful and state of the art way of doing object detection
before the deep learning era. HOGs are widely known for their use in pedestrian detection.[4]

Before knowing how H.O.G works let us know what gradients in this context are. Take the
following image for example:

When you step from left to right pixel by pixel, you will find that after some steps, there is a
sudden change in the pixel value i.e, from a black lower pixel number to a white higher pixel
number. This sudden change in the color is called a gradient and going from a darker tone to
a lighter tone is called a positive gradient and vice versa. Going from left to right gives us the
horizontal gradient and as expected going from top to down gives a vertical gradient

4.1 How the H.O.G works

HOG works with something called a block which is similar to a sliding window. A block is
considered as a pixel grid in which gradients are constituted from the magnitude and direction
of change in the intensities of the pixel within the block.

1-So the first step would be to convert an RGB image to grayscale.

Fig 4.2: RGB to gray scale

6
 To get a closer look, let's focus on one such grid of size 8*8. Look at the following picture.

In the block of 64 pixels, for each pixel, horizontal and vertical gradients are calculated.
Like in the above picture, horizontal and vertical gradients are calculated as :
Horizontal Gradient: 120 –70 = 50
Vertical Gradient : 100 –50 = 50

Once we get the gradients, we try to calculate something called gradient magnitude and
gradient angle for each of 64 pixels.
Now with those 64 gradient vectors, we try to compress them to 9 vectors, trying to retain the
maximum structure. To do this we try to plot a histogram of magnitudes and angles. Here x-
axis is angles and they are binned into 9 bins each with a size of 20 degrees.

Fig 4.4 Vector representation

The above results are for one 8*8 grid and we compressed the representation to 9 vectors.

7
 When we slide that 8*8 grid along the whole image and try to interpret the histogram
results we get something like below.

Fig 4.5: Hog features

And by plotting the HOG features we will find that the structure of the object or face is
well maintained, losing all the insignificant features.

8
 Chapter 5
PCA

PCA (Principal Component Analysis) is a dimensionality reduction technique that was

proposed by Pearson in 1901. It uses Eigenvalues and Eigenvectors to reduce dimensionality
and project a training sample/data on small feature space. Let’s look at the algorithm in more
detail (in a face recognition perspective).[4]

5.1 Training Algorithm:

 Let’s Consider a set of m images of dimension N*N (training images)

Fig 5.1: N*N Training image

9
 We first convert these images into vectors of size N2 such that:

<x1,x2,x3...xm >

Now we calculate the average of all these face vectors and subtract it from each

Vector, Σm

ψ = (1/m)Σmi=1 xi..............(i)

ai = xi – ψ................... (ii)

Fig 5.2: Average face

Now we take all face vectors so that we get a matrix of size of N2 * M.

A= a1, a2, a3.... am

10
 > Now, we find Covariance matrix by multiplying A with AT. A has dimensions N2
* M, thus AT has dimensions M * N2. When we multiplied this gives us matrix of N2*
N2, which gives us N2 eigenvectors of N2 size which is not computationally efficient to
calculate. So we calculate our covariance matrix by multiplying AT and A. This gives us
M * M matrix which has M (assuming M << N2) eigenvectors of size M.

Cov = AT A

In this step we calculate eigen values and eigenvectors of above covariance matrix
using the formula below

AT Avi = ƛivi
AATAvi = ƛiAvi
C’μi = ƛiμi......................... (iii)

where,
’
C = AAT and μi = Avi

’
From the above statement It can be concluded that C and C have same eigenvalues
and their eigenvectors are related by the equation μi = Avi. Thus, the M eigenvalues (and eigenvectors)
’
of covariance matrix gives the M largest eigenvalues (and eigenvectors) of C
Now we calculate Eigenvector and Eigenvalues of this reduced covariance matrixand map them
’
into the C by using the formula μi = Avi
Now we select the K eigenvectors of Cǀ corresponding to the K largest
eigenvalues (where K < M). These eigenvectors have size N2.
In this step we used the eigenvectors that we got in previous step. We take the
normalized training faces (face – average face) Xi and represent each face vectors in
the linear of combination of the best K eigenvectors (as shown in the diagram
below).

Xi -ψ = Σkj=1 ωiui...................(iv)

11
 Fig 5.3: Linear Combination of K

In this step, we take the coefficient of eigenfaces and represent the training faces in
the form of a vector of those coefficients.

12
 Fig 5.4: Coefficient of Eigen Faces

Given an unknown face y, we need to first preprocess the face to make it centered in the
image and have the same dimensions as the training face
Now, we subtract the face from the average face Ψ

Ɵ= y – Ψ.................................. (v)

Now, we project the normalized vector into eigenspace to obtain the linear
combination of eigenfaces.

Φ = Σki=1ωiui................ (vi)

From the above projection, we generate

the vector of the coefficient such that

13
 We take the vector generated in the above step and subtract it from the training image
to get the minimum distance between the training vectors and testing vectors

er = mini||Ω................ (vii)

Fig 5.5: Test images – Average Images

Limitations:
 Proper centered face is required for training/testing.
 The algorithm is sensitive to lightening, shadows and also scale of face in the image .
 Front view of the face is required for this algorithm to work properly

14
 Chapter 6

Fisher faces Method

Fisherface is one of the popular algorithms used in face recognition and is widely believed to
be superior to other techniques, such as eigenface because of the effort to maximize the
separation between classes in the training process. The purpose of this research is to establish
a program of face recognition application using fisherface method by utilizing GUI
applications and databases that are used in the form of a Papuan facial image. Image
recognition using fisherface method is based on the reduction of face space dimension using
Principal Component Analysis (PCA) method, then apply Fisher's Linear Discriminant
(FDL) method or also known as Linear Discriminant Analysis (LDA) method to obtain
feature of image characteristic
Principal Component Analysis (PCA) method, then apply Fisher's Linear Discriminant
(FDL) method or also known as Linear Discriminant Analysis (LDA) method to obtain
feature of image characteristic.

Fisherfaces algorithm extracts principle components that separates one individual from
another. So, now an individual's features can't dominate another person's features.

Image recognition using this algorithm is based on reduction of face space dimensions using
PCA method and then applying LDA method also known as Fisher Linear Discriminant
(FDL) method to obtain characteristic features of image.

LDA is used to find a linear combination of features that separates two or more classes or
objects. It can be used for dimension reduction before further classification. It attempts to
model the difference between classes of data.

This method doesn't capture illumination variations as obviously as Eigenfaces method.

 Data is assumed to be uniformly distributed in each class.

 Aim is to maximize the ratio of between-class scatter matrix and the within-class
scatter matrix.
 It can produce good results even in varying illumination.

6.1 The Algorithm

Let X be a random vector with samples drawn from c classes:

X = {X1, X2, X3,……, Xc}

X1 = {x1, x2, x3,……., xn}

15
 The scatter matrices S_{B} and S_{W} are calculated as:

SB = ∑ ∁= �i (μi-μ) (μi-μ)T
T............................................. (i)
1� ∑��=1 �° (xi-μi) (xi-μi)
SW = ∑
=1

,where μ is the total mean:

1
μ = ∑N X
N i=1 i

And �� is the mean of class i ∈ {1,2. . . , c} :

1
= ∑ � ……………(ii)
| | ��∈�� �

Fisher’s classic algorithm now looks for a projection W, that maximizes the class
separability criterion:
= | | ..……………...(iii)
| |

A solution for this optimization problem is given by solving the General Eigenvalue Problem:

�−1���� = ����............................................................ (iv)

The optimization problem can then be rewritten as:

= | |

| |
= ……………….(v)
| |

The transformation matrix W, that projects a sample into the (c-1)-dimensional space is then given by:

W = WT WT............................................................ (vi)
fld pca

16
6.1.1 FISHERFACE PROCESS

Step 1: Retreive data

Collection of data is done in form of face images . Collection can be done using photographs
already saved or from a webcam. Face must be fully visible and must be facing forward.

Step 2: Image Processing

a) Preprocessing stage: Getting images using camera or saved images and conversion from
RGB to grayscale. Image data is divided into training and test data.

b) Processing stage: Fisherface method will be applied to generate feature vector of facial image
data used by system and then to match vector of traits of training image with vector characteristic
of test image using euclidean distanceformula

Step 3: Feature generation

Features of the faces are extracted.

Step 4: Recognition Process:

After the training is done, the next stage is image recognition process. The goal is to
successfully recognize the test image.

If training image is the same as the testing image: In this case system can successfully
identify the test image correctly up to 100%.

If training image is not the same as the testing image: The test image and the training
image must come from the image of the same person's face. System can now successfully
identify the test image correctly upto 90%.[6]

17
 Chapter 7

Face Recognition using Haar Cascade

It consists of following features: -

• Haar feature
• Adaboosting
• Cascading

7.1 Haar Feature: -

Consider the following feature image showing the Haar features Haar feature are
basically similar to the convolution karnel, which are used to detect the presence of the
features in an image. used in Viola- Jones face detection algorithm.[5]

Fig 7.1: Haar Feature

A black region is replaced by +1 and the white region is replaced with -1. When we
apply this mask to an input image, we just subtract the pixel values under black region and
output will be a single value.

Haar features have some sort of resemblance to some facial feature or characteristics of
faces. Viola- Jones algorithm uses a 24*24 sub window from an image, and it calculates these
features all over the image.

18
7.2 Adaboost:-

Adaboost eliminates all the redundant features and it narrows it down to several thousands of
features that are very useful. We have all the combinations of all the 160000+ feature. But out
of them some are relevant and some are irrelevant. For example the below feature, which is
used to detect bridge of the nose, gives us maximum values only on the nose bridge region of
face.

Fig 7.2 Adaboost

For rest of the face region it is irrelevant. This relevance and irrelevance is determined by
Adaboost. It select only few features which are relevant. Adaboost is a machine learning
algorithm which helps in finding only the best feature among all the 160000+ features. After
these features are found, a weighted combination of all these features are used in evaluating
and deciding any given window has a face or not.

7.3 Cascading: -
After performing Adaboost and take a linear combination of all those 2500 outputs to
detect whether a face is detected or not. Now even though an image is detected ai an input
image, an excessive large amount of evaluated sub window would still be negative.

In simple terms , instead of using these 2500 features all time every single 24*24
window, we use cascades. Out of 2500 features we make sets of features. Lets say first 10
features are kept in one classifiers, and next 10-20 features in another classifiers and so on.
though complexity increased, but the advantages is that, when we applied this cascade on a
certain window 24*24 size on a given image, based on the output of the first classifier, we can
check

19
 Fig: 7.3 Cascading

7.3.1 How Face DetectionWorks

There are many techniques to detect faces, with the help of these techniques, we can
identify faces with higher accuracy. These techniques have an almost same procedure for Face
Detection such as OpenCV, Neural Networks, Matlab, etc. The face detection work as to
detect multiple faces in an image. Here we work on OpenCV for Face Detection, and there
are some steps that how face detection operates, which are as follows-

 Firstly the image is imported by providing the location of the image.

 The picture is transformed from RGB to Grayscale because it is easy to detect
faces in the grayscale.

Fig 7.4: Converting RGB image to Grayscale

20
 After that, the image manipulation used, in which the resizing, cropping, blurring and
sharpening of the images done if needed.
 The next step is image segmentation, which is used for contour detection or segments
the multiple objects in a single image so that the classifier can quickly detect the
objects and faces in the picture.
 The next step is to use Haar-Like features algorithm, which is proposed by Voila
and Jones for face detection. This algorithm used for finding the location of the
human faces in a frame or image. All human faces shares some universal properties
of the human face like the eyes region is darker than its neighbour pixels and nose
region is brighter than eyeregion.

Fig: 7.5: Using Haar Feature

21
 Fig 7.6: Haar-like features for face detection

 The haar-like algorithm is also used for feature selection or feature extraction for
an object in an image, with the help of edge detection, line detection, centre
detection for detecting eyes, nose, mouth, etc. in the picture. It is used to select
the essential features in an image and extract these features for face detection.

 The next step is to give the coordinates of x, y, w, h which makes a rectangle box
in the picture to show the location of the face or we can say that to show the
region of interest in the image. After this, it can make a rectangle box in the area
of interest where it detects the face. There are also many other detection
techniques that are used together for detection such as smile detection, eye
detection, blink detection, etc.

22
 Conclusion

In this approach, a face recognition based automated student attendance system is thoroughly described. The
proposed approach provides a method to identify the individuals by comparing their input image obtained
from recording video frame with respect to train image. This proposed approach able to detect and localize
face from an input facial image, which is obtained from the recording video frame. Besides, it provides a
method in pre-processing stage to enhance the image contrast and reduce the illumination effect. Extraction
of features from the facial image is performed by applying Haar cascade.

23
 References

[1] Pooja G.R, et al. (2010). An automated Attendance System Using Image Processing. International
Journal Of Advanced Networking & Applications.
[2] Margaret Rouse. (2012). What is facial recognition? - Definition from WhatIs.com. [online] Available
at: http://whatis.techtarget.com/definition/facial-recognition [Accessed 25 Mar. 2018].
[3] Viola, P and M.J. Jones, 2004. “Robust Real-Time Face Detection”, International Journal of Computer
Vision, 57(2): 137-154.
[4] Priyanka Dhoke and M.P. Parsai, “A MATLAB Based Face Recognition Using PCA with Back
Propagation Neural network” International Journal of Innovative Research in computer and Communication
Engineering, 8(2): 89-95
[5] Yi-Qing Wan, 2014. “An Analysis of the Viola-Jones Face Detection Algorithm”, Image Processing
OnLine, pp: 128-148
[6] Bhumika G. Bhatt, Zankhana H. Shah “Face Feature Extraction Techniques: A Survey”, National
Conference on Recent Trends in Engineering & Technology, 13-14 May 2011.

24