FACE RECOGNITION SECURITY SYSTEM

A Practical Journal Submitted in Fulfillment of the Degree of

MASTER In COMPUTER APPLICATION

Year 2023-2024

By

Mr. Amit Dattaram Patil

(Application Id: - 81940)

Semester- 2
Under the Guidance of

Prof. Dnyaneshwar Deore

Institute of Distance and Open Learning

Vidya Nagari, Kalina, Santacruz East – 400098.
University of Mumbai

PCP Center
[Satish Pradhan Dyanasadhana College, Thane]

1
 Institute of Distance and Open Learning,
Vidyanagari, Kalina, Santacruz (E) -400098

CERTIFICATE
This to certify that, Mr. Patil Amit Dattaram Drupada appearing Master in Computer
Application (Semester II) Application ID: 81940 has satisfactory completed the prescribed
practical of Mini Project as laid down by the University of Mumbai for the academic year
202324

Teacher in charge Examiners Coordinator

IDOL, MCA
University of Mumbai

Date: -
Place: - TH

ABSTRACT

We present an approach to secure homes using facial recognition technology

combined with the existing locking mechanism to improve the security with
the motive of cost efficiency. Our approach uses sub-machine vector to identify
the face and capture a two-dimensional image by taking specific points on the
face and comparing it with the stored images. Our major goal is to improve
security using software that is everyday being developed for the better.

2
With every passing day, we are becoming more and more dependent upon
technology to carry out even the most basic of our actions. Facial detection and
Facial recognition help us in many ways, be it sorting of photos in our mobile
phone gallery by recognizing pictures with their face in them or unlocking a
phone by a mere glance to adding biometric information in the form of face
images in the country's unique ID database (Aadhaar) as an acceptable
biometric input for verification.

This project lays out the basic terminology required to understand the
implementation of Face Detection and Face Recognition using Intel's
Computer Vision library called 'OpenCV'.

It also shows the practical implementation of the Face Detection and Face
Recognition using OpenCV with Python embedding on both Windows as well
as macOS platform.

The aim of the project is to implement Facial Recognition on faces that the
script can be trained for. The input is taken from a webcam and the recognized
faces are displayed along with their name in real time.
This project can be implemented on a larger scale to develop a biometric
attendance system which can save the time-consuming process of manual
attendance system.
INDEX

CHAPTER TITLE PAGE NO.

NO.
1 INTRODUCTION 5

2 WORKING 6
3 FLOW DIAGRAM 8
4 DESIGN WITH AUTOCADE 9
5 CODING 11

3
 6 RESULT 22

7 CONCLUSION 23

8 REFERENCE 24
CHAPTER-1

INTRODUCTION

The world that we live in has improved drastically with the

recent technological advancements and most of these
technological advancements have been achieved because of the
development in software which in turn improves the hardware.

• DEEP LEARNING
Deep learning is an artificial intelligence function that imitates the
workings of the human brain in processing data and creating patterns for
use in decision making. Deep learning is a subset of machine learning in
artificial intelligence (AI) that has networks capable of learning
unsupervised from data that is unstructured or unlabelled. Also known as
deep neural learning or deep neural network.

• FACIAL RECOGNITION
Facial recognition is a biometric software application capable of uniquely
identifying or verifying a person by comparing and analyzing patterns
based on the person's facial contours. Facial recognition is mostly used
for security purposes, though there is increasing interest in other areas of
use. In fact, facial recognition technology has received significant
attention as it has potential for a wide range of application related to law
enforcement as well as other enterprises

As the need for security is becoming mandatory,facial recognition has

slowly become one of the most secure platforms for protection from
threats.It uses deep learning or machine learning techniques to identify
the characteristics of a person’s face.This technology became a huge
success in the smartphone industry and now we have implemented this
technology in homes for unlocking doors.
CHAPTER-2 WORKING
Application of deep learning and OpenCV together to Detect faces

1. Compute 128-d face embeddings to quantify a face

4
2. Train a Support Vector Machine (SVM) on top of the embeddings
3. Recognize faces in images and video streams
All of these tasks will be accomplished with OpenCV, enabling us to obtain a
“pure” OpenCV face recognition pipeline.

Working of OpenCV’s face recognition

An overview of the OpenCV face recognition pipeline. The key step is a CNN
feature extractor that generates 128-d facial embeddings.
In order to build our OpenCV face recognition pipeline, we’ll be applying deep
learning in two key steps:

1. To apply face detection, which detects the presence and location of a face in an
image, but does not identify it
2. To extract the 128-d feature vectors (called “embeddings”) that quantify each
face in an image
• The model responsible for actually quantifying each face in an image is
from the OpenFace project, a Python and Torch implementation of face
recognition with deep learning.
• Reviewing the entire Face Net implementation is outside the scope of this
tutorial, but the gist of the pipeline can be seen in Figure 1 above.

5
 • First, we input an image or video frame to our face recognition pipeline.
Given the input image, we apply face detection to detect the location of a
face in the image.

• Optionally we can compute facial landmarks, enabling us to pre-process

and align the face
• Face alignment, as the name suggests, is the process of (1) identifying the
geometric structure of the faces and (2) attempting to obtain a canonical
alignment of the face based on translation, rotation, and scale.

• While optional, face alignment has been demonstrated to increase face

recognition accuracy in some pipelines.

• After we’ve applied face alignment and cropping, we pass the input face
through our deep neural network

How the
deep learning face recognition model computes the face embedding.
The FaceNet deep learning model computes a 128-d embedding that quantifies
the face itself.

COMPUTATION OF FACE EMBEDINGS BY THE NETWORK

• The input data to the network
• The triplet loss function
• To train a face recognition model with deep learning, each input batch of
data includes three images:

• The anchor
• The positive image

6
 • The negative image
• The anchor is our current face and has identity A.
• The second image is our positive image — this image also contains a face
of person A.
• The negative image, on the other hand, does not have the same identity,
and could belong to person B, C, or even Y!
• The point is that the anchor and positive image both belong to the same
person/face while the negative image does not contain the same face.

• The neural network computes the 128-d embeddings for each face and
then tweaks the weights of the network (via the triplet loss function) such
that:

• The 128-d embeddings of the anchor and positive image lie closer
together
• While at the same time, pushing the embeddings for the negative image
father away
• In this manner, the network can learn to quantify faces and return highly
robust and discriminating embeddings suitable for face recognition.

WORKING OF THE MECHANISM

• If the face captured is matched with the stored image, then a signal is sent
from the Arduino to the servo motor

• Now the servo motor rotates the spur gear(pinion) which in turn moves
the linear gear (rack) in clockwise direction and now the door is unlocked.

• If the image captured does not match the required percentage of success
or when no image is matched, the door remains locked.

FLOW DIAGRAM

7
 DETECTION OF FACE

CAPTURE IMAGE

COMPARE REMAINS
IMAGE LOCKE
D
UNLOCKS

DESIGN WORK USING AUTOCAD

8
9
 CODE WORK

extract_embeddings.py

# USAGE
# python extract_embeddings.py --dataset dataset --embeddings
output/embeddings.pickle --detector face_detection_model --embedding-
model openface_nn4.small2.v1.t7

10
 # import the necessary
packages from imutils import
paths import numpy as np
import argparse import
imutils import pickle import
cv2
import os

# construct the argument parser and parse the

arguments ap = argparse.ArgumentParser()
ap.add_argument("-i", "--dataset", required=True,
help="path to input directory of faces + images")
ap.add_argument("-e", "--embeddings", required=True,
help="path to output serialized db of facial embeddings")
ap.add_argument("-d", "--detector", required=True,
help="path to OpenCV's deep learning face detector")
ap.add_argument("-m", "--embedding-model", required=True,
help="path to OpenCV's deep learning face embedding model")
ap.add_argument("-c", "--confidence", type=float,
default=0.5, help="minimum probability to filter weak
detections") args = vars(ap.parse_args())

# load our serialized face detector from disk print("[INFO]

loading face detector...") protoPath =
os.path.sep.join([args["detector"], "deploy.prototxt"])
modelPath = os.path.sep.join([args["detector"],
"res10_300x300_ssd_iter_140000.caffemodel"])
detector = cv2.dnn.readNetFromCaffe(protoPath, modelPath)

# load our serialized face embedding model from disk

print("[INFO] loading face recognizer...")
embedder =
cv2.dnn.readNetFromTorch(args["embedding_model"])

11
 # grab the paths to the input images in our dataset
print("[INFO] quantifying faces...")
imagePaths =
list(paths.list_images(args["dataset"]))

# initialize our lists of extracted facial embeddings and

# corresponding people
names knownEmbeddings =
[] knownNames = []

# initialize the total number of faces processed

total = 0

# loop over the image paths for (i, imagePath)

in enumerate(imagePaths): # extract the person
name from the image path
print("[INFO] processing image {}/{}".format(i + 1,
len(imagePaths)))
name = imagePath.split(os.path.sep)
[-2]

# load the image, resize it to have a width of 600 pixels (while

# maintaining the aspect ratio), and then grab the image
# dimensions image =
cv2.imread(imagePath) image =
imutils.resize(image, width=600)
(h, w) = image.shape[:2]

# construct a blob from the image

imageBlob =
cv2.dnn.blobFromImage( cv2.resize(image,
(300, 300)), 1.0, (300, 300),
(104.0, 177.0, 123.0), swapRB=False,
crop=False)

12
 # apply OpenCV's deep learning-based face detector to
localize
# faces in the input image
detector.setInput(imageBlob
) detections =
detector.forward()

# ensure at least one face was

found if len(detections) > 0:
# we're making the assumption that each image has only
ONE # face, so find the bounding box with the largest
probability i = np.argmax(detections[0, 0, :, 2])
confidence = detections[0, 0, i,
2]

# ensure that the detection with the largest probability

also # means our minimum probability test (thus helping
filter out
# weak detections) if
confidence >
args["confidence"]:
# compute the (x, y)-coordinates of the bounding box for
# the face box = detections[0, 0, i, 3:7] *
np.array([w, h, w, h])
(startX, startY, endX, endY) = box.astype("int")

# extract the face ROI and grab the ROI

dimensions
face = image[startY:endY, startX:endX]
(fH, fW) = face.shape[:2]

# ensure the face width and height are sufficiently large

if fW < 20 or fH < 20:
continue

13
 # construct a blob for the face ROI, then pass
the blob
# through our face embedding model to obtain the
128-d
# quantification of the face faceBlob =
cv2.dnn.blobFromImage(face, 1.0 / 255,
(96, 96), (0, 0, 0), swapRB=True,
crop=False) embedder.setInput(faceBlob)
vec = embedder.forward()

# add the name of the person + corresponding face

# embedding to their respective lists
knownNames.append(name)
knownEmbeddings.append(vec.flatten())
total += 1

# dump the facial embeddings + names to disk

print("[INFO] serializing {}
encodings...".format(total)) data = {"embeddings":
knownEmbeddings, "names": knownNames} f =
open(args["embeddings"], "wb")
f.write(pickle.dumps(data))
f.close()

train_model.py
# USAGE
# python train_model.py --embeddings
output/embeddings.pickle -- recognizer output/recognizer.pickle --le
output/le.pickle

# import the necessary packages from

sklearn.preprocessing import LabelEncoder
from sklearn.svm import SVC import
argparse import pickle

14
 # construct the argument parser and parse the
arguments ap = argparse.ArgumentParser()
ap.add_argument("-e", "--embeddings",
required=True,
help="path to serialized db of facial embeddings")
ap.add_argument("-r", "--recognizer", required=True,
help="path to output model trained to recognize faces")
ap.add_argument("-l", "--le", required=True,
help="path to output label encoder")
args = vars(ap.parse_args())

# load the face embeddings

print("[INFO] loading face
embeddings...")
data = pickle.loads(open(args["embeddings"],
"rb").read())

# encode the labels

print("[INFO] encoding
labels...") le =
LabelEncoder()
labels =
le.fit_transform(data["names"])

# train the model used to accept the 128-d embeddings of the face
and
# then produce the actual face recognition
print("[INFO] training model...") recognizer =
SVC(C=1.0, kernel="linear", probability=True)
recognizer.fit(data["embeddings"], labels)

# write the actual face recognition model to

disk f = open(args["recognizer"], "wb")
f.write(pickle.dumps(recognizer))
f.close()

15
# write the label encoder to
disk f = open(args["le"], "wb")
f.write(pickle.dumps(le))
f.close()

recognize_video_3.py

# USAGE
# python recognize_video_3.py --detector face_detection_model --
embedding- model openface_nn4.small2.v1.t7 --recognizer
output/recognizer.pickle --le output/le.pickle

# import the necessary packages

from urllib.request import
urlopen from imutils.video import
VideoStream from imutils.video
import FPS import numpy as np
import argparse import imutils
import serial import pickle
import time
import cv2
import os

# construct the argument parser and parse the

arguments ap = argparse.ArgumentParser()
ap.add_argument("-d", "--detector",
required=True,
help="path to OpenCV's deep learning face detector")
ap.add_argument("-m", "--embedding-model", required=True,
help="path to OpenCV's deep learning face embedding model")

16
 ap.add_argument("-r", "--recognizer",
required=True, help="path to model trained to
recognize faces") ap.add_argument("-l", "--le",
required=True,
help="path to label encoder")
ap.add_argument("-c", "--confidence", type=float, default=0.5,
help="minimum probability to filter weak detections")
args = vars(ap.parse_args())

# load our serialized face detector from disk print("[INFO]

loading face detector...") protoPath =
os.path.sep.join([args["detector"], "deploy.prototxt"])
modelPath = os.path.sep.join([args["detector"],
"res10_300x300_ssd_iter_140000.caffemodel"])
detector = cv2.dnn.readNetFromCaffe(protoPath,
modelPath)

#serial monitor obj ser=

serial.Serial('COM5',9600,timeout=1)
ser.write(b'L')

# load our serialized face embedding model from disk

print("[INFO] loading face recognizer...") embedder =
cv2.dnn.readNetFromTorch(args["embedding_model"]) #
load the actual face recognition model along with the
label encoder recognizer =
pickle.loads(open(args["recognizer"], "rb").read())
le = pickle.loads(open(args["le"],
"rb").read())

# initialize the video stream, then allow the camera sensor to

warm up print("[INFO] starting video stream...") #url =

17
"http://192.168.1.2:8080/shot.jpg" vs =
VideoStream(src=0).start()
time.sleep(2.0)

# start the FPS throughput estimator

fps = FPS().start()

# loop over frames from the video file stream

while True:
# grab the frame from the threaded video
streaur frame = vs.read() key =
cv2.waitKey(1) & 0xFF if key == ord("q"):
ser.write(b'L')
break

msg = ser.readline()
if msg == b'open\r\n':
continue
#imgResp = urlopen(url)
#imgNp = np.array(bytearray(imgResp.read()),dtype=np.uint8)
#frame = cv2.imdecode(imgNp,-1)

# resize the frame to have a width of 600 pixels (while

# maintaining the aspect ratio), and then grab the image
# dimensions
frame = imutils.resize(frame, width=600)
(h, w) = frame.shape[:2]

# construct a blob from the image

imageBlob =
cv2.dnn.blobFromImage( cv2.resize(frame,
(300, 300)), 1.0, (300, 300),

18
 (104.0, 177.0, 123.0), swapRB=False,
crop=False)

# apply OpenCV's deep learning-based face detector to

localize
# faces in the input image
detector.setInput(imageBlob
) detections =
detector.forward()

# loop over the detections for i in

range(0, detections.shape[2]):
# extract the confidence (i.e., probability) associated with
# the prediction
confidence = detections[0, 0, i,
2]

# filter out weak detections if

confidence >
args["confidence"]:
# compute the (x, y)-coordinates of the bounding box for
# the face box = detections[0, 0, i, 3:7] *
np.array([w, h, w, h])
(startX, startY, endX, endY) = box.astype("int")

# extract the face ROI face =

frame[startY:endY, startX:endX]
(fH, fW) = face.shape[:2]

# ensure the face width and height are

sufficiently large
if fW < 20 or fH < 20:
continue

# construct a blob for the face ROI, then pass

the blob

19
 # through our face embedding model to obtain the
128-d
# quantification of the face faceBlob =
cv2.dnn.blobFromImage(face, 1.0 / 255,
(96, 96), (0, 0, 0), swapRB=True,
crop=False) embedder.setInput(faceBlob)
vec =
embedder.forward()

# perform classification to recognize the

face preds =
recognizer.predict_proba(vec)[0] j =
np.argmax(preds) proba = preds[j]
name = le.classes_[j]

# draw the bounding box of the face along with

the
# associated probability text = "{}: {:.2f}
%".format(name, proba * 100) y = startY - 10 if
startY - 10 > 10 else startY + 10
cv2.rectangle(frame, (startX, startY), (endX,
endY),
(0, 0, 255), 2)
cv2.putText(frame, text, (startX,
y),
cv2.FONT_HERSHEY_SIMPLEX, 0.45, (0, 0, 255), 2)
#sending output to arduino
if not name == "unknown":
ser.write(b'H')

# update the FPS counter

fps.update()

# show the output frame

cv2.imshow("Frame", frame)

20
 key = cv2.waitKey(1) & 0xFF

# if the `q` key was pressed, break from the loop

if key == ord("q"):
ser.write(b'L')
break

# stop the timer and display FPS information

fps.stop() print("[INFO] elasped time:
{:.2f}".format(fps.elapsed()))
print("[INFO] approx. FPS:
{:.2f}".format(fps.fps()))

# do a bit of cleanup
cv2.destroyAllWindows()
vs.stop()

CHAPTER-3

RESULT

Prototype of the safe works as it was expected and successfully detects the
owners face with the implementation of software code and opens the
mechanical latch, designed with precision to increase the security of the
system and closes only when prompted by the owner.
The motive of the project is achieved by this simple prototype with the
objective of improving the security and safeguard the properties of the
owner from theft and burglary.

21
 CHAPTER-4

CONCLUSION

We have designed and fabricated the prototype of our project (Biometric Face
Recognition Security System) and we are here to conclude that the methodology
used in this project helps to the further development of the existing security
systems in various fields.
Though we made the design as a prototype, in our view it strengthens the
disadvantages in the ancient methodology and converted to an ergonomical
product as it does not normally involve any physical locking mechanism
rather the whole process is happening in the central computer system which
no one other than the owner has access to.
The application of IOT which enables the owner to control the system over the
Internet can be implemented to access the camera of the security system from
any side of the world.
By incorporating the latest solenoid enabled lock other than the mechanical
rack and pinion can ensure the system works at any situation, by reducing the
error of servos system.
Using Raspberry Pi can help in faster processing of the face and increase the
speed of detection and recognition of the face.
Instead of using a 64 points 128 embeddings of the face, we can implement the
use of 208 points embeddings which captures the image at 3D format and
quantifying them enables the system to identify the difference between an image
and a real person.
REFERENCES
• Rack and pinion variable ratio steering gear Google Patent Search.
Retrieved 2007-03-22.
• American National Standards Institute, Gear Nomenclature,
Definitions of Terms with Symbols (ANSI/AGMA 1012-G05 ed.),
American Gear Manufacturers Association
• M. Turk and A. Pentland. Eigenfaces for recognition.
• Bingo, Y.; Courville, A.; Vincent, P. (2013). "Representation
Learning: A Review and New Perspectives". IEEE Transactions on
Pattern Analysis and Machine Intelligence. 35 (8): 1798–1828
22
• Schmidhuber, J. (2015). "Deep Learning in Neural Networks: An Overview".
Neural Networks
• Support Vector Machines — scikit-learn 0.20.2
documentation". Archived from the original on 2017-11-08.
Retrieved 2017-11-08.
• Support Vector Machines — scikit-learn 0.20.2
documentation". Archived from the original on 2017-11-08.
Retrieved 2017-

23