Seminar Report on

PAY WITH FACE

Submitted in Partial fulfillment of requirements of the award of the degree

Bachelor of technology
In
Department of Computer Science and Engineering
By

SRAVANTHI PADAVALA
15241A05M1

Department of Computer Science and Engineering

Gokaraju Rangaraju Institute of Engineering and Technology

(Autonomous)
(Approved by AICTE, Affiliated to JNTUH)
2018-19
 GOKARAJU RANGARAJU
INSTITUTE OF ENGINEERING AND TECHNOLOGY
(Autonomous under JNTUH, Hyderabad)
Bachupally, Kukatpally, Hyderabad-500090.

Department of Computer Science and Engineering

CERTIFICATE

This is to certify that the Technical seminar entitled “PAY WITH FACE”is submitted by
SRAVANTHI PADAVALA (15241A05M1) in partial fulfillment of the requirement for the
award of the degree in BACHELOR OF TECHNOLOGY in Computer Science and
Engineering during the academic year 2018-2019.

Coordinator Incharge
(Mr. M. Sridhar) (Dr. P. VaraprasadRao)

HOD
(Dr. K. Madhavi)
 INDEX

1 Introduction 1
1.1 Existing System 1
1.2 Proposed System 3
2 Literature Review 5
3 Need for New Technology 7
4 Details of Paying with your face 9
4.1 Definition
4.2 Architecture
4.3 Working
4.4 Approach with Example
4.5 Live Scenarios
5 Limitations of Proposed Technology 15
6 Conclusion 20
7 Future Enhancement 21
8 References 22
 1. INTRODUCTION
The ongoing digital and technology revolution led by the ever-increasing penetration of
smart phones and internet on mobile, has revolutionised digital payments. The payments space
has witnessed the entry of several non-banking institutions offering payment services and
solutions. Customers are becoming more demanding expect instantaneous and one- touch
payment solutions. Finally, there have been several progressive changes in the regulatory
framework. Now-a-days digital payments are displaying exponential growth. Digital payments
will provide access to customer transaction data. Led by the evolution of digital technologies,
payments are no longer forte of banks. Credit cards, online wallets and cryptocurrencies are just
some of the numerous ways of payment in the modern world. Yet, there has been a recent
addition to this list. Meet a completely new technology: paying with your face. No fingerprints
or retinal scans. Just smile into the camera and have the payment done. Also the over the past
few years, computers have become incredibly good at recognizing faces, and the technology is
expanding quickly in China in the interest of both surveillance and convenience. Face
recognition is transforming everything from policing to the way people interact every day with
banks, stores, and transportation services. It will make faster and secure money transactions. It is
currently being used in China to pay for your goods and services. Near future, all most countries
in the world are going to implement paying with your face.
1.1 EXISTING SYSTEM
Payment methods are the ways that your customers can pay for your goods and services.
It is good to consider what payment methods you will offer when you start your business as well
as periodically after you start [Link] starting your business, you’ll need to decide how you will
collect payment from your customer as part managing of your business finances. This is an
important aspect of ensuring you manage the cash flow of your business effectively. The
payment methods you choose may determine what extra equipment and facilities you require in
your business. You should also periodically review the payment methods you offer to minimise

1
unnecessary costs and make the most of business opportunities. As your business grows, you
may find you need to expand or change your payment options to meet the needs of your business

and customers. Some examples of payment methods include: • Cash • Cheque • Money order
payments • Credit and debit card payments • Direct debit payments • Online payments (ex
:PayPal,Paytm) • EFTPOS payments • Gift cards and vouchers • Bitcoin and digital currencies.
Each payment method has different advantages and disadvantages. No one type of payment is
best. The best payment method depends on the needs of your business. When choosing payment
methods, think about how their advantages and disadvantages affect your customers and your
business operations. For example, cash is private and reliable, but it is expensive to handle and
has a higher risk of theft. Some payment methods increase the risk of theft. Cash is easy to steal
since it doesn’t go directly into your bank account. Some customers might prefer to pay cash for
certain goods and services, such as medication, for privacy reasons. Credit cards automatically
record transactions. On the other hand, EFTPOS is quick and has a lower risk of theft, but
requires electricity and a phone connection. EFTPOS and credit card providers often charge
services. Bitcoin and other cryptocurrencies has a threat of ransomware. In the payments
industry there is a lot of jargon spoken and a lot of unnecessary complication that try to make the
whole world seem more than it is. Take for example the Single European Payments Area (SEPA)
initiative. Here the governing body, has perpetuated this state of affairs by creating more and
more ‘types’ of payments so now we have SEPA credit transfers, direct debits, business to
business direct debits, card transactions, e-payments, m-payments... the list goes on. Among all
the existing payment modes, some of them secure and some of them are convenient while some
of them are may be used to illegalize earning such as cash, because they do not have proper
records. So, digital payments are secure and convenient .
1.2 PROPOSED SYSTEM
We have different payment modes, out of which some are secure, private, and
convenient. Another addition to the mode of payments is “Paying with your Face”. Paying with
your face is convenient for the users. This mode of payment is convenient to the users rather than
security. Usually, now-a-days we use either cash or debit/credit cards for paying for our goods
and services. This may delay the transaction time. The customer has to wait for long time in
queue to finish the payment. So, this new payment mode enables us faster transaction. Instead of

2
waiting for long time in queues, each customer can finish his/her transaction in nearly 30
seconds. It is time saving.

2. LITERATURE REVIEW BACKGROUND

Face recognition is not a new technology, but now it became good enough that we can
rely on its search results. It was just one of running series of Google’s attempts to remedy the
Face Unlock feature introduced in the Ice Cream Sandwich version of Android: a feature that
was tricked by holding up a photo to the phone. Google responded by introducing a technique
called “Liveness Check” that required users to blink to prove they were alive and not just a
photo. Google hoped that the funny-face technology it patented three years ago would be harder
to crack, since it could ask for any of a number of gestures, forcing an intruder to do a quite a lot
of grimacing or photo-editing in order to illicitly use another’s Android phone. Now, this face
recognition is used for payments. Facial recognition has the potential to be far more convenient
than fingerprint scanning could ever be, which is why there’s a chance that the technology could
eventually become the new standard for paying. But, it needs more than four to five years for
facial recognition to be solely used for financial transactions.
Facial recognition is a category of biometric software that maps an individual's facial
features mathematically and stores the data as a faceprint. The software uses deep learning
algorithms to compare a live capture or digital image to the stored faceprint in order to verify an
individual's identity. High-quality cameras in mobile devices have made facial recognition a
viable option for authentication as well as identification. Apple’s iPhone X, for example,
includes Face ID technology that lets users unlock their phones with a faceprint mapped by the
phone's camera. The phone's software, which is designed with 3-D modeling to resist being
spoofed by photos or masks, captures and compares over 30,000 variables. As of this writing,
Face ID can be used to authenticate purchases with Apple Pay and in the iTunes Store, App Store
and iBooks Store. Apple encrypts and stores faceprint data in the cloud, but authentication takes
place directly on the device. Developers can use Amazon Rekognition, an image analysis service
that's part of the Amazon AI suite, to add facial recognition and analysis features to an
application. Google provides a similar capability with its Google Cloud Vision API. The

3
technology, which uses machine learning to detect, match and identify faces, is being used in a
wide variety of ways, including entertainment and marketing.

The Kinect motion gaming system, for example, uses facial recognition to
differentiate among players. Smart advertisements in airports are now able to identify the gender,
ethnicity and approximate age of a passersby and target the advertisement to the person's
demographic.
Facebook uses facial recognition software to tag individuals in photographs. Each time an
individual is tagged in a photograph, the software stores mapping information about that person’s
facial characteristics. Once enough data has been collected, the software can use that information
to identify a specific individual's face when it appears in a new photograph. To protect people's
privacy, a feature called Photo Review notifies the Facebook member who has been identified.
Currently, there are no laws in the United States that specifically protect an individual's
biometric data. Facial recognition systems are currently being studied or deployed for airport
security and it's estimated that more than half the United States population has already had their
faceprint captured. According the Department of Homeland Security, the only way to avoid
having biometric information collected when traveling internationally is to refrain from
traveling. The General Data Protection Regulation (GDPR) for European Member States does
address biometric data.
HOW IT WORKS:
The software identifies 80 nodal points on a human face. In this context, nodal points are
endpoints used to measure variables of a person’s face, such as the length or width of the nose,
the depth of the eye sockets and the shape of the cheekbones. The system works by capturing
data for nodal points on a digital image of an individual’s face and storing the resulting data as a
faceprint. The faceprint is then used as a basis for comparison with data captured from faces in
an image or video.
Even though the facial recognition system only uses 80 nodal points, it can quickly and
accurately identify target individuals when the conditions are favorable. However, if the
subject’s face is partially obscured or in profile rather than facing forward, or if the light is
insufficient, this type of software is less reliable.

4
Uses of facial recognition technology:
 A research team at Carnegie Mellon has developed a proof-of-concept iPhone app that
can take a picture of an individual and -- within seconds -- return the individual's name,
date of birth and social security number.
 The Google Arts & Culture app uses facial recognition to identify museum
doppelgangers by matching a real person's faceprint with portrait's faceprint.
 Professor Shen Hao of the Communications University of China uses facial recognition
technology to track students’ attendance.
 Amazon, MasterCard and Alibaba have rolled out facial recognition payment methods
commonly referred to as selfie pay.

OPTIMISTIC STARTUP
A company in China is selling service products related to face recognition that can be
applied to huge platforms, like banking security to authorize payments. The developers of the
new app called Face++ are located in Beijing, China. Since then, a number of improvements
have been added to it. One of the most important of them is moving to a completely new
platform with computer vision algorithms at its core. The company is called Face++. The system
seems to work quite easily, just like a well- designed system should do. It is used for paying with
your face. Once a picture is uploaded into the system, your access to buildings, payments
confirmations or identity verification goes immediately in place by itself. The Face++ startup
has a partnership with several commercial banks: Ant Financial and a subsidiary of Alibaba -
Alipay. Many people already use Alipay to make their online payments. Here is how it works:
when users upload their photos to the system, it integrates facial biometric data and tracks up to
83 different key points on the face from various angles at the same time.

5
 Identity verification happens immediately with an extremely high accuracy. This
technology is used in Alipay’s “smile-to-pay”application.

Figure:2.1 Sketch of working of Face++

Face++ payment confirmations can improve the financial security of any person. Together with
an overall image of an individual’s face, this app also captures some specific facial features. The
software can identify faces with 99 percent accuracy. In addition to ID verification, Face++ can
do other things: analysis of age, race, face comparison, gender detection and identification of
emotional expressions. All of that based on just one photo. The app can also monitor people’s
movements inside a room. That makes this technology quite helpful for banks or big stores.
From the Bitcoin and the Block chain points of view, this facial recognition technology can be
helpful too. For example, possible applications include wallet login systems or creating unique
addresses for ICO campaigns. Banks can also be mentioned here, as this technology can be
applied to provide security to authorize payments and also even to track down robbers.

6
 3. NEED FOR NEW TECHNOLOGY
In this modern era, people are looking for convenient ways to make payments. Such a
convenient mode of payment is “paying with your face”. Facial recognition has the potential to
be far more convenient than fingerprint scanning could ever be, which is why there’s a chance
that the technology could eventually become the new standard for paying.
It is convenient because:
It Slims Your Wallet:
If you don’t like carrying around stacks of credit cards and other documents, paying through
your face is a great way to slim down while still having your important information on hand.
Instead, you need not carry your smartphone or any other credit cards, digital wallets, payment
with your face option that allows you to make purchases with any of your accounts from your
face.
Purchases are More Convenient :
While it’s certainly convenient to save space in your pocket, payment with your face is more
convenient than that. In addition to credit card information, they can also store loyalty cards and
coupons. For example, Face++ helps companies convert these options into digital versions to
help customers go paperless. With other apps, payment through your face users can take
advantage of their faces and receive digital coupons based on where they’re shopping.
Budgeting Has Never Been Easier:
With the rise of payment through your face comes the incredible concept of digitizing your
budget, too. No more recording transactions by hand, saving receipts, or manually entering your
expenses and income into computer programs. Now you can link your card or wallet directly to a
budgeting app that does all the work for you.
Time Saving:
Paying with your face is time saving. Customers need not wait for longer time in queues. They
can finish their transactions quickly. Simply scan their purchased goods and pay the amount just
by scanning your face. It takes just 30 sec for completing the transaction.

7
99% Accuracy:
The key to facial recognition software is accuracy. Without supplemental forms of
identification, facial recognition software must be extremely accurate to have any use at all. As
an added level of security, facial recognition software often includes a “liveness test.” To prevent
foul play, users are often asked to turn their head slightly and/or talk while the software analyzes
their face. This stops someone from simply holding up a picture of someone they wish to
impersonate. While there are numerous options available, On Budget is a great model of what
payment through face is capable of. The app works alongside a prepaid card or simply transfer
money onto the card and make purchases with it. The app will automatically do all the
calculations for you just as you do with other digital payment modes. It delivers graphs and other
reports to help you stay on budget. Other similar apps sync with your bank, credit card, PayPal,
and other accounts to help you stay on budget by keeping track of your income and expenses.

8
 4. DETAILS OF PAYING WITH YOUR FACE
4.1 DEFINTION:
From facial recognition ATMs to restaurants that use it for making suggestions, there is a
wide variety of use cases that have been successful – even a toilet that dispenses toilet paper
based on facial recognition. Now, the technology behind face recognition has evolved to a great
extent, and it can now be securely used for financial transactions. It is the very next evolution in
the individual payments space. Newer methods like artificial intelligence and machine learning
are actively employed to identify a person with spot-on accuracy. When people pay with their
face, the payment is generally faster than using cash or a credit card.
4.2 ARCHITECTURE:
In the scope of security and privacy in payments people are using facial recognition
payments. As such no replicas of same face can be created and the whole face can be made. The
user is identified using image information which is processed utilizing facial recognition. The
device verifies that the image information corresponds to a living human using one or more
human-verification processes. The device prompts the user to perform an action to confirm the
transaction, and causes the transaction to be performed after verifying performance of the action
by the identified user. The sensors, combined with neural networks and machine learning
techniques, help to make a mathematical model of your face. This serves as your unique
authentication mechanism to unlock your device. The system will work in the dark by
illuminating your face, and it will adapt to your changing face over time –meaning, you can grow
a beard and it won’t get confused. In order for facial recognition to be solely used for financial
transactions, it would make more than four years considering the current camera and deep
learning technology levels. Face detection occurs first. The algorithms typically cycle through
various boxes, looking for faces with a certain dimension.

Inside those boxes, the system detects facial landmarks and assigns a score, providing a
confidence level regarding whether the image is a face. Once confirmed as a face, the technology

9
creates a template, generally based on factors such as the relative distance between the eyes, the
spot just under the nose and above the lip, and ear to ear .As such 83 points are identified on
face.

Figure 4.1: Recognition points on face

The mathematical representation developed is then compared to other detected faces. The
similarity in ratios between distances on various points of the face, typically focused around
anchors, such as the nose, the eyes, the ears and the mouth, yields a score on a logarithmic scale.
Close matches range from 3 to 5, and definite non matches are less than 1. When the same image
serves as both probe and target, a score of 40+ is possible.
4.3 WORKING:
The data upon which the payment through face recognition system works is no more than
a set of discernable pixel-level patterns for the system, that is, the pattern recognition system
does not perceive meaningful “faces” as a human would understand them. Nevertheless, it is
very important for these systems to be able to locate or detect a face in a field of vision so that it
is only the image pattern of the face (and not the background “noise”) that is processed and
analyzed.

10
Figure 4.2: Working of paying with your face
The way system that is used for payment through face can typically be used for three different
tasks, or combinations of tasks: verification/detection, identification, and watch list (Confirming
the payment). Each of these represents distinctive challenges to the implementation.
Verification
Verification or authentication is the simplest task for a FRS. An individual with a pre-
existing relationship with an institution (and therefore already enrolled in the reference database
or gallery) presents his or her biometric characteristics (face or probe image) to the system,
claiming to be in the reference database or gallery (i.e. claiming to be a legitimate identity). The
system must then attempt to match the probe image with the particular, claimed template in the
reference database. This is a one-to-one matching task since the system does not need to check
every record in the database but only that which corresponds to the claimed identity (using some
form of identifier such as an employee number to access the record in the reference database).
There are two possible outcomes: (1) the person is not recognized or (2) the person is
recognized. If the person is not recognized (i.e., the identity is not verified) it might be because
the person is an imposter (i.e., is making an illegitimate identity claim) or because the system
made a mistake (this mistake is referred to as a false reject). The system may also make a
mistake in accepting a claim when it is in fact false (this is referred to as a false accept).

11
Identification
Identification is a more complex task than verification. In this case, the FRS is provided a
probe image to attempt to match it with a biometric reference in the gallery (or not). This
represents a one-to-many problem. In addition, we need to further differentiate between closed-
set identification problems and open-set identification problems. In a closed-set identification
problem we want to identify a person that we know is in the reference database or gallery (in
other words for any possible identification we want to make we know beforehand that the person
to be identified is in the database). Open-set identification is more complex in that we do not
know in advance whether the person to be identified is or is not in the reference database. The
outcome of these two identification problems will be interpreted differently. If there is no match
in the closed-set identification then we know the system has made a mistake (i.e., identification
has failed (a false negative)). However in the open-set problem we do not know whether the
system made a mistake or whether the identity is simply not in the reference database in the first
instance. Real-world identification applications tend to be open-set identification problems
Watch list
The watch list task is a specific case of an open-set identification task. In the watch list
task, the system determines if the probe image corresponds to a person on the watch list and then
subsequently identifies the person through the match (assuming the identities of the watch list are
known). It is therefore also a one-to-many problem but with an open-set assumption. When a
probe is given to the system, the system compares it with the entire gallery (also known in this
case as the watch list). If any match is above the operating threshold, an alarm will be triggered.
If the top match is identified correctly, then the task was completed successfully. If however the
person in the probe image is not someone in the gallery and the alarm was nonetheless triggered,
then it would be a false alarm (i.e., a false alarm occurs when the top match score for someone
not in the watch list is above the operating threshold). If there is not an alarm then it might be
that the probe is not in the gallery (a true negative) or that the system failed to recognize a person
on the watch list (a false negative).

12
Face Recognition Model
As you can see from the diagram below, face recognition requires a wide range of
technologies:

Figure 4.3: Face Recognition Processing Flow

Face recognition systems in general, and access control systems based on face authentication in
particular, use a “learning” mechanism to collect data on facial characteristics if users. Hence,
the first important point to care about in a face recognition model is the Face Database storing
this information. When the system finishes scanning a video or photo of a user’s face, the
digitalized information will go through these following modules one after another: Face
Detection: locating the face in the photo or video and removing unnecessary details on the
background. Feature Extraction: extracting facial characteristics needed for recognition. Feature
Match: comparing scanned information with database to decide if it matches some user’s face. If
the face matched, the ID of the corresponding is returned.

13
4.4 EXAMPLE
Google’s Hands Free app lets you make payments with your face. Here is how it works:
Google will notify you of a nearby participating store using your phone’s Bluetooth, Wi-Fi and
location data. When you are ready to pay at one of these stores, instead of taking out your wallet,
just tell the cashier “I want to pay with Google.” The cashier will then confirm your identity by
making sure you look like the person logged into the app. They will then ask for your initials,

Figure 4.4: Working of Google pay

4.5 LIVE SCENARIOS

Paying through your face is being currently used in China. A China based online wallet
company “Alipay” has introduced “smile to pay” which allows you to pay through your face.
Nowadays, over 120 million people in China use this app to confirm their payments. A health-
food KFC concept restaurant in Hangzhou, China has installed an Alipay facial recognition
payment system that lets diners pay for their meals with a smile and their mobile phone number.
KFC's KPRO is the first restaurant to use the "Smile to Pay" system designed by Ant Financial
Services Group, an Alibaba Group affiliate.

14
In addition to scanning their faces at an ordering kiosk, KPRO diners must enter their phone
numbers, a safeguard against fraud. “Smile to pay" is the first service of its type, according to
Ant, though it's certainly not the first use of facial
recognition for payment services. Uniqul, a startup allows the users to register with their faces
and creates an account with which the customers pay with their face in the Uniqul points.
France-based Safran demonstrated technology that can use the existing card payment networks to
replace the credit card pin with the more secure and graceful facial recognition. This will
virtually help us get rid of the task of remembering card pins. PayPal is currently testing a new
mobile-payment system based on facial-recognition and geolocation.

15
 5. LIMITATIONS TO THE PROPOSED TECHNOLOGY
The biggest issue is for lower-income people and lower-income countries. These are
countries that typically have less well-developed banking systems, where credit cards are
typically not as available. Now they can bypass credit cards. The financial services offered to
relatively poor people in the U.S. are very expensive—including cashing your paycheck, or
getting a payday loan, or even cashing a government check several. Factors limit the
effectiveness of paying with your face:
Image Quality
Image quality affects how well facial-recognition algorithms work. The image quality of
scanning video is quite low compared with that of a digital camera. Even high-definition video
is, at best, 1080p (progressive scan); usually, it is 720p. These values are equivalent to about
2MP and 0.9MP, respectively, while an inexpensive digital camera attains 15MP. The difference
is quite noticeable.

Image Size
When a face-detection algorithm finds a face in an image or in a still from a video
capture, the relative size of that face compared with the enrolled image size affects how well the
face will be recognized. An already small image size, coupled with a target distant from the
camera, means that the detected face is only 100 to 200 pixels on a side. Further, having to scan
an image for varying face sizes is a processor-intensive activity. Most algorithms allow
specification of a face-size range to help eliminate false positives on detection and speed up
image processing.
Face Angle
The relative angle of the target’s face influences the recognition score profoundly. When
a face is enrolled in the recognition software, usually multiple angles are used (profile, frontal
and 45-degree are common). Anything less than a frontal view affects the algorithm’s capability
to generate a template for the face. The more direct the image (both enrolled and probe image)
and the higher its resolution, the higher the score of any resulting matches.

16
Processing and Storage
Even though high-definition video is quite low in resolution when compared with digital
camera images, it still occupies significant amounts of disk space. Processing every frame of
video is an enormous undertaking, so usually only a fraction (10 percent to 25 percent) is
actually run through a recognition system. To minimize total processing time, agencies can use
clusters of computers. However, adding computers involves considerable data transfer over a
network, which can be bound by input-output restrictions, further limiting processing speed.
Ironically, humans are vastly superior to technology when it comes to facial recognition. But
humans can only look for a few individuals at a time when watching a source video. A computer
can compare many individuals against a database of thousands. As technology improves, higher-
definition cameras will become available. Computer networks will be able to move more data,
and processors will work faster. Facial- recognition algorithms will be better able to pick out
faces from an image and recognize them in a database of enrolled individuals. The simple
mechanisms that defeat today’s algorithms, such as obscuring parts of the face with sunglasses
and masks or changing one’s hairstyle, will be easily overcome. An immediate way to overcome
many of these limitations is to change how images are captured. Using checkpoints, for example,
requires subjects to line up and funnel through a single point. Cameras can then focus on each
person closely, yielding far more useful frontal, higher-resolution probe images. However, wide-
scale implementation increases the number of cameras required. Clearly, privacy concerns
surround this technology and its use. Finding a balance between national security and
individuals’ privacy rights will be the subject of increasing discussion, especially as technology
progresses.

17
 6. CONCLUSION
To sum up, the “paying with your face” system works well in China and people from
other countries would probably join the modern wave soon as it’s rather convenient. This
technology could bring quite some convenience to our life. It could ensure more safety than our
traditional passwords. But at the same time, it could increase the control over individuals. We are
standing at a critical point where we have to make sure this technology is used for good
purposes. We want to see our technological advances to truly enhance our life, to ensure a
brighter future, not a darker one. In the near future, paying with your face can be a more
common and widespread feature for convenience.

18
 7. REFERENCES
1.[Link]
2.[Link]
with-your-face/
3.[Link]
4.[Link]
5.[Link]
6.[Link]
7.[Link]
8.[Link]
[Link]

19