Touchless Touchscreen
Touchless Touchscreen
Bachelor of Technology
In
Computer Science Engineering
Submitted by
M.NITHESH
(21H55A0512)
(Batch: 2020-2024)
2022 - 2023
CERTIFICATE
This is to certify that the technical seminar report entitled “TOUCHLESS TOUCH
SCREEN TECHNOLOGY" being submitted by MADAM NITHESH (21H55A0512) in
partial fulfilment for the award of Bachelor of Technology in COMPUTER SCIENCE
AND ENGINEERING is a record of bonafide work carried out his/her under my guidance
and supervision.
The results embodied in this technical seminar report have not been submitted to any
other University or Institute for the award of any Degree.
ACKNOWLEDGMENT
I would like to thank the Teaching & Non- teaching staff of Department of
Information Technology for their co-operation
MADAM NITHESH
21H55A0512
DECLARATION
I hereby declare that that work which is being presented in this dissertation entitled
“TOUCHLESS TOUCHSCREEN TECHNOLOGY” submitted towards the partial
fulfilment of the requirements for the award of degree of Bachelor of Technology in
Computer Science Engineering from CMR College of Engineering and Technology,
Hyderabad, is an authentic record of my own work carried out under the supervision Mr.
SAIDHULU, Assistant Professor, Department of CSE, CMRCET, Hyderabad.
To the best of our knowledge and belief, this Technical Seminar bears no resemblance
with any report submitted to CMRCET or any other University for the award of any degree or
diploma.
.MADAM NITHESH
21H55A0512
ABSTRACT
It was the touch screens which initially created great furore.Gone are the days when
you have to fiddle with the touch screens and end scratching up. Touch screen displays are
ubiquitous worldwide. Frequent touching a touchscreen display with a pointing device such as
a finger can result in the gradual de-sensitization of the touchscreen to input and can
ultimately lead to failure of the touch-screen. To avoid this a simple user interface for
Touchless control of electrically operated equipment is being developed. EllipticLabs
innovative technology lets you control your gadgets like Computers, MP3 players or mobile
phones without touching them. A simple user interface for Touchless control of electrically
operated equipment. Unlike other systems which depend on distance to the sensor or sensor
selection this system depends on hand and or finger motions, a hand wave in a certain
direction, or a flick of the hand in one area, or holding the hand in one area or pointing with
one finger for example. The device is based on optical pattern recognition using a solid state
optical matrix sensor with a lens to detect hand motions. This sensor is then connected to a
digital image processor, which interprets the patterns of motion and outputs the results as
signals to control fixtures appliances, machinery, or any device controllable through electrical
signals.
CONTENTS
CHAPTER-1
INTRODUCTION TO TOUCHSCREEN
A touchscreen is an important source of input device and output device normally layered
on the top of an electronic visual display of an information processing system. A user can give
input or control the information processing system through simple or multi-touch gestures by
touching the screen with a special stylus and/or one or more fingers. Some touchscreens use
ordinary or specially coated gloves to work while others use a special stylus/pen only. The
user can use the touchscreen to react to what is displayed and to control how it is displayed;
for example, zooming to increase the text size. The touchscreen enables the user to interact
directly with what is displayed, rather than using a mouse, touchpad, or any other intermediate
device (other than astylus, which is optional for most modern touchscreens).
Touchscreens are common in devices such as game consoles, personal computers, tablet
computers, electronic voting machines, point of sale systems, and smartphones. They can also
be attached to computers or, as terminals, to networks. They also play a prominent role in the
design of digital appliances such as personal digital assistants (PDAs) and some e-readers.
Historically, the touchscreen sensor and its accompanying controller-based firmware have
been made available by a wide array of after-market system integrators, and not by display,
chip, or motherboard manufacturers. Display manufacturers and chip manufacturers
worldwide have acknowledged the trend toward acceptance of touchscreens as a highly
desirable user interface component and have begun to integrate touchscreens into the
fundamental design of their products.
1
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
touching object can be calculated. This technology is growing in popularity due to its
scalability, versatility, and affordability for larger touchscreens.
2
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
CHAPTER-2
WORKING OF TOUCHSCREEN
A resistive touchscreen panel comprises several layers, the most important of which are
two thin, transparent electrically resistive layers separated by a thin space. These layers face
each other with a thin gap between. The top screen (the screen that is touched) has a coating
on the underside surface of the screen. Just beneath it is a similar resistive layer on top of its
substrate. One layer has conductive connections along its sides, the other along top and
bottom.
A voltage is applied to one layer, and sensed by the other. When an object, such as a
fingertip or stylus tip, presses down onto the outer surface, the two layers touch to become
connected at that point: The panel then behaves as a pair of voltage dividers, one axis at a
time. By rapidly switching between each layer, the position of a pressure on the screen can be
read.
3
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
Unlike a resistive touchscreen, one cannot use a capacitive touchscreen through most types
ofelectrically insulating material, such as gloves. This disadvantage especially affects
usability in consumer electronics, such as touch tablet PCs and capacitive smartphones in cold
weather. It can be overcome with a special capacitive stylus, or a special-application glove
with an embroidered patch of conductive thread passing through it and contacting the user's
fingertip.
4
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
CHAPTER-3
INTRODUCTION TO TOUCHLESS
TOUCHSCREEN
Touch less control of electrically operated equipment is being developed by Elliptic Labs.
This system depends on hand or finger motions, a hand wave in a certain direction. The
sensor can be placed either on the screen or near the screen. The touchscreen enables the user
to interact directly with what is displayed, rather than using a mouse, touchpad, or any other
intermediate device (other than a stylus, which is optional for most modern
touchscreens).Touchscreens are common in devices such as game consoles, personal
computers, tablet computers, electronic voting machines, point of sale systems ,and
smartphones. They can also be attached to computers or, as terminals, to networks. They also
play a prominent role in the design of digital appliances such as personal digital assistants
(PDAs) and some e-readers.
The touch less touch screen sounds like it would be nice and easy, however after closer
examination it looks like it could be quite a workout. This unique screen is made by
TouchKo, White Electronics Designs, and Groupe 3D.The screen resembles the Nintendo Wii
without the Wii Controller. With the touchless touch screen your hand doesn’t have to come
5
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
in contact with the screen at all, it works by detecting your hand movements in front of it.
This is a pretty unique and inter esting invention, until you break out in a sweat. Now this
technology doesn’t compare to the hologram-like IO2 Technologies Helio display M3, but
thats for anyone that has $18,100 laying around.
You probably wont see this screen in stores any time soon. Everybody loves a touch screen
and when you get a gadget with touch screen the experience is really exhilarating. When the
Iphone was introduced, everyone felt the same. But gradually, the exhilaration started fading.
While using the phone with the finger tip or with the stylus the screen started
getting lots of finger prints and scratches. When we use a screen protector; still dirty marks
over such beautiful glossyscreen is a strict no-no. Same thing happens with I-pod touch. Most
of the time we have to wipe the screen to get a better unobtrusive view of the screen.
Thanks to EllipticLabs innovative technology that lets you control your gadgets like
Computers, MP3 players or mobile phones without touching them. Simply point your finger
in the air towards the device and move it accordingly to control the navigation in the device.
They term this as “Touchless human/machine user interface for 3D navigation”.
6
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
Sure, everybody is doing touchscreen interfaces these days, but this is the first time I’ve seen
a monitor that can respond to gestures without actually having to touch the screen. The
monitor, based on technology from TouchKo was recently demonstrated by White
Electronic Designs and Tactyl Services at the CeBIT show. Designed for applications
where touch may be difficult, such as for doctors who might be wearing surgical gloves, the
display features capacitive sensors that can read movements from up to 15cm away from the
screen. Software can then translate gestures into screen commands.
Touchscreen interfaces are great, but all that touching, like foreplay, can be a little bit of
adrag. Enter the wonder kids from Elliptic Labs , who are hard at work on implementing a
touchless interface . The input method is, well, in thin air. The technology detects motion in
3D and requires no special worn-sensors for operation. By simply pointing at the screen,
users can manipulate the object being displayed in 3D. Details are light on how this actually
functions, but what we do know is this:
It obviously requires a sensor but the sensor is neither hand mounted nor present on the
screen. The sensor can be placed either on the table or near the screen. And the hardware
setup is so compact that it can be fitted into a tiny device like a MP3 player or a mobile
phone. It recognizes the position of an object from as 5 feet.
7
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
CHAPTER-4
4.1-BLOCK DIAGRAM
The system is capable of detecting movements in 3-dimensions without ever having to put
your fingers on the screen. Sensors are mounted around the screen that is being used, by
interacting in the line-of-sight of these sensors the motion is detected and interpreted into
onscreen movements. The device is based on optical pattern recognition using a solid state
optical matrix sensor with a lens to detect hand motions.
This sensor is then connected to a digital image processor, which interprets the patterns of
motion and outputs the results as signals to control fixtures, appliances, machinery, or any
device controllable through electrical signals. You just point at the screen (from as far as 5
feet away), and you can manipulate objects in 3D.It consists of three infrared lasers which
scan a surface. A camera notes when something breaks through the laser line and feed that
information back to the Plex software.
8
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
The Leap Motion controller sensor device that aims to translate hand movements into
computer commands. The controller itself is an eight by three centimeter unit that plugs into
the USB on a computer. Placed face up on surface, the controller senses the area above it and
is sensitive to a range of approximately one meter. To date it has been used primarily in
conjunction with apps developed specifically for the controller. One factor contributing to the
control issues is a lack of given gestures, or meanings for different motion Controls when
using the device, this means that different motion controls will be used in different apps for
the same action, such as selecting an item on the screen. Leap Motion are aware of some of
the interaction issues with their controller, and are planning solutions. This includes the
development of standardized motions for specific actions, and an improved skeletal model of
the hand and fingers.
9
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
A movement of part of the body, especially a hand or the head, to express an idea or
meaning Based graphical user interphase.
A Leap Motion controller was used by two members in conjunction with a laptop and the
Leap Motion software development kit. Initial tests were conducted to establish how the
controller worked and to understand basic interaction. The controller is used to tested for there
cognition of sign language. The finger spelling alphabet was used to test the functionality of
the controller. The alphabet was chosen for the relative simplicity of individual signs, and for
the diverse range of movements involved in the alphabet. The focus of these tests is to
evaluate the capabilities and accuracy of the controller to recognize hand movements. This
capability can now be discussed in terms of the strengths and weaknesses of the controller.
10
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
CHAPTER-5
TOUCHLESS UI
The basic idea described in the patent is that there would be sensors arrayed around the
perimeter of the device capable of sensing finger movements in 3-D space.
Figure 5.1-Touchless UI
UI in their Redmond headquarters and it involves lots of gestures which allow you to take
applications and forward them on to others with simple hand movements. The demos included
the concept of software understanding business processes and helping you work. So after
reading a document - you could just push it off the side of your screen and the system would
know to post it on an intranet and also send a link to a specific group of people.
11
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
5.1-Touch-less SDK
In a nutshell, the Touchless SDK enables touch without touching. Well, Microsoft
OfficeLabs has just released “Touchless,” a webcam-driven multi-touch interface SDK that
enables “touch without touching.” Using the SDK lets developers offer users “a new and
cheap way of experiencing multi-touch capabilities, without the need of expensive hardware
or software. All the user needs is a camera,” to track the multi-colored objects as defined by
the developer. Using the SDK lets developers offer users “a new and cheap way of
experiencing multi-touch capabilities, without the need of expensive hardware or software.
All the use r needs is a camera,” to track the multi-colored objects as defined by the
developer. Just about any webcam will work. Using the SDK lets developers offer users “a
new and cheap way of experiencing multi-touch capabilities, without the need of expensive
hardware or software.
All the user needs is a camera,” to track the multi-colored objects as defined by the developer.
12
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
5.2-Touch-less Demo
The Touch less Demo is an open source application that anyone with a webcam can use to
experience multi-touch, no geekiness required.
The demo was created using the Touch less SDK and Windows Forms with C#. There are
4 fundemos: Snake - where you control a snake with a marker, Defender up to 4 player
version of a pong-like game, Map - where you can rotate, zoom, and move a map using 2
markers, and Draw the marker is used to guess what….draw! Mike demonstrated Touch less
at a recent Office Labs’ Productivity Science Fair where it was voted by attendees as “most
interesting project.” If you wind up using the SDK, one would love to hear what use you
make of it!
13
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
5.3-Touch wall
Touch Wall refers to the touch screen hardware setup itself; the corresponding software to
run Touch Wall, which is built on a standard version of Vista, is called Plex. Touch Wall and
Plex are superficially similar to Microsoft Surface, a multi-touch table computer that was
introduced in 2007 and which recently became commercially available in select AT&T stores.
14
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
software.Early prototypes, say Pratley and Sands, were made, simply, on a cardboard screen.
A projector was used to show the Plex interface on the cardboard, and the system worked
fine. Touch Wall certainly isn’t the first multi-touch product we’ve seen (see iPhone). In
addition to Surface, ofcourse, there are a number of early prototypes emerging in this space
But what Microsoft has done with a few hundred dollars worth of readily available hardware
is stunning. It’s also clear that the only real limit on the screen size is the projector, meaning
that entire walls can easily be turned into a multi touch user interface. Scrap those white
boards in the office, and make every flat surface into a touch display instead. You might even
save some money.
15
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
CHAPTER-6
MINORITY REPORT INSPIRED TOUCHLES TECHNOLOGY
Tobii Rex is an eye-tracking device from Sweden which works with any computer
running on Windows 8. The device has a pair of infrared sensors built in that will
track the user’s eyes.
Elliptic Labs allows you to operate your computer without touching it with the Windows
8Gesture Suite.
16
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
III. AirWritting
Airwriting is a technology that allows you to write text messages or compose emails by
writing in the air.
Figure 6.3-AirWritting
IV. Eyesight
EyeSight is a gesture technology which allows you to navigate through your devices by
just pointing at it.
Figure 6.4-Eyesight
V. MAUZ
MAUZ is a third party device that turns your iPhone into a trackpad or mouse.
17
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
Figure 6.5-MAUZ
Figure6.6-Point Grab
18
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
Figure6.7LeapMotion
VIII.
Myoelectric Armband
Myoelectric armband or MYO armband is a gadget that allows you to control your other
bluetooth enabled devices using your finger or your hands.
19
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
It detects and recognizes a user’s body movement and reproduces it within the video game
that is being played.
20
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
CHAPTER-7
ADVANTAGES
No de-sensitization of screen.
Can be controlled from a distance.
Usefull for physically handicapped people.
Easier and satisfactory experience.
Gesturing and cursor positioning.
No drivers required
21
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
CONCLUSION.
Touchless Technology is still developing. Many Future Aspects. With this in few
years our body can become a input device. The Touch less touch screen user interface
can be usedeffectively in computers, cell phones, webcams and laptops. May be few
years down the line, our body can be transformed into a virtua l mouse, virtual
keyboard ,Our body may be turned in to aninput device. It appears that while the
device has potential, the API supporting the device is notyet ready to interpret the full
range of sign language. At present, the controller can be used withsignificant work for
recognition of basic signs, However it is not appropriate for complex signs,especially
those that require significant face or body contact. As a result of the significantrotation
and line-of sight obstruction of digits during conversational signs become inaccurate
andindistinguishable making the controller (at present) unusable for conversational
However, whenaddressing signs as single entities there is potential for them to be
trained into Artificial Neural Networks.
22
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
FUTURE SCOPE
Many personal computers will likely have similar screens in the near future. But touch
interfaces are nothing new -- witness ATM machines. How about getting completely out of
touch? A startup called LM3Labs says it's working with major computer makers in Japan,
Taiwan and the US to incorporate touch less navigation into their laptops, Called Airstrike;
the system uses tiny charge-coupled device (CCD) cameras integrated into each side of the
keyboard to detect user movements.
You can drag windows around or close them, for instance, by pointing and gesturing in midair
above the keyboard. You should be able to buy an Airstrike-equipped laptop next year, with
high-end stand-alone keyboards to follow.
Any such system is unlikely to replace typing and mousing. But that's not the point. Airstrike
aims to give you an occasional quick break from those activities.
23
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
REFERENCES
24
CMR COLLEGE OF ENGINEERING & TECHNOLOGY
lOMoARcPSD|24870217
25
CMR COLLEGE OF ENGINEERING & TECHNOLOGY