OS design based systematic review of Human Computer

Interaction
Prakash kumar Kishan Kumar
School of Computer Science and Engineering School of Computer Science and Engineering
Lovely Professional University Punjab, Jalandhar Lovely Professional University
144411, India Punjab, Jalandhar 144411, India

Divyakolu Adithya Kirti

School of Computer Science and Engineering [email protected]
Lovely Professional University School of Computer Science and Engineering
Punjab , Jalandhar 144411, India Lovely Professional University
Punjab , Jalandhar 144411, India

Abstract possible with other interface paradigms. Voice

This paper explores the integration of HCI principles user interfaces (VUI) are utilized for voice
in OS design, focusing on enhancing user detection and synthesizing systems. Then a
experience, productivity, and satisfaction. description of current technology and more
In these different techniques, principles are designed recent developments in the subject is given. A
for the interaction of a human and a computer in description of the many architectures used in
such a manner that our expectations are met. HCI designs follows below. The concluding
Additionally, the author can define HCI as the parts provide an overview of certain HCI
area of study where only the approaches, applications and discuss possible future
principles, and techniques are applied to build a developments. Since the interface between
user-friendly interface between people and humans and computers is essential to
computers. Because people are all surrounded facilitating this contact, humans engage with
by many devices that make our jobs easier, HCI computers in a variety of ways.
is crucial in our daily lives. Therefore, HCI is It involves the research, design, implementation, and
the end result of ongoing testing and evaluation of computing systems that are
improvement of interface designs that may have implicated in behavior of human users with
an impact on the context of usage for users. software expert systems on the one hand.[3]

1 INTRODUCTION 1 HUMAN COMPUTER INTERACTION

The interface between humans and computers is
Most complex machines are useless unless men can
essential for supporting the various ways in
utilize them correctly. The major words that
which humans and computers communicate [1].
should be taken into account when designing
HCI may alternatively be referred to as
HCI are simply presented in this straightforward
computer-human interaction (CHI), man-
argument: both usability and functionality.[4]
machine interaction (MMI), or HMI (human-
What a system can do, or more specifically, how its
machine interaction) (CHI). The widely used
functions can contribute to the accomplishment
graphical user interfaces (GUI) of today are
of its objective, can ultimately be used to
found in desktop applications, internet
explain why it was really developed.
browsers, mobile computers, and computer
The range of actions or services that a system offers
kiosks.[2]
its users determines its functionality. On the
The evolving multi-modal and Graphical user
other hand, functionality only has value when it
interfaces (GUI) enable humans to interact with
can be effectively used by the user. The extent
embodied character agents in a way that is not
and scope to which a system with a certain
 feature can be utilised effectively and suitably over the past ten years. The acceleration of
to achieve specific objectives for specific users research and the ongoing innovations in
is the system's usability. When a system's marketing hastily make the new technology
functionality and usability are properly accessible to everyone. Not all of the available
balanced, the system can actually perform as technologies are, however, available to the
intended. general public or inexpensive. An overview of
In light of these ideas and the fact that the terms the technology that is more or less accessible to
"computer," "machine," and "system" are and used by the public is provided in the first
frequently used synonymously in this context, portion of this paragraph. HCI is the study of
human-computer interaction (HCI) is a design how to make machines and computers work
that aims to create a fit between the user, the better for us. Even though it seems
machine, and the necessary services in order to straightforward, this industry is currently
achieve a certain performance in terms of the evolving so quickly and is so fascinating that it
quality and efficiency of the services . The has contributed to the development of some of
criteria for what constitutes an effective HCI our most cherished creations, like virtual reality,
design are largely arbitrary and context-specific. self-driving cars, and all of your favourite
The study of computer technology's use and design touch screen technologies. The type of data
with a particular emphasis on human-computer that HCI professionals gather is used to
interfaces is known as human-computer continuously improve the UX design industry.
interaction. Researchers in human-computer [6]
interaction (HCI) study how people use Not only has the quality of interactions improved in
computers and create new technologies to the Human-Computer Interaction (HCI) sector
enable creative computer usage. Designing user- throughout time, but there have also been
friendly interfaces that people with various numerous branching’s. The diverse research
skills and levels of expertise may easily use fields have placed distinct emphasis on the ideas
requires HCI. Most importantly, groups lacking of multimodality rather than unimodality,
formal training and knowledge on interfacing intelligent adaptive interfaces rather than
with certain computing systems might benefit command/action-oriented ones, and lastly active
from human-computer interaction.[5] rather than passive interfaces. Using computers
A graphics editing program me may not require had always raised the issue of interacting with
the same level of precision as an aviation part them. The ways that people have interacted with
designing tool, for example, in terms of vision computers have evolved significantly over time.
and design of the parts. The numerous sorts of The trip is still ongoing, and new designs for
HCI that are created for the same goal may technologies and systems appear more
differ in design depending on the technology frequently every day.
that is available. One illustration is using HCI aides in our understanding of the
menus, instructions, graphical user interfaces differences between good and terrible software
(GUI), or virtual reality to access any particular products. Unfortunately, there is no surefire
computer's features. A more thorough recipe for producing a successful product. It is
description of the current tools and techniques comparable to product design or architecture in
for interacting with computers as well as recent this regard.
developments in the subject is provided in the
next section.
Although architects and product designers need to
2 DESCRIPTION OF HCI have a solid scientific understanding of the
It is practically hard to distinguish between a materials they use, the effectiveness of their
concept that is fiction and one that is or could be work depends on how creatively they put this
real thanks to the developments made in HCI knowledge to use. Working in a studio with a
 master designer or studying case studies of It demonstrates how various forms of
successful ideas are typically the best ways to interaction can be combined (Multi- Modal
gain this craft skill of creativity.[7] Interaction) and how each form can have its
The major goal of studying human-computer performance enhanced (Intelligent Interaction).
interaction is to create methods that improve The relative human sense that the gadget is built
user contact with computers and make them for essentially categorises the physical
more intuitive. As there is a significant physical technologies for HCI that are now available.
barrier between the user and the computer, the These gadgets primarily rely on the three senses
use of physical devices for human-computer of sight, hearing, and touch.
interaction, such as keyboards and mice, The more advanced techniques that rely on audition
hampers the interface's naturalness and frequently require some sort of speech
intuitiveness. As ubiquitous computing has recognition. These gadgets are far more
grown, user- computer contact is no longer only challenging to construct because they are
restricted to keyboard and mouse operations. An designed to make interaction as easy as
appealing alternative to conventional text-based possible. And it's simpler to construct output
interfaces through graphical user interfaces is auditory apparatus. Now, machines provide a
the direct use of hands as an input device to wide variety of non-speech and voice signals
provide natural human-computer interaction. and messages as output signals. Simple
Developing a reliable hand gesture detection examples include beeps, alarms, and turn-by-
system remains a difficult problem for turn directions from a GPS device.[9]
conventional vision-based techniques, despite Haptic devices are the most expensive and
the enormous market for hand gesture-based challenging to construct. Through touch,
interface design. Therefore, this hand gesture weight, and relative rigidity, these interfaces
recognition system would be a simple and produce sensations in the skin and muscles.
natural way for users to interact with their Typically, haptic gadgets are created for virtual
computers. It can effectively monitor both static reality or applications that support people with
and dynamic hand movements. disabilities. [10] The most recent approaches
and developments in HCI are attempting to
integrate earlier forms of interaction with other
cutting-edge technologies like networking and
3 HCI Technologies Currently Used animation.
Any design of HCI must be mindful of a variety of Wearable technology, wireless technology, and
human behaviour factors and be helpful. In virtual technology are three categories into
opposition to the seeming simplicity of the which these new developments can be divided.
interaction technique itself, the degree of human The boundaries between these new technologies
involvement in a machine interaction can are dissolving so quickly due to advancements
occasionally be quite complex. The level of in technology that they are blending together.
functionality/usability, as well as the financial These include, but are not limited to, personal digital
and economical aspects of the machine on the assistants (PDA), GPS navigation systems,
market, all influence how sophisticated the military super-soldier enhancing devices (such
present interfaces are. For instance, a as thermal vision , tracking other soldiers'
thermostatic on/off switch is sufficient for an movements using GPS, and environmental
electrical kettle's interface because the device's scanning), radio frequency identification
main purpose is to heat water, and adding (RFID) products, and virtual tours for real estate
further features would not be cost-effective. In businesses. Some of these new gadgets
order to give the user a better and simpler improved and incorporated earlier
interface, this paper focuses primarily on communication techniques. As an example,
advances in the physical aspects of interaction. image depicts the Canesta keyboard, a
 keyboarding solution provided by Compaq's with users (Flexible HCI) is a crucial aspect of
iPAQ, as a possible option. A QWERTY-like the current generation of interfaces. Intelligence
pattern is projected onto a solid surface using a HCI designs are user interfaces that include at
red light to create a virtual keyboard. A motion least a minimal level of intelligence in user
sensor on the device then attempts to monitor response and/or perception. A couple of
the user's finger movements while they type on examples include speech-enabled interfaces that
the surface, sending the keystrokes back to the communicate with users using natural language
device. and tools that visibly track users' eye and
4.1 Modern HCI Advancements movement motions.
The following sections discuss recent developments For example we can consider, the most notable
and directions in HCI research, including difference between intelligent and non-intelligent
intelligent and customizable interfaces and HCI solutions is that the latter tend to really be
ubiquitous computing. These user interfaces active interfaces, whilst the former only react
necessarily involve varying degrees of when the user calls for them to. One illustration
affectionate, cognitive, and physical exertion. is clever billboards or advertisements that
display themselves according to the preferences.
4.2 HCI that is intelligent and adaptive
Although the majority of the public still uses simple 4.3 Ambient Intelligence and Pervasive
command/action setups with relatively simple Computing
physical gear, research is moving in the Unmistakably, ubiquitous computing is the focus of
direction of designing intelligent and flexible the most recent HCI research (Ubicomp). The
interfaces. It is unknown or, at the very least, term, which is frequently used synonymously
not widely accepted what the determined the with ambient intelligence and embedded
significant meaning of intellect or being smart devices, refers to the most advanced form of
is. However, one can define these ideas by human-computer interaction, which involves
looking at how clearly the functionality and getting rid of the desktop and integrating the
usefulness of new technologies on the market computer into the environment so that it
have grown and improved. The creation of HCI encircles people everywhere and is invisible to
designs that provide users a simpler, more them. Hence, the term "ambient."[13]
enjoyable, and fulfilling experience is Mark Weiser first proposed the concept of
economically and technologically essential. The ubiquitous computing in 1998 while serving as
interfaces are becoming more user-friendly the head technologist at the Xerox PARC
every day in order to achieve this goal.[12] Computer Science Lab. His plan was to
Differentiating between employing intelligence in integrate computers into everything around us,
the creation of the interaction (Intelligent HCI) including common things, so that we could
and in the manner in which the interface interact with several computers at once while
engages between users (Adaptive HCI) is a they were invisible to us and wirelessly talking
crucial aspect of the current generation of with one another.
interfaces. Smart HCI designs are user
interfaces that include at a minimum level of 5 Systems Architecture for HCI
intelligence in user response and/or perception. The configuration of a Systems engineering is its
Examples include voice-activated user most crucial component. In actuality, the
interfaces that communicate with users using quantity and variety all inputs and outputs that a
everyday language and tools that detect a user's given interface offers serve as its primary
eye or movement and react accordingly. defining characteristics. An HCI system's
Differentiating between employing intelligence in architecture outlines these output signals as well
the creation of the interface (Smart HCI) or in as how they interact.
the manner in which the interface communicates Another aspect of intelligent interfaces to take into
 account is the fact that most non-intelligent HCI application, a clear overview of each area may
designs are passive in nature, responding only be drawn. In general, facial expression analysis
when the user calls for them to, whereas the focuses on recognising emotions visually . Body
most intelligent and adaptive interfaces tend to movement tracking and gesture detection are
be active interfaces. Smart billboards or adverts often the main areas of focus in this field.
that show themselves in a way that appeals to Although they can serve a variety of functions,
people' tastes are an example. The use of a they are typically utilised for direct human-
variety of HCI techniques in combination and computer interaction in command-and- action
how they might contribute to the creation of scenarios.[16] The primary function of gaze
intuitive, adaptive natural user interfaces are detection is to better comprehend a user's
covered in the following section.[14] attention, intent, or concentration in instances
The settings and designs that an interface is where it is important to take context into
based on are described in detail in the following account The exception to this rule is eye
sections. tracking technology used to assist people with
disabilities, in which eye monitoring is a key
5.1 Systems for Unimodal HCI component of command and action scenarios,
As was already mentioned, an interface primarily such as pointer movement and clicking with
depends on the quantity and variety of its inputs blinking . It is noteworthy that several
and outputs, which serve as communication researchers attempted to use visual approaches
channels and allow people to communicate with to supplement or even completely replace other
computers through this interface. A modality is types of interactions (audio- and sensor-based).
any of the various independent single channels . For instance , speech recognition mistake
Unimodal systems are those that have just one correction is known to be influenced by lip
modality as their foundation.[15] reading or lip movement tracking.
Different modalities can be categorized into three Wearable smart devices, such as smart lenses, use
groups according to their nature:
spectacles to display information, and gaze
1. Based on images
motions can be used as an interface for
2. Based on audio
navigation and other user-specified tasks.
3. Based on Sensors
Additionally, this technique has excellent eye-
The following subsections give examples and
tracking accuracy and can be applied as a
references for each modality as well as
drawing tool in a virtual world. These can be
descriptions of each category.
used to identify the user's drowsiness and, if
desired, can issue a warning. The ability to
5.1.1 Based on images HCI
identify which product has captured the viewer's
The most popular topic of HCI study is undoubtedly
attention while they are viewing an
the visual-based human-computer interaction.
advertisement is another situation in which user
Researchers attempted to address many facets of
eye tracking is crucial.[17]
human responses that can be identified as a
visual signal while taking into account the
5.1.2 Based on audio HCI
breadth of applications and range of open
Another crucial aspect of HCI systems involves
challenges and methodologies. The following
audio-based human-computer interaction. This
list includes some of the major study areas in
section deals with data collected by various
this section:
audio signals. Although the nature of auditory
• Gaze Detection
signals might not be as unpredictable as that of
• Gesture Recognition
visual signals, the information gleaned from
• Large-scale Body Movement Tracking
them can sometimes be more reliable, practical,
• Based On facial Analysis (Eyes Movement and even unique.[18] The following sections
Tracking) can be used to categorise the research fields in
Although the objectives of each category vary due to
 this section: and pressure sensors.
• Speaker identification, auditory emotion
analysis, and speech recognition 5.2 Systems for Multimodal HCI
• Noise and sign detections made by humans Multimodal refers to the blending of many
(Gasp, Sigh, Laugh, Cry, etc.) modalities. These modalities in MMHCI
In the past, researchers have mostly concentrated on systems primarily refer to the communication
voice activation and speaker identification . The channels through which the system is connected
attempts in analysing sentiments in audio to inputs . These channels' definitions are
signals were sparked by recent initiatives to derived from human communication kinds,
incorporate human emotions in smart human which are essentially his senses of sight,
computer interaction. Standard human auditory hearing, feel, smell, and taste. These sorts of
cues like sighs, gasps, and other sounds enabled engagement are possible with machines, but
emotion analysis for creating more intelligent they are not the only ones.[20]
HCI systems in addition to the pitch and pitch As a result, a multimodal interface facilitates
of speech data. A very recent field of human- human-computer interaction using two or more
computer interaction (HCI) that has applications input methods other than the standard keyboard
in the art business is music generation and and mouse. One multimodal system may differ
interaction. significantly from another in terms of the
precise number of allowed input modes, their
5.1.3 Based on sensor HCI nature, and how they interact. Possible
This section combines a wide range of topics with variations of voice, gesture, gaze, facial
several applications. These many regions have expressions, and other unconventional means of
one thing in common: the interaction between input are included in multimodal interfaces.
the user and the machine is provided by at least Gesture and speech input is one of the input
one physical sensor. These sensors, as seen method combinations that is most frequently
below, might be quite simple or extremely supported. The practical limits and unresolved
complex.[19] issues in each modality oppose restrictions on
1. Pencil-Based Communication the fusion of various modalities, despite the fact
2. A keyboard and mouse that an ideal modal HCI system should have a
3. Joysticks mixture of single modalities that interact
4. Motion Tracking Digitizers and Sensor correlatively. Despite all the advancements in
5. Haptic Sensors MMHCI, the majority of multimodal systems
still treat the individual modalities
6. Pressure sensors independently and only aggregate their results at
7. Smell-and-Taste Sensors the very end.
Some of these instruments are relatively new The rationale is that there is still to be done to obtain
technology, while others have existed for a a trustworthy tool for each sub-sector because
while. Pen-based sensors are particularly the open challenges in each field have not yet
relevant to mobile devices since they relate to been solved. In addition, the scientific
handwriting and gesture recognition. Modern responsibilities of various modalities and their
technology called motion tracking sensors and contributions to interaction are unknown.
digitizers has completely changed the video "However, humans communicate through
gaming, animation, and film industries. They multiple modes in redundant and
take the shape of joint sensors or wearable complimentary ways. The signals cannot be
clothing, and they have greatly improved the viewed as mutually independent and cannot be
ability of both computers and people to interact merged in a context-free fashion at the
with the real world. Industries in computers and conclusion of a intended analysis in order to
virtual reality are especially interested in haptic achieve a multimodal evaluation of various
 incoming signal acquired by different sensors.
Instead, the input data must be processed in a References
mutual subspace and in accordance with a 1. Huang, K. Y. (2009, October). Challenges
context-dependent model. in human-computer interaction design for
mobile devices. In Proceedings of the World
6. Applications of HCI Congress on Engineering and Computer
 Multimodal Systems for People with Science (Vol. 1, No. 6, pp. 1-6).
Disabilities
2. H. Sharp, Y. Rogers, and J. Preece,
 Multimodal Systems for Emotion
Recognition Interaction design: Beyond humancomputer
 Multimodal Applications Based on Maps interaction, 2nd ed. New York: John Wiley
 Medicine with Multi-Modal HCI & Sons, 2007.
 And many more 3. A. Sears and J. A. Jacko, Eds., The human-
computer interaction handbook:
Conclusion Fundamentals, evolving technologies and
A crucial component of systems design is human- emerging applications, 2nd ed. Boca Raton,
computer interaction. The user's representation FL: CRC Press, 2007.
and use of the system will determine how good 4. L. Wang and A. S. M. Sajeev, “Roller
it is. Better HCI designs have so received a interface for mobile device applications,” in
great deal of attention. The goal of the new Proceedings of the Eight Australasian
research path is to replace conventional regular Conference on User Interface, 2007, pp. 7–
techniques of contact with multimodal, 13.
adaptive, intelligent, and natural ones. The 5. E. T. Velde, D. E. Atsma, R. Hoekema, J. E.
Third Wave, also known as context - awareness Luijten, C. I. Buddelmeijer, H. J. Spruijt,
or ubiquitous computing, aims to blend and N. H. J. J. Putten, “A multicenter PDA
technology into the surroundings so that it project to support the clinical decision
appears more natural and unnoticeable. Another process,” in Proceedings of the IEEE
developing area of HCI that has the potential to Conference on Computers in Cardiology,
become the common protocol of the future is 2004, pp. 177–179.
virtual reality. Through a thorough reference 6. Dittmar, Anke, Toralf Hübner, and Peter
list, this paper tried to provide an overview of Forbrig. "HOPS: a prototypical specification
these difficulties as well as a survey of the tool for interactive systems." Interactive
available research. For most Americans and Systems. Design, Specification, and
countless of other people across the world, Verification: 15th International Workshop,
human contact has evolved into a necessary DSV-IS 2008 Kingston, Canada, July 16-18,
component of daily life. Studying these 2008 Revised Papers 15. Springer Berlin
websites with an eye toward user interaction has Heidelberg, 2008.
demonstrated the necessity of web designers 7. Gerlach, James H., and Feng-Yang Kuo.
putting a strong emphasis on human-computer "Understanding human-computer interaction
interaction when creating websites. However, for information systems design." MIS
their focus cannot be on enhancing interactions quarterly (1991): 527-549.
for the typical user; rather, it should be on
8. Klasnja, Predrag, Sunny Consolvo, and
enhancing interactions for all potential website
Wanda Pratt. "How to evaluate technologies
users. According to the websites we've analysed
for health behavior change in HCI
below, which are all well-designed for the
research." Proceedings of the SIGCHI
average user, there is still plenty that can be
conference on human factors in computing
done to make these websites simple to use for
systems. 2011.
everyone.
9. Bødker, Susanne. "Third-wave HCI, 10
 years later---participation and sensor-based system design: A
sharing." interactions 22.5 (2015): 24-31. survey." IEEE Transactions on Multi-Scale
10. Bødker, Susanne. "When second wave HCI Computing Systems 3.2 (2017): 124-138.
meets third wave challenges." Proceedings 20. Baig, Muhammad Zeeshan, and Manolya
of the 4th Nordic conference on Human- Kavakli. "Multimodal systems: taxonomy,
computer interaction: changing roles. 2006. methods, and challenges." arXiv preprint
11. Hernández, Benjamín, and Adriana arXiv:2006.03813 (2020).
Alvarado. "Using Modern Interaction
Devices for HCI and Interaction Design
Courses." Eurographics (Education
Papers). 2014.
12. Hou, Ming, Simon Banbury, and Catherine
Burns. Intelligent adaptive systems: An
interaction-centered design perspective.
CRC Press, 2014.
13. Wren, Christopher R., and Yuri A. Ivanov.
"Ambient intelligence as the bridge to the
future of pervasive computing." 2008 8th
IEEE International Conference on
Automatic Face & Gesture Recognition.
IEEE, 2008.
14. Alavi, Hamed S., et al. "Introduction to
human-building interaction (hbi) interfacing
hci with architecture and urban
design." ACM Transactions on Computer-
Human Interaction (TOCHI) 26.2 (2019): 1-
10.
15. Baig, Muhammad Zeeshan, and Manolya
Kavakli. "Multimodal systems: taxonomy,
methods, and challenges." arXiv preprint
arXiv:2006.03813 (2020).
16. Andersen, Martin. "Evaluating and
improving the user-interface of existing
GIS-applications by the use of HCI
evaluation techniques." (2015).
17. Costanza, Enrico. Designable Visual
Markers for Mobile Human-Computer
Interaction. Diss. École Polytechnique
Fédérale de Lausanne, 2010.
18. Hailpern, Joshua, et al. "A3: Hci coding
guideline for research using video
annotation to assess behavior of nonverbal
subjects with computer-based
intervention." ACM Transactions on
Accessible Computing (TACCESS) 2.2
(2009): 1-29.
19. Mosenia, Arsalan, et al. "Wearable medical