Bath Spa University, January 2021, Bath, UK Nishesh Jaiswal

An investigation into the Efficiency and Utility of Voice User

Interfaces Siri and Google Assistant for smartphones.
Nishesh Jaiswal
Bath Spa University
Bath, England

ABSTRACT
This paper’s primary objective is to focus on User Interfaces
of two Conversational UXs by comparing the two usability
criteria; efficiency and utility for iOS and Android smartphone
participants. A two-stage study has been conducted consisting
of a questionnaire and post-session interview, using Siri and
Google Assistant as the Conversational UXs with their respec-
tive VUIs in question. The second part of the questionnaire
consisted of certain tasks which the participants had to per- Figure 1. History of the VUI and how it progressed. [6]
form. Two groups collectively totalling 10 participants have
been selected for an interview; one group being iOS partici-
pants and the other group being Android participants with a
mixture of both English and non-native English speakers, and
thus ensuring that diverse results are achieved. Approximately VUIs. The primary advantage of a VUI is that it allows for
4 hours of study data have been collected and analysed for a hands-free, eyes-free way in which participants can inter-
the differences observed. There are two hypotheses to this act with a product while focusing their attention elsewhere
study: the UI efficiency is expected to show iOS participants [5]. Some examples are Siri in Apple smartphones, Google
to have less intrusive VUI interface, and iOS smartphones are Assistant in Android smartphones, Alexa in Amazon devices,
expected to have a better the VUI utility when it comes to task Bixby in Samsung devices. These interfaces are divided into
completion. The key findings confirm 1 of those hypotheses three elements: prompts, grammars and dialogue logic [12].
and reject 1. The study confirms the correlations between all participants are able to give voice commands and in return
the variables mentioned above. get a response and thus are able to control with the help of
VUI for example in Apple smartphones, Siri can be given a
Author Keywords command, and it responds with a reaction. The elements can
Voice User Interface; efficiency; utility; Siri; Google be detailed as Prompts which are shown to the participant as
Assistant; Task completion. a dialogue message which may or may not be a part of a con-
versation, grammars are participant’s responses to the prompt
CCS Concepts and the dialogue logic is the action taken by the system (here
•Computing Methodologies ! Speech Recognition; smartphone) to execute the participant’s request. To achieve
•Human-centered computing ! Human Computer this type of interaction, voice-based interfaces use automatic
Interaction (HCI); speech recognition (ASR), natural language processing (NLP)
- including natural language understanding (NLU), pattern-
INTRODUCTION action mapping and natural language generation (NLG), and
A Conversational UX is a participant experience that combines text-to-speech (TTS) technology [10].
chat, voice or any other natural language-based technology to VUI with great Conversational UX has become very popular
mimic a human conversation. [13] It is generally a visual inter- and pervasive (Google reports that 27 percent of the online
face commonly called as a VUI (Voice User Interface). Voice global population is using voice search on mobile. [8]). The de-
User Interfaces (VUIs) allow the participant to interact with a velopment of VUI can be dated back to 1952 where engineers
system through voice or speech commands. Virtual assistants, are Bell Labs developed an automatic digit recognizer called
such as Siri, Google Assistant, and Alexa, are examples of Audrey. It could recognize numbers from 0-9 by perform-
Permission to make digital or hard copies of all or part of this work for personal or ing pattern matching on predefined classes of voices [2][6].
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
Then, after a few years, The first-gen VUI was introduced by
on the first page. Copyrights for components of this work owned by others than the SpeechWorks and Nuance in 1984 through Interactive Voice
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or Response (IVR) systems. These IVRs were able to recognize
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from [email protected]. human voice over call and performed the tasks given to them.
CHI’20, April 25–30, 2020, Honolulu, HI, USA [2][6]. Although the VUI was developed later, IBM in 1961,
© 2020 Copyright held by the owner/author(s). Publication rights licensed to ACM. introduced the first digital speech recognition tool, the IBM
ISBN 978-1-4503-6708-0/20/04. . . $15.00 Shoebox [7]. In 2002, Microsoft integrated speech recogni-
DOI: https://doi.org/10.1145/3313831.XXXXXXX
Bath Spa University, January 2021, Bath, UK Nishesh Jaiswal

Figure 3. Chart showing where Siri performs better and where Google
assistant performs better [9]

Figure 2. Chart of current hold of VUI in the US. [1]

tion technology into its office products [14]. In 2011, Siri was Figure 4. Spectogram of human voice pronouncing the vowels. [10]
introduced by Apple being the first modern smartphone VUI
[14]. In the modern world, VUIs with Conversational UXs are
available in many digital devices ranging from smartphones,
smart speakers and even cars [3]. With the digitization of the To examine which VUI has a better grasp at efficiency and
world, not only the pervasiveness but also the importance of utility aspects of HCI, the study in this research uses a sam-
VUIs and conversational UXs has been increasing with even ple of two participant groups: iOS participants and Android
possible plans to use them in AR and VR. Nonetheless, as participants. The two usability criteria chosen to compare the
the VUIs regular expansion and improvements continue, there selected interfaces are efficiency and utility. In this research
is one area which deserves some exploration which is task "efficiency" is measured as the resources expended by the
completion and intrusiveness, which will form the main focus participant in relation to the accuracy and completeness of
of this research. This topic has been selected to explore the po- goals achieved [4]. The "utility" is defined as a measure of
tential differences in the selected VUIs usability and efficiency the feature provision based on participant’s needs [16, 15] –
criteria based on the aspects of task completion. The two VUIs the more features fulfilling the participant’s needs the program
chosen are Apple’s Siri and Google’s Google Assistant due provides, the higher the utility level. The research explores
to the following conditions: first, those VUIs were chosen two hypotheses. Firstly, the VUI efficiency is expected to
based on their mobile accessibility, as the experiment section be better on iOS as compared to Android. Lastly, the VUI
within this research utilized an iPhone with a built-in Siri and task completion efficiency is expected to be better on an iOS
Android Smartphones with built-in Google Assistant. These Smartphone than an android smartphone. The objective is to
criteria could have been met by other VUIs such as Alexa, examine the potential gap in VUI efficiency in design usability
Bixby or Cortana; however, Siri was selected because it was for iOS and Android smartphone participants by conducting a
the first VUI to be introduced in a smartphone and Google 2 staged study following a single-method approach.
Assistant was selected because of how much hold the android
operating system has over the smartphone market which is RELATED WORK
72.48 percent [21].
In order to explore VUI effectiveness and utility, two research
MNCs like Google and Apple are exploring and expanding areas need to be investigated: the way VUIs work and the VUI
their VUI interface, language, and accent support. However, design process.
multiple language support is not present on a lot of their ser- To understand VUI usability and its working of how it is able
vices or devices. For example, Google Home supports 44 lan- to understand the human language, we need to understand
guages on smartphones. Apple’s Siri supports 21 languages. the participant input and how human speech works. When
[11]. A wide language and dialect support is considered as an
humans speak, they create two sounds, namely "voiced" and
essential aspect as this is what limits companies creating VUIs
"unvoiced". It depends on whether our vocal cords are vibrat-
with global outreach [11]. The VUIs popularity is directly
ing or not while speaking. Whereas human speech consists of
affected by the language and the accent support offered by
audio segments called "phones" having characteristic frequen-
the same, which depends from one geographical location to cies called "Formants". Formants are used to identify vowels
another, which in turn directly affects the task completion rate. as they have a distinctive sound, obtained by using FFT. It can
To improve their language support, Google in 2018 launched be seen the figure 4.
multilingual support for the Google Assistant allowing mul-
tilingual participants to switch between the used languages Humans create "voiced", and "unvoiced" sounds when speak-
without having to change the language support settings [20]. ing. This is determined by whether or not the vocal cords
Bath Spa University, January 2021, Bath, UK Nishesh Jaiswal

Figure 5. The ASR pipeline. [10]

Figure 7. prompts and responses [18]

deals with if we actually need to have an interface for the

output to be seen. It explores the possibility that the task
can be achieved through voice interaction and if an interface
interaction will be required. The second step is to compile
Figure 6. The Design Process for Conversational Interface. [19] conversation, which can be related to making an information
architecture for the VUI, but the only drawback is that the
participant cannot go back to the previous interaction. The
are vibrating when producing the sound. Human speech con- next step involves exploring possibilities where brainstorming
sists of audio segments called "phones" with characteristic ideas around conversations is required. The last step involves
frequencies called "Formants". The latter can be used to iden- model interactions where, instead of using hotspots, the VUI’s
tify vowels, which in the English language are voiced sounds. language is used to create interactions for the output to bee
As Figure 4 shows, each vowel has several unique Formant seen. [19]
frequencies [10]. Every VUI is built on a certain set of design principles, and
In addition to Formants, human speech also consists of we will be discussing them in detail now. There are four
"phonemes", which are defined as those small parts of any sets of fundamental design principles around which the con-
word which help distinguish one word from another if changed. versational UX of a VUI is built. The first is talking to the
For example, /d/ and /h/ in the words “den” and “hen”. These participant, which involves prompts and statements [18]. Al-
two words are differentiated by a single phonological element, though for a developer, the output is similar in terms of code,
here, in this case, is a phoneme, and thus forms a basic pair. the design needs to be different for both the prompt and the
[10] code. These two form the basis of every human conversation,
with a statement being spoken, and the response is in the form
ASRs are used by VUI to digitise the speech input received of a prompt for the conversation to continue.
(phones, formants and gaps that consist of "utterances,") by
breaking them into smaller pieces and applying machine learn- It can be seen in figure 7, which is a prompt and a statement.
ing to identify those smaller pieces and produce the output This generally starts with an invocation phrase to wake up the
into probable words that is most likely being spoken. VUI and is followed by a statement and results in a prompt.
Prompts have a unique property called "re-prompt", which is
Next we discuss the design process for VUI. From figure 6, we when there is no input detected, the VUI issues a re-prompt
can see that when the speech enters the front end device, the which asks for the input again. Next step involves listening to
job of the front end device is to convert that analogue signal to the participant. Here the participant has his turn to speak, and
a digital waveform using signal processing. It also maximises whatever is spoken is transformed into an intent. Intents are of
the signal to noise ratio. This process is carried out in small two types, the local intent and the global intent. Local intents
"chunks" of 10 milliseconds consisting of about 5 frames. show the participant inputs which are expected as a response
These features are fed from the front end to the decoder. The to a prompt. To a question like "are you going?", a Yes or
decoder or the back end consists of three different components, No intent will be expected. In this case, Yes and No are local
namely, acoustic model, lexicon and language model. The intents. The follow-up to a local intent is straightforward: it
acoustic model selects the most probable phones from speech is the expected next step in the conversation. [18] participant
databases and selects those string of phones which are then inputs that should be managed at any point in the conversation
matched using the lexicon. The language model takes the are called Global intents. Intents like Help, Cancel and Stop
words which are most likely to make sense and outputs it. that VUI requires every skill to handle are included. Never-
theless, following a global intent is tricky as it requires the
Nevertheless, for the speech to be outputted, the VUI needs
design to back into the conversation. The third step consists
a User Interface which is able to show the output data in a
presentable manner. This is where the design of the VUI of Fallbacks, Interjections Segues. The fallback can be ex-
comes in. plained as when none of the input matches the intent which
the application is listening to. An example can "Sorry I could
As shown in figure 6, the design of a VUI consists of four not catch that". More advanced Fallbacks can be triggered on
different stages, namely, Identify value, Compile conversation, the second and third try as well. Interjections are treated as
Explore possibilities and model interaction. Identify value global intents. They are present when the intents do not go
Bath Spa University, January 2021, Bath, UK Nishesh Jaiswal

as per the path required but add to the personality of the VUI. followed by questions. All the tasks were the same for all the
They are generally used to fill in vague questions and continue participants.
the conversation or get back into the conversation. Moving
on to the next step, we come onto the states and conditional This study used a participant size of 10, with both iOS and
branching. These are used to personalise the experience of Android smartphone participants. To get the answers for the
the VUI. States allow to keep track of what the participants research questions and explore the possible differences in VUI
say and do, generally ranging from questions like yes or no, efficiency and utility for iOS and Android participants the
whereas conditional branching is the ability to make the condi- participant size was split into two equal groups of 5, with one
tions or decisions based on what state it was in [18]. We will group consisting on iOS participants. In contrast, the other
therefore now discuss in brief about the two VUIs. group consisted of android participants.
The single method system research methods used for this
Siri study included a survey and a post survey interview. During
In 2011, when iPhone 4s was released, it was also launched the pre-study stage, participants were required to fill out the
with their own VUI called "Siri". They called it as your own survey which featured questions regarding which operating
personal assistant. In short, Siri is a voice-controlled personal system they had on their smartphones, if they were aware of
assistant for iOS participants. Siri is designed in a way that the the VUI in their phones, how often they use it and if they
participant can seamlessly interact with the phone by speaking have encountered any problems while using it, and if they can
to it, and it will speak back with a response. Accessing Siri is access it easily. The answers to these questions were used to
easy, by either pressing the power button for a short amount of review the demographics of the sample group.The answers to
time until the UI pops up on the screen or by just saying "Hey these questions about prior knowledge about VUI was used to
Siri" which acts as a trigger for the VUI to wake up. This also understand the participant’s background and familiarity with
depends on which model of the phone you are having, saying both the VUIs in general as well as with Siri and Google
that if it is an old model, pressing the home button instead Assistant. It was also used to get an insight into what kind
of the power button will activate Siri. The best part of Siri of tasks do they generally assign the VUIs with and what
is how seamlessly it connects across all your devices in the problems do they encounter. Following this the participants
ecosystem. Siri has access to all the apps in your phone and were asked to perform certain tasks on their respective phones
can be tasked to perform any of the tasks like "sending a mail, by using voice commands on Siri and Google Assistant.
playing music, calling someone" etc. It has multiple languages
and accent support, as well. [17] The task sets used in this stage focused on gathering data on the
VUIs efficiency and utility. This formed the Similarly, in the
Google Assistant post-study stage the participants attended an interview which
Google Assistant is Google’s answer to Apple’s Siri as their included questions concerning the effectiveness and utility of
own VUI for android smartphones. Initially, when it was the used VUIs as well as participants’ prior experience with
launched in 2016, it was just an extension for google now VUIs in general. For each participant, the first activity of the
with basic voice controls, with google now pulling out only pre-study stage was the initial part of the survey. The survey’s
the relevant information that was needed for you. Although main purpose was to determine the participant’s technological
Google now has been dead for years, it transitioned into google background and ensure each participant chosen had met all
assistant with even better functionality, including voice-based the necessary conditions of the efficiency and utility knowl-
feedback and text-based feedback as needed by the participant. edge of the VUI. The answers to the questions regarding prior
There are hosts of functions that the google assistant can per- knowledge on the VUIs were used to understand participants’
form like "control devices in your smart home, send emails, background and familiarity with VUIs in general as well as
call someone, open apps, read notifications etc." The one thing Siri and Google Assistant. A lack of previous knowledge and
that makes google assistant stand out from the rest of the VUI handling experience with VUIs could cause a higher VUI task
is that it is available on all the leading mobile OS platforms. It completion error rate due to the participant’s unfamiliarity
can be accessed by the long press of the Home button or by on how to use the interface. To ensure that all participants
saying the trigger "hey google" to activate it. [22] were on the same page a brief introduction was given to each
participant about what is VUI and how to access it. In case
METHOD any participant would still fail to perform a question, he/she
The research was conducted using a single method system was free to ask questions to clear the doubt.
approach consisting of survey and observing the data col-
lected. Thus this can be implied that the methods that will During the experiment participants were required to perform
be employed to collect the data will be both quantitative and sets of pre-defined tasks by giving voice commands to two sep-
qualitative, with a focus on qualitative data more. This study arate Voice User Interfaces. Siri and Google Assistant were the
has been divided into two parts: the pre-study, in-study and provided VUIs for the participants to use in this research, util-
post-study stages (these are two combined stages into one). ising an iPhone with built-in Siri and an Android smartphone
The pre-study stage focused more on collecting quantitative with Google Assistant built in throughout the experiment. The
data to get data for the research. In contrast, the in-study and two task sets comprised of a utility task set (with commands
post-study stage was focused more on collecting qualitative such as ‘Perform a calculation’ or ‘Play a song’) and an effi-
data to review the earlier stage results. The stages were com- ciency task set including commands like ’deliberately muffling
pleted by all the participants, which included a set of tasks your voice to perform a certain action’ or ’giving commands
Bath Spa University, January 2021, Bath, UK Nishesh Jaiswal

in a row’. The utility task set comprised of 6 questions and Survey Results for OS
requests, while the efficiency task set included 2 commands OS India UK USA
and requests, with one task being common for both the cri-
terion. The data was collected in the form of screenshots or iOS 1 3 1
consecutive screenshots if needed. The experiment has been Android 5 0 0
conducted in order to investigate the role of the participant Table 1. Table showing the values of where people’s geographical loca-
tion who were the part of the survey.
smartphone VUI type (iOS or android smartphone) on the
effectiveness and utility of the chosen VUIs. Moreover, the
utility and Efficiency rate is expected to differ between the iOS
and Android participant groups. The iOS participant group they have encountered problems while using the VUI whereas
is expected to receive higher utility and effectiveness than the remaining 7 said (70%)they were comfortable with the
the Android participant group. The latter part of the survey UI and had no problems with it whatsoever. The mean of the
is paraphrased in order to collect more qualitative and some participants who said they rarely used the VUI was 3, who
quantitative data about the participant’s self evaluated utility used 2-3 times the mean was 1.5 and the mean for participants
and efficiency and explore the reasons behind the results of who used it sometimes was 0.5. The standard deviation for the
the experiments that were achieved.The interview consisted of same order was 1.4, 0.7 and 0.7. This can be seen in the Table
18 questions in total which was divided into 3 categories. First 2.
one was general questions asking the participants about their
familiarity with VUIs. The second part consisted of them to Table 2. First task results
give commands to the VUI and record the responses in form Rarely 2-3 times sometimes
of screenshots.The third and last part included post survey Valid 2 2 2
questions. The second part where the participants were asked Missing 0 0 0
to give commands to the VUI consisted of mixed commands Mean 3.000 1.500 0.500
which included both the utility and efficiency testing param- Median 3.000 1.500 0.500
eters. This was also present in the third part where the post Mode 2.000 1.000 0.000
survey questions were designed in a specific way where the Std. Deviation 1.414 0.707 0.707
participant’s responses could be judged for what that felt how
the VUI performed in terms of both utility and efficiency. The
questions that were asked later were based on their experience Second task results
and they were focused around the VUI interface design and The next question was what did they use the VUI for in general.
task completion abilities. The answers varied with each and every participant ranging
from "asking for directions, reminders, general search results,
FINDINGS weather updates, calling someone in their contact list, playing
This study has been completed by 10 participants, this group in- songs, searching for an e-commerce product and image search).
cluded 5 iOS and 5 android smartphone participants who were Out of all the participants 2 participants (20%) agreed to asking
a mixture of native English and Non native English speakers. directions frequently and 3 (30%) different participants agreed
Both groups consisted of mixed participants. The non-native on searching for something on the internet.
language categories were Indian whereas the native English
speaking categories were from UK and USA . All of the An- Table 3. Third task results
droid participants were from India (100%) whereas the iOS yes no
participants were a mixture from India (20%), UK (60%) and
the USA (20%). Although it was never asked but the partici- Valid 2 2
pants age range varied from 22 years to 30 years. This can be Missing 0 0
seen Table 1. Mean 3.500 1.500
Median 3.500 1.500
The activities which which the participants performed helped Mode 3.000 1.000
me prove the hypothesis and confirm the other one wrong. Std. Deviation 0.707 0.707
The tasks were to perform a calculation, search for weather,
play music, look for other device in the ecosystem, translate
from language to another, Speak in a muffled voice and try to Third task results
see if the VUI was able to understand it and ask questions in The third question that was asked was if the participants ever
succession and see the response. Here we will discuss what encountered any problems while using the VUI. 7 of the 10
the participants experienced after they gave commands to the (70%) participants recorded saying yes, they did find problems
VUI and got the responses. while using the VUI. Remaining 3 participants (30%) said
that they encountered no problems at all. Out of the 7 partici-
First task results pants who agreed that they did find problems while using the
Out of the 10 participants 6 participants (60%)said they rarely VUI 3 of them (42.8%) were iOS participants and 4 of the
used the VUI functionality in their mobile phones. 2 of them participants were Android participants (57.14%). The mean
said that they used it 2-3 times a week(20%) and 2 of them said for participants saying yes was 3.5 and for no was 1.5, with
that they used it sometimes (20%). 3 of them (30%) said that standard deviation of 0.7. This can be seen in Table 3.
Bath Spa University, January 2021, Bath, UK Nishesh Jaiswal

Table 4. Fourth task results the 10 participants 4 (40%) participants agreed that it was
easily difficult intrusive. The remaining 6 (60%) participants did not find it
Valid 2 2 intrusive. Out of the 4 participants who found the interface to
Missing 0 0 be intrusive, only 1 iOS participant found it that way (20%)
Mean 4.000 1.000 whereas 3 out of 5 android smartphone participants found the
Std. Deviation 1.414 1.414 VUI to be intrusive. The mean of the participants who said
Minimum 3.000 0.000 that the VUI was intrusive was 2 and who said that it wasnt
Maximum 5.000 2.000 intrusive was 3. The standard deviation for the same was 1.4.
This can be seen in table 6.
Table 7. Seventh task results
Fourth task results
The fourth task required the participants to answer how quickly allowed not allowed
it was for them to access the VUI in their smartphone. 8 (80%) Valid 2 2
of the participants said they were able to access the VUI "with Missing 0 0
the touch of a button" whereas the remaining 2 (20%) said Mean 3.000 2.000
they have had trouble accessing the VUI and they both were Std. Deviation 2.828 2.828
android smartphone participants. The mean for participants Minimum 1.000 0.000
saying that it was easy was 4 and for saying that it was difficult Maximum 5.000 4.000
wsa 1. The standard deviation for the same was 1.4. This can
be seen in the Table 4.
Table 5. Fifth task results Seventh task results
solved not solved Seventh task required the participants to tell if the VUI al-
lowed them to move to another task while using the VUI or
Valid 2 2 when they get the search result from the VUI. 4 out of the
Missing 0 0 10 participants (40%) said that it didn’t allow to move to an-
Mean 3.500 1.500 other task or multitask. Whereas the remaining 6 out of 10
Std. Deviation 2.121 2.121 (60%) agreed that the VUI allowed them to multitask. All 5
Minimum 2.000 0.000 android participants were able to multitask while using the
Maximum 5.000 3.000 VUI whereas only 1 (20%) of the iOS participants was able
to multitask while accessing the VUI. The mean of the par-
ticipants who said that the VUI allowed to multitask was 3
Fifth task results and those who said that it was not allowing to multitask was
The fifth task required the participants to tell if the VUI had 2, with the standard deviation of 2.2. This can be seen in the
solved their problems, which here means that if the VUI had table 7.
correct reponses to their questions. Out of the 10 participants
3 (30%) participants did not agree that they got what they Table 8. Eighth Task results
were looking for. The remaining 7 (70%) agreed stating that it change no change
solved the problems. Out of the 3 participants who said that
the VUI did not solve their problems were all iOS participants. Valid 2 2
2 of 5 (40%) iOS participants agreed that Siri was able to solve Missing 0 0
their problems. The mean for the participants saying that it Mean 3.000 2.000
solved their problems was 3.5 and for the participants whose Std. Deviation 1.414 1.414
problems werent solved was 1.5. The standard deviation for Minimum 2.000 1.000
the same was 2.12. This can be seen in Table 5. Maximum 4.000 3.000

Table 6. Sixth task results

intrusive not intrusive Eighth task results
Valid 2 2 The last task was more of an efficiency task where the par-
Missing 0 0 ticipants were asked if the VUI offered them more freedom
Mean 2.000 3.000 and control of how it functioned like changing the language,
Std. Deviation 1.414 1.414 accent. 6 out of 10 (60%) of the participants said that they
Minimum 1.000 2.000 were able to change the language and accent and 4 out of the
Maximum 3.000 4.000 10 (40%) of the participants said that they were unable to do so.
It is to be noted that only 1 iOS participant could not change
the language and accent and 3 android participants faced the
Sixth task results similar issue. The mean of the participants who said that the
The sixth task required the participants to tell if the Interface VUI offered them more freedom and control was 3 and those
was intrusive or not. Like if it covered the entire screen or who said it didn’t was 2 with a standard deviation of 1.4. This
caused visual disturbance while accessing the VUI. Out of can be seen in the table 8.
Bath Spa University, January 2021, Bath, UK Nishesh Jaiswal

DISCUSSION AND CONCLUSION GOOGLE DRIVE LINK

Having established the statistical results gathered from the The google drive link for viewing the survey details are: https:
experiment and the survey, the overall result of this research //cutt.ly/Zj8G5kA
proves one hypothesis to be correct and one hypothesis to be
wrong. I had originally stated that iOS participants have less REFERENCES
intrusive VUI interface, and according to the given data and [1] cibelle dardi. 2018. "Siri owns already 46% of the
the table I can confirm my hypothesis that it is true with par- mobile voice assistant market — almost 2 times more
ticipants found the VUI of Siri less intrusive than the android market share than Google Assistant". Web article.
participants. I had also originally stated that iOS smartphones (2018). Retrieved December 26, 2020 from
are expected to have a better VUI utility when it comes to task https://vuix.io/blog/
completion, but the average amount of participants indicate a-home-so-uncluttered-that-it-almost-looks-empty/.
that it is the android participants whole found that Google
[2] Daffodil. 2019. "The Rise of Voice User Interface
assistant VUI had a correct and satisfactory response to their
(VUI)". Web article. (2019). Retrieved December 28,
commands. Similarly reflected in the survey, most participants
2020 from https://insights.daffodilsw.com/blog/
who did not find any problems with the VUI were iOS partici-
the-rise-of-voice-user-interface-vui.
pants and android participants found the most problems with
the VUI. Also according to the table, we can also see that all [3] Saptarshi Das. 2019. "Designing voice user interface
the iOS participants were able to access the VUI and had a fair (VUI) for providing that “WOW” user experience!".
idea of how to access it, whereas the android participants had Web article. (2019). Retrieved December 28, 2020 from
problems accessing the VUI. This helped support my hypothe- https://www.rapidops.com/blog/
sis that iOS VUI Siri, has a better VUI efficiency by having a designing-voice-user-interface-vui-\
less intrusive UI and easy to access VUI. for-providing-that-wow-user-experience/.

Similarly reflecting on the survey, most of the participants [4] Interaction design foundation. 2016. "Efficiency". Web
stated that Siri was easy to use and access with having little to article. (2016).
no problems in accessing the VUI. Although, the tasks were
[5] Interaction design foundation. 2018. "Voice User
performed better by google assistant in android smartphones.
Interfaces". Web article. (2018). Retrieved December 30,
Participants also responded in saying that Siri gave them more
2020 from https://www.interaction-design.org/
freedom in selecting languages and choosing the accent of the
literature/topics/voice-user-interfaces.
response given, whereas the Google Assistant didn’t offer that
much freedom. Lastly the participants who used Siri as their [6] Jonathan Engstrom. 2017. "The Rise of VUI". Web
VUI also noted that Siri UI was not intrusive and didn’t cover article. (2017). Retrieved January 2, 2021 from
the entire screen and allowed the participants to see what else https://journal.theoneoff.com/trends/the-rise-of-vui.
was on screen, whereas Google Assistant didn’t offer the same.
[7] IBM. 2016. "IBM Shoebox". Web article. (2016).
This helps cement the hypothesis that VUI efficiency of the
Retrieved December 28, 2020 from
iOS VUI "siri" is better than Google Assistant.
https://www.ibm.com/ibm/history/exhibits/
Similarly when performing the tasks the participants using specialprod1/specialprod1_7.html.
Google Assistant noticed that it gave correct responses to most
[8] DBS Interactive. 2020. "Voice Search Statistics and
of their queries. Also android participants mentioned that it
Emerging Trends". Web article. (2020). Retrieved
allowed the participants to multitask easily and come back to it
January 2, 2020 from https:
at a later stage which was not in the case of Siri, thus proving
//www.dbswebsite.com/blog/trends-in-voice-search/.
my initial hypothesis wrong that iOS devices running Siri had
a better VUI utility. [9] BRET KINSELLA. 2018. "Siri Does Better Than
Expected in Voice Assistant Face Off, But Google Still
Although the data gathered points towards Siri having a better Leads". Web article. (2018). Retrieved December 28,
efficiency in VUI and Google Assistant have a better utility, 2020 from https://voicebot.ai/2018/02/12/
there were issues around the data gathered within this study siri-better-expected-voice-assistant-face-off-\
that causes the data to not be usable for a general populace. google-still-leads/.
The first point is the data sample size which was originally
thought to be around 20 people but was limited to just 10 peo- [10] Charles McLellan. 2016. "How we learned to talk to
ple later because of the lack of volunteers and test participants. computers, and how they learned to answer back". Web
Time constraint was also a big factor in not getting the required article. (2016). Retrieved December 28, 2020 from
amount of data. Therefore, there is an argument that the given https://www.techrepublic.com/article/
data set is not substantial enough to provide definitive results. how-we-learned-to-talk-to-computers/.
The second point of concern is the specificity of the study, as [11] Global me. 2021. "Language Support in Voice Assistants
the study was centered around just smartphones and their VUI Compared". Web article. (2021). RetrievedJanuary 12,
a broad section of devices were left out which could have been
2021 from https://www.globalme.net/blog/
compared with and gathered a much larger and accurate data
language-support-voice-assistants-compared/.
set. Also given that not all participants have access to all kinds
of different technology and the required hardware, collecting
data from just a survey couldn’t gather enough points.
Bath Spa University, January 2021, Bath, UK Nishesh Jaiswal

[12] James P. Giangola Jennifer Balogh Michael H. Cohen, 2020 from https://www.pocket-lint.com/apps/news/
Michael Harris Cohen. 2004. "Voice user interface apple/112346-what-is-siri-apple-s-personal-voice-\
design". Book. (2004). Retrieved December 28, 2020 assistant-explained.
from https://www.google.co.uk/books/edition/_/PI_
[18] Guillaume Privat. 2018. "Fundamental Elements of VUI
n2EcJfT0C?hl=en&gbpv=1&pg=PR17&dq=Giangola+J.+P.
Design". Web article. (2018). Retrieved January 10,
+Balogh+J.+Cohen,+M.+H.+2004.+Voice+User+Interface+
2021 from https://blog.prototypr.io/
Design.+Addison-Wesley+Professional.
fundamental-elements-of-vui-design-8630077a7009.
[13] Erika Morphy. 2018. "What Is Conversational User
Experience (UX)". Web article. (2018). Retrieved [19] Sourabh Purwar. 2019. "Exploring the new realms of
December 29, 2020 from VUI: Conversational Design". Web article. (2019).
https://www.cmswire.com/digital-experience/ Retrieved December 28, 2020 from https://uxplanet.
what-is-conversational-user-experience-ux/. org/exploring-the-new-realms-of-vui-conversational-\
design-bf3fcc210ff .
[14] AVA MUTCHLER. 2017. "Voice Assistant Timeline: A
Short History of the Voice Revolution". Web article. [20] Johan Schalkwyk and Ignacio Lopez Moreno. 2018.
(2017). Retrieved December 28, 2020 from "Teaching the Google Assistant to be Multilingual".
https://voicebot.ai/2017/07/14/ Web article. (2018). Retrieved December 28, 2020 from
timeline-voice-assistants-short-history-voice-revolution/. https://ai.googleblog.com/2018/08/
Multilingual-Google-Assistant.html.
[15] Jakob Nielsen. 2012a. "How we learned to talk to
computers, and how they learned to answer back". Web [21] Statcounter. 2020. "Mobile Operating System Market
article. (2012). Retrieved January 12, 2020 from Share Worldwide". Web article. (2020). Retrieved
https://www.techrepublic.com/article/ January 20, 2020 from https://gs.statcounter.com/
how-we-learned-to-talk-to-computers/. os-market-share/mobile/worldwide.

[16] Jakob Nielsen. 2012b. "Usefulness, Utility, Usability". [22] Maggie Tillman. 2020. "What is Google Assistant and
Web article. (2012). Retrieved January 12, 2020 from what can it do?". Web article. (2020). Retrieved January
https://www.nngroup.com/videos/ 10, 2021 from
usefulness-utility-usability/. https://www.pocket-lint.com/apps/news/google/
[17] Britta O’Boyle. 2020. "What is Siri and how does Siri 137722-what-\is-google-assistant-how-does-it-work-\
work?". Web article. (2020). Retrieved December 28, and-which-devices-offer-it.