Opportunities and Challenges in Using AI Chatbots in
Higher Education
Shanshan Yang Chris Evans
Warwick Manufacturing Group Warwick Manufacturing Group
University of Warwick University of Warwick
Coventry, UK Coventry, UK
[email protected] [email protected]
ABSTRACT AI chatting robot, rather than a human individual [2].
Artificial intelligence (AI) conversational chatbots have gained AI conversational chatbots have gained popularity over time, and
popularity over time, and have been widely used in the fields of have been widely used in the fields of e-commerce, online
e-commerce, online banking, and digital healthcare and well- banking, and digital healthcare and well-being, among others
being, among others. The technology has the potential to provide [3,4]. The technology has the potential to provide personalised
personalised service to a range of consumers. However, the use service to a range of consumers [5,6]. However, little work has
of chatbots within educational settings is still limited. In this been done in the context of using the technology in educational
paper, we present three chatbot prototypes, the Warwick settings. In this paper, we present three chatbot case studies
Manufacturing Group, University of Warwick, are currently recently conducted by the Warwick Manufacturing Group,
developing, and discuss the potential opportunities and technical University of Warwick. Three AI virtual agents are under
challenges we face when considering AI chatbots to support our development, with potential opportunities: 1) to support the
daily activities within the department. Three AI virtual agents are delivery of a taught Master’s course simulation game; 2) to
under development: 1) to support the delivery of a taught support the training and use of a newly introduced educational
Master’s course simulation game; 2) to support the training and application; 3) to improve the processing of helpdesk requests
use of a newly introduced educational application; 3) to improve within a university department.
the processing of helpdesk requests within a university
department. We hope this paper is informative to those interested 2. RELATED WORK
in using chatbots in the educational domain. We also aim to Chatbots have become popular in both business and healthcare
improve awareness among those within the chatbot development settings. Liao and her team have developed a fashion chatbot to
industry, in particular the chatbot engine providers, about the help customers to search for fashion products based on their style
educational and operational needs within educational institutes, preferences [3]. Ko and Lin have developed a chatbot to manage
which may differ from those in other domains. business cards [4]. Madhu and his colleagues [5] have developed
a medical agent to a medical agent capable of suggesting possible
CCS Concepts ailments based on the person’s stated symptoms, and give a list of
•Information systems ➝ Information retrieval ➝ Users and potential treatments. Fadhil and Gabrielli have developed a
interactive retrieval ➝ personalisation chatbot to provide nutrition education and promote healthy
lifestyles [6].
Keywords
Chatbot; AI in education; personalised educational service; AI in The use of chatbots within educational settings is still limited.
educational simulation; AI in application training; AI in helpdesk Some works have been conducted to support both online and
support; intelligent systems. mobile learning: For example, some chatbots have been created
as intelligent tutors to teach foreign languages, such as English
1. INTRODUCTION [7]. Verleger and Pembridge have created a chatbot to teach
A chatbot is an artificial intelligence (AI) automated software computer programming language [8]. Lee and Fu have used
tool that simulates a conversational interaction’between the user chatbots to support peer assessment [9].
and a computer, using natural language [1]. Where chatbot
Some chatbots have been created which embed elements of other
technology is enabled, the end user is able to ‘talk’ to a pre-built
latest technologies: For instance, Dascalu and Bodea [10] have
Permission to make digital or hard copies of all or part of this work for developed a job interview training chatbot which has used both
personal or classroom use is granted without fee provided that copies are VR and AI technologies. Park and Jeong have developed an
not made or distributed for profit or commercial advantage and that indoor dialog chatbot working in a mixed reality environment
copies bear this notice and the full citation on the first page. Copyrights [11].
for components of this work owned by others than ACM must be
honored. Abstracting with credit is permitted. To copy otherwise, or 3. METHODOLOGY
republish, to post on servers or to redistribute to lists, requires prior
Our pilot study investigated the feasibility of introducing chatbots
specific permission and/or a fee. Request permissions from
[email protected]. in higher educational contexts through the following research
ICEEL 2019, November 5-7, 2019, Barcelona, Spain. questions:
© 2019 Association for Computing Machinery.
• How can chatbots be used within a typical university in the
ACM ISBN 978-1-4503-7225-1/19/11…$15.00
United Kingdom?
https://doi.org/10.1145/3371647.3371659
79
• What are the potential benefits and technical challenges of as it is a newly introduced educational application. The chatbot
introducing chatbots in a university setting? will act as a virtual tutor to instruct teachers and students on how
to use the software, and train them to solve basic technical issues
We considered three case studies in the areas of educational on their own as necessary. A selection of chat examples are
simulation, educational software’ training, and helpdesk support, presented in Figure 2 below.
to support our study. In each case study, an AI virtual chatbot
prototype has been proposed.
4. CHATBOT PROTOTYPES
In this section, we will briefly present the prototypes we are
developing, together with the approach we adopted to build these
chatbots.
4.1 Chatbot in Educational Simulation Game
We created a simulation bot to support the delivery of a
simulation game within a taught Master’s module. In this
simulation game, the chatbot acts as a virtual customer who aims
to do business with a different company from the one they
currently use. Students act as the current company’s salesperson;
they must talk to this virtual customer and do their best to retain
them as a client. In order to retain the customer, students must
successfully present four selling points that the current company
provides. A selection of chat massages are displayed in Figure 1.
Figure 2. Reading list chatbot response snapshot
4.3 Chatbot in Helpdesk Support
As one of the largest departments within the university, we
receive hundreds of requests from our students and staff every
day. At times, the helpdesk cannot respond to these requests
quickly enough, due to large workloads and staff shortages
during holiday periods. A helpdesk bot was introduced to answer
frequently asked questions, to respond to basic requests, and to
guide users in effectively submitting helpdesk requests to the
correct teams. A selection of chat messages are shown in Figure 3.
Figure 1. Simulation chatbot response snapshot
4.2 Chatbot in Educational Application
Training
A reading list bot has been developed to support the training of
users of a newly introduced educational software—the digital
reading list. A digital reading list system was recently introduced
to many UK universities, in order to allow library reading
resources to be better managed by librarians, teachers, and
students [12]. Teachers can generate a digital reading list for each Figure 3. Helpdesk chatbot response snapshot
of their courses. Students can view these lists, which inform them
exactly where, when, and how to access each book in the library, 4.4 Methods for Creating Chatbots
based on the information provided in the list. However, a In order to develop each chatbot, we followed the four main steps
significant number of users do not know how to use this software, below.
80
Step one: collect pairs of sample questions and answers A4 & A5: There are always questions that chatbots cannot
answer. In addition, it is impossible to predicate all potential
For each chatbot prototype, we collected roughly 30 sample scenarios for interaction while building a bot (see Table 1). Some
question types and their corresponding sample responses from chatbot-building platforms, such as IBM Watson, are currently
teachers and support staff. developing solutions to this by introducing search skills that
Since end users may ask the same types of questions but frame connect to internal databases and redirect questions to human
them differently, we generated 150 pairs of questions and agents as messages via mailing systems, such as Intercom [13,14].
answers based on the original 30. However, these are not ready to use by bot developers yet.
Step two: decide on the conversational response types B2: To change and update responses provided in the chatbot
requires necessary AI knowledge and technical skills. In most
Based on the length of conversations, we decided how the cases, teaching and support staff responsible for maintaining or
chatbots would answer each type of question. Some questions can updating responses in the chatbots do not currently have these
be answered directly in-chat, while others will need to be directed skills, as shown in Table 2. User friendly interfaces, which would
to human support; a select number of questions will need to be allow multiple users to access the chatbot-building framework,
directed to other teams, both internal and external to the would be very useful to enable less technically knowledgeable
department. staff to make changes to the content of chatbots easily, without
Step three: develop the chatbot prototypes damaging the whole product. However, currently, there is no
such technology available in the market.
Next we selected and used a bot development engine to create
and host the chatbots we had designed. B5: It would be advantageous if the chatbots were able to
consider users’ feelings during interactions as one of our targets
Step four: review the chatbot prototypes is to provide positive service experiences to our students and staff
(see Table 2). Some providers have proposed the idea of having
In order to review the chatbots, we evaluated the prototypes
‘tone analyses’ which could measure the users’ potential
against with the potential benefits we have identified from
emotional state [15,16], and provide appropriate responses during
literatures and pervious work experience, to explore if current
the interaction. However, this feature would require high-level AI
technologies can actually support the primary benefits we have
programming skills in order to build it into a bot, which most bot
identified.
creators do not possess.
5. OPPORTUNITIES AND CHALLENGES Table 2. Review of the reading list bot
In this section, we first present the potential benefits that our
chatbots are trying to achieve. Subsequently, we discuss gaps and Can
technical challenges chatbot developers are currently facing. We Can this be this be
will discuss each of the unsolved challenges in detail. met in the effected
Potential benefits / opportunities
current in the
Table 1. Review of the simulation bot prototype? near
Can future?
Can this
this be B1: Answers provided by the
be met in
effected chatbot are short enough to read Yes NA
Potential benefits / opportunities the
in the and follow easily.
current
near
prototype?
future? B2: Training staff can easily and
A1: By talking to the chatbot quickly change or update answers No yet Not sure
students can retain the customer by contained in the chatbot.
Yes NA
presenting four must-have selling
points that the current company has. B3: Further support, such as email
contact and links, are provided Yes NA
A2: The chatbot can understand and when users need extra assistance.
differentiate between the must-have
Yes NA
and optional selling points students B4: Inputs with grammatical and
raise in the interaction. typing mistakes can still be Yes NA
A3: The chatbot can prompt students understood by the chatbot.
if they become stuck during the Yes NA
interaction. B5: The chatbot is able to be aware
of the user’s feelings during the
A4: Students will not realise they are No yet Not sure
conversations, and provide suitable
talking to a robot as opposed to a No yet Not sure emotional care as necessary.
human.
A5: Difficult questions asked by
Possibly
students can immediately be No yet
yes
redirected to course teachers.
81
Table 3. Review of the helpdesk bot chatbots together with the technical challenges faced currently
within a university department.
Can this
Can this
be We hope this paper is informative and be able to offer inspiration
be met in
effected to educational colleagues, with teaching, research or
Potential benefits / opportunities the
in the administration roles, who are interested in the use of chatbots in
current
near education.
prototype?
future?
We also wish to improve awareness within the chatbot
C1: The chatbot is able to guide development industry, in particular among chatbot engine
users to submit requests to the Yes NA providers, of the educational and operational needs of educational
correct teams. institutes, which differ from other domains. Most of the popular
chatbot development platforms are not originally designed to
C2: The chatbot is able to redirect support educational activities.
Possibly
submitted requests to the correct No yet
yes In conclusion, these chatbot prototypes are not yet ready for use,
teams/persons automatically.
as a number of significant technical challenges still remain. The
C3: The chatbot is able to guide next phase of our work will focus on addressing the technical
users to browse helpdesk-related challenges identified, potentially in collaboration with the
Yes NA university’s technical teams, or third-party industry partners.
websites in order to save them
time. After these challenges are resolved, a potential next step will be
to invite further stakeholders to experiment with these prototypes
C4: The chatbot is able to in order to explore how to better design personalised educational
automatically fill in the request No yet Not sure services via AI chatbots for students, teachers and staff in a
form while interacting with users. university environment.
7. REFERENCES
C5: The chatbot can be fully [1] Maroengsit,W., Piyakulpinyo, T., Phonyiam,K.,
integrated with current university No yet Not sure Pongnumkul,S., Chaovalit,P. and Theeramunkong,T. 2019.
systems. A Survey on Evaluation Methods for Chatbots. In
Proceedings of the 7th International Conference on
Information and Education Technology (ICIET 2019). ACM,
C2: The result in Table 3 shows that, staff and students may be New York, NY, USA, 111-119.
unsure which teams in the department their helpdesk requests DOI=10.1145/3323771.3323824
need to be submitted to. This creates extra workload for the
helpdesk team, who must redirect their inquiries to the correct [2] Molnár,G. and Szüts, Z. 2018. The Role of Chatbots in
team. This requires the helpdesk staff to be very experienced in Formal Education, IEEE 16th International Symposium on
order to know the relevant team for referral. Some chatbot Intelligent Systems and Informatics (SISY), Subotica, 197-
building platforms have stated that they have features which can 202. doi= 10.1109/SISY.2018.8524609
auto-assign user requests to different teams as pre-specified [3] Lizi, L., You, Z., Yunshan, M., Richang H., and Tat-Seng C.
[13,14,15,16,17,18]. However, this feature is not yet ready to use 2018. Knowledge-aware Multimodal Fashion Chatbot. In
by bot developers. Proceedings of the 26th ACM international conference on
Multimedia (MM '18). ACM, New York, NY, USA, 1265-
C4: Time would be saved and workload reduced significantly if
1266. DOI=10.1145/3240508.3241399
chatbots could automatically fill in users’ request forms and
redirect the requests to the right team while interacting with [4] Meng-Chieh, K. and Zih-Hong, L. 2018. CardBot: A
customers (see Table 3). Most chatbots have the potential to Chatbot for Business Card Management. In Proceedings of
collect extra information from users by asking additional the 23rd International Conference on Intelligent User
questions, or allowing users to make selections from a number of Interfaces Companion (IUI '18 Companion). ACM, New
options provided by the chatbot [13,14,15,16,17,18]. However, York, NY, USA, Article 5, 2 pages.
we are currently facing difficulties with auto-forwarding user DOI=10.1145/3180308.3180313
requests to the right team, as mentioned earlier. [5] Madhu, D., Jain, C. J. N., Sebastain, E., Shaji,S. and
C5: After the chatbots are built, they need to be able to fully Ajayakumar, A.2017. A novel approach for medical
integrate with universities’ current systems, in line with their assistance using trained chatbot, International Conference
policies and regulations regarding to data privacy and security. on Inventive Communication and Computational
Typical systems include universities’ websites, VLEs, mailing Technologies (ICICCT), Coimbatore,243-246.
systems and helpdesk systems. However, currently, these DOI=10.1109/ICICCT.2017.7975195
chatbots are still standalone applications which have not been [6] Ahmed, F. and Silvia, G. 2017. Addressing challenges in
successfully integrated with existing systems (see Table 3). promoting healthy lifestyles: the al-chatbot approach. In
Proceedings of the 11th EAI International Conference on
6. CONCLUSIONS AND FURTHER WORK Pervasive Computing Technologies for Healthcare
In this paper, we have proposed and discussed the opportunities (PervasiveHealth '17). ACM, New York, NY, USA, 261-
of having three different AI chatbots in the areas of learning and 265. DOI=10.1145/3154862.3154914
teaching, educational application training, and helpdesk support,
[7] Xuan, L. P., Thao, P., Quynh, M. N., Thanh, H. N., and Thi,
and have shared our practical experiences of developing these
T. H. C. 2018. Chatbot as an Intelligent Personal Assistant
82
for Mobile Language Learning. In Proceedings of the 2018 Annual International Conference on Mobile Systems,
2nd International Conference on Education and E-Learning Applications, and Services (MobiSys '19). ACM, New York,
(ICEEL 2018). ACM, New York, NY, USA, 16-21. NY, USA, 708-709. DOI=10.1145/3307334.3328595
DOI=10.1145/3291078.3291115 [12] Talis Aspire Reading List. https://talis.com/talis-aspire 2019.
[8] Verleger, M. and Pembridge, J. 2018. A Pilot Study [13] IBM Watson. https://www.ibm.com/watson/ 2019.
Integrating an AI-driven Chatbot in an Introductory
Programming Course, IEEE Frontiers in Education [14] Dialogflow. https://dialogflow.com/ 2019.
Conference (FIE), San Jose, CA, USA, 1-4. DOI= [15] Wit.ai. https://wit.ai/ 2019.
10.1109/FIE.2018.8659282
[16] Pandorabots. https://home.pandorabots.com/home.html 2019.
[9] Yi-Chieh, L. and Wai-Tat, F. 2019. Supporting peer
[17] Gonda,D. E., Luo,J., Wong, Y. and Lei,C. 2018.
assessment in education with conversational agents. In
Evaluation of Developing Educational Chatbots Based on
Proceedings of the 24th International Conference on
the Seven Principles for Good Teaching, IEEE International
Intelligent User Interfaces: Companion (IUI '19). ACM,
Conference on Teaching, Assessment, and Learning for
New York, NY, USA, 7-8. DOI=10.1145/3308557.3308695
Engineering (TALE), Wollongong, NSW, 446-453.
[10] Stanica, I., Dascalu, M., Bodea, C. N. and Bogdan, M. A. D. doi=10.1109/TALE.2018.8615175
2018. VR Job Interview Simulator: Where Virtual Reality
[18] Ranoliya,B. R., Raghuwanshi,N.and Singh,S. 2017. Chatbot
Meets Artificial Intelligence for Education, Zooming
for university related FAQs, International Conference on
Innovation in Consumer Technologies Conference (ZINC),
Advances in Computing, Communications and Informatics
Novi Sad, 9-12. DOI=10.1109/ZINC.2018.8448645
(ICACCI), Udupi, 1525-1530.
[11] KyuHwon, P. and Young-Seob, J. 2019. Indoor Dialog doi=10.1109/ICACCI.2017.8126057
Agent in Mixed Reality (video). In Proceedings of the 17th
83
