University of the Philippines Manila

College of Arts and Sciences

Department of Physical Sciences and Mathematics

Developing a Real-Time 3D AI Primary School

English Teacher Using OpenAI and Unreal
Engine

A special problem in partial fulfillment

of the requirements for the degree of
Bachelor of Science in Computer Science

Submitted by:

Will Nigel C. De Jesus

June 2023

Permission is given for the following people to have access to this SP:

Available to the general public Yes

Available only after consultation with author/SP adviser No

Available only to those bound by confidentiality agreement No

 ACCEPTANCE SHEET

The Special Problem entitled “Developing a Real-Time 3D AI Primary

School English Teacher Using OpenAI and Unreal Engine” prepared and submitted
by Will Nigel C. De Jesus in partial fulfillment of the requirements for the degree of
Bachelor of Science in Computer Science has been examined and is recommended for
acceptance.

Avegail D. Carpio, M.Sc.

Adviser

EXAMINERS:
Approved Disapproved

1. Richard Bryann L. Chua, M.Sc.

2. Perlita E. Gasmen, M.Sc. (cand.)
3. Ma. Sheila A. Magboo, Ph.D. (cand.)
4. Vincent Peter C. Magboo, M.D.
5. Marbert John C. Marasigan, M.Sc. (cand.)
6. Geoffrey A. Solano, Ph.D.

Accepted and approved as partial fulfillment of the requirements for the degree
of Bachelor of Science in Computer Science.

Vio Jianu C. Mojica, M.Sc. Marie Josephine M. De Luna, Ph.D.

Unit Head Chair
Mathematical and Computing Sciences Unit Department of Physical Sciences
Department of Physical Sciences and Mathematics
and Mathematics

Maria Constancia O. Carrillo, Ph.D.

Dean
College of Arts and Sciences

i
 Abstract

Recent advancements in AI chatbot technology have brought about significant im-

provements. This research paper explores the application of OpenAI’s GPT-3 and
Unreal Engine to develop a real-time primary school English teacher. The system
aims to enable teachers to input various primary school English topics, receive AI-
generated content, and observe the interactive 3D avatar delivering the material.
Similarly, students can pose relevant questions to the lesson and observe the avatar’s
delivery of the content.
Two primary school English teachers were engaged as domain experts in use test-
ing to evaluate and provide feedback on the system. The results indicate that the
system received positive feedback from the domain experts. They emphasized the
system’s vast knowledge base in English education, the quality of its content, and
the adaptability of content generation based on prompts. However, the experts also
noted the need for additional interactive features, such as the ability to pause and
encourage student participation.
Overall, the system demonstrates significant potential as an educational tool
within our existing education system.
Keywords: AI chatbots, OpenAI’s GPT-3, Unreal Engine, real-time primary school En-

glish teacher, education technology, content generation, virtual avatars, interactive learning,

teacher-student interaction, language processing

Contents

Acceptance Sheet i

Abstract ii

List of Figures vi

I. Introduction 1
A. Background of the Study . . . . . . . . . . . . . . . . . . . . . . . 1
B. Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . 1
C. Objectives of the Study . . . . . . . . . . . . . . . . . . . . . . . . 2
D. Significance of the Project . . . . . . . . . . . . . . . . . . . . . . 2
E. Scope and Limitations . . . . . . . . . . . . . . . . . . . . . . . . 2
F. Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

II. Review of Related Literature 4

A. AI Content Generation . . . . . . . . . . . . . . . . . . . . . . . . 4
B. Human-AI Interaction . . . . . . . . . . . . . . . . . . . . . . . . . 5
C. AI Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

III. Theoretical Framework 10

A. AI Chatbot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
B. Natural Language Processing . . . . . . . . . . . . . . . . . . . . . 10
C. OpenAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
C..1 GPT-3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
C..2 Prompt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
D. Unreal Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
D..1 Metahuman . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
E. Text-to-Speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

iii
 E..1 Google TTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
F. Oculus Lipsync . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
F..1 Visemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
G. MetahumanSDK . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

IV. Design and Implementation 14

V. Results 19
A. Overview of the System’s Final Design . . . . . . . . . . . . . . . 19
B. Evaluation Process . . . . . . . . . . . . . . . . . . . . . . . . . . 21

VI. Discussions 25
A. Comparative Analysis of Expert Feedback . . . . . . . . . . . . . 25
A..1 Performance and Instructional Aspects . . . . . . . . . . . . . 25
A..2 User Interface and Functionality . . . . . . . . . . . . . . . . . 26
A..3 Adaptability and Customization . . . . . . . . . . . . . . . . . 26
A..4 Prompt Customization . . . . . . . . . . . . . . . . . . . . . . 26

VII. Conclusions 28

VIII. Recommendations 29

IX. Bibliography 31

X. Appendix 34
A. Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
A..1 Evaluation Form of Expert 1 . . . . . . . . . . . . . . . . . . . 34
A..2 Evaluation Form of Expert 2 . . . . . . . . . . . . . . . . . . . 38
B. Source Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
B..1 Initialize Level . . . . . . . . . . . . . . . . . . . . . . . . . . 43
B..2 Generate AI Lecture . . . . . . . . . . . . . . . . . . . . . . . 43

iv
 B..3 Generate AI Answer . . . . . . . . . . . . . . . . . . . . . . . 44
B..4 Start Animation . . . . . . . . . . . . . . . . . . . . . . . . . 44
B..5 Edit and Save Settings . . . . . . . . . . . . . . . . . . . . . . 45

XI. Acknowledgment 46

v
List of Figures

1 A news article written by AI translated to English. . . . . . . . . . . 5

2 Replika: AI Chatbot . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 Conversation of Jiwo and a Preservice Teacher. . . . . . . . . . . . . 9
4 Algiola customer support that uses gpt-3 . . . . . . . . . . . . . . . . 11
5 Gpt3 sample prompt and response for OpenAI Q and A model. . . . 11
6 A real-time 3D scene made in unreal engine for virtual production . . 12
7 Metahuman 3D Characters . . . . . . . . . . . . . . . . . . . . . . . . 12
8 Table showing visemes based on phonemes. . . . . . . . . . . . . . . . 13
9 Overview of the project implementation considering the user teacher . 14
10 Overview of the project implementation considering the user student 15
11 Sample UE5 programming nodes. . . . . . . . . . . . . . . . . . . . . 16
12 Sample Prompt for OpenAI and its response . . . . . . . . . . . . . . 17
13 System Main Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
14 System Main Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 20
15 Lecture Prompt Engineering . . . . . . . . . . . . . . . . . . . . . . . 21
16 Question Prompt Engineering . . . . . . . . . . . . . . . . . . . . . . 21
17 Prompt Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
18 Domain experts exploring the proposed system. . . . . . . . . . . . . 23
19 Domain experts answering the interview questions . . . . . . . . . . . 24
20 Expert 1’s Evaluation Form, page 1 . . . . . . . . . . . . . . . . . . . 34
21 Expert 1’s Evaluation Form, page 2 . . . . . . . . . . . . . . . . . . . 35
22 Expert 1’s Evaluation Form, page 3 . . . . . . . . . . . . . . . . . . . 36
23 Expert 1’s Evaluation Form, page 4 . . . . . . . . . . . . . . . . . . . 37
24 Expert 2’s Evaluation Form, page 1 . . . . . . . . . . . . . . . . . . . 39
25 Expert 2’s Evaluation Form, page 2 . . . . . . . . . . . . . . . . . . . 40
26 Expert 2’s Evaluation Form, page 3 . . . . . . . . . . . . . . . . . . . 41

vi
27 Expert 2’s Evaluation Form, page 4 . . . . . . . . . . . . . . . . . . . 42
28 Initialize level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
29 Generate AI Lecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
30 Generate AI Answer . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
31 Start Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
32 Edit and Save Settings . . . . . . . . . . . . . . . . . . . . . . . . . . 45

vii
I. Introduction

A. Background of the Study

In recent years, there has been an increasing interest in using artificial intelligence
(AI) to create digital assistants (chatbots). These AI chatbots are used in wide variety
of fields such as ecommerce [1], medicine [2], education [3], etc.
Most chatbots in the Philippines are focused on ecommerce, assisting users on
their queries. Even IskOU, an educational chatbot of University of the Philippines
Open University, primarily deals with assisting users with their admission questions
[4].
A milestone in the development of chatbot technology is the AI’s ability to gen-
erate its own content [5]. OpenAI is a research and deployment company that has
made publicly available state of the art AI models that can create essays, answer
questions, and summarize topics based on a rapidly-growing knowledge base [6]. The
technology inspired educators to utilize AI in the field of education [7].
Integrating a 3D avatar and environment to a chatbot was previously a hurdle due
to technological limitations. But through technological advancements in graphical
processing as well as the emergence of efficient 3D rendering softwares like Unreal
Engine, it is now possible to create a real-time 3D AI chatbot [8] [9].

B. Statement of the Problem

One important concern in utilizing AI chatbots in the field of education is the quality
of their outputs [10]. GPT-3 excels as an English language model because it was
trained on a very large English dataset. However, integrating it as the core content
generator for an AI teacher requires extensive observation and evaluation by domain
experts.

1
C. Objectives of the Study

The study aims to develop a real-time 3D primary school English teacher using Ope-
nAI and Unreal Engine and evaluate the quality of its contents. In particular, the
system will:

1. Allows teachers to

(a) Input any primary school English topic

(b) Review the AI-generated response and edit if necessary

(c) Watch the AI-generated lecture content delivered by a 3D avatar

2. Allows students to

(a) Input any question they have about the topic

(b) Receive an AI-generated answer content delivered by a 3D avatar

D. Significance of the Project

The findings of the study will be a contribution to the field of education as developing
a 3D primary school English teacher can reinforce our current education system.
There are remote areas that human teachers cannot reach, and thus developing the
technology that can extend to these areas can greatly help both teachers and students.
Moreover, the study will help create a more individualized learning environment.
There are students that find it hard to cope up with learning in a class environment,
and deploying teachers to individual students is not possible due to lack of manpower.
AI can be utilized to cater to an individual’s learning needs.

E. Scope and Limitations

1. The intended users are primary school English teachers and students.

2
 2. The study will focus on the quality of the content and delivery of the 3D AI
teacher by having domain experts evaluate the system.

3. The system will not be tested on an actual class setup.

F. Assumptions

1. The users are primary school teachers and students.

2. The domain experts will assume that the system is to be used on an actual
English class.

3. The users will only enter relevant topics on primary school English subject.
Some examples would be short story discussions, basic grammar, etc.

4. The user will enter topics based on the official syllabus.

3
II. Review of Related Literature

It is apparent that AI technology is widely used in various fields. In particular, this

paper aims to further improve its use in the field of education. With the goal of
developing a real-time 3D AI teacher, there are three main ideas that the researcher
would like to explore in this section.

A. AI Content Generation

At the early stages of AI content generation, a person can easily distinguish what
was created by a human from an AI. However, AI generated content has drastically
improved and has reached the point where it is challenging to determine which con-
tent was created by an AI. This technology can be greatly utilized in the field of
journalism as news articles can ideally be generated efficiently without human inter-
vention. However, it is important to determine how effective these contents are from
the perspective of its readers.
Researchers conducted an experiment to determine if readers can distinguish be-
tween human and AI-generated contents. Results showed that on a purely text-based
content, readers cannot distinguish between human and AI-generated content. More-
over, readers tend to find AI-generated content more objective and informative. This
proves that AI is capable of generating human-like content [5].
On the other hand, the participants of the experiment were still able to distinguish
between human and AI in terms of audio delivery of information. This means that
although the content itself is human-like, the technology behind text-to-speech is yet
to reach the same level of realism. The researchers were unable to include video
contents in the experiment due to technical limitations.
Both content generation and information delivery are crucial for developing a 3D
AI teacher. This paper aims to create a system that integrates AI-generated content

4
 Figure 1: A news article written by AI translated to English.

to a real-time 3D environment with the use of Unreal engine, which was previously
not feasible for the aforementioned.

B. Human-AI Interaction

In addition to AI being able to generate content on its own, there are now AI-chatbots
that can interact with humans. Similar to content generation, AI-chatbots have also
improved and are now more human-like in terms of its interaction and responses to
its users.
AI-chatbots are often considered to be functional AI, whose purpose is to help
humans accomplish tasks. Business owners find that the use of AI-chatbots as virtual
assistants tend to help their customers with their sales as it improves their overall
virtual flow and decision making [1]. The ability of AI to interact and meaningfully
respond to users is a key factor for the customers’ flow experience.
Aside from functional AI, there are also AI-chatbots that are considered as social
companions. The COVID-19 pandemic has created a very stressful situation for a
lot of people. Replika, an AI-chatbot, was used by some people during the pandemic
in order to cope with the situation. Interestingly, researchers have observed five
types of digitally mediated empathy from the interaction of Replika and its users:
crafting normalcy, affirming identity anchors, maintaining and using communication
networks, putting alternative logics to work, and downplaying negative feelings while

5
foregrounding positive emotions [11]. Moreover, users felt that Replika was able to
help them cope with the pandemic as they felt connected with it despite it being just
an AI-chatbot.

Figure 2: Replika: AI Chatbot

Nonetheless, AI-chatbots as a social companion is still not accepted by some. Re-

searchers have found that people are still more comfortable with the idea of functional
AI rather than social AI [12]. This means that for some people, AI should just remain
as tools rather than social companions.
On the other end of the spectrum, there are people that develop romantic re-
lationships with AI-chatbots. Researchers have discovered that human-chatbot re-
lationships develop very similar to a human-human relationship [13]. People would
start communicating and interacting with the chatbot which increases each person’s
interest and trust.
As the intended AI teacher will mostly interact with its students, it is important to
determine how effective AI-chatbots are in communicating with humans. Previously
mentioned studies indicate that AI-chatbots can indeed meaningfully interact with
its users. The effectiveness of AI-chatbots is a proof of concept to the use of AI in
the field of education. However, it should also consider the social implications of
AI-chatbots, which raises questions on how the teachers, students, and parents will
perceive the AI teacher.
Still, AI utilization has been found to be lacking in the field of education com-
pared to medicine, ecommerce, etc [3]. However, AI can certainly be utilized in

6
education due to the advancements in deep learning methods as well as natural lan-
guage processing that allowed intent recognition. The researchers propose creating
an encoder-decoder AI chatbot system that will analyze the user’s input intents to
provide an appropriate response which is aimed to help student comprehension and
ease faculty workloads.
Intent recognition is a vital part of the proposed system. Natural language pro-
cessing can allow for the AI teacher to understand queries coming from students and
use them as a prompt for a response that is meaningful and relevant.

C. AI Education

As AI technology rapidly expands, AI-literacy has now become an important skill for
people. However, there is a huge gap for AI-based materials and learning in the early
childhood education curiculum as the existing educational system focuses on teaching
AI-education mostly in secondary and tertiary education [14]. The study discovered
effective ways of incorporating AI-based learning in early childhood education through
the use of social robots as learning companions. Moreover, problem-based learning
was deemed to be the most effective method of helping children develop AI-literacy.
Furthermore, there have been other attempts at integrating AI literacy in educa-
tion. Researchers have proposed the use of digital story writing as an inquiry-based
teaching strategy to further improve AI literacy of primary students [15]. Students
were given some lessons about artificial intelligence and from there are tasked to for-
mulate a story about it. This challenged the students to come up with stories based
on their understanding of AI. The approach considers whether a student comes up
with a fictional or non-fictional story. The program gathered 16 of the top-performing
students in digital story writing and observed that thiey were able to formulate au-
thentic stories where artificial intelligence could be utilized as a form of solution or
innovation. Thus, digital story writing was found to be an effective pedagogy to

7
reinforce students’ understanding of Artificial Intelligence.
In addition to the methods mentioned in the paper, having an actual AI interact
and discuss Artificial Intelligence education in class can also bolster children’s knowl-
edge and interest in Artificial Intelligence. Exposing children to AI technology can
help improve their interest in learning more about AI. Researchers have proposed the
use of AI chatbots to impact children’s interest in reading [16]. The study focuses
on the manner of interaction of children with an AI chatbot and its impact on their
interest in reading, utilizing book talk cues and social affective cues. The researchers
discovered that the chatbot was able to engage the children in conversations and that
the children developed interest in the AI chatbot.
Further, not only students but also teachers can greatly benefit from the use of
AI. Responsive teaching is a major skill every teacher must learn. Researchers aim
to utilize AI chatbots’ ability to act as students that will meaningfully engage with
pre-service teachers and create an interactive training environment [17]. The study
proved the feasibility of the concept as the participants were able to practice their
questioning skills on the chatbot. It also established the use of AI in the field of
education. However the AI was not perfect, especially since technical limitations
restricted the realism of the AI’s responses.
The rapid improvement of AI chatbot technology paved the way for more realistic
human-AI interaction, which is a very important aspect in the field of education.
In the stated paper, AI was utilized to act as a student to aid teachers enhance
their questioning skills. Given our current technology, it is not far-fetched to utilize
chatbots to act as teachers as well.
In summary, we have reached the point where AI can reliably and independently
generate its own content, meaningfully interact with users, which can and is being
harnessed in the field of education. To improve upon the previously mentioned uses
of AI-chatbots, this study aims to create a system that will integrate OpenAI’s AI

8
 Figure 3: Conversation of Jiwo and a Preservice Teacher.

models that can generate content and Unreal engine that can give a visual represen-
tation of the educational chatbot, and evaluate the quality of its content and delivery
of English lectures.

9
III. Theoretical Framework

A. AI Chatbot

AI chatbots are computer programs that utilizes machine learning to simulate a

human-like conversation with its user [18].

B. Natural Language Processing

Natural language processing is a branch of Artificial Intelligence that deals with the
development of AIs that have the ability to comprehend texts the same way as humans
[19]. This is a key factor in the development of AI chatbots as it allows such programs
to converse and understand its users.

C. OpenAI

OpenAI is a research and development company that focuses in artificial intelligence.

OpenAI provides various AI models like chatbot AI, code transformation AI, etc [6].

C..1 GPT-3

GPT-3 is the latest machine learning model developed by OpenAI which can generate
content in natural language based on prompts [20].

C..2 Prompt

Prompts are inputs given to an AI model that influences its behavior and output
[21]. OpenAI’s GPT-3 is a powerful model that requires prompt engineering to fully
utilize.

10
 Figure 4: Algiola customer support that uses gpt-3

Figure 5: Gpt3 sample prompt and response for OpenAI Q and A model.

D. Unreal Engine

Unreal Engine is a game engine developed by Epic Games that allows real-time 3D
rendering [8].

D..1 Metahuman

Metahuman is a 3D framework by Epic Games that allows creation of hyperrealistic

3D human characters [22].

E. Text-to-Speech

Text-to-speech is a technology that can generate audible speech from a series of text.
These technologies use machine learning model to synthesize voice, tone, and words

11
 Figure 6: A real-time 3D scene made in unreal engine for virtual production

Figure 7: Metahuman 3D Characters

from the input text and parameters [23].

E..1 Google TTS

Google TTS is an advanced text-to-speech AI model by Google used to synthesize

realistic human voice from texts.

12
F. Oculus Lipsync

Oculus Lipsync is a framework by Meta used to automatically create real-time lipsync

animations from audio which will then be applied to a 3D avatar. The process uses
machine learning to predict visemes from an audio stream which dictates mouth
movement and shapes [24].

F..1 Visemes

A viseme is a visual depiction of a mouth shape based on phonemes [25]. Oculus

Lipsync maps phonemes to visemes and generates data that can be mapped to a 3D
character’s mouth position.

Figure 8: Table showing visemes based on phonemes.

G. MetahumanSDK

Metahuman SDK is a set of tools developed to create high-quality immersive ani-

mations for Metahumans in Unreal Engine. The tool allows for generating lipsync
animations from audio file by utilizing the Oculus Lipsync framework.

13
IV. Design and Implementation

Figure 9: Overview of the project implementation considering the user teacher

Figure 9 shows the integration of Unreal Engine and OpenAI in the 3D AI teacher’s
system. The teacher will enter a simple text prompt that will include the topic they
want the AI teacher to discuss. On the input box inside Unreal Engine, the text input
will be processed to standardized the prompts to get consistent responses from the
OpenAI api. The processed prompt will then be passed to OpenAI which will perform
natural language processing to understand the prompt and generate an appropriate
response.
As per the prompt, the system would only take in the topic intended to be dis-
cussed. The formatting of the input would take place in Unreal Engine’s program-
ming. For example, a teacher would like the AI to discuss about the short story
”The Little Prince”, after inputting it on the text box, Unreal Engine will send an
API request in the form of: ”You are a grade 7 English teacher. Create a monologue

14
script of a lecture where you discuss the topic ’The Little Prince’. Explain and give
examples.” Indicationg to OpenAI’s GPT-3 model what it is (in this case, a grade 7
English teacher) is essential in getting a proper response from the AI. Asking it to
create a monologue script gave results that feels like the AI is conducting the lecture
rather than simply listing all information it knows about the topic.
The OpenAI’s response will then be used by Google TTS to generate an audio
speech following the text response. Unreal Engine will then process the audio file
generated to create and apply lipsync animation to the metahuman. Once everything
is set, the animation and audio will then play.

Figure 10: Overview of the project implementation considering the user student

Figure 10 shows the same process, but with a student user. The difference will
be on the prompt engineering, where instead of asking gpt-3 to generate a lecture, it
will be asked to generate an answer to the student’s question.
As per the prompt, it will follow the same format earlier with a few modification.
Instead of tasking it to create a lecture script, we are now going to use OpenAI’s

15
Q and A model where we input the question and the AI will answer based on its
knowledgebase. For example: ”You are a grade 7 English teacher. You discussed
about the topic ’The Little Prince’. A student asked the question: Question entered
in the text box. Create a dialogue script of you answering the question.”
In order to evaluate the content, the study will require the help of two primary
school English teachers. One teacher will evaluate the AI teacher’s content and de-
livery for topics within grades 1 to 3 curriculum, and the other for grades 4 to 6. The
English teachers will be interviewed to examine and evaluate the system after testing.

Figure 11: Sample UE5 programming nodes.

Unreal engine is based on the programming language C++. It is commonly used

in game engines because of its exceptional memory management. Nonetheless, Un-
real engine allows its users to do complex C++ programming easily by allowing the
option to perform a node-based programming. Figure 11 is an example of Google
Text-To-Speech programming integrated in an Unreal Engine Project. Node-based
programming allows users to visualize the flow of the programming and even makes
debugging C++ codes a lot simpler.
The integration of OpenAI’s chatbot will go through Unreal Engine’s program-
ming as well. An important process in the system would be the prompt-engineering
for the chatbot. OpenAI’s chatbot is trained over a very large dataset that includes

16
 Figure 12: Sample Prompt for OpenAI and its response

topics outside the scope of the research. Figure 12 shows a sample prompt for the
chatbot and its response in green highlight. In this example, we prompted the chat-
bot to assume a teacher’s role and create a script as if it was conducting a lecture.
Without those prompts, asking for it about fractions would get a direct answer in-
stead.
One ethical concern of using OpenAI as the backbone for the AI English teacher
is plagiarism. Since the AI model was trained on a dataset of already existing works,
some argue that using GPT-3 to automatically write essays for school papers is a
form of plagiarism. There were attempts at using plagiarism detectors at AI-written
contents in which the results show that the texts were not plagiarized [26]. However,
this does not excuse academic dishonesty as students use AI to get around writing
their own essays.
On the other hand, the intent of the study is to create an educational tool that
can be used to teach students rather than create a mechanism for students to commit

17
fraud. The contents produced by the AI teacher would resemble presenting a para-
phrased portion of a textbook, but automatically and animated. Ultimately, the AI
teacher is a tool, and the user is responsible for any form of misuse.

18
V. Results

This chapter aims to provide a comprehensive analysis of the system while providing
the domain experts’ feedback on the functionality, usability, and design of the 3D AI
teacher. The researcher will consolidate the data collected from the domain experts
and draw themes and patterns to their responses that can be used to further evaluate
the system.

A. Overview of the System’s Final Design

The system design is characterized by its simplicity and user-friendly interface. Upon
launching the 3D AI Teacher, a brief introduction is provided, followed by the appear-
ance of menu controls. The main menu prominently features two buttons: one for
creating a lecture and another for creating a question as shown in figure 13. Clicking
on either button opens the main interface, which includes the subject/question input,
send button, response text area, start audio lecture/answer button, prompt settings,
and a back button.

Figure 13: System Main Menu

19
 In the subject/question input, users can enter their desired topic or question, and
upon clicking the send button, the system prompts OpenAI to generate the script.
Notably, the ”Start a lecture” and ”Ask a question” features differ in the prompts
used, which are established on the backend. The response generated by OpenAI
is then displayed in the text area, allowing the teacher to review and refine it as
necessary. Clicking the ”Start audio lecture/answer” button processes the script and
generates a text-to-speech audio, which is accompanied by corresponding animations.
Figure 14 shows the main interface where users can interact with the 3D AI Teacher.

Figure 14: System Main Interface

While the system provides users with the ability to customize certain prompts,
the final prompt sent to GPT-3 follows a fixed formatting approach established on
the backend. For the lecture feature, the prompt includes specific instructions, such
as introducing oneself and creating a monologue script on the given topic. These
instructions are crucial in guiding GPT-3 to respond as a teacher conducting a lesson,
rather than providing a third-person summary. Similarly, for the question prompt,
the addition of the ”One of your students is asking a question” prompt prompts
GPT-3 to assume the role of answering a student’s query. Figures 15 and 16 shows

20
the backend implementation of prompt engineering GPT-3 to the system’s use-case.

Figure 15: Lecture Prompt Engineering

Figure 16: Question Prompt Engineering

In addition to the fixed parts of the prompts, the system allows users to adjust
the ”Role,” ”Situation,” ”Instructions,” and add ”Additional Information” to further
customize the prompt as shown in 17. This flexibility enables scenarios like greeting a
student on their birthday before starting the lecture or requesting GPT-3 to provide
a specific number of questions at the end.

B. Evaluation Process

Prior to the evaluation process, the researcher sought the assistance of two domain
experts in the field of Primary school English education. The first expert is a 50-

21
 Figure 17: Prompt Settings

year-old female teacher with experience teaching English to students in grades 4 to 6.

The second expert is a 27-year-old female teacher with experience teaching English to
students in grades 1 to 3. Both experts currently hold teaching positions at Jesus Is
Lord Colleges Foundation, Inc., located in Bocaue, Bulacan, which also served as the
study’s location. The selection of these domain experts was deliberate as it aimed to
incorporate perspectives from different generations of teachers, thereby providing the
researcher with a more comprehensive understanding of the system’s use-case that
aligns with the characteristics of the Philippines’ teaching personnel. The usability
testing was conducted at April 13, 2023 at Jesus Is Lord Colleges Foundation, Inc.
The evaluation process consisted of several key steps to effectively gather feedback
from the experts. First, a brief explanation and quick demo of the 3D AI Teacher’s
system was provided, giving the experts an overview on its features and functionali-
ties. Once the experts had a basic understanding of the system, they were allowed to
freely explore and interact with the 3D AI, assuming the roles of instructor and even
students at some point. Throughout the usability testing, the researcher was present
to answer queries of the experts regarding the system and its implementation. The

22
experts were mostly curious with how it the system is able to generate the responses
and perform its features, and very few questions were raised with regards to making
something work. The experts had the opportunity to interact with various compo-
nents of the system and provide their feedback based on their experience. After the
exploration phase, the experts were asked to complete questionnaires to provide their
feedback on specific aspects of the system. Figure 18 shows the experts conducting
the usability test of the 3D AI Teacher’s system.

Figure 18: Domain experts exploring the proposed system.

The interview questions focused on evaluating the performance, functionality, and

usability of the 3D AI Teacher. The questions covered various aspects, including
the application of content knowledge, proficiency in using English language, effec-

23
tiveness in communication, adaptability to students’ learning styles, organization of
instructions, attitude in creating a positive learning environment, visual design and
navigation of the interface, responsiveness and functionality of the system, as well
as the effectiveness of prompt customization in generating high-quality English les-
son content. These questions were consulted with the experts beforehand, ensuring
that they effectively evaluate the aspects of the system from the perspective of its
intended users. The questions provided a rating scale of 1-10, where 1 indicates the
lowest rating and 10 indicates the highest. The experts’ responses provided valuable
insights into these areas, highlighting the strengths and areas for improvement of the
system, which will be discussed further in the subsequent sections. Figure 19,

Figure 19: Domain experts answering the interview questions

24
VI. Discussions

A. Comparative Analysis of Expert Feedback

This section will provide an in-depth analysis on the experts’ feedback regarding the
system. The goal is to identify common themes, patterns, and discrepancies in their
assessments. Doing so will give valuable insights into the strengths and areas of
improvement of the system from the perspective of two different experts.

A..1 Performance and Instructional Aspects

Both experts highly rated the application of content knowledge (9 and 10 respectively)
and language proficiency (10 for both) demonstrated by the 3D AI teacher. They
acknowledged the system’s ability to provide useful and precise information making
it a valuable resource for teaching English in primary school classrooms.
After engaging in deeper conversations and providing detailed explanations about
the fundamental elements of the system, the two domain experts reached a consensus
on the remarkable effectiveness of incorporating OpenAI’s GPT-3. They acknowl-
edged that the utilization of GPT-3, with its access to extensive data pertaining to
English education, offers a substantial advantage. This advantage stems from the fact
that a human teacher would not be capable of retaining the same breadth and depth
of knowledge as the AI system.
However, one expert noted that the real-time skill practice application was not
possible due to the lack of an option to pause the 3D AI Teacher from its discussions.
By skill practice, the expert means the question and answer part of the teacher. What
happens during lectures is, even if the users add the prompt ”provide questions to
test student understanding,” the AI teacher will just read on those questions without
allowing the students time to respond, which the expert emphasized to be an impor-
tant part of English education. This feedback highlights one lacking feature of the

25
system that could have greatly improved its interactivity.

A..2 User Interface and Functionality

In terms of the user interface, both experts found the system’s design to be visually
appealing (9 for both) and the navigation to be easy (9 for both). They appreciated
the straightforward approach of the design, allowing even the non-tech-savvy teacher
to use it despite having a harder time coping with new technology. Despite some
loading times for generating response and animation, both experts highly rated the
system’s responsiveness and functionality (10 for both), indicating that the features
and functions worked smoothly.

A..3 Adaptability and Customization

Regarding adaptability to students’ learning styles and individual needs, both experts
provided a positive rating (9 for both). They recognized the system’s potential to
cater to different learning preferences and adjust the level of instruction accordingly.
However, one expert expressed that there is still a need for teacher instructions and
oversight when using the system. This still stems to the fact that there is a lacking
pause feature, which can hinder students from processing one information or question
effectively before the next one is given by the AI teacher.

A..4 Prompt Customization

Both experts highly appreciated the effectiveness of prompt customization in gen-

erating high-quality English lesson content (10 for both). They acknowledged the
flexibility offered by the system in customizing prompts, allowing for the inclusion
of additional information or specific instructions to tailor the lesson to their require-
ments. In the conduct of the usability testing, the experts tried different prompts like
”start the class with a prayer”, ”start the class with a joke”, and the likes. Although

26
the system was not able to provide some requirements that are very specific Filipino
context as GPT-3 is not trained for that. Still, prompt customization proved very
useful especially when instructing the AI to emphasize a point.

27
VII. Conclusions

In conclusion, the study successfully developed a real-time 3D primary school English

teacher using OpenAI and Unreal Engine, accomplishing the objectives set forth. The
system allows a teacher to input the desired primary school English topic, review and
make necessary edits to the AI-generated response, and witness the delivery of the
AI-generated lecture content through a 3D avatar. Similarly, students are provided
with the opportunity to ask any question they have about the topic and receive AI-
generated answer content delivered by the 3D avatar.
The system was able to demonstrate its capability to support and contribute to
both teachers and students in the English learning process. The feedback obtained
from the domain experts further highlights the system’s potential as they expressed
their satisfaction with its performance, user interface, and functionality. The experts
provide valuable feedback and gave recommendations on features that could help solve
some of its weakness in the area of interactivity. Moreover, the experts recognized
the effectiveness of OpenAI’s GPT-3 model in providing vast amounts of English
education-related data that can greatly help teachers in their field of work.

28
VIII. Recommendations

While the system received positive feedback from the domain experts, there are still
areas that require further development and testing to enhance its overall effectiveness
and usability. The following are recommendations proposed for future improvements:

1. Implement the pause feature: One notable suggestion from the experts is the
inclusion of a pause feature within the system. This feature would allow for
increased interactivity and engagement, enabling students to pause the AI
teacher’s lecture or question session to provide their answers or discuss concepts
with their peers. The addition of this feature would foster a more interactive
learning environment and promote student participation.

2. Conduct further testing in an actual classroom scenario: While the evaluation

process involved the domain experts exploring and providing feedback on the
system, it is essential to conduct further testing in a real classroom setting.
By implementing the system in an authentic teaching environment, it will be
possible to gather feedback from a larger group of students and teachers. This
will provide valuable insights into the system’s performance, usability, and its
impact on student learning outcomes. Additionally, conducting user testing
with diverse student populations and English proficiency levels will help identify
any potential challenges or areas for improvement.

3. Explore the ethics of using AI as teachers: The use of AI incurred a bad traction
especially at the onset if its popularity as reports of students using AI to cheat
on their school activities. There are a lot of gray areas with the use of AI,
especially issues on intellectual property. Thus, further study on the ethics of
the use of AI in education is necessary in our path towards formal use of AI in
education.

29
 Furthermore, it is important to acknowledge the limitations of the current lan-
guage model provided by OpenAI. While the model is highly capable and knowledge-
able in various English subjects, it may lack the ability to provide content in a true
Filipino context. For instance, aspects such as Filipino-style jokes, prayers, or greet-
ings may not be accurately reflected in the AI-generated responses. As such, future
work should focus on refining the language model to incorporate a more authentic
Filipino context, taking into account cultural nuances and specific linguistic features
unique to the Filipino language.

30
IX. Bibliography

[1] A. M. Baabdullah, A. A. Alalwan, R. S. Algharabat, B. Metri, and N. P. Rana,

“Virtual agents and flow experience: An empirical examination of ai-powered
chatbots,” Technological Forecasting and Social Change, vol. 181, p. 121772,
2022.

[2] A. Khadija, F. F. Zahra, and A. Naceur, “Ai-powered health chatbots: Toward

a general architecture,” Procedia Computer Science, vol. 191, pp. 355–360, 2021.
The 18th International Conference on Mobile Systems and Pervasive Computing
(MobiSPC), The 16th International Conference on Future Networks and Com-
munications (FNC), The 11th International Conference on Sustainable Energy
Information Technology.

[3] T. A. BAHA, M. E. HAJJI, Y. ES-SAADY, and H. FADILI, “Towards highly

adaptive edu-chatbot,” Procedia Computer Science, vol. 198, pp. 397–403, 2022.
12th International Conference on Emerging Ubiquitous Systems and Pervasive
Networks / 11th International Conference on Current and Future Trends of In-
formation and Communication Technologies in Healthcare.

[4] upopenu, “Meet iskou and iska: Upou chatbot at your service,” Jul 2019.

[5] J. Kim, S. Shin, K. Bae, S. Oh, E. Park, and A. P. del Pobil, “Can ai be a con-
tent generator? effects of content generators and information delivery methods
on the psychology of content consumers,” Telematics and Informatics, vol. 55,
p. 101452, 2020.

[6] OpenAI, “About openai,” Sep 2020.

[7] “Using gpt-3 for education: Use cases.”

[8] EpicGames, “The most powerful real-time 3d creation tool.”

31
 [9] A. Kurniawan, W. Fachri, A. Elevanita, Suryadi, and D. Agushinta R., “Design
of chatbot with 3d avatar, voice interface, and facial expression,” in 2015 Inter-
national Conference on Science in Information Technology (ICSITech), pp. 326–
330, 10 2015.

[10] A. Saravanan, “The limitations of chat-gpt,” Dec 2022.

[11] Q. Jiang, Y. Zhang, and W. Pian, “Chatbot as an emergency exist: Mediated

empathy for resilience via human-ai interaction during the covid-19 pandemic,”
Information Processing & Management, vol. 59, no. 6, p. 103074, 2022.

[12] J. Kim, K. Merrill Jr., and C. Collins, “Ai as a friend or assistant: The medi-
ating role of perceived usefulness in social ai vs. functional ai,” Telematics and
Informatics, vol. 64, p. 101694, 2021.

[13] M. Skjuve, A. Følstad, K. I. Fostervold, and P. B. Brandtzaeg, “My chatbot

companion - a study of human-chatbot relationships,” International Journal of
Human-Computer Studies, vol. 149, p. 102601, 2021.

[14] J. Su and Y. Zhong, “Artificial intelligence (ai) in early childhood education:

Curriculum design and future directions,” Computers and Education: Artificial
Intelligence, vol. 3, p. 100072, 2022.

[15] D. T. K. Ng, W. Luo, H. M. Y. Chan, and S. K. W. Chu, “Using digital story

writing as a pedagogy to develop ai literacy among primary students,” Computers
and Education: Artificial Intelligence, vol. 3, p. 100054, 2022.

[16] C.-C. Liu, M.-G. Liao, C.-H. Chang, and H.-M. Lin, “An analysis of children’
interaction with an ai chatbot and its impact on their interest in reading,” Com-
puters & Education, vol. 189, p. 104576, 2022.

32
[17] D. Lee and S. Yeo, “Developing an ai-based chatbot for practicing responsive
teaching in mathematics,” Computers & Education, vol. 191, p. 104646, 2022.

[18] J. Ventoniemi, “What is an ai chatbot? here’s what you should know (+ info-
graphic).”

[19] IBM, “What is natural language processing?.”

[20] OpenAI, “Gpt-3 powers the next generation of apps,” May 2022.

[21] Microsoft, “What is prompt?.”

[22] EpicGames, “What is metahuman?.”

[23] M. C. Borntrager, “What is text-to-speech?,” 2015.

[24] C. L. Sam Johnson, “Tech note: Enhancing oculus lipsync with deep learning.”

[25] Meta, “Viseme reference: Unity: Oculus developers.”

[26] L. Salem and S. Fiore, “Temple university.”

33
X. Appendix

A. Forms

A..1 Evaluation Form of Expert 1

Figure 20: Expert 1’s Evaluation Form, page 1

34
Figure 21: Expert 1’s Evaluation Form, page 2

35
Figure 22: Expert 1’s Evaluation Form, page 3

36
Figure 23: Expert 1’s Evaluation Form, page 4

37
A..2 Evaluation Form of Expert 2

38
Figure 24: Expert 2’s Evaluation Form, page 1

39
Figure 25: Expert 2’s Evaluation Form, page 2

40
Figure 26: Expert 2’s Evaluation Form, page 3

41
Figure 27: Expert 2’s Evaluation Form, page 4

42
B. Source Code

B..1 Initialize Level

Figure 28: Initialize level

B..2 Generate AI Lecture

Figure 29: Generate AI Lecture

43
B..3 Generate AI Answer

Figure 30: Generate AI Answer

B..4 Start Animation

Figure 31: Start Animation

44
B..5 Edit and Save Settings

Figure 32: Edit and Save Settings

45
XI. Acknowledgment

I became interested in exploring AI and 3D graphics, which led to the idea for this
paper. However, I soon discovered that developing the system required was very com-
plex. Despite this, the unwavering support and encouragement I received motivated
me to persevere and complete this research successfully.
First and foremost, I express my gratitude to God, my rock and savior, for sus-
taining me and guiding me throughout this research paper. I am immensely thankful
to my parents, Jessie and Estrellita De Jesus, and my entire family for their un-
derstanding and unwavering support, providing for my needs without hesitation. I
would also like to extend my heartfelt appreciation to my research adviser, Avegail
D. Carpio, for her invaluable guidance throughout the entire process.
To Nicole Bernal, your words of encouragement kept me going during times when
I felt like giving up. To Masahiro Kobayashi, I am grateful for your insights and fresh
perspective that greatly contributed to the development of my paper. Lastly, I want
to acknowledge my classmates, for their unwavering support while we work on our
respective papers.
I cannot emphasize enough how vital each and every one of you has been in this
journey. It would have been impossible for me to achieve this without your support.
From the bottom of my heart, thank you.

46