Multimodal Technologies

and Interaction

Article
Effects of Kahoot! on K-12 Students’ Mathematics Achievement
and Multi-Screen Addiction
Nikolaos Pellas

Department of Primary Education, University of Western Macedonia, 3rd km Niki-Florina, 53100 Florina, Greece;
[email protected]

Abstract: Digital platforms are increasingly prevalent among young students in K-12 education,
offering significant opportunities but also raising concerns about their effects on self-assessment and
academic performance. This study investigates the effectiveness of Kahoot! compared to traditional
instructional methods in enhancing mathematics achievement and its impact on multiple screen
addiction (MSA) among Greek students aged 9 to 12 during a STEM summer camp. A quasi-
experimental design was employed with a purposefully selected sample of one hundred and ten
(n = 110) students, who were non-randomly divided into two groups: (a) an experimental group of
fifty-five students (n = 55) who engaged with Kahoot! (using dynamic visual aids and interactive
content) and (b) a control group of fifty-five students (n = 55) who received traditional instruction
(using digital textbooks and PowerPoint slides with multimedia content) on laptops and tablets. The
findings revealed a statistically significant difference in MSA scores, with the experimental group
exhibiting lower MSA scores compared to their counterparts, indicating a positive impact on reducing
screen addiction levels. While Kahoot! led to lower MSA levels, it significantly improved overall
mathematical achievement, with a substantial effect size, suggesting a strong positive impact on
learning outcomes. The current study highlights the importance of aligning educational tools with
the intended outcomes and recommends further research to explore the broader impact of gamified
learning on student engagement, screen addiction, and learning outcomes.

Keywords: student performance; Kahoot!; primary education; mathematics; multiple screen addiction

Citation: Pellas, N. Effects of Kahoot!

on K-12 Students’ Mathematics
Achievement and Multi-Screen
1. Introduction
Addiction. Multimodal Technol.
Interact. 2024, 8, 81. https://doi.org/ Instructional design is essential for creating effective learning experiences across vari-
10.3390/mti8090081 ous formats, including online and face-to-face courses, workshops, and training programs,
among K-12 students [1]. It facilitates learning by ensuring that students gain knowledge,
Academic Editors: Jerry Alan Fails
improve skills, or change behavior while remaining engaged and motivated [2]. This
and Julius Nganji
involves setting clear instructional objectives, delivering accurate and relevant content
Received: 22 August 2024 through different media (text, images, videos, etc.), and incorporating activities and assess-
Revised: 13 September 2024 ments to apply and evaluate learning [3]. Activities like quizzes, discussions, case studies,
Accepted: 14 September 2024 or other exercises help learners apply their knowledge and skills. Finally, assessments
Published: 16 September 2024 evaluate how well learners have achieved their learning [4,5].
For subjects like mathematics, well-designed content moves beyond passive lectures
to include interactive activities and problem solving, catering to varied learning styles and
making abstract concepts more tangible [6–8]. This approach helps address the limitations
Copyright: © 2024 by the author.
of traditional methods and the challenges posed by constant digital stimulation [9]. By
Licensee MDPI, Basel, Switzerland.
considering students’ age, knowledge, and learning styles, the content fosters relevance
This article is an open access article
and appropriate challenge. Some researchers [7,8] advocated replacing passive lectures,
distributed under the terms and
which inherently favor teacher-centered instruction and rote memorization, with interactive
conditions of the Creative Commons
Attribution (CC BY) license (https://
activities, problem solving, and collaboration. This shift transforms math from memoriza-
creativecommons.org/licenses/by/
tion into a process of exploration and discovery, aligning perfectly with the way students
4.0/).
naturally engage with the world around them. While traditional methods like textbooks

Multimodal Technol. Interact. 2024, 8, 81. https://doi.org/10.3390/mti8090081 https://www.mdpi.com/journal/mti

Multimodal Technol. Interact. 2024, 8, 81 2 of 21

and chalkboards remain valuable resources, their reliance on passive learning approaches
can limit engagement and fail to compete with the constant stimulation readily available
on multiple screens [9]. This highlights the need for innovative instructional design that
not only delivers content effectively but also fosters active participation and a sense of
enjoyment in the learning process [4].
Students today are inseparable from their smartphones, tablets, and laptops, constantly
interacting with digital platforms throughout the day. This presents both challenges
and opportunities for education. Educators can leverage this constant engagement by
incorporating interactive quizzes, simulations, and educational apps alongside traditional
materials. These approaches provide to different learning styles, boost engagement, and
create an interactive learning environment that competes with, without succumbing to,
the allure of other digital distractions [5]. A wide array of technological devices and the
increased time spent using them have led to a tendency to addiction to multiple devices,
applications, and services. By blending visual and auditory features with text, video,
and animated graphics, multimedia tools can enhance understanding and retention of
information [10]. Consequently, individuals often use multiple devices simultaneously,
exhibiting multi-screen consumption behavior that involves interacting with various types
of features and elements projected across different platforms [11].
Multiple screen addiction (MSA) is a significant issue because MSA not only affects
students’ engagement but also their overall academic performance. It specifically refers to
the uncontrollable and compulsive use of various screen devices, resulting in detrimental
effects on multiple aspects of an individual’s life [12]. This addiction shows not only in
excessive screen use but also in the discomfort and negative emotions experienced when
access to these devices is limited [13]. Additionally, the concept of screen addiction en-
compasses a broader range of problematic media behaviors involving multiple screens,
extending from excessive consumption to a more severe, pathological level of use, where
the behavior significantly disrupts daily life [14]. This unveils challenges, particularly in
subjects requiring sustained focus like mathematics, which require sustained attention and
problem solving [15]. A promising approach to address these challenges in mathematics
education lies in interactive learning platforms like Kahoot! (https://kahoot.com/, ac-
cessed on 8 May 2024). It offers a promising approach to enhance student engagement
and achievement by allowing them to use a variety of computing devices, such as laptops,
tablets, or smartphones [16]. This platform leverages game-based learning and dynamic
environments to promote active participation and enjoyment specifically in the learning
of mathematical concepts [17]. Kahoot!’s “playable” features and tasks create an engag-
ing and stimulating learning experience for mathematics, potentially fostering a deeper
understanding of concepts through a more enjoyable and interactive approach [18,19].
Despite the widespread adoption of technology in mathematics, there is a need to
empirically assess the impact of specific digital tools on student MSA and achievement. On
the one hand, traditional instructional design methods, often reliant on passive learning
approaches like lectures with (digital) slideshows using PowerPoint (ver. 365), can struggle
to compete with the stimulation readily available on multiple screens [8,20]. This lack of
engagement might inadvertently contribute to MSA as students seek more stimulating
distractions on their devices. On the other hand, popular game-based learning platforms
like Kahoot! can be used on various mobile devices to promote interactive learning expe-
riences in instructional design content creation for mathematics education. Its potential
benefits for engagement and knowledge retention are widely recognized, while concerns
exist regarding its contribution to screen time overload and the development of MSA
among students [11,21]. Therefore, this study seeks to determine the impact of Kahoot!
in comparison to traditional teaching methods, addressing these questions through the
following two research questions (RQs):
• RQ1: Is there any significant difference between the MSA of students who followed Kahoot!-
supported instruction and their counterparts who received traditional instruction?
Multimodal Technol. Interact. 2024, 8, 81 3 of 21

• RQ2: Is there any significant difference between the performance of students who followed
Kahoot!-supported instruction and their counterparts who received traditional instruction?
This study aimed to investigate the impact of integrating Kahoot! as an instructional
tool on students’ achievements and its potential influence on MSA, focusing on mathematics
education. The findings will contribute to a deeper understanding of the complex interplay
between technology, learning, and student success in the era of pervasive screen use.

2. Background
2.1. Multiple Screen Addiction (MSA)
MSA, characterized by the excessive and simultaneous use of multiple devices, has
become a growing concern, especially among younger generations. People with MSA
struggle to control their screen usage, experiencing withdrawal symptoms like anxiety, loss
of productivity, and feelings of emptiness when unable to access their devices [14]. This
excessive and compulsive media consumption can be seen on televisions, laptops, tablets,
and smartphones. The visually appealing process of using multimedia content and other
non-educational content readily available on multiple screens can significantly distract
students from their studies. This distraction can lead to lower achievements. Frequent
interruptions caused by checking other screens can also result in incomplete or subpar
assignments as students struggle with time management. Multitasking itself can lead to
mistakes and lower-quality work due to divided attention and effort put into educational
tasks [13]. The constant entry of information and pressure to respond to multiple stimuli
from various screens can increase stress and anxiety levels in students. Overuse of screens,
even for educational purposes, can lead to burnout, where students feel overwhelmed and
disengaged [12].
Multiple screen use can negatively affect cognitive functions crucial for effective learn-
ing. Regular switching between devices, such as web-based platforms and smartphones,
can reduce the ability to concentrate for extended periods, leading to a decreased attention
span. This fragmented attention can result in students missing critical information or
instructions presented on educational platforms [11,22]. Other studies [14–16] have also
pointed out that divided attention can impair memory retention. When students constantly
switch between devices during educational sessions, they may retain less information
overall. Furthermore, multiple screen usage often promotes “surface learning”—focusing
on quick answers rather than deep, meaningful engagement with the material. There is
a growing body of research on MSA with different perspectives. Lin et al. [14] found a
positive correlation between media multitasking (using multiple devices simultaneously)
and screen addiction. Their study also suggests that screen use mediates the effect of media
multitasking on addiction, meaning screen time itself plays a significant role. Saritepeci [12]
advocated that MSA is a behavioral addiction, such as smartphone addiction or internet
addiction. A key indicator of such addictions is the distress caused by being unable to
access the desired object or activity (e.g., feeling uncomfortable without a smartphone).
MSA, where individuals simultaneously engage with multiple digital devices, can lead to
fragmented attention and reduced performance. This behavior often results in diminished
concentration and lower achievement, counteracting the positive effects of educational
technologies. Previous research [2,23] has also demonstrated the potential of educational
platforms and applications to enhance student engagement and classroom participation.
These tools provide immediate feedback and encourage active involvement, fostering
enthusiasm and stimulating student discussions. Given the complex nature of MSA fac-
tors and their potential interconnections, a comprehensive investigation is essential to
understand more about their impact on students’ performance in regular learning subjects.
While research has extensively documented the negative consequences of excessive
screen time on student learning and well-being, there is still a significant research “gap”
in understanding how educational technologies can be leveraged to mitigate this issue.
The potential of gamified learning platforms, such as Kahoot!, to address the effects of
MSA remains largely unexplored. This study aims to investigate the relationship between
Multimodal Technol. Interact. 2024, 8, 81 4 of 21

Kahoot! usage and excessive screen time levels among students using different devices.
Consequently, research should also consider the broader implications of digital platforms
on student engagement and investigate whether there is a link with students’ learning
performance to inform educators about the design and development of more effective
instructional content.

2.2. The Kahoot! Platform

Kahoot! (https://kahoot.com/, accessed on 8 May 2024) headquartered in Oslo,
Norway. It is a Norwegian online game-based learning platform that offers learning games,
known as ‘kahoots’, which are us-er-generated multiple-choice quizzes accessible via a web
browser or the Kahoot! application, available on Google Play and Apple Store. Kahoot!
is an interactive online learning platform that blends education with fun. Accessible on
both mobile devices and computers, Kahoot! uses interactive quizzes and games to make
learning engaging and enjoyable.
Students can access a wide range of educational content anytime, anywhere. The inter-
active and competitive nature of the platform, with points, leaderboards, and timers, makes
learning fun and motivates students to participate actively [10]. Kahoot!’s user-friendly
interface and simple setup make it easy for both teachers and students to use. This seamless
integration enhances the learning experience. Incorporating Kahoot! within the assessment
strategy offers the additional benefit of immediate feedback. This real-time feedback allows
students to grasp and learn from their errors promptly, fostering a more effective learning
experience and enhancing comprehension and retention of the material [24]. Furthermore,
the regular use of Kahoot! for quizzes and reviews reinforces learning and promotes better
memory retention through repeated exposure to the information.
While Kahoot offers several benefits, there still exist potential drawbacks to ensure
its effective use inside the classroom. Students using the platform on mobile devices
alongside other applications might get distracted by notifications, games or social media
in multitasking settings, hindering their focus on educational content. While competition
can be motivating, it can also lead to anxiety or stress for some students. This can have a
negative impact on their learning experience, especially for those who struggle with test
anxiety. The fast-paced nature of Kahoot! quizzes can lead to “surface learning”, where
students prioritize getting quick answers for points rather than deeply understanding the
material [10]. Furthermore, the platform’s format might not always allow for in-depth
exploration of complex topics, potentially limiting students’ ability to develop a thorough
comprehension of the subject matter. Students without reliable access to mobile devices or
the internet may be disadvantaged when using Kahoot! This can create unequal learning
opportunities within the classroom. Furthermore, prolonged use of Kahoot! on screens can
contribute to digital fatigue, especially if used extensively without breaks [23].
Recent research on Kahoot!’s impact on performance has shown various perspectives
about its potential usage. According to Göksün and Gürsoy [9], Kahoot! has a more
positive—though not statistically significant—impact on achievement and engagement
compared to other groups. Quizizz had a lower impact than the control group in both areas.
Despite these findings, pre-service teachers expressed positive views about gamification,
noting issues related to MSA, reinforcement, entertainment, competition, infrastructure,
and tools. Mada and Anharudin [15] pointed out that using Kahoot! in learning helps
students recall material, increases their excitement and engagement, and encourages active
participation and competition. It also enhances interest and MSA, particularly in Internet-
related subjects, leading to improved knowledge. However, challenges include the lack of
suitable devices and reliable internet connections for all students. A study by Malak [16]
found that the experimental group that used Kahoot! experienced reduced stress and
anxiety and improved self-efficacy and achievement compared to the control group. The
findings suggest integrating Kahoot! into education can enhance both psychological health
and performance.
Multimodal Technol. Interact. 2024, 8, 81 5 of 21

Several recent studies have additionally investigated the effectiveness of Kahoot! in im-
proving student engagement and achievement in mathematics education. For instance, Um-
boh et al. [25] conducted a classroom action research study focusing on elementary-school
math scores. Their findings, based on a three-cycle intervention with 22 fourth-graders in
Indonesia, suggest that Kahoot! can be a valuable tool. Student activity levels and learning
outcomes improved significantly, with the average score reaching 81% by the final cycle.
Prieto’s [18] research investigated the effectiveness of Kahoot! in reducing student MSA
and science anxiety and fostering greater engagement in secondary classrooms. Students
responded to teacher-created quizzes using their mobile devices and received immediate
feedback. Positive student evaluations suggest Kahoot!’s potential to enhance learning
through interactive quizzes. Wahyuni et al. [19] investigated Kahoot!’s impact on under-
standing mathematical symbols among pre-service math teachers unfamiliar with them.
The same study involved 100 participants who completed pre-tests and post-tests after
using Kahoot! for symbol learning. Statistical analysis showed a significant improvement
in understanding, highlighting Kahoot!’s potential as a tool for grasping mathematical
symbols. Lastly, AlAli and Wardat’s [24] research focused on the challenges of Kahoot!
as a potential solution and examined its impact on achievement and student enjoyment.
The findings contributed to understanding Kahoot!’s suitability for gifted students and
provided insights into the broader use of gamification techniques in educational contexts.
While digital platforms like Quizizz, Mentimeter, and Socrative offer engaging features
for K-12 students, Kahoot! can be considered as the most relevant due to the following
reasons [9,10,21]:
• Accessibility: While all platforms offer varying degrees of accessibility features, Ka-
hoot! stood out due to its user-friendly interface, mobile compatibility, and extensive
support resources, making it more inclusive for a wider range of participants compared
to Quizizz, Mentimeter, and Socrative.
• Economic cost: Considering the budget constraints of our project, Kahoot!’s free tier
offered a robust set of features, making it a more cost-effective option compared to the
paid tiers required for similar functionalities on Quizizz and Mentimeter. Socrative,
while offering a free version, presented limitations in terms of question types and
participant capacity compared to Kahoot!.
• Alignment with research objectives: Kahoot!’s emphasis on gamification and real-time
feedback aligned more closely with our aim to enhance student engagement and
knowledge retention than Quizizz, which primarily focuses on formative assessment.
Mentimeter and Socrative, while offering interactive elements, did not provide the
same level of engagement and immediate feedback as Kahoot!.
In summary, Kahoot! has the potential to positively influence learning performance
by increasing engagement, motivation, and retention. Given its interactive and engaging
nature, it is hypothesized that Kahoot! can redirect students’ attention from passive screen
consumption to active learning, thereby mitigating the negative impacts of excessive screen
time. By incorporating gamification and providing immediate feedback, Kahoot! may
promote a more balanced and productive screen use experience.
However, potential distractions, excessive use, and accessibility issues must be con-
sidered to maximize its benefits and minimize drawbacks. This study aimed to evaluate
Kahoot!’s effectiveness in enhancing motivation and achievement among students creating
instructional design content for mathematics education. By analyzing Kahoot!’s impact
on both motivation and success, this research sought to address the issue of declining
motivation and improve overall classroom performance. By implementing strategies to
mitigate potential challenges, educators can leverage Kahoot!’s potential to design and
develop learning tasks for all students.
Multimodal Technol. Interact. 2024, 8, 81 6 of 21

3. Materials and Methods

3.1. Research Design
A quasi-experimental design was employed to investigate the causal relationship using
a non-equivalent control group design with pre- and post-test measurements. Employing a
non-randomized sampling approach with matched group sizes and gender distribution
allow an effective control of potential confounding variables, thereby reducing the risk of
biased findings [26]. Specifically, practical constraints or existing classroom structures also
influenced group allocation, making the randomization imperfect in a STEM summer camp.
This approach was chosen because it allowed for the comparison of outcomes between an
experimental group (EG) using an alternative gamified platform and a control group (CG)
receiving traditional instruction. The former group utilized Kahoot!, a digital platform
offering dynamic visual aids and interactive content, while the latter group received
traditional instruction through digital textbooks and PowerPoint slides with multimedia
content. Participants in the experimental group utilized iPads, iPhones (with Kahoot! app
version 6.07), or laptops (with the web platform) to monitor their rankings and evaluate
their performance. In contrast, the control group solely employed laptops and tablets to
view PowerPoint presentations (version 365).
To mitigate potential threats to the validity of this study, such as the “novelty effect”,
where initial excitement about Kahoot! could influence results, several measures were
implemented. First, participants’ prior experience with digital learning resources and
mobile devices for self-directed activities was assessed. This included their familiarity
with digital textbooks, multimedia presentations, and webcasts, which helped ensure
that all participants, regardless of group assignment, had a similar baseline experience
with educational technology. Second, this study aimed to moderate the novelty effect by
carefully considering the participants’ pre-existing familiarity with digital learning tools
and adjusting the instructional design to account for these factors. By collecting detailed
information on participants’ prior experience with technology, this study aimed to control
for any differences that could affect their engagement and performance outcomes.
Overall, the quasi-experimental design provided valuable insights into the effective-
ness of Kahoot! compared to traditional methods while acknowledging the limitations
inherent in non-randomized group assignments. The approach enabled a meaningful
comparison of the two instructional methods while recognizing the potential influence of
participants’ prior experiences with digital learning tools.

3.2. Participants
This study recruited a sample of convenience consisting of one hundred and ten
students (n = 110) who participated voluntarily and were non-randomly divided into two
groups. One class (n = 55) was designated as the EG. During a STEM summer camp after
the spring semester of 2023–2024, this group received instruction in mathematics using the
Kahoot! platform. The remaining class (n = 55) served as the control group and received
“traditional” instruction (PowerPoint slides with multimedia content).
The gender distribution was even, with 48% females (n = 53) and 52% males (n = 57).
The average student age was 10.5 years old (range: 9–12 years, SD = 1.99). Most participants
were 11 or 12 years old (combined: 66%), followed by 10-year-olds (26%) and 9-year-olds
(8%). Participants categorized their daily computer usage frequency: 35% reported being
light users, 50% reported moderate usage, and 15% identified as heavy users. This process
provided insights into the extent to which the internet is integrated into their daily lives.
Additionally, the current study investigated the types of devices participants exten-
sively used. Smartphones were the most popular choice (82%), followed by laptops (70%)
and tablets (48%). This information helped establish the participants’ comfort level with
technology and their preferred methods of digital interaction (e.g., mobile learning vs.
desktop learning). Finally, to assess existing experience with educational technology, this
study measured participants’ familiarity with educational platforms besides Kahoot!. This
allowed for a more comprehensive understanding of how participants might approach
Multimodal Technol. Interact. 2024, 8, 81 7 of 21

and compare the intervention (Kahoot!) to other learning tools they might have encoun-
tered previously.
By gathering these details, this study paints a clearer picture of the participants’
digital fluency and preparedness for engaging with the computer-supported learning
environment. Students came from the school of education’s diverse departments, focusing
on instructional content creation in mathematics education. All participants did not have
prior experience using Kahoot! or other relevant platforms in previous projects.
Before the intervention, the equivalence of the CG and EG was assessed using the
independent-samples t-test. The results, presented in Table 1 along with corresponding
analysis outcomes, indicated no statistically significant differences between the two groups’
pre-test scores on the achievement and MSA scales. This suggests that before the interven-
tion, the EG and the CG demonstrated similar baseline levels of achievement and MSA.

Table 1. Equivalence of CG and EG on pre-test measures.

Tool Group n Mean S.D. T n−1 p

Test EG 55 10.43 6.87 0.622 53 0.577 *
CG 55 11.30 7.11 0.622 53 0.577
MSA scale EG 55 2.07 1.61 1.541 53 0.101
CG 55 1.93 1.55 1.541 53 0.101
Note: * p < 0.05.

Equivalence in pre-test scores plays a critical role in experimental studies. This en-
sures that any subsequent differences observed in the post-test scores can be confidently
attributed to the intervention, in this case, the use of Kahoot!, rather than pre-existing
group disparities [26]. By establishing similar pre-test scores, this study strengthened the
foundation for attributing any observed changes in post-test scores to the impact of Kahoot!
on students’ MSA and achievements.
Ethical approval for this study was obtained from the administrator of the STEM
campus, ensuring that the research adhered to the ethical guidelines outlined in the Dec-
laration of Helsinki and relevant national standards for research involving participants.
Given the age range of the participants (9 to 12 years), special attention was given to
the process of obtaining consent. Informed consent was first obtained from the parents
or legal guardians of all participating students before the commencement of this study.
This ensured that parents were fully aware of this study’s objectives, procedures, and any
potential risks involved. Additionally, assent was obtained from the child participants.
Assent is particularly important when working with minors to ensure they understand the
nature of their involvement.
The participants were informed about this study’s purpose and their role in a way that
was understandable and reassuring, ensuring that they felt comfortable participating. Both
written and verbal assent were sought, depending on the individual student’s comprehen-
sion level. This study maintained strict confidentiality and anonymity of all participant
data. All personal information was anonymized, and participants were given the option to
withdraw at any point without any consequence.

3.3. Procedure
3.3.1. Teaching Intervention
The teaching intervention for this study was designed to align with the mathematical
content covered in the primary school curriculum, which typically encompasses Grades
5–10 and focuses on developing foundational math skills such as counting, basic arithmetic
(addition, subtraction, multiplication, and division), fractions, decimals, geometry, and
basic measurement. Each quiz was designed to assess students’ understanding of key
mathematical concepts, problem-solving abilities, and critical thinking skills. The quizzes
were structured to align with the learning objectives outlined in the curriculum, ensuring
coherence and relevance to classroom instruction. Instructors administered the quizzes
Multimodal Technol. Interact. 2024, 8, 81 8 of 21

during dedicated sessions, guiding students as needed. Students participated in the quizzes
using various computing devices, while instructors monitored both administration and
student progress. To evaluate progress in learning outcomes and MSA, both groups were
closely monitored throughout the intervention. Instructors tracked student performance,
participation rates, and response patterns. Regular assessments, including pre-tests and
post-tests, were implemented to measure achievement and MSA scores.
Limiting device usage to desktop laptops and tablets for both the control and experi-
mental groups was a strategy designed to control device-related variables and ensure that
comparisons between the two groups remained fair and unbiased. Minimum specifica-
tions for the devices, such as screen size, processing power, and operating system, were
established to ensure a consistent user experience across all participants. Additionally,
brief training sessions were conducted for participants unfamiliar with the devices to
minimize potential learning curves and reduce frustration, contributing to more reliable
data collection. Device-related issues or challenges faced by participants were documented
to identify any potential biases that might have impacted this study’s outcomes.
The CG received traditional instruction using PowerPoint and/or digital textbooks.
Although typically displaying static content, these materials also included interactive
elements such as animations and multimedia. This could blur the lines between traditional
and technology-enhanced instruction, potentially influencing this study’s results. Therefore,
it was crucial to detail device usage and the instructional context to differentiate the CG
from the EG with Kahoot!. Furthermore, mathematical concepts were taught in the CG
through lectures using static visual aids (e.g., charts, graphs, and models) projected in class.
Students accessed digital readings from the course textbook on laptops and tablets, allowing
for easy reading and annotation. Classroom activities, guided by the teacher, provided
limited interaction through desktop computers and tablets. Independent study assignments
involved PowerPoint presentations to reinforce lecture material, with instructor feedback
provided through delayed written messages on tablets.
In contrast, the EG used Kahoot! to enhance engagement and interactivity. The instruc-
tor integrated Kahoot! into lectures to introduce new mathematical concepts and reinforce
learning through interactive quizzes. Students accessed these quizzes using laptops, cre-
ating a more interactive and engaging lecture experience. The instructor managed and
displayed quiz results instantly using a tablet. Kahoot! also enabled access to digital texts
in a readable format, featuring dynamic visual aids and interactive elements that made the
learning experience more engaging and visually stimulating. These aids were accessed
on laptops, tablets, and smartphones, with the screen size and capabilities of each device
enhancing the visual experience.
Students in the EG were also assigned additional Kahoot! quizzes to complete outside
of class, providing ongoing engagement and practice opportunities. These quizzes were
accessible on various devices (laptops and tablets), ensuring that students could complete
their assignments conveniently, whether at home, in the library, or elsewhere. Kahoot!
quizzes presented questions and answers in a game-like format, actively engaging students.
Immediate feedback was provided through Kahoot!, allowing students to see their scores
and correct answers right after each quiz. This instant feedback helped students address
misunderstandings quickly and reinforced key learning points. Feedback was accessible
on the same devices used for the quizzes, ensuring a seamless experience.
The use of multiple devices, such as a laptop for the main quiz interface and a smart-
phone for additional interaction, encouraged active participation and sustained interest
by allowing students to interact dynamically with the material. By specifying the types
of devices and their usage in different instructional contexts, this description provides a
detailed understanding of how the EG engaged with Kahoot! in a flexible and interactive
learning environment. The EG accessed Kahoot!, while the CG used PowerPoint on various
devices, creating a versatile learning environment. Laptops were primarily used during
instructor-led sessions and for completing quizzes that required larger screens and more
detailed interaction. Tablets were used for both in-class activities and homework, providing
Multimodal Technol. Interact. 2024, 8, 81 9 of 21

a portable, user-friendly interface for interactive learning, quick access to quizzes, and
instant feedback, especially useful for on-the-go learning and immediate participation in
gamified activities.

3.3.2. Material Development

To understand better the design rationale of the material development of this study,
the pedagogical underpinnings of Kahoot! for the EG and PowerPoint slides and digital
textbook design materials were carefully considered. The alignment of the chosen question
formats, difficulty levels, and multimedia elements with this study’s goals and target
learning outcomes has been clearly outlined.
The Kahoot! materials were developed with a focus on fostering critical thinking
and analytical skills, aligned with this study’s goal of enhancing problem-solving abilities.
For instance, questions in Kahoot! were crafted to challenge students’ problem-solving
capabilities, e.g., through a multiple-choice question designed to test arithmetic skills: “If
you have 5 apples and you buy 7 more, how many apples do you have in total?” This approach
emphasizes the application of mathematical concepts in various contexts to promote deeper
learning. In contrast, the CG utilized traditional PowerPoint slides that presented content
in a more static format. These slides were structured to convey fundamental concepts and
examples, such as through a slide that introduces multiplication with an explanation and a
sample problem: “If you have 4 groups of 3 apples each, how many apples are there in total?”
While the PowerPoint slides or digital textbooks with multimedia content aimed to
provide clear and direct instruction, they lacked the interactive and engaging elements
found in Kahoot!. The integration of multimedia elements in Kahoot!, including images,
videos, and audio clips, was designed to enhance student engagement and make learning
more dynamic. Meanwhile, the CG materials incorporated basic multimedia, such as static
images and brief video clips, to support content delivery but did not offer the same level of
interaction. This deliberate design choice highlights the contrast between the experimental
and control conditions and underscores the potential impact of interactive digital tools on
student learning outcomes.
The key differences between the PowerPoint presentations and Kahoot! in terms of
interactivity, learner control, and feedback mechanisms have been articulated. For example,
Kahoot! provided interactive elements and real-time feedback, while the PowerPoint
presentations were more static and did not include such features. Some of the most
important for the EG using Kahoot! are the following:
• Animated avatars: The types of animated avatars available to players, such as char-
acters, animals, and fantasy figures, have been specified. The contribution of these
avatars to student engagement and motivation has been described. For example,
avatars might include a cartoon cat that celebrates correct answers with a cheer or a
superhero that gives encouragement for incorrect answers.
• Colored titles and text: The use of color in the question titles and answer options
has been explained. The enhancement of readability and visual appeal through color
coding has been discussed. For instance, different colors might be used to differentiate
between answer choices, making it easier for students to follow along.
• Visual elements: The use of images, graphics, and other visual elements within the
Kahoot! questions have been detailed. The support these visuals provided for content
understanding and engagement has been explained. For example, a picture of a math
problem might be used to illustrate a question about fractions.
• Sound effects and music: The incorporation of sound effects and music into the Kahoot!
game has been described. The enhancement of the overall gaming experience and
contribution to the learning environment through these auditory elements have been
discussed. For instance, a sound effect for a correct answer might be a celebratory
chime, while a motivational tune might play during the game.
• Question variety: Specific examples of different question types used within the Kahoot!
games, such as multiple choice, true/false, open-ended, and picture choice, have been
Multimodal Technol. Interact. 2024, 8, 81 10 of 21

provided. The use of these question types to assess various learning outcomes has
been explained. For example, a picture choice question might ask students to identify
which image represents a fraction.
• Feedback mechanisms: The visual and auditory feedback provided to players, includ-
ing correct/incorrect answer animations, celebratory sound effects, and informative
messages, has been described. This feedback is intended to guide and motivate
students throughout the game.
Some of the most important design elements for the CG using PowerPoint slides and
digital textbooks that were considered are the following:
• Slide content depth: The level of detail covered in each PowerPoint slide, including
key concepts, examples, and visuals, has been specified. The support these details
provided for instructional goals was described.
• Learning activities: Any accompanying activities or worksheets used in conjunction
with the PowerPoint presentations have been outlined. The role of these activities in
reinforcing the material presented has been explained.
• Teacher guidance: The level of teacher guidance provided during the control group
sessions has been explained. The methods used by teachers to facilitate learning and
support students with multimedia materials have been described.
On the one hand, students in the EG were given the option to select from a variety of
animated avatars, including animals, superheroes, and fantasy characters. These avatars
displayed dynamic reactions to players’ answers, such as cheering for correct responses or
providing encouraging messages for incorrect ones. Preliminary observations suggested
that these features contributed to a more engaging and motivating learning environment.
On the other hand, the PowerPoint presentations for the CG did not include animated
avatars or interactive elements. The multimedia aspects were limited to static images,
videos, and animations designed to present mathematical concepts in a structured manner.

3.3.3. Data Collection and Analysis

To collect data from participants, all questionnaires were self-reported and delivered
via email. Completion time was capped at 40 min to avoid categorizing participants as
novices or experts, ensuring equal opportunity and unbiased responses regardless of expe-
rience. The main author also developed weekly lesson plans that guided the entire process.
The data analysis encompassed several key steps and statistical techniques to thoroughly
examine this study’s variables. Initially, the subscales were translated into Greek, this
student’s native language, using the back-translation method proposed by Brislin [27].
The survey instrument utilized a 5-point Likert scale, with response options ranging from
1 (strongly disagree) to 5 (strongly agree). Likert-based analyses were conducted to evaluate
the participants’ responses. Overall scores were calculated by determining the proportion
of correct responses across all assessments. Descriptive statistics, including means (Ms)
and standard deviations (SDs), were employed to analyze these overall scores.
To assess the normality assumption of the data, the Shapiro–Wilk test was employed.
The findings of this test are presented in Table 2.
To maintain participant anonymity, the authors, both computer science teachers,
handled the data processing. Performance metrics were analyzed using an adaptive scoring
method by Bridgeman and Cline [28], assigning 0 points for incorrect answers, 0.50 points
for partially correct answers, and 1 point for correct answers. To facilitate interpretation,
overall assessment scores per topic were calculated by averaging associated test question
scores. These performance results were used exclusively for this study and did not affect
participant grades. To analyze the learning experience, subscale scores were calculated by
averaging responses to corresponding items. The statistical software IBM® SPSS® Statistics
(version 27) was used to conduct the data analysis, ensuring a rigorous examination and
accurate interpretation of the findings.
Multimodal Technol. Interact. 2024, 8, 81 11 of 21

Table 2. Normality test.

Group Time Point Shapiro–Wilk Statistic p-Value Conclusion

Normally
CG Pre-test 0.823 0.155
distributed
Normally
CG Post-test 0.856 0.299
distributed
Normally
EG Pre-test 0.877 0.144
distributed
Normally
EG Post-test 0.899 0.354
distributed

3.4. Instruments
3.4.1. Demographics
To better understand the relationship between technology use and its impact on daily
life, participants were required to complete a web-based form consisting of seven questions.
This form highlighted several key topics reflected in this study:
1. Participant demographics: This study captured the age, gender distribution, and
grade levels of the participants, including how these demographics influenced tech-
nology use.
2. Technology usage patterns: This study explored the frequency and intensity of daily
computer usage among participants, categorizing them into light, moderate, and
heavy users.
3. Device preferences: This study investigated the types of devices participants frequently
used, such as smartphones, laptops, and tablets, and how these preferences impacted
their digital interactions.
4. Digital fluency: This study assessed participants’ comfort levels with technology and
their familiarity with various digital platforms, focusing on their ability to engage
with computer-supported learning environments.
5. Educational technology experience: This study examined participants’ prior experi-
ence with educational technology platforms, excluding Kahoot!, to understand how
their background might affect their engagement with new tools.
6. Readiness for technology-enhanced learning: This study evaluated participants’ over-
all preparedness and ability to effectively use technology in educational settings,
particularly in the context of instructional content creation for mathematics education.
7. Impact of no prior experience: This study considered the implications of participants
not having prior experience with Kahoot! or similar platforms on their learning
experience and engagement with the intervention.
The demographics form was designed to be completed within approximately 10 min. All
responses remained anonymous, ensuring privacy while allowing for collective data analysis.

3.4.2. Student Performance in Mathematics

To understand how well the students aged 9–12 learned mathematics, pre-and post-tests
were used, following guidelines from Terwel et al. [29], which focused on three main areas:
• Retention: How well the students remembered what they learned.
• Near transfer: How well the students could apply what they learned to similar problems.
• Far transfer: How well the students could use their knowledge in new and differ-
ent situations.
Pre-test: Checking initial knowledge.
Before starting the treatment, students took a pre-test with multiple-choice questions
on the following topics to understand what they already knew:
1. Arithmetic: Basic operations like addition, subtraction, multiplication, and division,
and understanding number properties.
Multimodal Technol. Interact. 2024, 8, 81 12 of 21

2. Geometry: Recognizing shapes and understanding basic measurement concepts like

perimeter and area.
3. Fractions: Understanding fractions and how to work with them.
4. Introductory algebra: Solving simple equations and recognizing patterns.
Post-test: Evaluating learning in mathematics.
After the lessons, students took a post-test with more challenging problems. This test
included the following areas:
• Retention: Questions to see how well the students remembered what they learned.
• Near transfer: Problems similar to those practiced during lessons to see how well the
students could apply what they learned.
• Far transfer: New and different problems to test if the students could use their knowl-
edge in unfamiliar situations.
The post-test topics included:
1. Arithmetic: Multi-step word problems involving various operations.
2. Geometry: Complex problems on shape properties and calculating the area of com-
posite shapes.
3. Fractions: Real-life problems involving fractions.
4. Introductory algebra: Word problems with simple algebraic equations.

3.4.3. The MSA Scale

The current study employed the MSA scale developed by Sarıtepeci [12] to assess MSA
levels among primary education students. This scale utilizes a 5-point Likert format to
measure participants’ agreement with statements related to multi-screen use. It comprises
three sub-dimensions: Compulsive Behavior (8 items), Loss of Control (3 items), and
Excessive Screen Time (4 items). Each subdimension captures a distinct aspect of multi-
screen addiction.
However, for this study, a modified 10-item version of the MSA was administered.
This version included 3 items from the Excessive Screen Time dimension, focusing on the
duration of screen use. Additionally, 4 items from the Compulsive Behavior dimension
were retained to assess the uncontrollable urge to use multiple screens. Finally, 3 items
from the Loss of Control dimension evaluated difficulties in regulating screen time.
The relevance of the sample items in reflecting the sub-dimensions of the MSA scale
plays a crucial role in establishing the content validity of the scale. Content validity refers
to the extent to which the items on a scale adequately cover and represent the construct
being measured. In the case of the MSA, the sub-dimensions are Compulsive Behavior,
Loss of Control, and Excessive Screen Time, each representing a distinct aspect of multi-
screen addiction.
1. Compulsive Behavior: Items within this subdimension assess the uncontrollable urge
to engage with multiple screens. For instance, a sample item like “I feel the need to
be constantly interacting with some type of screen” directly taps into the compulsive
nature of screen use, capturing the essence of this dimension. The relevance of this
item lies in its ability to measure the behavioral patterns associated with addiction,
such as the persistent and compulsive engagement with screens despite potential
negative consequences.
2. Loss of Control: This subdimension evaluates difficulties in regulating screen time. A
sample item such as “I cannot control the amount of time I spend in front of any screen”
is highly relevant, as it reflects the individual’s struggle with self-regulation. This item
contributes to content validity by addressing the core issue of loss of control, which is
a hallmark of addictive behaviors. It ensures that the scale accurately measures the
inability to limit screen time, a critical component of multi-screen addiction.
3. Excessive Screen Time: Items in this subdimension focus on the duration and extent
of screen use. An example might be “I spend more time on screens than I intend to”,
which is relevant because it captures the excessive nature of screen engagement, often
Multimodal Technol. Interact. 2024, 8, 81 13 of 21

leading to negative outcomes such as neglect of responsibilities or social isolation.

The inclusion of such items ensures that the scale covers the dimension of excessive
use, further strengthening its content validity.
Overall, the relevance of these sample items is critical in ensuring that each subdi-
mension is adequately represented. By directly addressing the key aspects of multi-screen
addiction, these items contribute to the content validity of the MSA scale, making it a
comprehensive tool for assessing the various facets of screen addiction. This thorough
representation of the construct across multiple dimensions allows the scale to accurately
measure and differentiate between the different components of multi-screen addiction.
The internal consistency of the modified 10-item MSA was deemed acceptable based
on a Cronbach’s alpha coefficient of 0.83, based on Cortina’s [30] recommendations. For
illustration purposes, two sample items from the scale are presented: “I feel the need to be
constantly interacting with some type of screen” and “I cannot control the amount of time I spend
in front of any screen”. These examples provide insight into the types of questions students
encountered when taking the modified MSA.

3.5. Reliability and Validity

The reliability of the students’ performance tests was assessed using two methods.
Internal consistency was evaluated using the Kuder–Richardson 20 (KR-20) formula on a
separate sample of 15 students to estimate the extent to which the test items measure the
same construct. Test–retest reliability was determined by administering the test to a group
of students twice, with a three-week interval, and calculating Cronbach’s alpha to assess
the consistency of scores over time.
The MSA scale was employed in this study. Pearson’s correlation coefficient was used
to examine the scale’s correlational validity with teacher evaluations, a criterion considered
a valid indicator of MSA in this context. Additionally, Cronbach’s alpha was calculated
to assess the scale’s internal consistency. The reliability of the MSA scale was assessed
using both Cronbach’s alpha and McDonald’s omega coefficients. While Cronbach’s alpha
yielded an excellent value of 0.93, indicating high internal consistency, McDonald’s omega,
which is generally considered a more robust estimate of reliability, particularly for ordinal
data, provided a value of 0.91. Although both coefficients suggest strong reliability, the
slightly lower omega value might indicate a more complex factor structure underlying the
scale than initially suggested by Cronbach’s alpha.
Table 3 presents the reliability coefficients (Cronbach’s alpha, KR-20, and test–retest)
for both the achievement test and the MSA scale. High values for all coefficients suggest
strong internal consistency and reliability, implying that the instruments accurately measure
their intended constructs.

Table 3. Reliability coefficients.

Authenticity Coefficient Calculated Reliability Factor

Cronbach’s alpha Codder–Richardson Half-partition
Compulsory
reliability coefficient reliability coefficient 20 reliability coefficient
0.91 0.92 0.92 0.88

As shown in Table 3, the reliability coefficients for both the achievement test and
the MSA scale were high (KR-20 = 0.92, Cronbach’s alpha = 0.91 and 0.88, respectively),
indicating strong internal consistency and reliability. These results suggest that both
instruments accurately measure the intended constructs.
Table 4 presents the achievement test item difficulty and discrimination coefficients.
Difficulty coefficients ranged from 0.33 to 0.71, and discrimination coefficients ranged from
0.21 to 0.86. Following established criteria by Al-Zboon et al. [31], the main researcher
applied the following selection process:
Multimodal Technol. Interact. 2024, 8, 81 14 of 21

• Items with discrimination coefficients between 0.19 and 0.39 were considered accept-
able but might require revision to improve their ability to differentiate between high
and low performers.
• Items with discrimination coefficients exceeding 0.35 were deemed strong and retained.
• Difficulty coefficients between 0.35 and 0.80 were considered acceptable.

Table 4. Difficulty and discrimination coefficients for achievement test items.

Items Difficulty Coefficient Discrimination Coefficient

1. 0.54 0.83
2. 0.46 0.26
3. 0.69 0.71
4. 0.46 0.46

Following these criteria, a 9-item achievement test was deemed suitable and included
in this study.

3.6. Data Analysis

This section presents the statistical methods employed to address the RQs:
• MSA (RQ1): To assess the impact of Kahoot! on MSA, descriptive statistics (means and
standard deviations) were calculated for MSA scale scores. Subsequently, a one-way
ANCOVA (analysis of covariance) was conducted, following the recommendations by
Lüdtke and Robitzsch [32]. Pre-test MSA scores served as the covariate to control for
pre-existing differences in MSA between the EG and CG groups. This analysis aimed
to identify statistically significant differences in post-intervention MSA levels.
• Students’ performance (RQ2): Descriptive statistics (means and standard deviations)
were calculated for achievement test scores. Another one-way ANCOVA was con-
ducted, controlling for pre-existing achievement levels using pre-test scores as the
covariate. This analysis aimed to identify significant differences in students’ achieve-
ments between the groups after the intervention.
• MSA scale validity: Pearson’s correlation coefficient assessed the construct validity of
the MSA scale by measuring the interrelatedness of items within the scale.
• MSA scale reliability: Cronbach’s alpha coefficient evaluated the scale’s internal
consistency. Moreover, the Kuder–Richardson equation (KR-20) provided further
reliability assessment.
By employing these statistical methods, the current study aimed to comprehensively
evaluate the effectiveness of Kahoot! in enhancing MSA and achievement among primary
school students.

4. Results
The analysis of this study’s findings aimed to identify the effects of Kahoot! on
students’ MSA levels in mathematics to answer RQ1. The independent t-test revealed no
significant difference in pre-intervention MSA scores between the EG and CG (t = −0.75,
p = 0.456, Cohen’s d = 0.23). This small effect size indicates that the groups were similar in
their MSA levels before the intervention, allowing for a more accurate assessment of the
impact of Kahoot! on MSA in the post-intervention analysis.
The means and standard deviations for the MSA scale scores of both groups (EG and
CG) were calculated, as shown in Table 5. The results revealed a notable difference in
mean scores between the two groups. Students who used Kahoot! in their mathematics
learning (EG) achieved a lower average MSA score (2.50) compared to those who received
traditional instruction (CG), who had a mean score of 3.51.
Multimodal Technol. Interact. 2024, 8, 81 15 of 21

Table 5. Post-achievement MSA scale.

Groups n Mean SD Adjusted Mean

CG 55 3.51 1.04 3.22
EG 55 2.50 0.95 1.99

Further analysis using a one-way ANCOVA, as detailed in Table 6, confirmed a

statistically significant difference in MSA achievement between the two groups, favoring
the CG. The eta-squared (η2 ) value of 0.356 indicates a moderate effect size, suggesting
that approximately 35.6% of the variance in MSA scores can be attributed to the type of
instruction received.

Table 6. One-way ANCOVA (analysis of covariance) on the MSA scale.

Source Sum Squares df Mean Square F Sig. Partial Eta Square

Pre-test 0.357 1 0.322 0.797 0.678
Group 21.083 1 20.077 25.761 0.001 * 0.356
Error 41.717 47 0.675
Total 66.979 49
* p < 0.001.

These findings highlight that traditional instruction, as implemented in the CG, was
less effective in reducing students’ MSA levels in mathematics compared to the use of
Kahoot!. The significant difference in mean scores and the moderate effect size emphasize
the positive influence of Kahoot! in terms of lowering screen addiction. Although the
CG exhibited higher MSA levels, suggesting greater screen addiction, Kahoot! effectively
reduced MSA and significantly improved overall mathematical achievement. This demon-
strates Kahoot!’s potential as an educational tool that not only enhances learning outcomes
but also helps address screen addiction. According to this study’s results, the CG had
higher MSA levels, which suggests that traditional instruction was less effective in reducing
MSA scores, not less effective overall. Therefore, the lower MSA scores observed in the EG
do not directly imply a lack of motivation or engagement with the subject. It is of great
importance to carefully consider the balance between using innovative tools like Kahoot!
and ensuring that these tools effectively contribute to the intended educational outcomes.
The independent t-test revealed no significant difference in pre-intervention mathe-
matics achievement scores between the EG and CG (t = −0.71, p = 0.487, Cohen’s d = 0.21).
This small effect size suggests that any pre-existing differences in mathematics achievement
between the groups were minimal, ensuring that subsequent analysis can more accurately
reflect the impact of the intervention on post-intervention outcomes.
To evaluate the impact of Kahoot! on students’ mathematics achievement, Table 7
presents the analysis based on the post-application mathematics achievement scores of
students in the EG and CG while controlling for any pre-existing achievement differences
to answer RQ2.

Table 7. Post-achievement of the students’ achievement test.

Group Sample Size Mean SD Adjusted Mean

EG 55 28.55 4.15 26.66
CG 55 18.55 4.22 17.88

The ANCOVA, set at a significance level of α = 0.05, was employed to determine

the statistical significance of the observed achievement disparities between the groups
(Table 8).
Multimodal Technol. Interact. 2024, 8, 81 16 of 21

Table 8. One-way ANCOVA for students’ mathematics achievement test.

Source Sum Squares df Mean Square F Sig. Partial Eta Square

Pre-test 15.724 1 15.724 1.866 0.155
Teaching
1375.221 1 1533.410 177.689 0.001 * 0.655
strategies
Error 377.887 47 7.512
Total 1881.322 49
* p < 0.001.

Teaching strategies, as shown by the ANCOVA results, had a statistically significant

effect on math achievement scores (F = 177.689, p < 0.001). This indicates that students
taught with different strategies achieved significantly different scores on the math test,
even after accounting for their pre-test scores. A substantial portion (65.5%) of the variance
in achievement scores is explained by the different teaching strategies, highlighting their
significant impact on student learning.
The statistical analysis in Table 8 revealed significant differences (p < 0.001) in the
post-achievement test scores between the groups (α = 0.05). Students in the EG, who were
taught using Kahoot!, achieved a significantly higher average score (28.55) compared to the
CG (18.55). Furthermore, the eta-squared (η2 ) value of 0.655 indicates that approximately
65.5% of the variance in achievement scores can be attributed to the use of Kahoot!.
Overall, this study’s results provide strong evidence that the use of Kahoot! in the
EG resulted in a statistically significant improvement in students’ achievement scores on
the post-achievement test compared to the CG. The effect size, measured by eta squared
(η2 ), suggests that 65.5% of the variation in achievement scores between the groups can
be explained by the use of Kahoot!, indicating a large and impactful effect on student
learning achievements.

5. Discussion
This study investigated the impact of integrating Kahoot! as an instructional tool on K-
12 students’ achievements in mathematics and its potential influence on MSA. Specifically,
it aimed to determine whether the interactive features of Kahoot! could enhance both
academic performance and reduce screen addiction compared to traditional instruction
using PowerPoint slides or digital textbooks.
In response to RQ1, the analysis of this study’s findings revealed a statistically signifi-
cant difference between the groups. The CG had a higher mean MSA score of 3.51 compared
to the EG, which had a mean MSA score of 2.50. This result indicates that students in
the CG underwent higher levels of screen addiction than those in the EG. Contrary to
initial expectations, the data suggest that exposure to Kahoot! was associated with lower
MSA scores, contradicting the common perception that traditional instruction might better
manage screen time [9]. While the term MSA might imply a focus on screen use, in this
context, it specifically relates to students’ self-evaluation of their mathematical abilities.
This finding stresses an important caveat, which is that the MSA does not directly measure
motivation or engagement in the learning process [15,23,24].
Although Kahoot! is designed to enhance student motivation and engagement through
its competitive and interactive features, this analysis focuses on its impact on screen
addiction. The results of this study include the discrepancy between the lower MSA scores
observed with Kahoot! and the significant improvement in mathematical achievement. The
difference in effect sizes between MSA and mathematics achievement suggests that Kahoot!
might have a more substantial impact on the latter. Therefore, the lower MSA scores
observed in the Kahoot! group, alongside the significant improvement in mathematical
achievement, suggest that Kahoot! not only reduces screen addiction but also enhances
academic performance.
The variance between the MSA scale and its specific focus on screen addiction high-
lights the importance of selecting assessment tools that align closely with the constructs
Multimodal Technol. Interact. 2024, 8, 81 17 of 21

they aim to measure. This study reveals that while Kahoot! effectively reduces screen ad-
diction (resulting in lower MSA scores) and enhances academic achievement, the tool used
to measure these outcomes may not fully capture the constructs intended. This discrepancy
challenges previous findings [9,15,25], which did not account for the dual impact of Kahoot!
on reducing screen addiction while simultaneously improving academic performance. Even
though Kahoot! is recognized for its ability to create a competitive learning environment,
this study underscores its effectiveness in both reducing screen addiction and enhancing
students’ mathematical achievements, despite limitations in assessing self-assessed math
skills directly. This finding emphasizes the need for researchers to diligently select and
align assessment tools with the specific constructs they aim to measure to ensure reliable
and meaningful results.
To answer RQ2, the one-way ANCOVA on the mathematics achievement test showed
a statistically significant difference between groups (EG = 28.55 vs. CG = 18.55), sug-
gesting a clear advantage for students utilizing Kahoot!, as previous studies have also
advocated [17,25]. While Kahoot! reduced students’ self-assessed MSA scores, it signifi-
cantly enhanced their actual performance in mathematics, as evidenced by the one-way
ANCOVA results. The computed F-value of 177.689 yielded a p-value of 0.001 at α = 0.05,
firmly establishing the statistical significance of the differences in achievement between
the two groups. This finding underlines the robustness of Kahoot! as a tool for improving
mathematical achievement, reinforcing its utility in educational settings where student
performance is a primary focus. Moreover, Kahoot! significantly improved overall mathe-
matical achievement, with a substantial effect size (η2 = 0.655), suggesting a strong positive
impact on learning outcomes. In other words, a substantial 65.5% of the variance in the
performance achievement of participants in the EG on the post-achievement test can be
linked to the use of Kahoot! Such a significant effect size is indicative of the powerful
role that interactive, gamified learning tools like Kahoot! can play in educational settings,
particularly in enhancing students’ understanding and retention of complex subjects such
as mathematics at a younger age. The observed positive influence on student achievement
might be attributed to Kahoot!’s interactive design, competitive elements, and engaging
approach. These features could potentially foster a deeper understanding and retention of
mathematical concepts through several mechanisms. For instance, the immediate feedback
provided by Kahoot! after each question not only reinforces learning but also helps stu-
dents quickly correct misconceptions, thereby setting their grasp of the material. Kahoot!’s
interactivity may enhance focus and active learning, while the competitive aspect could
encourage students to review and consolidate mathematical knowledge in preparation
for challenges [10]. Moreover, the use of Kahoot! might help students develop better
study habits and time management skills, as the anticipation of competitive quizzes can
motivate them to engage with the material more consistently and thoroughly, consistent
with Nguyen et al. [17].
This study provides evidence for Kahoot!’s potential to improve students’ mathematics
achievement. The combination of lower MSA scores and significant improvements in
mathematical achievement observed in the Kahoot! group suggests that Kahoot! not
only reduces screen addiction but also enhances academic performance. Nonetheless, this
study also acknowledges certain limitations, particularly the lack of measures for student
interest and engagement, which are critical to fully understanding Kahoot!’s impact. Using
validated instruments, such as the MSA scale, could help clarify Kahoot!’s influence and
its potential effects on other cognitive and affective outcomes, such as problem-solving
skills and confidence in mathematics [13]. Additionally, exploring the specific mechanisms
by which Kahoot! facilitates learning—such as its competitive elements and immediate
feedback—could provide valuable insights into its effectiveness. Understanding these
mechanisms in greater detail could inform the development of more refined instructional
strategies that maximize the benefits of gamified learning tools in mathematics education.
A more comprehensive approach to evaluating Kahoot!’s effectiveness should consider
not only student achievement but also their perceptions of the learning process. This
Multimodal Technol. Interact. 2024, 8, 81 18 of 21

approach would also explore how Kahoot! supports learning through its interactive
features. By integrating these considerations, researchers can better understand how
Kahoot! contributes to educational outcomes, considering both cognitive achievements
and the broader behavioral patterns it may influence.
The findings of this study regarding Kahoot!’s positive impact on student achievement
in mathematics align with previous research [16,18,24]. These studies suggest that Kahoot!
facilitates deeper conceptual understanding and knowledge retention through its engaging
and interactive design. The alignment of these findings with the broader body of research
further validates the use of Kahoot! as an instructional platform, especially in contexts
where active learning and student engagement are prioritized. Statistical analyses, includ-
ing independent-sample t-tests for MSA and one-way ANCOVA for performance, revealed
that students using Kahoot! exhibited lower self-assessed mathematical competence but
significantly outperformed the control group in actual mathematics achievement, with a
substantial effect size (η2 = 0.655). These findings suggest that while Kahoot! enhances
mathematical performance, it may not necessarily improve students’ self-assessment of
their mathematical abilities. The current study highlights the importance of aligning as-
sessment tools with the intended constructs and recommends further research to explore
the broader impact of gamified learning on student engagement and learning mecha-
nisms. This comprehensive approach will provide a more complete understanding of
Kahoot!’s effectiveness and guide the development of more effective instructional strategies
in mathematics education.

6. Conclusions
In conclusion, this study demonstrates the importance of selecting appropriate mea-
surement tools in educational research. The Kahoot! platform has significant benefits, but
careful consideration must be given to the constructs being measured to ensure that out-
comes are accurately interpreted. The findings showed a statistically significant difference
in MSA scores, with the EG displaying lower scores than the CG, suggesting a positive
effect on reducing screen addiction. Although Kahoot! contributed to lower MSA levels,
it also significantly boosted overall mathematical achievement, with a notable effect size,
indicating a strong positive influence on learning outcomes. This suggests Kahoot! not
only effectively enhances mathematical performance but also reduces MSA levels. This
study underlines the importance of aligning educational tools with desired outcomes and
recommends further research to explore the broader effects of gamified learning on student
engagement, screen addiction, and academic success.
The current study contributes to the growing body of literature advocating for the
integration of digital tools in education, particularly those that actively engage students
and foster a competitive yet supportive learning environment.
To expand our knowledge of how gamified learning tools can enhance education, this
study proposes the following areas for exploration:
• Deeper exploration of MSA: Given the unexpected findings regarding MSA, future
research should delve deeper into the relationship between gamified learning tools
and overall screen time to examine the long-term effects of such tools on students’
digital habits.
• Focus on motivational aspects: Research should explore the motivational factors un-
derlying students’ engagement with Kahoot! beyond competition. Intrinsic motivation
and its relationship to achievement could be a fruitful area of investigation.
• Mechanism studies: To better understand how Kahoot! impacts learning, studies
should focus on identifying the specific mechanisms at play, such as the role of imme-
diate feedback, competition, and interactivity.
• Combination with other tools: Investigating how Kahoot! can be integrated with
other instructional strategies or technologies to maximize its effectiveness would
be beneficial.
Multimodal Technol. Interact. 2024, 8, 81 19 of 21

• Qualitative research: Incorporating qualitative methods to capture students’ perspectives

on Kahoot! can provide rich insights into their experiences and learning processes.

7. Implications for Design and Practice

The findings of this study have significant implications for the use of interactive
learning tools like Kahoot! in mathematics education, including the following:
• Enhanced student achievements: The improvement in mathematics education achieve-
ments for students using Kahoot! (EG = 28.55 vs. CG = 18.55) highlights the potential
of gamified learning tools to significantly boost student performance. The large effect
size (η2 = 0.655) indicates substantial gains in performance, making these tools valu-
able for instructional strategies. This suggests Kahoot! not only aids in reinforcing
mathematical concepts but also promotes a competitive learning environment that
motivates students to excel. Such tools can be particularly effective in diverse edu-
cational settings where student engagement is a challenge, offering a structured yet
enjoyable pathway to mastery.
• MSA and learning: This study revealed a significant improvement in MSA scores,
suggesting that Kahoot!’s interactive and competitive format can effectively reduce
students’ multi-screen distractions. This positive correlation indicates Kahoot! may
indirectly enhance learning outcomes. However, this study emphasizes the need for
validated questionnaires and research instruments to accurately measure the impact
of Kahoot! on students’ motivation for learning mathematics. Without such tools,
the interpretation of Kahoot!’s influence on student engagement and interest remains
incomplete. Future research should explore the relationship between multi-screen
attention and actual learning outcomes more deeply, ensuring that increased screen
interaction translates to meaningful educational benefits.
• Instructional design content: Kahoot!’s interactive elements, real-time feedback, and
competitive aspects appear to foster a deeper understanding and retention of math-
ematical concepts. These features align with best practices in instructional design,
which advocate for active learning, immediate feedback, and student engagement
as cornerstones of effective education. Educators should consider integrating such
tools into their curricula to promote active learning and sustained focus. Immediate
feedback helps students quickly address knowledge gaps, enhancing their learning
process. Moreover, the competitive nature of Kahoot! can drive students to engage
more thoroughly with the material, as they prepare for quizzes and challenges, thus
reinforcing their learning outside of the classroom.
This study demonstrates the significant potential of interactive learning tools like
Kahoot! to enhance students’ achievements in mathematics education. It further empha-
sizes the role of well-designed digital tools in creating dynamic and interactive learning
environments that can cater to a wide range of student needs and learning styles. However,
it also highlights the need for precise measurement tools in educational research to accu-
rately assess the impact of these innovations on student motivation and learning outcomes.
By integrating engaging and interactive tools into the curriculum, educators can foster a
more dynamic and effective learning environment for student engagement. To sum up, the
findings suggest that, when used thoughtfully, tools like Kahoot! can transform traditional
educational practices, leading to more meaningful and sustained academic success.

8. Limitations and Future Work

There are several important limitations to consider in this study. First, the reliance on
self-reported data from Greek students introduces a potential for subjectivity, which could
skew the results, particularly given the focus on elementary school contexts. Incorporat-
ing standardized assessments alongside Kahoot! activities would offer a more objective
measure of learning outcomes. Second, this study’s small sample size (n = 110) and limited
demographic diversity constrain the generalizability of its findings. The observation that
Multimodal Technol. Interact. 2024, 8, 81 20 of 21

higher MSA scores do not necessarily correlate with increased motivation for math learning
further complicates the interpretation of the results.
These limitations underline the need for additional research. Future studies should
investigate the long-term effects of Kahoot! on student achievement and retention in math-
ematics, as well as its effectiveness across various subjects. Moreover, exploring the impact
of different teacher training approaches and how educators adapt Kahoot! for diverse
learning objectives and student needs would provide valuable insights. Understanding if
Kahoot! provides a wide range of learning styles is also crucial. By addressing these gaps,
future research could better illuminate Kahoot!’s potential to enhance student learning
across disciplines and educational settings.

Funding: This research received no external funding.

Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Informed consent was first obtained from the parents or legal
guardians of all participating students before the commencement of this study.
Data Availability Statement: The data used are publicly available and their citation is provided in
the manuscript.
Acknowledgments: I would like to express my gratitude to all participants who voluntarily partici-
pated in this study.
Conflicts of Interest: The author declares no conflicts of interest.

References
1. Mensah, F.S.; Ampadu, E. Benefits, challenges, and opportunities of using computer-assisted instruction in mathematics education.
In EAI/Springer Innovations in Communication and Computing; Springer Nature: Dordrecht, The Netherlands, 2024; pp. 31–49.
[CrossRef]
2. Göksu, I.; Özcan, K.V.; Çakir, R.; Göktas, Y. Content analysis of research trends in instructional design models: 1999–2014. J. Learn.
Des. 2017, 10, 85–109. [CrossRef]
3. Büscher, C.; Prediger, S. Teachers’ practices of integrating challenging demands of inclusive mathematics education in a
professional development program. J. Math. Teach. Educ. 2024, 27, 209–233. [CrossRef]
4. Demir, Ö.; Birgili, B. The mediating role of instructional design and video length between grade level and pupil-content interaction
in instructional mathematics videos on YouTube. Educ. Inf. Technol. 2024, 29, 5599–5629. [CrossRef]
5. Parsons, B.; Curry, J.H. Can ChatGPT pass graduate-level instructional design assignments? Potential implications of artificial
intelligence in education and a call to action. TechTrends 2024, 68, 67–78. [CrossRef]
6. Llinares, S. Instructional quality of mathematics teaching and mathematics teacher education. J. Math. Teach. Educ. 2021, 24, 1–3.
[CrossRef]
7. Salami, O.O. A flipped classroom application: Undergraduate students’ perception in mathematics. Math. Educ. J. 2024, 8, 11–20.
[CrossRef]
8. Stylianides, G.J.; Stylianides, A.J.; Moutsios-Rentzos, A. Proof and proving in school and university mathematics education
research: A systematic review. ZDM–Math. Educ. 2024, 56, 47–59. [CrossRef]
9. Göksün, D.O.; Gürsoy, G. Comparing success and engagement in gamified learning experiences via Kahoot and Quizizz. Comput.
Educ. 2019, 135, 15–29. [CrossRef]
10. Pellas, N. Exploring learning outcomes and psycho-emotional experiences of undergraduate students in digital literacy training
and support using web-based assessment platforms. J. Res. Technol. Educ. 2024, in press. [CrossRef]
11. Yildiz Durak, H.; Hopcan, S.; Polat, E.; Özüdoğru, G.; Atman Uslu, N. Investigating undergraduates’ online engagement
behaviors predictors: The role of multiple screen addictions, MSA, Academic Success, and Autonomous Learning. Technol. Knowl.
Learn. 2024, in press. [CrossRef]
12. Saritepeci, M. Multiple screen addiction scale: Validity and reliability study. Instr. Technol. Lifelong Learn. 2021, 2, 1–17. [CrossRef]
13. Saritepeci, M.; Durak, Y.H.; Atman Uslu, N. A latent profile analysis for the study of multiple screen addiction, mobile social
gaming addiction, general mattering, and family sense of belonging in university students. Int. J. Ment. Health Addict. 2022,
in press. [CrossRef]
14. Lin, T.T.C.; Kononova, A.; Chiang, Y.H. Screen addiction and media multitasking among American and Taiwanese users. J. Comput.
Inf. Syst. 2020, 60, 583–592. [CrossRef]
15. Mada, R.D.; Anharudin, A. How Online Learning Evaluation (Kahoot) Affecting Students’ Achievement and MSA (Case Study
on its Students). Int. J. Educ. Vocat. Stud. 2019, 1, 5. [CrossRef]
Multimodal Technol. Interact. 2024, 8, 81 21 of 21

16. Malak, M.Z. Effect of using gamification of “Kahoot!” as a learning method on stress symptoms, anxiety symptoms, self-efficacy,
and academic achievement among university students. Learn. MSA 2024, 87, 101993. [CrossRef]
17. Nguyen, T.T.T.; Yukawa, T. Kahoot with smartphones in testing and assessment of language teaching and learning, the need of
training on mobile devices for Vietnamese teachers, and students. Int. J. Inf. Educ. Technol. 2019, 9, 286–296. [CrossRef]
18. Prieto, M.C.; Palma, L.O.; Tobías, P.J.B.; León, F.J.M. Student assessment of the use of Kahoot in the learning process of science
and mathematics. Educ. Sci. 2019, 9, 55. [CrossRef]
19. Wahyuni, M.; Fauziddin, M.; Rizki, L.M. The effects of using Kahoot! on understanding the concept of mathematical symbols in
Higher education. Al-Ishlah J. Pendidik. 2021, 13, 1539–1545. [CrossRef]
20. Goudeau, S.; Sanrey, C.; Stanczak, A.; Manstead, A.; Darnon, C. Why lockdown and distance learning during the COVID-19
pandemic are likely to increase the social class achievement gap. Nat. Hum. Behav. 2021, 5, 1273–1281. [CrossRef]
21. Wang, A.I.; Tahir, R. The effect of using Kahoot! for learning—A literature review. Comput. Educ. 2020, 149, 103818. [CrossRef]
22. Zhang, J.; Zeng, Y. Effect of college students’ smartphone addiction on academic achievement: The mediating role of academic
anxiety and moderating role of sense of academic control. Psychol. Res. Behav. Manag. 2024, 17, 933–944. [CrossRef] [PubMed]
23. Figuccio, M.J.; Johnston, M. Kahoot! Predicts exam scores and promotes student engagement. J. Res. Innov. Teach. Learn. 2021,
15, 170–177. [CrossRef]
24. AlAli, R.; Wardat, Y. Exploring the impact of Kahoot! as a collaborative gamified mathematics learning platform for Jordanian
junior school gifted students. J. Asian Sci. Res. 2024, 14, 227–236. [CrossRef]
25. Umboh, D.; Tarusu, D.; Marini, A.; Sumantri, M.S. Improvement of student mathematics learning outcomes through Kahoot
learning games application at elementary school. J. Phys. Conf. Ser. 2021, 1869, 012124. [CrossRef]
26. Privitera, G.J.; Ahlgrim-Delzell, L. Research Methods for Education; Sage Publications: Thousand Oaks, CA, USA, 2018.
27. Brislin, R.W. Back-translation for cross-cultural research. J. Cross-Cult. Psychol. 1970, 1, 185–216. [CrossRef]
28. Bridgeman, B.; Cline, F. Effects of differential Time-Consuming tests on Computer-Adaptive test scores. J. Educ. Meas. 2004,
41, 137–148. [CrossRef]
29. Terwel, J.; van Oers, B.; van Dijk, I.; van den Eeden, P. Are representations to be provided or generated in primary mathematics
education? Effects on transfer. Educ. Res. Eval. 2009, 15, 25–44. [CrossRef]
30. Cortina, J.M. What is the coefficient alpha? An examination of theory and applications. J. Appl. Psychol. 1993, 78, 98–104.
[CrossRef]
31. Al-Zboon, H.S.; Alrekebat, A.F.; Abdelrahman, M.S.B. The effect of multiple-choice test items’ difficulty degree on the reliability
coefficient and the standard error of measurement depending on the Item Response Theory (IRT). Int. J. High. Educ. 2021, 10, 22.
[CrossRef]
32. Lüdtke, O.; Robitzsch, A. ANCOVA versus Change Score for the Analysis of Two-Wave Data. J. Exp. Educ. 2023, 1–33. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.