Modeling Development and Validation of

Metaverse Attitude Scale

Ömer Kırbaş
Ministry of National Education, Turkey

Fatih Doğan
Çanakkale Onsekiz Mart University, Turkey

www.ijte.net

To cite this article:

Kirbas, O. & Dogan, F. (2023). Modeling Development and validation of Metaverse Attitude
Scale. International Journal of Technology in Education (IJTE), 6(2), 155-186.
https://doi.org/10.46328/ijte.363

The International Journal of Technology in Education (IJTE) is a peer-reviewed scholarly online journal.
This article may be used for research, teaching, and private study purposes. Authors alone are responsible
for the contents of their articles. The journal owns the copyright of the articles. The publisher shall not be
liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever
caused arising directly or indirectly in connection with or arising out of the use of the research material. All
authors are requested to disclose any actual or potential conflict of interest including any financial, personal
or other relationships with other people or organizations regarding the submitted work.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
 International Journal of Technology in Education
2023, Vol. 6, No. 2, 155-186 https://doi.org/10.46328/ijte.363

Modeling Development and Validation of Metaverse Attitude Scale

Ömer Kırbaş, Fatih Doğan

Article Info Abstract

Article History This study is an attempt to develop a reliable and valid measurement tool to
Received: measure secondary school students' attitudes towards metaverse use. Therefore,
23 September 2022
the study reported the analyses related to metaverse attitude scale (MVAS). The
Accepted:
01 April 2023
content validity of the scale was ensured by an expert’s view, and Exploratory
Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) designed
according to the Technology Adaptation Model (TAM) were employed to ensure
its construct validity. . The structural equation model was used to reveal the
Keywords
relationship across the variables. EFA was carried out with the data obtained from
Metaverse
251 5th, 6th, 7th, and 8th-grade secondary school students enrolled in public schools
Attitude
Validity in a city in northwest Turkey. The results of EFA confirmed a 22-item three-factor
Reliability model. Then, the three-factor model obtained from EFA was cross-validated
Scale development through the use of CFA that proposed a 21-item 3-factor model. The fit indices for
CFA were found as .05 for RMSEA, .049 for SRMR, .889 for GFI, .871 for AGFI,
.9146 for NFI, .965 for CFI, and .916 for RFI. Besides, the results affirmed the
proposed 21-item three-factor model theoretically and statistically. Structural
equation modeling was used to determine the relationship across variables.
Accordingly, secondary school students were identified to have positive responses
towards cognitive and affective fields related to the metaverse. However, they
could not transform these responses into behavioral field. As a result, the
metaverse may be used in education since it is likely to observe the independent
variables such as perceived usefulness, perceived ease of use and attitude towards
use regarding the metaverse in secondary school students.

Introduction

Education plays a major role in the development of societies. While technological developments, which
accelerated with the information revolution, affect people's daily lives, it is almost impossible not to get their share
from these changes in education (Gökçe, 2008). Encountered in all areas of human life, technology paves the way
for new trends with its use for educational purposes (Demirli, 2002). Distance education has been a hot topic with
the developments in communication technologies in terms of making education more efficient, spreading,
individualizing, and meeting the demand of the increasing population (Aydın, 2002). Students' access to
educational resources is facilitated by supporting educational environments with technology (Miller, 1996). Thus,
the interaction between students and teachers also becomes stronger. Educators and administrators also need to
work together to boost student learning and streamline technology integration (Ibrahim and Shiring, 2022). In

155
Kirbas & Dogan

addition, it motivates students to construct knowledge and learn to learn in environments where teachers serve as
guides (Nitko, 1996). Thus, technology offers new opportunities for individual learning. Innovations in
informatics play a significant role in daily life as they enrich human interaction, communication, and social
connectedness. Three major waves of technological innovation have occurred around personal computers, internet
access, and mobile devices for today's users. Technologies such as spatial, immersive virtual, and augmented
reality, which is the fourth wave, are currently on the agenda (Kamenov, 2022). This wave is expected to create
the paradigm of spatially ubiquitous computing with the potential to transform online education, online business,
and entertainment. This paradigm is the metaverse (Mystakidis, 2022). Metaverse, in the sense of superiority,
refers to a virtual world in which social-cultural activities are combined, consisting of the combined words of the
meta and universe. Besides, the concept was used for the first time in Neal Stephenson's science fiction novel
"Snow Crash" in 1992.Today, it also refers to a world where virtual and reality come together and gain value
through social activities (Stephenson, 1992). Augmented reality and virtual reality are technological tools that
have been used in every field recently (Batdı & Talan, 2019). Metaverse is expected to be used in various areas
like these technologies (Tas and Bolat, 2022). The metaverse distinguishes itself in three ways from
interconnected virtual reality and augmented reality. Firstly, working via VR has a physical and processing-
oriented approach, yet metaverse has a strong side to socially serving sustainable content. Secondly, metaverse
does not necessarily have to contain AR and VR technology, the system can maintain itself without them. Thirdly,
the metaverse has an environment that can accommodate large numbers of people to strengthen social meaning
(Park and Kim, 2022a).

Education is one of the most significant areas for the development of the metaverse universe and its spread to
wider areas. Audio-visual-based education shows high potential with metaverse. Information can be transferred,
and the information can be made permanent by repeating it with the training provided to the students. However,
the most important factor in completing the education is to feel what is seen in writing while living. Kanematsu
et al (2014) stated that it is difficult and dangerous to experience radiation. Moreover, they reported that
radioactivity can be analyzed technically and scientifically thanks to the metaverse. As a result, they concluded
that difficult and dangerous experiments and activities in education may be possible with metaverse. In addition,
Tas and Bolat (2022) noted that Metaverse may be effective in learning expensive and dangerous experiments or
subjects that cannot be directly observed as augmented reality (AR). Gartner (2022) suggested that approximately
30 % of people would spend 2 hours a day in the Metaverse environment by 2027 for education, socializing and
entertainment. Therefore, the metaverse universe is of great importance to complete learning in education. Hence,
more and more education researchers are expected to actively participate in meta-universe studies in education in
the near future (Zhang et al., 2022).

Considering the studies on the metaverse universe, Park and Kim (2022a) examined the basic techniques and
approaches of the metaverse universe and the basic methods of metaverse representatives in the fields of movies,
games, and studies. They also summarized the limitations of the metaverse in terms of social implications and
challenges. Choi (2022) demonstrated how different types of teleworking (with metaverse, without metaverse)
could reduce the population pressure in megacities in different ways and offered practical suggestions to
politicians to support these practices for strategy and managers. Park and Kim (2022b) identified the types of

156
 International Journal of Technology in Education (IJTE)

worlds in the metadata to provide a gaming experience to users. They also highlighted the positive relationship
between learning motivation and game experience. They presented examples of 5 different virtual worlds on the
metaverse base. These virtual worlds are considered as survival, maze, multiple choice, race/jump, and an escape
room. The results suggested that education has changed significantly with the COVID-19 epidemic and that new
methods to satisfy educators and students should be explored here. The results also showed that metaverse is the
method that can be presented as an alternative. With the use of the metaverse in education, the absence of time
and space restrictions may provide equal opportunities for those who suffer from physical problems and
environments (such as disaster areas). Suh and Ahn (2022) conducted a study with primary school students on
employing the metaverse for a student-centered constructivist education after the pandemic period. Examining
the related literature on the subject, the researchers reached primary school students in Korea through using 18
items for the measurement of each factor in the meta database and made statistical analyzes including the
difference of means with the t-test. The analysis results revealed that 97.9% of the students had experiences with
the metaverse and that 95.5% of them thought it was closely related to their daily lives. Gomes and Klein (2013)
reported a design study in metaverse with 6 volunteer students who were requested to have a design with the
opportunities. Gomes and Klein (2013) stated that the students had some problems while they were working and
they also stated that the students had little competence for teamwork and that the lack of devices suitable for the
metaverse second life environment led to some problems. Jovanovic and Milosavljevic (2022) introduced the
VoRtex application they developed to create an educational experience in the virtual world. The purpose of
designing VoRtex was to support collaborative learning activities with the virtual environment. VoRtex was an
accessible application in which modern technology via metaverse is used with the support of education standards.
Finally, the researchers analyzed the advantages and disadvantages of collaborative learning with the metaverse
platform and real-world classroom environments. Barry et al. (2015) introduced a blink system for avatars in the
“problem-based learning” class in the metaverse environment. The emotional reactions of university students to
various problems they were requested to discuss in the project, which included simple and difficult problems,
were measured. In this study including mathematical problems, students were left alone with question sessions of
10 minutes. After completing the study, the students were asked to respond the questions by conducting a
questionnaire. Accordingly, the results affirmed that difficult questions would make the students' emotions
unstable and the number of blinks increased. Lee and Hwang (2022) proposed a system that includes virtual reality
and metaverse in education to compensate for the deficiencies in the distance applied education models available
today. An aircraft maintenance simulation was developed for the system proposed in the study. The findings
suggested that 40 volunteers did not have preliminary knowledge about aircraft maintenance. The application was
conducted and the data were obtained. The analysis results indicated that the recommendations were suitable for
the use of the systems they confirmed a sense of spatial presence.

Kye et al. (2021) described four types of the metaverse universe to explain the potential and limitations of
metaverse in educational applications. They asserted the potential of the metaverse to be a new educational
environment as follows; “A space for social communication, greater freedom to create and share, and new
experiences through virtualization”. Locurcio (2022) emphasized the necessity of using metaverse and artificial
intelligence in the field of dental education. Jaramillo-Mujica et al. (2017) put forward the idea of designing a 3D
environment and metaverse environment to encourage the learning of physics engineering students. Besides, Huh

157
Kirbas & Dogan

(2022) analyzed the computer-based test application for Korea's medical licensing exam to contribute to the
metaverse in medical education. Diaz (2020) investigated the metaverse universe for hybrid and mobile learning.
The researchers found that the traditional model varied from being static to more dynamic, and the students are at
the center in the inverted classroom. In addition, Tas and Bolat (2022) presented an bibliometric report on
metaverse. Accordingly, the number of studies on the metaverse has generally increased in recent years.
Furthermore, they also found that the keywords such as virtual reality, second life and augmented reality are
mostly used in the studies conducted on the use of metaverse in education. Likewise, Saritas and Topraklıkoğlu
(2022) carried out a literature review with 22 different documents on the concepts of Metaverse and identified the
aim of each articles in the research and their contributions to the field of education. While some researchers were
working to introduce the metaverse, others created a metaverse universe themselves. The study also depicted the
presence of studies on postgraduate-undergraduate education in the field of education. The relevant literature
involves a scale development (5th and 6th grade) study for primary school students in South Korea. However, there
is no such a study specifically published on developing a measurement tool to examine the use, approaches, and
attitudes of secondary school students for metaverse.

Problem Status

In recent years, there have been tendencies in education for students to receive education independent of time and
place. The reason why this model, known as distance education, is preferred is that students do not fall behind due
to their current education or health reasons. Many regions have provided distance education especially during the
covid19 period. Digital education environments used in distance education activities have emerged during the
epidemic process. Distance education has become a part of life during this period when digitalization has reached
its peak (Durak et al, 2020). Today, countries in an effort to move forward in the field of education combine
education and technology as a whole. The reason why these countries use technology more actively in learning
environments is to provide students with an opportunity to learn in practice, research-based, and more fun ways.
Thus, technological developments and teaching materials are changing and developing day by day. Thus, there is
a need to search on the use of metaverse, a product of technology that is prone to use in education.

Offering a realistic experience among educational materials, metaverse allows students to get to know virtual
reality better. Although there are no widespread studies on the metaverse in related literature yet, several education
institutions in different countries have conducted studies on the metaverse field. The lack of studies on secondary
school students in the relevant literature and the inability to obtain measurement tools for students' intentions,
behaviors, and attitudes towards the metaverse made it necessary to develop a new scale to contribute to the
literature. Accordingly, the problem statement of the study was noted as “What is the relationship across the
variables affecting the secondary school students' use of metaverse as a learning environment?”
On the other hand, the following sub-problems were provided,
 Is the scale developed for the secondary school students’ metaverse attitudes according to the TAM
model valid and reliable?
 What are the variables in the scale developed for the secondary school students’ metaverse attitudes
according to the TAM model?

158
 International Journal of Technology in Education (IJTE)

Purpose and Importance of the Research

It is widely known that technology is an inseparable part of our daily lives and plays a significant role in facilitating
lives through diversifying and differentiating according to their needs. Rapid advances in technology are not only
related to science, but also have a high connection with education (Yıldırım, 2020). Various concepts such as
AR/VR and Metaverse, which are technological tools, have emerged with the changes in education systems from
the past to the present. Due to the increasing importance of education with technology, societies have always
attempted to create more satisfactory and contemporary education systems. Thus, the adaptation of technological
products to the field of education has always been prioritized. Davis (1989) argued that user acceptance of new
technologies depends on the variables such as perceived usefulness and perceived ease of use. Hence, the success
of information systems may not only be evaluated according to technical and managerial qualities, but also vary
across the demographic characteristics, expectations and perceptions of the people who use the system, and the
perceptions of the users can primarily have an effect on this success. Thus, individuals’ attitudes and intentions
may directly affect the behaviors. As a result, behavioral changes with the determination of an attitude or intention
can be predictable.

However, psychological variables such as attitude, intention, perception, and anxiety, which are used to predict
behaviors, cannot be measured directly. Such psychological variables can only be determined by observing
behaviors, question lists, interviews, projective techniques, or by individuals' responses to various questions in a
measurement tool. Thus, individuals’ behavioral changes can be predicted through measuring psychological
variables and detailed information can be obtained about adaptation processes. Therefore, this study aims at
developing a measurement tool to examine the processes of secondary school students' acceptance of the
metaverse, one of the latest products of technology, as a new learning environment. The dependent and
independent main variables of the TAM model were taken into account during the development of the
measurement tool. The tool is capable of measuring individuals' responses in cognitive, affective and behavioral
fields related to the metaverse. Accordingly, perceived usefulness, which is one of the main variables of TAM, is
defined as the degree of belief that a person will increase in performance through using a certain system. Perceived
ease of use, another main variable, is described as the degree of personal belief that a person's use of a particular
system requires effort. Attitude, which is another dimension of the model and which is formed as a result of the
accumulation of emotions and ideas, refers to the positive or negative reaction of the individual to the system.
Intention is the main determinant of an individual's behavior and indicates the possibility of an individual to
perform a certain behavior. The model suggests that ease of use, perceived usefulness, and attitude may be
effective on individuals' behavioral intentions.

Selection of TAM

There are some important models in the relevant literature to examine the assimilation of information technologies
such as the Theory of Reasoned Action, the Theory of Planned Behavior (Mathieson, 1991), the Technology
Acceptance Model (Davis, 1989), and the Unified Theory of Acceptance and Use of Technology (Guo and Liu,
2013). The foundations of these models are based on the Theory of Reasoned Action (TRA). TRA suggests that

159
Kirbas & Dogan

an individual can predict their actions based on pre-existing attitudes and behavioral intentions. TAM used in the
study is also a model that includes the acceptance of information technologies based on the TRA model (Moon
and Kim, 2001). Landry et al. (2006) noted that TAM is suitable for use in academic environments. In addition,
many factors should be investigated, including the variables suggested by TAM, when examining whether users
are satisfied with applications for computers and their derivatives (Adams et al., 1992). Therefore, the TAM model
was deployed to examine the research problem regarding students' attitudes, intentions, and perceived benefits.

Method

This research is based on the exploratory sequential mixed research method, in which quantitative and qualitative
research is used together for middle school students' attitudes towards the metaverse. Cresswell (2013) stated that
the mixed method should combine or integrate qualitative and quantitative data. The mixed method suggests that
qualitative and quantitative data can be used simultaneously to eliminate the lack of a data group that arises from
using only the data source. According to Cresswell (2013), the exploratory sequential mixed method, which is a
theme of the mixed method, starts with the data collection method of the research and explores the participants’
perspectives. A quantitative measurement tool is created to be used in the second stage with the analysis of the
elicited data (Cresswell, 2013). In this regard, focus group interviews were conducted with a group of secondary
school students to create an item pool. At the end of the interview, opinions that might be attitude sentences about
metaverse were noted and contributed to the creation of the items. Some items were added by a literature review
and panel system. Expert opinion was sought to determine the extent (scope validity) of the items in the scale and
to help the measurement tool reach the target (Tekin, 1977). Factor analysis was performed to identify the
construct validity of the scale. Factor analysis is a method that reduces the number of items related to each other
by bringing them together. The purpose of factor analysis is to reduce the number of variables and classify them
according to the relationship across variables (Kalaycı, 2010).

Research Prospectus Outlined

This research aims to develop a valid and reliable tool that will serve to measure secondary school students'
attitudes towards the metaverse. The data were collected from a total of 251 5th, 6th,7th, and 8th-grade students
studying in a public secondary school in a city located in Northwest Turkey. Table 1 displays the steps followed
for the development of the tool, which will help to understand the research more easily.

Table 1. The Steps followed in the Study

Sub problem Pathway Stage Process
Items were collected from the TAM literature and through focus
Stage 1 group interviews. The draft MVAS was created with 54 items
Sub problem 1

Content
and its content validity was ensured by expert opinions.
Validity
The CVR and CVI values of each item in the form were
Stage 2 calculated as a result of the opinions of the experts. Items
numbered “7-18-24-32-51” were removed from the draft

160
 International Journal of Technology in Education (IJTE)

Sub problem Pathway Stage Process

MVAS. Face validity was also provided with a group of
secondary school students.
The construct validity of the form was provided through EFA. It
Construct
was conducted with251 secondary school students. Items "1,2,
validity Stage 3
3, 5, 6, 7, 8, 9, 10, 12, 14, 15, 17, 24, 26, 27, 29, 30, 35, 42, 43,
44, 46, 47, 48 and 49” were removed from the draft MVAS
ANOVA Tukey's Nonadditivity analysis was carried out to
check their relationship with each other and the homogeneity of
Control of the items constituting the MVAS. Whether the phenomenon can
items Stage 4 be measured appropriately with the measurement tool was
determined by Hotelling's T-Squared analysis. Sequence
validity of the items in the measurement tool was performed by
Intraclass correlation coefficient
The confirmation cross-validity of the MVAS was performed
Confirmation with 211 secondary school students. The data were obtained
of construct from a different sample from the sample group used in the pilot
Stage 5
validity application. Item “18”, which could not meet the construct
Subproblem 2

validity assumptions, was removed from the MVAS.

Convergent and discriminant validity was also ensured.
Cronbach’s alpha coefficients were computed for each sub-
Reliability Stage 7
factor and overall, the scale.
Modeling The hypotheses were checked according to TAM and the data-
Stage 8
model fit was tested
A valid and reliable MVAS was prepared. The MVAS
Result Stage 9
contained 22 productive Likert-type items

The final MVAS was obtained by applying the procedures depicted in Table 1 respectively. Table 2 shows the
MVAS containing 22 productive Likert-type items.

Table 2. Metaverse Attitude Scale for the Secondary School Students

Neither agree
nor disagree
Disagree
Strongly

Strongly
disagree

Dimensions Code Items

Agree
agree

PU1 Item45: With Metaverse technology, more

1 2 3 4 5
realistic virtual environments can be created.
PU

PU2 Item40: With Metaverse, experiments in

1 2 3 4 5
science class become quite easy.
PU3 Item4: Gaming is fun in the Metaverse 1 2 3 4 5

161
Kirbas & Dogan

Neither agree
nor disagree
Disagree
Strongly

Strongly
disagree
Dimensions Code Items

Agree
agree
universe
PU4 Item25: With the Metaverse, experiments
1 2 3 4 5
can be performed more easily.
PU5 Item33: We can be more creative with
1 2 3 4 5
Metaverse.
PU6 Item23: Metaverse requires the Internet. 1 2 3 4 5
PU7 Item11: Metaverse is an emerging
1 2 3 4 5
technology.
PU8 Item22: Metaverse is a virtual reality tool. 1 2 3 4 5
PU9 Item34: I am aware that the Metaverse will
1 2 3 4 5
be more prevalent in the future.
PU10 Item13: I think Metaverse will expand to
1 2 3 4 5
wider areas in the future.
PEU1 Item41: Thanks to the metaverse universe, I
1 2 3 4 5
can be more successful in my lessons.
PEU2 Item28: I want to study with Metaverse. 1 2 3 4 5
PEU3 Item16: My interest in the lesson increases
1 2 3 4 5
when the lesson is taught with Metaverse.
PEU

PEU4 Item21: Lesson topics covered in Metaverse

1 2 3 4 5
are more fun.
PEU5 Item19: Thanks to Metaverse, I can study
1 2 3 4 5
without going to school.
PEU6 Item20: The knowledge which learn with
1 2 3 4 5
Metaverse is quite permanent.
UA1 Item32: We get lazy with the metaverse. 1 2 3 4 5
UA2 Item31: Metaverse is a waste of time. 1 2 3 4 5
UA3 Item37: Metaverse can interfere with the
1 2 3 4 5
privacy of people.
UA4 Item39: A negative situation that may arise
UA

in the metaverse universe can cause the death 1 2 3 4 5

of a person.
UA5 Item38: Metaverse can cause health
1 2 3 4 5
problems in humans.
UA6 Item36: The psychology of people using
1 2 3 4 5
metaverse applications may be impaired.

162
 International Journal of Technology in Education (IJTE)

Research Model and Hypotheses

This sections covers the relations across the variables in terms of examining the reflections of the secondary
school students' attitudes towards the use of Metaverse as a learning environment in education within the scope
of the TAM. Figure 1 displays the model used in the present study.

Figure 1. Research Model and Hypotheses

The research model was designed by means of the dependent and independent variables in TAM. Accordingly,
the hypotheses (H1, H2, H3 and H4) were structured as follows:
 Hypothesis 1 (H1): Secondary school students' perception of ease of use regarding metaverse as a
learning environment has a positive effect on their attitudes towards the use of this technology (PEU's
effect on UA)
 Hypothesis 2 (H2): Secondary school students' perception of ease of use regarding metaverse as a
learning environment has a positive effect on their perceived usefulness (PEU's effect on PU).
 Hypothesis 3 (H3): Secondary school students' attitudes towards the use of metaverse as a learning
environment have a positive effect on their intention to accept this technology (UA’s effect on BI).
 Hypothesis 4 (H4): Secondary school students' perceived usefulness regarding metaverse as a learning
environment has a positive effect on their attitudes towards the use of the metaverse (PU's effect on UA).

TAM is a model that explains the causal relationships between perceived usefulness, perceived ease of use,
attitude, intention to use, and actual use. The model points that the perception of ease of use and the perceived
usefulness have a significant effect on individuals' intention to use information technology (Davis,1989). Various
external variables such as social factors, design features, image, job fit, output quality, demonstrability, experience
and volunteering were added to the model and the effect of subjective norms on the dimensions of the classical
model was explained (Venkatesh ve Davis, 2000). Thus, more than one TAM with many independent variables
in the literature was created. In the present study, it was excluded from the model as it may lead to Fishbein
paradigms in external variables (Fishbein, 1966). Since metaverse technology is not widely used in education
activities today, the dependent variable actual use representing the behavioral field of TAM was excluded from
the research model. Therefore, the simplest TAM was used in the creation of the research model and the
establishment of the relevant hypotheses.
Data Collection Process for EFA

163
Kirbas & Dogan

This study examined secondary school students' attitudes towards the metaverse. To start the study, firstly, the
publications on metaverse in the literature were examined. However, the related literature lacks a measurement
tool whose validity and reliability have been proven. For this reason, the MVAS was developed and used to
determine the secondary school students’ attitudes towards the metaverse. Before starting the data collection
process, secondary school students were informed on semantic WEB and metaverse for a short time in each
classroom within the scope of Information and Technology course. Semantic WEB and meta-universe were
clarified and students were provided awareness of their functions. Great attention was paid to include students
who use VR Box in their lessons, in any project or in any game, or those who are familiar with augmented reality
technology. A group of students with VR Box experience, who are not included in the sample group, were
requested to write an essay describing their usage skills, emotions, thoughts, and their relationship with technology
regarding metaverse. The texts written by 12 students were read. The sentences suitable for the scale were arranged
and included in the draft MVAS. In addition, criteria items suitable for the research by the TAM were searched
in the literature. Special importance was attached to reflect the cognitive, affective, and behavioral dimensions of
the items thought to be included in the draft MVAS. Negative items were also included in the scale. The items in
the study were designed according to the operational variables of the TAM model. TAM is a moving model that
depends on the motive of intention, belief, and behavior (Sun and Zhang, 2006). In this regard, phenomena that
may affect behavior were modeled within the scope of technology use. These variables were cognitive responses
(variables: perceived usefulness, PU and perceived ease of use, PEU), affective responses (variable: user attitude,
UA), and behavioral responses (variable: behavioral intention, BI). Afterwards, an item pool consisting of 54
Likert-type attitude items was created. The draft MVAS was presented to field experts of 14 people. The items in
the draft MVAS were evaluated by an expert in terms of language, scope, and psychometrics. The draft MVAS,
whose content validity was also ensured, was presented to the students for pilot application along with a personal
information form containing some categorical variables.

Respondents’ Demographic Profile for EFA

Population can be expressed as a community consisting of units that fit a certain definition, on which research
will be conducted, or that constitute the study area of the research and can generalize the results (Özmen, 2000).
The population consisted of 3650 secondary school students studying in the Dilovası district of Kocaeli province
in Turkey during the 2021-2022 academic year. The sample of the study held 251 students who were chosen by a
simple random sampling method. It is a sampling type in which each unit forming the population has an equal
probability of being included in the sample or each unit has an equal chance of being selected and the units do not
affect each other (Altunışık et al., 2012). The following formula prepared by Yamane (1967) based on the
population size was used in determining the sample size.
(𝑁𝑡 2 𝑝𝑞)
𝑛= ……………………………………………………………………………….....(1)
(𝑑 2 (𝑁−1)+𝑡 2 𝑝𝑞)

In Equation (1), N: Number of individuals in the target population (main mass), n: Number of individuals to be
sampled, p: Probability of occurrence of the investigated event (0.10), q: the probability of not happening of the
investigated event (0.90), d: ± the sampling error accepted according to the incidence of the event. (0.05), t: the
theoretical value (1.96) found according to the t-table at a certain significance level. With the above-mentioned
formula (1), the sample number was obtained as 134. However, 251 secondary school students were included in

164
 International Journal of Technology in Education (IJTE)

the study. Accordingly, 119 (47.4%) of the 251 secondary school students participating in the study were girls
and 132 (52.6%) were boys. Of all the students, 45 (17.9%) were the 5th graders, 47 (18.7%) the 6th graders, 60
(23.9%) the 7th graders, and 98 (39.4%) the 8th graders. The students were also asked whether they had personal
computers and internet access at home. 213 (85%) of the students stated that they had the internet at home, and
38 (15%) did not have the internet at home. 97 (38.6%) students had a computer while 154(61.4%) did not.
Besides, an answer was sought by asking about the level of students' knowledge about metaverse. It was ranked
on 5 variables such as very little, little, moderate, good, and very good. The number of students who answered
“very little” was 85 (33.7%), those who gave “a little” answer was 54 (21.4%), those with “moderate” answers
were 83 (32.9%), the number of those who gave “good” answers 19 (7.5%) and those who stated it as “very good”
was 11 (4.4%).

Data Collection Process for CFA

CFA analysis was conducted with 211 participants. The data were collected via google forms. Since data were
collected to validate the scale, the number of questions about categorical data was not reduced in the study. It is
assumed that scale development is suitable for its purpose as the research process is related to secondary school
students.

Respondents’ Demographic Profiles for CFA

Data were obtained from the sample group with the following demographic characteristics for CFA. 94 (44.54 %)
of the secondary school students who provided feedback to the CFA analyses were girls and 117 (55,46 %) boys.
The secondary school students giving feedback were in the 5th (44 students, 20.85 %), 6th (43 students, 20.37 %),
7th (51 students, 24.17 %) and 8th (73 students, 34.59 %) grades. When 130 (61.61 %) of the students did not have
a computer at home, 81 (38.39 %) had computers in their homes. While 188 (89.09 %) students had permanent
internet at home, 23 (10.91 %) did not. 67 of the students(31.75%) knew little about metaverse. While 73 (34.59%)
students knew moderately. 5 students (2.36%)stated that their metaverse knowledge was very good.

Results and Discussion

Overview of Statistical Analyses

Statistical analysis in the study was carried out in 4 steps. In the first step, an expert opinion of 14 people was
taken to ensure the validity of the draft MVAS content. In the second step, EFA was provided to explore the factor
structure underlying the measurement tool. EFA was performed with principal component analysis through use
of SPSS 21.0, and the interactions of the items with each other were also examined. In the third step, the structures
of the factors obtained from EFA were confirmed by CFA (Yong and Pearce, 2013). CFA was achieved with the
Analysis of Moment Structure (AMOS) 21 program using the maximum likelihood estimation (MLE) method.
Hence, the combination and reflection validity of the variables were determined. Besides, the cronbach alpha
coefficient was calculated for Likert items. In the last step of the study, hypotheses related to the variables of the
TAM model were established and the validity of the hypotheses was examined.

165
Kirbas & Dogan

Content Validity

Content validity ensures that items on the scale are necessary and that unwanted items are removed from the scale
(Boudreau et al., 2001). In addition, it determines to what extent the developed items represent the attitude which
is expected to measure. Thus, instead of using irrelevant concepts, it may be possible to reveal concepts with
stronger representation (Ayre and Scally, 2014). In content validity, which is a method used to determine whether
each item in the test is a sufficient or appropriate question to measure the behavior that is intended to be measured,
one of the ways used for examination is expert opinions (Büyüköztürk, 2009). As a result, it is vital to cover the
needs-oriented data with the expert opinions of each item in the scales (Yeşilyurt and Çapraz, 2018). Expert
opinions were sought to ensure content validity. Gaps were left where the experts could make explanations in
addition to the questions and they were requested to reflect their opinions. The draft of the experts whose opinions
were taken to ensure the content validity of the scale is as follows;
 For the simplicity and intelligibility of the written language, one lecturer working at the Department of
Turkish Education, Faculty of Education, Çanakkale Onsekiz Mart University (ÇOMU)
 One lecturer working in the Measurement and Evaluation Department, Department of Educational
Sciences, Faculty of Education, ÇOMU,
 For the analysis of the items, four instructors working at the Department of Science Education,
Department of Mathematics and Science, Faculty of Education, ÇOMU
 It was presented to a total of fourteen experts, eight of whom were science teachers working in
institutions affiliated with the Ministry of National Education.

The experts in the study were determined by a convenient sampling method. The draft of the questions presented
to the experts was also arranged as suggested by Yusoff (2019).
 The item is irrelevant to the measurement subject, it should be removed (1 point)
 The item is somewhat relevant to the subject being measured; it should be reviewed (2 points)
 The item is relevant to the subject being measured, but it should be slightly corrected (3 points)
 The item is very relevant to the subject being measured; the item must be found as a constant (4 points)

The higher the validity of the content on a scale, the more accurately the measurement of the targeted state will
emerge. For this reason, it was tried to prevent taking wrong steps in the development of the scale with the
qualitative and quantitative opinions taken from the experts. It is essential to find a quantitative criterion for
estimating the validity of the scale. In content validity, the content validity index (CVI) and content validity ratio
(CVR) are used as quantitative criteria. CVR is an internationally accepted criterion used to decide whether to
include items in separate scales. CVI is defined as the average content validity rate of all items at the end of the
scale. In short, CVR is used to determine whether each item is valid, while CVI is used to determine the
relationship of each item to the scale (Shi et al., 2012). The content validity studies were carried out in six steps
with the recommendation of Polit and Beck (2006). These six steps and their order are as follows;
 Preparation of content verification form
 Selecting a review panel with expert staff
 Content verification

166
 International Journal of Technology in Education (IJTE)

 Examination of areas and items

 Calculation of scores for each item
 Finding CVR, I-CVI, and S-CVI values

Ayre and Scally (2014) report was used to calculate the CVR value, and Lynn (1986) and Polit and Beck (2006)
reports were also used to find the CVI value. In addition, the CVR proposed by Lawshe (1975) in the development
of the draft MVAS was calculated. Data were analyzed through using the interpretations suggested by Ayre and
Scally (2014). Accordingly, the following equation was used
𝐴
CVR= 𝑁 −1
2

N; The total number of experts, A; According to Yusoff's scoring (2019), it is the number of experts who make
the “relevant” evaluation by giving 3 or 4 points to any item. The CVR value obtained at this stage is a statistical
tool used as a criterion for the rejection or acceptance of some items While taking the opinions of the experts who
gave 3-4 points in the calculations, the opinions of the experts who gave 1-2 points were examined. The question
“What is your suggestion?” to the expert who answered that the item should be corrected (2 points) was asked. In
addition, the expert who answered that the item should be removed from the scale (1 point) was asked "why".
According to the evaluations of Ayre and Scally (2014), CVRcritical=critical CVR, a positive value at the level of
α=.05 was used in the interpretation of the CVR value. Accordingly, the limit value for the CVR critical value
was accepted as .51 in the evaluations of the experts. Accordingly, the CVR values of items 7, 18, 24, 32, and 51
were determined to be below the critical value and these items were removed from the draft MVAS under the
strength of the opinions of fourteen experts at the α=.05 significance level. On the other hand, since the CVR
statement, which was first put forward by Lawshe (1975) and later revised by Ayre and Scally (2014), is based
on the empirical approach, the calculations of content validity have been expanded by taking Yusoff (2019)'s
suggestion into account in the present study. Yusoff (2019) suggested two separate CVI forms. These are the item-
level content validity index (I-CVI VR), which helps define the scope index of the item, and the scale-level content
validity index (S-CVI) that refers to general content validity. I-CVR was used to identify whether each item in
the scale could be used as a criterion. On the other, the S-CVI value was calculated to determine whether the
experts were compatible with each other. The S-CVI value, which examines the agreement of the expert opinions,
can be calculated by using two methods. The first of these is S-CVI/Ave, where the mean of I-CVI of all items is
known, while the other is S-CVI/UA, which belongs to the items marked as 3-4 points on the scale. S-CVI/UA is
named as the universal agreement method content validity index. Lynn (1986) and Polit and Beck (2006) stated
that the I-CVI value should be at least .75 or greater in studies involving five or more experts. In addition, the
literature shows that S-CVI/Ave and S-CVI/UA values should be at least .8 for the general validity of the scale
(Orts-Cortés et al., 2013). Herein, the I-CVI values of items 7,18,24,32, and 51 in the draft MVAS were obtained
as less than .78. Moreover, the S-CVI/Ave and S-CVI/UA values were found to be .93 and .84, respectively.
Finally, the experts’ views were converted into kappa values to eliminate the chance effect after the calculations
and to prevent the occurrence of a situation where the experts could predict the I-CVI. The Kappa index (k*) is a
fit index indicating that the items are subject-oriented, clear, and understandable, beyond the possibility of being
something other than their motivating feature (Wynd et al., 2003). In addition, the evaluation of the kappa value
was carried out with the Kappa sequence suggested by Fleiss (1971). The Fleiss kappa scale describes as
“Excellent ≥.74”, “good between .60-.73”, “moderate between .40-.59”, and “poor ≤.39” for each item in the

167
Kirbas & Dogan

measurement tool. The equations used to calculate Kappa are as follows;

𝑁! 𝐼−𝐶𝑉𝐼−𝑝𝑐
𝑝𝑐 = [ ] 0,5𝑁 ve 𝑘 =
𝐴!(𝑁−𝐴)! 1−𝑝𝑐

Here k; kappa coefficient, pc; random correlation coefficient, N; the number of experts, A; the number of experts
who made a “relevant” assessment. Based on this, it was determined that the kappa values of items 7,18,24,32,
and 51 were lower than .48. As a result of the calculations, five items from the 54-item draft measurement tool
were removed from the scale since they could not provide the CVR, I-CVI, and kappa values. Thus, a 49-item
MVAS was prepared in a 5-point Likert type with content validity (see Appendix A for expert system in content
validity of draft MVAS).

Face Validity

After the content validity, face validity was performed to examine the simplicity of the language of the form and
the clarity of the structure (Yusoff, 2019). The 49-item draft MVAS which ensured the content validity was
presented to a panel group of 30 secondary school students (Yusoff, 2019). The suggestions in the form were
evaluated as follows
 the item is not clear (1 point)
 the item is somewhat clear (2 points)
 the item is clear enough (3 points)
 and the item is very clear (4 points)
In addition, students were requested for information about each item. In Item-level face validity index (I-FVI) and
scale-level face validity index based on the average method (S-FVI/Ave), and scale-level face validity index based
on the universal agreement method (S-FVI/UA) indexes were calculated in face validity calculations. I-FVI value
is the ratio of the respondents who give 3-4 points in the evaluation of the openness of an item to all respondents,
and the S-FVI/Ave value is the average of the I-FVI value of all the items in the form and S-FVI/UA value is
also the ratio of respondents who give 3-4 points to the clarity of an item in the whole form. If all respondents
agree on an item, the universal agreement (UA) value is 1. The minimum acceptable values for I-FVI and S-FVI
are .8 and .83, respectively. Ozair et al. (2017)'s recommendations were used in all calculations of face validity.
I-FVI, S-FVI, S-FVI/Ave, and S-FVI/UA values were obtained as .91, .91, .93, and .83, respectively. Since there
is no item eliminated from the draft MVAS as a result of face validity, a comparative table is not included here.

Construct Validity

After providing content validity, normality analyzes were performed to verify the structural validity of the 49-
items draft MVAS. This study examined whether the data demonstrated normal distribution through using the
skewness and kurtosis statistics, which are descriptive analysis methods. In normality tests, skewness and kurtosis
values should be between -2 and +2 (George and Mallery, 2010). When these conditions are met in the scale, the
data are considered to have a norm. The skewness and kurtosis coefficients of the data obtained from the pilot
study carried out with 251 secondary school students were calculated as -1.093 ± .157 and 1.223 ± .306.
Accordingly the data showed a normal distribution.

168
 International Journal of Technology in Education (IJTE)

Factor Load Analysis

The suitability of items in the draft MVAS for Factor analysis was tested by performing the sample adequacy test
(Kaiser-Meyer-Olkin, KMO) and the Barlett Sphericity Test. Herein while the Barlett Test of Sphericity was
found to be significant, the KMO coefficient was determined as .912 (χ2=2749.526, df=253, p<0,01). For the data
set to be suitable for factor analysis, the KMO coefficient should be greater than 0.7 (Leech et al., 2005). A KMO
value above .9 indicates perfect sample adequacy (Büyüköztürk, 2009). These results show that the data set is
acceptable for EFA. MLE method was used as it is parallel with the Structural Equation Model (SEM) in providing
EFA. The study employed Varimax for rotation and the Listwise Selection method for removing missing data.
EFA showed three factors with an eigenvalue greater than 1 and all sub-factors explained 57.27% of the total
variance. This value is greater than 40%, which is accepted as the minimum. However, Field (2009) reported that
the significance of a factor loading depends on the sample size. Field (2009) suggested that the factor load
threshold for a sample of 100 and 200 subjects should be .512 and .364, respectively. Here, the factor load
threshold value was taken as .512, taking into account the standardized regression weights of the items with
AMOS output. Also, Hair et. al. (2010) reported that a sample size of 100-150 people (the number of respondents)
is sufficient for EFA when the factor load threshold is kept 0.6. Accordingly, the draft MVAS scale consisted of
49 items clustered under three sub-factors. In addition, the scree plot was also used in factoring the items in the
draft MVAS. In the scree plot, the sharp decrease was identified to continue until the third factor. In the related
component matrix chart obtained from the factor analysis, items with load values less than .6 and items with factor
loads less than .1 among the items clustered under the same sub-factor were considered overlapping (Büyüköztürk,
2009). Accordingly, 27 items were removed from the 49-item draft MVAS with content validity, and a 22-item
draft MVAS with a 3-factor was obtained. The lowest factor load in draft MVAS is .607 and the highest factor
load is .800.. In addition, Pallant (2007) suggested that the communality value indicating the compatibility of an
item in the factor with other items should not be less than .3. Accordingly, the analysis results showed that the
communality values ranged between .471 and .657. In addition, the factors were named according to TAM, taking
into account the expressions of the items. As a result of EFA, items 4,11,13,22,23,25,33,34,40 and 45 in PU, items
16,18,19,20,21,28 and 41 in PEU, and items 31,32,36,37,38 and 39 in UA was clustered. Finally, three factors
(PU, PEU and UA) were extracted from the data as the independent variables. However, no dependent variable
could be extracted from the EFA. Table 3 shows the results of EFA for the 49-item MVAS.

Table 3. Results of EFA for the 49-item MVAS

Factor Loading
Code Items Com* Eigen Value Cumulative %
PU PEU UA
Item13 .801 .650
Item34 .757 .627
Item22 .753 .596
Item11 .736 .559
PU Item33 .711 .622 8.122
35.311
Item23 .697 .502
Item25 .675 .558
Item4 .671 .502
Item40 .664 .519

169
Kirbas & Dogan

Factor Loading
Code Items Com* Eigen Value Cumulative %
PU PEU UA
Item45 .664 .591
Item19 .729 .548
Item21 .677 .622
PEU Item16 .666 .657
Item20 .665 .521 3.366
Item18 .654 .474
Item28 .626 .653
Tem41 .607 .591 49.945
Item36 .785 .597
Item38 .745 .585
UA Item39 .741 .553 57.278
Item37 .730 .575 1.687
Item31 .717 .503
Item32 .694 .492
Com*: Communalities; Total variance explained: 57.24 %

Herein, PU is the variable associated with the belief that the use of the metaverse will enhance learning. PEU is
the variable using the effortless metaverse. UA is the variable related to positive or negative feelings towards the
use of metaverse. However, unfortunately, none of the items were clustered under BI, which is the variable related
to the desire to use metaverse. While cognitive and affective responses could be determined in EFA, behavioral
responses could not.

Internal Consistency Analysis

Internal Consistency Analysis is an indication of the measurement status of the scale. A reliable test or scale
should present similar results in the same situations. The reliability of the MVAS was determined by using the
split half model, equivalent halves, and Cronbach alpha values. Spearman-Brown, Guttman split half, and
Cronbach alpha reliability coefficients were examined to calculate the two-half test reliability coefficient of the
scale. Table 4 shows the results of split-half reliability analyzes for MVAS

Table 4. Results of Split-half Reliability Analyses for MVAS

Confidence Coefficients (N:23)
Correlation Between Forms =.454 Equal Length Spearman-Brown= .679
Guttman Split-Half Coefficient = .592 Unequal Length Spearman-Brown = .679
a b
Alfa= .917 (N:11 ) for Part1 Alfa = .714 (N:12 ) for Part2
a
Items: item4, item11, item 13, item 22, item 23, item 25, item 33, item 34, item 40, item 45, item 16
b
Items: item 19, item 20, item 21, item 28, item 31, item 32, item 37, item 38, item 39, item 41, item 36, item
18

As in Table 4, alpha values for the first and second parts were determined to be greater than .7. These results

170
 International Journal of Technology in Education (IJTE)

showed that the items was consecutive and reliable (Özdamar, 2013). Likewise, the correlation value between the
forms was calculated as .454. The correlation coefficient was found to be .592 with the Guttman half split formula,
and the reliability of the two halves with the Spearman-Brown formula was determined as .679. The structure
similarity, collectability, and homogeneity of the items in the scale were tested with ANOVA and the results were
depicted in Table 5. As observed in Table 5, the items in the scale were found to have a homogeneous structure
and they were related to each other (F=54.252, p<.001). Besides, the scale was collectible (F=340.259, p<.001).

Table 5. The ANOVA Results regarding MVAS

b c
KT df OK F p
Between People 2584.258 250 10.337
Between Items 1387.137 21 66.054 54.252 .000
Nonadditivity 389.131a 1 389.131 340.259 .000
Within
Residual Balance 6002.915 5249 1.144
People
Total 6392.045 5250 1.218
Total 7779.182 5271 1.476
Total 10363.439 5521 1.877
Grand Mean = 3.3391
a.
Tukey's estimate of power to which observations must be raised to achieve additivity = -1.584
b
KT: Sum of Squares, cOK: Mean squares

Özdamar (2013) questioned whether the situation that is desired to be measured with Hotelling's T-Squared
analysis can be measured appropriately with the measurement tool. Accordingly, Hotelling's T-Squared Test was
conducted to determine whether the test design of MVAS was appropriate. The results demonstrated that the
MVAS model was in a suitable structure (F=11,136, p<.05). The results of Hotelling's T-Squared analysis were
shown in Table 6.

Table 6. Hotelling's T-Squared Analysis Results for MVAS

Hotelling's T-Squared F df1 df2 Sig
254.199 11.136 21 230 .000

The ICC value is classified as follows in the relevant literature (Ridout et all, 1999):
 If the ICC value is less than .40, the in-class relationship is weak.
 If the ICC value is between .4-.59, the in-class relationship is moderate.
 If the ICC value is between .6-.74, the in-class relationship is at a good level.
 If the ICC value is greater than .74, the in-class relationship is at a very good level.

MVAS was found to be consistent in terms of individual items (ICC=.254, p<.05) and have a reliable construct
validity in terms of mean criteria (ICC=.882, p<.05). It may be wise to mention that the items in the scale are valid
and reliable in terms of their arrangement and structural features (Özdamar, 2013). The measurement results
showed that the intraclass correlations were weak for single measures and very good for average measures. The

171
Kirbas & Dogan

data including the ICC results of the MVAS was given in Table 7.

Table 7. The ICC Results of MTS

95% Confidence
F Test with True Value 0
Intraclass Interval
Correlationb Lower Upper
Value df1 df2 Sig
Bound Bound
Single Measurements .254a .218 .296 8.490 250 5250 .000
c
Average Measurements .882 .860 .902 8.490 250 5250 .000
Two-way mixed effects model where people effects are random and measures effects are fixed.
a
The estimator is the same, whether the interaction effect is present or not.
b
Type C intraclass correlation coefficients using a consistency definition. The between-measure variance is
excluded from the denominator variance
c
This estimate is computed assuming the interaction effect is absent because it is not estimable otherwise

Cronbach alpha value was also checked for internal consistency of MVAS. Although many techniques are used
for reliability, Cronbach alpha technique is one of the most used methods in scale development processes and the
determination of internal consistency (Sharma, 2016). The results regarding Cronbach alpha value are suggested
in Table 8.

Table 8. Cronbach's Alpha Coefficients for MVAS and its Subdimensions

Number of
Dimension Item Numbers Cronbach
Items
Alpha

item4, item11, item13, item22, item23, item25, item33,

PU 10 .913
item34, item40, item45
PEU 7 item19, item20, item21, item16, item18, item28, item41 .859
UA 6 item37, item38, item39, item31, item32 .834
All Scale .884

Table 8 summarizes that the Cronbach alpha values for each sub-dimension obtained as a result of the EFA
analysis were quite high. The results also indicated that the internal consistency and reliability of the scale were
very high. Table 8 depicts that it the Cronbach alpha value for the overall scale was .884. Accordingly, it is most
likely that the internal consistency of the scale is at a good level (Murphy and Davidshofer, 1988; Nunnally, 1978).

Confirmatory Factor Analysis

The data regarding the sample group participating in the pilot study were used for CFA. At first, missing data
were checked for each item, and the most repeated option in that series was replaced with missing variables.

172
 International Journal of Technology in Education (IJTE)

AMOS 24.0 program was used to determine the level of agreement between the three-factor structure determined
as a result of the EFA and the sample data. The analysis of the data was provided by applying the likelihood
model. Structural equation modeling, which is a method used in psychology, sociology, educational research,
political science, marketing, etc., is a hybrid model of factor analysis and regression analysis (Dow et al., 2008).
CFA examines the conformity of the estimated covariance matrix created according to the theoretical model to
the covariance matrix of the observed data (Hox and Bechger, 1995). It is used when determining the construct
validity of a data set and checking the hypotheses developed for the relations between variables (Tabachnick and
Fidell, 2014). The fit statistics are related to the data disclosure dimension of the predetermined models. Many fit
statistics test the fit of the models. These fit statistics analyze the fit of the parameters of the proposed models and
the statistics obtained from the sample data. If the model does not fit the data, it is rejected. If the proposed model
cannot be rejected, the model can explain the causal structure underlying the observed data (Özdamar, 2013).

First Level Confirmatory Factor Analysis

First-level confirmatory factor analysis incorporates the relationship between latent variables into the model. The
path diagram of the CFA concerning the analysis of the data-model fit of the model obtained from the EFA was
shown in Figure 2.

Figure 2. Unstandardized and Standardized Factor Loads in Path Diagram of MVAS

The results revealed 10 items in the PU sub-dimension, 6 items in the PEU sub-dimension, and 6 items in the UA
sub-dimension. Accordingly, item 18, having low factor loading values in EFA analysis, was excluded from the
MVAS. Path coefficients of the items belonging to all sub-dimensions were found to be statistically significant.
Considering the standardized path coefficients, item 22 was determined to have the highest effect on PU. The
item 21 had the highest effect on PEU,and the item 36 had the highest effect on UA. CFA provides information

173
Kirbas & Dogan

on the level at which each item in the MVAS represents its latent variable (see Appendix B for all standardized
values obtained from the first level model diagram provided with the AMOS 24.0 program using the MLE
method). CFA analyses suggested that all standardized factor loadings were quite high. Accordingly, it is
remarkable that the proposed model is at an excellent level and within the limits of fit. As a result of CFA, the
Cmin /df value of 22 items and the three-factor scale was found to be 1.712 (Cmin: 309 df:204, p<.05). According
to Kline (2011) the proposed model is excellent if the Cmin /df value is below 2. if the Cmin /df value is below 5,
the model is at an acceptable level. In addition, other goodness-of-fit indices such as Normed Fit Index (NFI ≥
.95, good; Bentler and Bonett, 1980), Comparative Fit Index (CFI≥.97, good; Hooper et al. 2008), Goodness of
Fit Index (GFI≥.90, acceptable; Schermelleh and Moosbrugger, 2003), Adjusted Goodness of Fit Index (AGFI ≥
.90, good; Schermelleh and Moosbrugger, 2003), Relative Fit Index (RFI≥.90, good; Schermelleh and
Moosbrugger, 2003), Root Mean Square Error of Approximation (RMSEA ≤.005, good; Hooper et al., 2008),
Standardized Root Mean Square Residual ( SRMR ≤ .005, good; Schermelleh and Moosbrugger, 2003) were also
examined to determine the degree of fit between model and data. Accordingly, RMSEA, SRMR, GFI, AGFI, NFI,
CFI and RFI values for model-data fit were determined as .05, .049, .889, .871, .9146, .965, .916, respectively.
These values supported the proposed three-factor model theoretically and statistically. Accordingly, the results
revealed that the model and data had a good fit.

Convergent and Discriminant Validity

As regards the TAM, which was created for the research, a first-level confirmatory analysis was carried out to
reveal the interrelationships across the variables. Thus, the correlation values between the UA, PU, and PEU
variables were found to be acceptable and significant (see Table 9). Convergent and discriminant validity was
carried out to determine whether the observed variables were a part of the latent structures (Fornell and Larcker,
1981). Convergent validity indicates whether the observed variables measure the latent variable, while convergent
validity determines the relationship between the observed variables and the latent variable (Hair et al., 2010).
Convergent validity requires CR>.70, AVE>50, and CR>AVE. MSV<AVE and ASV<AVE are required to ensure
discriminant validity. Besides, the square root of the AVE value should be greater than the correlation value
between the variables (Hu and Bentler, 1999).

Table 9. Composite Reliability and Explained Mean-variance Values of the Variables

CR AVE MSV ASV MaxR (H) UA PU PEU Cronbach’s Alpha
UA .953 .670 .379 .3493 .956 .818a .953
a
PU .889 .675 .319 .2564 .920 .600 .821 .911
a
PEU .924 .634 .379 .2865 .910 .550 .440 .796 .898
CR: Composite reliability, AVE: Average shared variance, MSV: Maximum shared variance, ASV: Average
shared variance, Note: Diagonal values (a) are the square roots of AVE values

On analyzing Table 9, the lowest value of the calculated AVE for latent variables was noted as .634 and the
lowest calculated CR value as .889, meaning that convergent validity was provided for all latent variables in the
measurement model. With regard to the square roots of AVE values and correlations between variables,

174
 International Journal of Technology in Education (IJTE)

discriminant validity was ensured for all latent variables. The results showed that the MSV and ASV values are
smaller than the AVE value and the values are sufficient and acceptable.

Testing the Structural Model

This study aims at revealing the relations between the variables through examining the reflections of secondary
school students' attitudes towards the use of metaverse as a learning environment within the scope of TAM. At
that point, the relationships among the variables were tested through structural modeling after verification with
the first level CFA. Besides, the relationship between structures in model was evaluated. In other words, the effect
of PEU on PU and UA, and the effect of PU on UA were investigated. This is the basis for the TAM model. Path
analysis with observed variables was deployed to test the mutual effects in the analysis. Figure 3 shows the non-
standardized and standardized path diagram of the metaverse scale.

Figure 3. Non-standardized and Standardized Path Diagram of MVAS

The results of SEM analysis displayed that RMSEA, SRMR, GFI, AGFI, NFI, CFI and NFI values were
determined to be .050, .049, .889, .862, .916, .969, .915, respectively. These values verified the accuracy of the
proposed three-factor model. The results also revealed that the model and the data in the model have a good fit
(see Appendix C for parameter estimates of the analysis). Here, the results showed that the constructed structural
model was compatible and the model fit indexes remained within the specified limits, and all standardized and
non-standardized path coefficients were positive and significant.

Testing the Hypotheses

The research model and hypotheses were tested through studies of construct validity. The obtained data revealed
the accuracy of the research model. Cognitive and affective variables regarding motivation were extracted from
EFA. However, the variable of intention to use regarding the behavioral field could not be extracted from the
EFA. Therefore, the H3 hypothesis could not be tested in the research model. On the other hand, the H1, H2 and
H3 hypotheses were welcomed as the path coefficients had a positive and significant value.

175
Kirbas & Dogan

Conclusions

This study is an attempt to develop a valid and reliable metaverse scale for secondary school students. The content
and construct validity of the MVAS were examined during the scale development process. The content validity
of the MVAS was initially carried out with 6 steps specified by Polit and Beck (2006). These successive steps
were implemented as follows.
 Preparing the content verification form,
 Selecting a review panel with expert staff,
 Performing content verification,
 Examining the area and items,
 Calculating scores for each item,
 Finding I-CVI and S-CVI values.
The calculations were completed in line with the reports of Ayre and Scally (2014) for the CVR value and Lynn
(1986) and Polit and Beck (2006) for the CVI values. A 54-item draft form was created after the literature review
and panel discussions were completed to develop a scale suitable for secondary school students. 14 experts’ views
were received in line with the scaling-scoring suggestion of Yusoff (2019). As a result of the scoring, five items
with kappa values below .48 were removed from the test. The CVI/Ave and S-CVI/UA values of the remaining
49-item draft MVAS were calculated to be .93, and .84, respectively. Following content validity, face validity
was performed to examine the simplicity of the language of the form and the clarity of the structure (Yusoff,
2019). The 49-item draft form was presented to a panel group of 30 secondary school students. All calculations
and determinations of face validity were performed with the recommendations of Ozair et al. (2017). I-FVI, S-
FVI, S-FVI/Ave, and S-FVI/UA values were obtained as .91, .91, .93, and .83, respectively. There was no item
eliminated as a result of face validity. Secondly, EFA and CFA were used for construct validity. The present study
confirmed whether the data demonstrated normal distribution through using the skewness and kurtosis statistics.
The skewness and kurtosis coefficient was calculated as 1.093 ± .157and 1.223 ± .306, respectively. The suitability
of the draft MVAS for factor analysis was analyzed by performing the KMO sample adequacy test and the Barlett
Sphericity test. The Barlett Test of Sphericity (χ2=2749.526, df=253, p<0,01) was significant and the KMO
coefficient was determined as .912. This result showed that the data set is acceptable for EFA. In EFA, the data
set was found to have three factors according to the TAM model. Three factors with an eigenvalue greater than 1
explained 52.27% of the total variance of all sub-factors. The analysis results indicated that 26 items were removed
from the draft MVAS. Thus, 49 items in the draft MVAS were reduced to 23 items. In addition, the scree plot was
also used in factoring the items in the MVAS. A sharp decrease in the scree plot was determined to continue until
the third factor. In the internal consistency analysis of MVAS, the reliability of the scale was determined by using
the Split Half model and using Cronbach alpha values. Cronbach alpha values were calculated as .917 for the first
half and .714 for the second half. These results show that the items are consecutive and reliable (Özdamar, 2013).
Similarly, the correlation value between the forms was calculated as .454. Besides, the correlation coefficient with
the Guttman half split formula was found as .592. In addition, the reliability of the two halves with the Spearman-
Brown formula was determined to be .679. In addition, ANOVA Tukey's Nonadditivity analysis was carried out
to determine their relationship with each other and the homogeneity of the items in MVAS. As a result, the items
in MVAS were found to be homogeneous (F=54.252, p<0.001) and the test were collectible (F=340.259, p<.001).

176
 International Journal of Technology in Education (IJTE)

On the other hand, Hotelling's T-Squared analysis showed that MVAS consisted of homogeneous, strong, and
unique items (F=11.136, p<.05) and was effective in measuring. Each item of the MVAS was individually
consistent (ICC=0.254, p<.05) and had a reliable structure in terms of mean criteria (ICC=0.882, p<.05) (Ridout
et all, 1999).The internal consistency of MVAS was extremely high in terms of single and average measurements.
Finally, the Cronbach alpha value for the overall MVAS was found to be 0.884.

The AMOS 24.0 program was used to determine the level of agreement between the three-factor structure. The
fit indices for model-data fit were determined as .05 for RMSEA, .049 for SRMR, .889 for GFI, .871 for AGFI,
.9146 for NFI, .965 for CFI and .916for RFI. These values supported the proposed three-factor model theoretically
and statistically. Accordingly, the results revealed that the model and the data in the model had a good fit. As a
result of the analysis, an item in the EFA was excluded from the structure. It was found to be 6 items under the
UA sub-dimension, 10 items under the PU sub-dimension, and 6 items under the PEU sub-dimension. Upon
analyzing the standardized path coefficients,, the items with the highest effect on UA, PEU, and PU were item31,
item24, and item34, respectively. The AVE, MSV, and ASV values calculated for the latent variables provided
validity in convergent and discriminant validity. The relationship between the model and the structures was
provided by the second-level confirmatory factor analysis. As a result of SEM analysis, RMSEA, SRMR, GFI,
AGFI, NFI, CFI, NFI values were found as .050, .049, .889, .862, .916, .969, and .915, respectively. These values
verified the accuracy of the proposed three-factor model. All standardized and non-standardized path coefficients
were positive and significant.

Herein this study aims at measuring the adoption and using levels of the metaverse of secondary school students.
There is a dearth of studies conducted on the adoption of metaverse studies. Besides, there is no such a study
specifically published on investigating metaverse in terms of TAM. Hence, this study can be a reference source
for further studies. The results pinpointed that the most significant factors affecting the adoption and use of
metaverse by secondary school students were perceived ease of use and perceived usefulness related to the
application. The results also highlighted that the hypotheses determined by the research model were accepted.
Namely, ease of use influenced the perceived usefulness of the metaverse. In addition, perceived ease of use and
perceived usefulness of metaverse had a positive effect on the intention to use. Although secondary school
students reacted positively to cognitive responses (perceived ease of use, perceived usefulness of use) and
affective responses (user attitude) about the metaverse, they could not transform these responses into behavioral
responses (behavioral intention). This may be due to the technological inadequacies, the need for expert
experience of many metaverse programs, and new technology. On the other hand, the difference in behavior of
individuals can be observed depending on the strength of the attitude. Namely, the presence of attitude sometimes
may not be enough to observe the behavior. In such cases, attitude may influence behavior only when the
individual interacts with the environment (Atkinson et all, 1995). The students did not spend much time in the
meta-universe and thus, the attitude did not affect the behavior. However, the TAM model revealed that secondary
school students had absolute motivation for the use of metaverse. In this context, Suh and Ahn (2022) stated that
the use of metaverse for educational purposes had a positive effect on most students. Guo and Gao (2022)
suggested that the metaverse could improve students' learning activities and improve students' cognitive feelings
of interaction in teaching English. On the other hand, Kye et al. (2021) defined metaverse as a new educational

177
Kirbas & Dogan

environment. Lee et al. (2022) designed a scale for distance education with Metaverse based on aircraft
maintenance simulation and emphasized that the system they proposed had an effect on learning in the field of
technical education such as aircraft maintenance. Talan and Kalınkara (2022) examined the computer engineering
students’ views towards Metaverse. They concluded that the students had not used Metaverse before, but they
were eager to use it in their lessons. Reyes (2020) examined high school students' perceptions towards the use of
Metaverse in mathematics lessons. Accordingly the use of virtual reality in the classroom increased the perception
of changes in learning.

Recommendations

Based on the research findings, following recommendations were provided:

 MVAS scale addresses all grade levels of secondary education. There is no need to develop a separate
scale for each grade level. Therefore, it can be used as an important assessment tool in various metaverse
applications at the secondary education level.
 MVAS scale can be used as a data collection tool to examine students' attitudes in experimental studies
and the effect of metaverse on the secondary school level.
 Students’ behavioral reactions should be evaluated to popularize the use of metaverse at the secondary
school level. Technological infrastructures should be developed for this.
 Similar studies may examine how teachers and students understand the metaverse. Teachers should
design classrooms that allow students to solve problems or collaborate. Educational metaverse platforms
should be developed to prevent students from misusing data.

References

Adams, D., Nelson, R., & Todd, P. (1992). Perceived usefulness, ease of use, and usage of information technology:
A replication. MIS quarterly, 227-247.
Altunışık, R., Coşkun, R., Bayraktaroğlu, S., & Yıldırım, E. (2012). Research methods in social sciences: SPSS
Applied (7th Edition). Istanbul: Avcı Ofset.
Atkinson, R., Atkinson, R. C. and Hilgard, E. R. (1995). Introduction to psychology (14th Edition). New York:
Harcourt.
Aydın, C. (2002). Past and present of educational environments. Kurgu Anadolu University Faculty of
Communication Sciences International Refereed Journal of Communication, 19(19), 151-161.
Ayre, C., & Scally A. J. (2014). Critical values for Lawshe’s content validity ratio: revisiting the original methods
of calculation. Measurement and Evaluation in Counseling and Development, 79-86.
Bagozzi, R., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of
Marketing Science, 16, 74-94.
Barry, D., Ogawa, N., Dharmawansa, A., Kanematsu, H., Fukumura, Y., Shirai, T., . . . Kobayashi, T. (2015).
Evaluation for students' learning manner using eye blinking system in metaverse. Knowledge-Based And
Intellıgent Informatıon & Engıneerıng Systems 19th Annual Conference (pp. 1195-1204, DOİ:
10.1016/j.procs.2015.08.181). Kes-2015.

178
 International Journal of Technology in Education (IJTE)

Batdı, V., & Talan, T. (2019). Augmented reality applications: A Meta-analysis and thematic analysis. Turkish
Journal of Education, 8(4), 276-297. doi:http://dx.doi.org/10.19128/turje.581424
Bentler, P., & Bonett, D. (1980). Significance tests and goodness of fit in analysis of covariance structures.
Psychological Bulletin, 88(3), 588-606.
Boudreau, M., Gefen, D., & Straub, D. (2001). Validation in information systems research: A state-of-the-art
assessment. MIS quarterly, 1-16.
Buyukozturk, S. (2009). Handbook of data analysis for social sciences. (10th ed.). Ankara: Pegem.
Büyüköztürk, Ş. (2007). Manual of data analysis for social sciences. Ankara: Pegem A Publishing.
Choi, H. (2022). Working in the metaverse: Does telework in a metaverse office have the potential to reduce
population pressure in megacities? Evidence from young adults in seoul south Korea. Sustainability,
14(6), DOİ: 10.3390/su14063629.
Creswell, J. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.).
CA:Sage: Thousand Oaks.
Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology.
MIS Q., 13, 319.
Demirli, C. (2002). The effect of Web based teaching on students achievement in instructional technologies and
material development course (The sample Technical Education Faculty of FU). (Thesis of Doctoral),
122047.
Diaz, J. (2020). Virtual world as a complement to hybrid and mobile learning. Internatıonal Journal of Emergıng
Technologıes In Learnıng 15(22), 267-274 DOİ: 10.3991/ijet.v15i22.14393.
Dow, K., Jackson, C., Wong, J., & Leitch, R. (2008). A Comparison of structural equation modeling approaches:
The Case of User Acceptance of Information Systems. Journal of Computer Information Systems
(Summer).
Durak, G., Çankaya, S., & İzmirli, S. (2020). Investigation of distance education systems of universities in Turkey
during the COVID-19 pandemic period. Necatibey Faculty of Education electronik journal of Science
and mathematics education, 14(1), 787-809.
Field, A. P. (Discovering statistics using SPSS). 2009. London: England Sage.
Fishbein, M. (1966). The Relationships between beliefs,attitudes and behavior. In S. Feldman (ed), Cognitive
Consistency. New York: Academic Press
Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological bulletin, 378.
Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and
measurement error. Journal of Marketing Research, 18(1), 39–50.
Gartner. (2022). Predicts 2022: 4 Technology bets for building the digital future. Retrieved January 12, 2023,
from https://www.businesswire.com/news/home/20220207005085/en.
George, D., & Mallery, M. (2010). SPSS for Windows step by step: A Simple guide and reference, 17.0 update
(10a ed.) . Boston: Pearson.
Gomes, A., & Klein, A. (2013). The Development of distance teamwork competence through the use of tthe
second life (R) Metaverse. Administracao-Ensino e Pesquisa, 14 (2), 343-375 DOİ:
10.13058/raep.2013.v14n2.68.
Gökçe, A. (2008). Distance education in the process of globalization. Journal of Dicle University Ziya Gökalp

179
Kirbas & Dogan

Faculty of Education, (11), 1-12.

Guo, F., & Liu, F. (2013). A study on the factors influencing teachers’ behaviour of internet teaching research.
Int. J. Contin. Eng. Educ. Life-Long Learn., 23, 267-281.
Guo, H., & Gao, W. (2022). Metaverse-powered experiential situational english-teaching design: An emotion-
based analysis method. Frontıers In Psychology, DOI: 10.3389/fpsyg.2022.859159.
Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R. (2010). Multivariate data analysis. Prentice Hall. Upper
Saddle River, NJ
Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: Guidelines for determining model
fit. The Electronic Journal of Business Research Methods, 6(1), 53-60.
Hox, J., & Bechger, T. (1995). An Introduction to structural equation modeling. Family Science Review, 11.
Hu, L.-T., & Bentler, P. (1999). Cut off criteria for fit indexes in covariance structure analysis: Conventional
criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55.
Huh, S. (2022). Application of computer-based testing in the Korean Medical Licensing Examination, the
emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to
reviewers and volunteers. Journal of Educatıonal Evaluatıon For Health Professıons, DOİ:
10.3352/jeehp.2022.19.2.
Ibrahim, A., & Shiring, E. (2022). The Relationship between educators’ attitudes, perceived usefulness, and
perceived ease of use of instructional and web-based technologies: Implications from Technology
Acceptance Model (TAM). International Journal of Technology in Education (IJTE), 5(4), 535-551.
doi:https://doi.org/10.46328/ijte.285
Idris, F., Hassan, Z., Ya’acob, A., Gill, S., & Awal, N. (2012). The role of education in shaping youth’s national
identity” Procedia. Procedia - Social and Behavioral Sciences, 443-450.
Jaramillo-Mujica, J., Morales-Avella, L., & Coy-Mondragon, D. (2017). An experience using metaverses for
teaching mechanical physics to engineering students. Revısta Educacıon En Ingenıerıa 12(24), 20-30
DOİ: 10.26507/rei.v12n24.778.
Jondeau, E., & Rockinger, M. (2003). Conditional volatility, skewness, and kurtosis: existence, persistence, and
comovements. Journal of Economic Dynamics & Control, 27, 1699-1737.
Jondeau, E., & Rockinger, M. (2003). Conditional volatility, skewness, and kurtosis: existence, persistence, and
comovements. Journal of Economic dynamics and Control, 27(10), 1699-1737.
Jovanovic, A., & Milosavljevic, A. (2022). VoRtex metaverse platform for gamified collaborative learning.
Electronics, 11 (3), DOİ: 10.3390/electronics11030317.
K., Ö. (2016). Scale and test development in education, Health and behavioral sciences structural equation
modeling IBM SPSS, IBM SPSS AMOS and MINITAB applied. Eskişehir: Nisan Bookstore.
Kalaycı, N. (2010). An analysis of project-based learning in higher education in terms of students who manage
the project. Education and Science, 33(147), 85-105.
Kamenov, K. (2022, May 20). Immersive experience—The 4th wave in tech: Learning the ropes. retrieved from
https://www.accenture.com/gb-en/blogs/blogs-immersive-experience-wave-learning-ropes
Kanematsu, H., Kobayashi, T., Barry, D., Fukumura, Y., Dharmawansa, A., & Ogawa, N. (2014). Virtual STEM
class for nuclear safety education in metaverse. Proc. Comput. Sci., 35, 1255-1261.
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York London: The

180
 International Journal of Technology in Education (IJTE)

Guilford Press.
Kye, B., Han, N., Kim, E., Park, Y., & Jo, S. (2021). Educational applications of metaverse: possibilities and
limitations. Journal Of Educatıonal Evaluatıon For Health Professıons, 1-13 DOİ:
10.3352/jeehp.2021.18.32.
Landry, B., Griffeth, R., & Hartman, S. (2006). Measuring student perceptions of blackboard using the technology
acceptance model. Decision Sciences Journal of Innovative Education, 4(1), 87-99.
Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel psychology, 536-575.
Lee, H., & Hwang, Y. (2022). Technology-enhanced education through VR-making and metaverse-linking to
foster teacher readiness and sustainable learning. Sustainability, 14(8), DOİ: 10.3390/su14084786.
Leech, N., Barrett, K., & Morgan, G. (2005). SPSS for Intermediate statistics, use and interpretation. (2nd ed.).
Mahwah: Lawrence Erlbaum Associates Inc.
Locurcio, L. (2022). Dental education in the metaverse. Brıtısh Dental Journal 232(4), 191 DOİ: 10.1038/s41415-
022-3990-7.
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing research.
Mathieson, K. (1991). Predicting user intentions: comparing the technology acceptance model with the theory of
planned behavior. Inf. Syst. Res. 2, 173–191. http://dx.doi.org/10.2307/23010882.
Miller, C. (1996). A historical review of applied and theoretical spatial visualization publications in engineering
graphics. The Engineering Design Graphics Journal, 60(3), 12-33.
Moon, J., & Kim, Y. (2001). Extending the TAM for a World-Wide-Web context. Inf. Manag., 38, 217-230.
Murphy, K., & Davidshofer, C. (1998). Psychological testing: Principles and applications ( 4th edn.). New Jersey:
Prentice-Hall.
Mystakidis, S. (2022). Metaverse. Encyclopedia, 2(1), 486-497.
N., B. (2013). Introduction to structural equation modeling. Ankara: Ezgi Bookstore Publishing.
Nitko, A. (1996). Educational assessment of students (211 d ed.). Englewood: Cliffs:Prentice hall.
Nunnally, J. (1978). Psychometric Theory. New York: McGraw-Hill.
Orland, K. (2022, May 20). Ars Technica. Retrieved from wired: https://www.wired.com/story/what-is-the-
metaverse/#:~:text=Broadly%20speaking%2C%20the%20technologies%20companies,the%20digital%
20and%20physical%20worlds.
Orts-Cortés, M. I., Moreno-Casbas, T., Squires, A., Fuentelsaz-Gallego, C., Maciá-Soler, L., & González-María,
E. (2013). Content validity of the Spanish version of the practice environment scale of the nursing work
index. Applied Nursing Research, 5-9.
Ozair, M., Baharuddin, K., Mohamed, S., Esa, W., & Yusoff, M. (2017). Development and validation of the
knowledge and clinical reasoning of acute asthma management in emergency department (K-CRAMED).
Education in Medicine Journal, 9(2), 1-17.
Özdamar, K. (2010). Biostatistics with PASW. Eskişehir: Kaan Bookstore.
Özdamar, K. (2013). Statistical data analysis with package programs-II, (Multivariate Methods), 9th Edition.
Eskişehir: Nisan Bookstore.
Özmen, A. (2000). Sampling methods in applied research. Eskişehir: Anadolu University Publishing.
Pallant, J. (2007). SPSS survival manual: A step by step guide to data analysis using SPSS for Windows (3rd ed.).
Berkshire: Open University Press.

181
Kirbas & Dogan

Park, S., & Kim, S. (2022). Identifying world types to deliver gameful experiences for sustainable learning in the
Metaverse. Sustainability, 14(3), 1361 DOİ:10.3390/su14031361.
Park, S., & Kim, Y. (2022). A Metaverse: Taxonomy, components, applications, and open challenges. IEEE
ACCESS 10, 4209-425 DOİ:10.1109/ACCESS.2021.3140175.
Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported?
Critique and recommendations. Research in nursing & health, 489-497.
Reyes, C. (2020). Perception of high school students about using Metaverse in augmented reality learning
experiences in mathematics. Pıxel-Bıt- Revısta De Medıos Y Educacıon, 143-159, DOİ:
10.12795/pixelbit.74367.
Ridout, M., Demetrio, C., & Firth, D. (1999). Estimating intraclass correlation for binary data. Biometrics, 55 (1),
137-148.
S., B. (2010). Professional problems encountered by social workers in working life: The case of Bursa. Journal
of Society and Social Work, 21(1), 99-109.
Saritas, M., & Topraklikoglu, K. (2022). Systematic literature review on the use of metaverse in education.
International Journal of Technology in Education (IJTE), 5(4), 586-607.
doi:https://doi.org/10.46328/ijte.319
Schermelleh, E., & Moosbrugger, H. (2003). Evaluating the fit of structural equation models: Tests of significance
and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8.
Sharma, B. (2016). A focus on reliability in developmental research through Cronbach’s Alpha among medical,
dental and paramedical professionals. Asian Pacific Journal of Health Sciences, 3, 271-278. Doi:
10.21276/apjhs.2016.3.4.43.
Shi, J., Mo, X., & Sun, Z. (2012). Content validity index in scale development. Zhong nan da xue xue bao. Journal
of Central South University. Medical Sciences, 152-155.
Stephenson, N. (1992). Snow Crash (2011th edt.). London: Penguin.
Suh, W., & Ahn, S. (2022). Utilizing the metaverse for learner-centered constructivist education in the post-
pandemic era: An Analysis of elementary school students. Journal of Intelligence 10 (1), DOİ:
10.3390/jintelligence10010017.
Sun, H., & Zhang, P. (2006). Causal relationships between perceived enjoyment and perceived ease of use: An
alternative approach. Journal of the Association for Information Systems, 7(1), 24.
Tabachnick, B., & Fidell, L. (2014). Using multivariate statistics (New International Ed.). Harlow: Pearson.
Talan, T., & Kalinkara, Y. (2022). Students' opinions about the educational use of the metaverse. International
Journal of Technology in Education and Science (IJTES), 6(2), 333-346.
doi:https://doi.org/10.46328/ijtes.385
Tas, N., & Bolat, Y. (2022). Bibliometric mapping of metaverse in education. International Journal of Technology
in Education (IJTE), 5(3), 440-458. doi:https://doi.org/10.46328/ijte.323
Tekin, H. (1977). Quantitation and evaluation in education. Ankara: Mars Publishing.
Venkatesh, V. & Davis, F.D. (2000). A Theoretical extension of the Technology Acceptance Model: Four
longitudinal field studies. Institute for Operations Research and the Management Sciences, 46, pp. 186–
204.
Wynd, C. A., Schmidt, B., & Schaefer, M. A. (2003). Two quantitative approaches for estimating content validity.

182
 International Journal of Technology in Education (IJTE)

Western journal of nursing research, 508-518.

Yamane, T. (1967). Statistics, An Introductory Analysis, 2nd Ed. New York: Harper Row.
Yeşilyurt, S., & Çapraz, C. (2018). A roadmap for content validity used in scale development studies. Journal of
Erzincan University Faculty of Education, 20(1), 251-264.
Yıldırım, I. (2020). The effect of augmented reality applications in science teaching on the academic achievement
and retention of 6th grade students. Master Thesis. Turkey: Institute of Educational Sciences, Eskişehir.
Retrieved January 13, 2023, from https://tez.yok.gov.tr/UlusalTezMerkezi/
Yong, A., & Pearce, S. (2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis.
Tutorials in Quantitative Methods for Psychology, 9(2), 79-94. DOI: 10.20982/tqmp.09.2.p79.
Yusoff, M. S. B. (2019). ABC of content validation and content validity index calculation. Education in Medicine
Journal. 11(2), 49-54.
Zhang, X., Chen, Y., Hu, L., & Wang, Y. (2022). The metaverse in education: Definition, framework, features,
potential applications,challenges, and future research topics. Frontiers in Psychology 13.
doi:10.3389/fpsyg.2022.1016300

Author Information
Ömer Kırbaş Fatih Doğan

https://orcid.org/0000-0003-2670-7365 https://orcid.org/0000-0001-5844-8893
(Corresponding Author) Çanakkale Onsekiz Mart University
Ministry of National Education Faculty of Education, 17100-Çanakkale
Kocaeli, Turkey
Turkey
Contact e-mail: [email protected]

183
Kirbas & Dogan

Appendix A. Expert System for the Content Validity of Draft MVAS

* NA: Number of Agreement; According to Ayre and Scally (2014), there is no item below the CVR=CVR critical value (.571);
I-CVI: Item content validity; Pc: the probability of random compromise; k*: kappa coefficient; Evaluation criteria of k*: poor
≤.39, weak = .40-.59; good = .60-.73; excellent ≥.74 according to Fleiss (1971), S-CVI/Ave* (based on proportion relevance):
average proportion of “relevant” scores through experts index; S-CVI/Ave (based on I-CVI): average I-CVI scores of all items

184
 International Journal of Technology in Education (IJTE)

Appendix B. First Level Confirmatory Factor Analysis for All Sub-dimensions

Items Latent Variable Β0 Β1 SH CR p

item45 <--- PU .807 1.000 .071 14.649
item 40 <--- PU .755 .901 .071 13.271 <.001
item 4 <--- PU .767 .922 .071 14.457 <.001
item 25 <--- PU .769 .954 .070 15.374 <.001
item 33 <--- PU .847 1.038 .070 15.024 <.001
item 23 <--- PU .790 .941 .072 15.022 <.001
item 11 <--- PU .843 1.032 .071 14.649
item 22 <--- PU .876 1.071 .071 13.271 <.001
item 34 <--- PU .861 1.058 .071 14.457 <.001
item 13 <--- PU .863 1.081 .070 15.374 <.001
item 41 <--- PEU .743 1.000 <.001
item 28 <--- PEU .821 1.106 .091 12.095
item 16 <--- PEU .840 1.080 .087 12.351 <.001
item 21 <--- PEU .879 1.156 .090 12.910 <.001
item 19 <--- PEU .745 .941 .086 10.886 <.001
item 20 <--- PEU .740 .882 .082 10.734 <.001
item 32 <--- UA .823 1.000
item 31 <--- UA .828 1.042 .076 13.648 <.001
item 37 <--- UA .755 ,918 .077 11.860 <.001
item 39 <--- UA .695 ,883 .083 10.643 <.001
item 38 <--- UA .717 ,876 .081 10.882 <.001
item 36 <--- UA .745 ,939 .082 11.462 <.001
β0: standard covariance values, β1: non-standardized covariance values, SE: Standard error, *p < .001 significant level

185
Kirbas & Dogan

Appendix C. SEM Analysis Results for MVAS

Items Latent Variable Β0 Β1 SH CR p

item45 <--- PU .446 1.000
item40 <--- PU .807 .901 .072 12.427 <.001
item4 <--- PU .755 .922 .073 12.671 <.001
item25 <--- PU .767 .954 .075 12.715 <.001
item33 <--- PU .769 1.038 .071 14.649
item23 <--- PU .847 .941 .071 13.271 <.001
item11 <--- PU .790 1.032 .071 14.457 <.001
item22 <--- PU .843 1.071 .070 15.374 <.001
item34 <--- PU .876 1.058 .070 15.024 <.001
item13 <--- PU .861 1.081 .072 15.022 <.001
item41 <--- PEU .743 1.000
item28 <--- PEU .821 1.106 .091 12.095 <.001
item16 <--- PEU .840 1.080 .087 12.351 <.001
item21 <--- PEU .879 1.156 .090 12.910 <.001
item19 <--- PEU .745 .941 .086 10.886 <.001
item20 <--- PEU .740 .882 .082 10.734
item32 <--- UA .823 1.000 <.001
item31 <--- UA .828 1.042 .076 13.648 <.001
item37 <--- UA .755 .918 .077 11.860 <.001
item39 <--- UA .695 .883 .083 10.643 <.001
item38 <--- UA .717 .876 .081 10.882 <.001
item36 <--- UA .745 .939 .082 11.462 <.001
SEM
PU <--- PEU .440 .490 .084 5.821 <.005
UA <--- PEU .353 .317 .065 4.865 <.005
UA <--- PU .446 .359 .059 6.102 <.005
β0: standard covariance values, β1: non-standardized covariance values, SE: Standard error, *p < .001 significant level

186