See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/374235062

Development of the “Scale for the assessment of non-experts’ AI literacy” –

An exploratory factor analysis

Article in Computers in Human Behavior Reports · September 2023

DOI: 10.1016/j.chbr.2023.100338

CITATIONS READS

76 2,003

4 authors:

Matthias Carl Laupichler Alexandra Aster

Universitätsklinikum Bonn Universitätsklinikum Bonn
19 PUBLICATIONS 623 CITATIONS 13 PUBLICATIONS 493 CITATIONS

SEE PROFILE SEE PROFILE

Nicolas Haverkamp Tobias Raupach

University of Bonn Universitätsklinikum Bonn
45 PUBLICATIONS 203 CITATIONS 233 PUBLICATIONS 4,937 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Matthias Carl Laupichler on 11 October 2023.

The user has requested enhancement of the downloaded file.

 Computers in Human Behavior Reports 12 (2023) 100338

Contents lists available at ScienceDirect

Computers in Human Behavior Reports

journal homepage: www.sciencedirect.com/journal/computers-in-human-behavior-reports

Development of the “Scale for the assessment of non-experts’ AI literacy” –

An exploratory factor analysis
Matthias Carl Laupichler a, *, Alexandra Aster a, Nicolas Haverkamp b, Tobias Raupach a
a
Institute of Medical Education, University Hospital Bonn, Bonn, Germany
b
Department of Psychology, University of Bonn, Bonn, Germany

A R T I C L E I N F O A B S T R A C T

Keywords: Artificial Intelligence competencies will become increasingly important in the near future. Therefore, it is
AI literacy essential that the AI literacy of individuals can be assessed in a valid and reliable way. This study presents the
AI competencies development of the “Scale for the assessment of non-experts’ AI literacy” (SNAIL). An existing AI literacy item set
AI literacy scale
was distributed as an online questionnaire to a heterogeneous group of non-experts (i.e., individuals without a
AI literacy questionnaire
Assessment
formal AI or computer science education). Based on the data collected, an exploratory factor analysis was
Exploratory factor analysis conducted to investigate the underlying latent factor structure. The results indicated that a three-factor model
had the best model fit. The individual factors reflected AI competencies in the areas of “Technical Under­
standing”, “Critical Appraisal”, and “Practical Application”. In addition, eight items from the original ques­
tionnaire were deleted based on high intercorrelations and low communalities to reduce the length of the
questionnaire. The final SNAIL-questionnaire consists of 31 items that can be used to assess the AI literacy of
individual non-experts or specific groups and is also designed to enable the evaluation of AI literacy courses’
teaching effectiveness.

1. Introduction technologies; communicate and collaborate effectively with AI; and use
AI as a tool online, at home, and in the workplace” (p. 2). Furthermore,
Artificial intelligence (AI) is having an increasing impact on various Ng et al. (2021a) state in their literature review that „instead of merely
aspects of daily life. These effects are evident in areas such as education knowing how to use AI applications, learners should be inculcated with
(Zhai et al., 2021), healthcare (Reddy et al., 2019), or politics (König & the underlying AI concepts for their future career, as well as the ethical
Wenzelburger, 2020). However, AI is not only used in niche areas that concerns of AI applications to become a responsible citizen” (p. 507).
require a high degree of specialization, but it is also integrated into Despite these and other attempts to define AI literacy, there is still no
everyday life applications. Programs like ChatGPT (OpenAI, 2023) clear consensus on which specific skills fall under the umbrella term AI
provide free and low-threshold access to powerful AI applications for literacy. However, researchers seem to agree that AI literacy is aimed at
everyone. It is already becoming apparent that the use of these AI ap­ non-experts, which are laymen who have not had specific AI or computer
plications requires a certain level of AI competence that enables a crit­ science training. These may be individuals who could be classified as
ical appraisal of the programs’ capabilities and limitations. consumers of AI, or individuals who interact with AI in a professional
manner (Faruqe et al., 2021). Because of this somewhat ambiguous
1.1. Defining AI literacy definitional situation, we propose the following AI literacy working
definition: The term AI literacy describes competencies that include basic
These competencies are often referred to in the literature as AI lit­ knowledge and analytical evaluation of AI, as well as critical use of AI ap­
eracy. There are several definitions of AI literacy, but one of the most plications by non-experts. It should be emphasized that programming
commonly used can be found in a paper by Long and Magerko (2020), skills are explicitly not included in AI literacy in this definition, since in
which lists 16 core AI literacy competencies. They define AI literacy as our view they represent a separate set of competencies and go beyond AI
“a set of competencies that enables individuals to critically evaluate AI literacy.

* Corresponding author. Institute of Medical Education, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany.
E-mail addresses: [email protected] (M.C. Laupichler), [email protected] (A. Aster), [email protected] (N. Haverkamp),
[email protected] (T. Raupach).

https://doi.org/10.1016/j.chbr.2023.100338
Received 18 April 2023; Received in revised form 7 August 2023; Accepted 25 September 2023
Available online 27 September 2023
2451-9588/© 2023 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

1.2. Assessing AI literacy assumptions from practical observations (i.e., participants’ responses to
the AI literacy items), we deliberately refrained from formulating
Although comparatively young, the field of AI literacy and AI edu­ hypotheses.
cation has been the subject of increasing research for several years However, three research questions can still be formulated that can
(Kandlhofer et al., 2016, Cetindamar et al., 2022). In addition, there are structure the development of the scale. First, we are interested in
many examples in the literature of courses and classes that strive to whether hidden (or latent) factors influence item responses. These could
increase AI literacy of individuals at different levels of education, e.g., be subconstructs that map different capabilities in the field of AI literacy.
kindergarten (Su & Ng, 2023), high school (Ng et al., 2022), or uni­ For example, it would be possible that AI literacy consists of the specific
versity (Laupichler et al., 2022). However, few attempts have been made subcategories of “awareness,” “usage,” “evaluation,” and “ethics,” as
to develop instruments for assessing individuals’ AI literacy. However, postulated by Wang et al. (2022). As a first step, it would therefore be
the existence of such instruments would be essential, for example, to interesting to determine how many factors there are and which items of
evaluate the teaching effectiveness of the courses described above. the item set can be assigned to each individual factor. Thus, research
Another advantage of AI literacy assessment tools would be the ability to question (RQ) 1 is:
compare the AI literacy of different subgroups (e.g., high school or
RQ1. How many factors should be extracted from the available data,
medical students), identify their strengths and weaknesses, and develop
and which items of the SNAIL-questionnaire load on which factor?
learning opportunities based on these findings. In addition, a scale
reliably assessing AI literacy could be used to characterize study pop­ While RQ1 can be answered mainly with statistical methods (more
ulations in AI-related research. It is important that such assessment in­ on this in the section 2), RQ2 is more concerned with the meaning of the
struments meet psychometric quality criteria. In particular, the factors in terms of factor content. Often, multiple items loading on a
reliability and validity of the instruments are vitally important and single factor follow a specific content theme. This theme can be identi­
should be tested extensively (Verma, 2019). fied and named, and the name can be used as the “title” for the
To our knowledge, there are currently three publications dealing respective factor.
with the development of psychometrically validated scales for AI liter­
RQ2. Can the items loading on a factor be subsumed under a particular
acy which allow a general and cross-sample assessment of AI literacy.
theme that can be used as a factor name?
The first published scale by Wang et al. (2022) found four factors that
constitute AI literacy: “awareness”, “usage”, “evaluation”, and “ethics”. Lastly, in most item sets there are certain items whose added value is
This scale was developed primarily to “measure people’s AI literacy for rather low. This could be due, to the fact that an item is worded
future [Human-AI interaction] research” (p. 5). The authors developed ambiguously or measures something other than what it is supposed to
their questionnaire based on digital literacy research and found that measure. Such items should be excluded from the final scale because
digital literacy and AI literacy overlap to some extent. Another study they can negatively influence the psychometric quality criteria. In
was published by Pinski and Benlian (2023). This study primarily pre­ addition, a scale is more efficient if it requires fewer items while
sents the development of a set of content-valid questions and supple­ maintaining the same quality.
ments this with a pre-test of the item set with 50 participants. Based on
RQ3. Do items exist in the original item set that can be excluded to
the preliminary sample, structural equation modelling was used to
increase the efficiency of the final SNAIL-questionnaire?
examine whether their notion of a general model of AI capabilities was
accurate. While the study is well designed overall, the results of the
2. Material and methods
pre-test based on only 50 subjects can indeed only be considered pre­
liminary. It is also interesting to note that the questionnaire is intended
This study was approved by the local Ethics Committee (application
to be used to assess general AI literacy, but in the pre-selection of par­
number 194/22), and all participants gave informed consent.
ticipants, a certain level of programming ability was required. The most
recent study in this area was published as a preprint by Carolus et al.
(2023) and is still in the peer-review process at this time. The authors 2.1. Variable selection and study design
generated a set of potential AI literacy items derived from the categories
listed in the review by Ng et al. (2021b). Afterwards, the “items were Laupichler et al. (2023) developed a preliminary item set for
discussed, rephrased, rejected, and finalised by [their] team of re­ assessing individuals’ AI literacy in a Delphi expert study. In this study,
searchers” (Carolus et al., 2023, p.6). They then tested the fit of the items 53 experts in the field of AI education were asked to evaluate
to the theoretical categories using confirmatory factor analysis. Of note, pre-generated items in terms of their relevance to an AI literacy ques­
this procedure corresponds to the top-down process of deduction, as the tionnaire. In addition, the experts were asked to contribute their own
authors derive practical conclusions (i.e., items) from theory. item suggestions as well as to improve the wording of the pre-generated
items. The relevance and the wording of 47 items were evaluated in
1.3. Developing the „scale for the assessment of non-experts’ AI literacy” three iterative Delphi rounds (for more information on the Delphi pro­
cess, see Laupichler et al., 2023). This resulted in a preliminary set of 39
The main objective of this paper is to present the development of the content-valid items designed to cover the entire domain of AI literacy.
“Scale for the assessment of non-experts’ AI literacy” (SNAIL), which The authors argued that the item set is preliminary because their psy­
aims to expand the AI literacy assessment landscape. It differs from chometric properties were not assessed in the study. The items were
existing AI literacy assessment tools in several essential ways. First, the formulated as “I can…" statements, e.g. “I can tell if the technologies I
focus of the scale is clearly on non-experts, i.e., individuals who have not use are supported by artificial intelligence”.
had formal AI training themselves and who interact or collaborate with We used an analytical, observational, cross-sectional study design.
AI rather than create or develop it (in contrast to Carolus et al. (2023)). All 39 items created by Laupichler et al. (2023) were presented to the
Second, we focused exclusively on AI literacy items, as the assessment of participants in an online questionnaire. Participants rated the corre­
AI literacy must be detached from related constructs such as digital sponding competency on a seven-point Likert scale from “strongly
literacy (in contrast to Wang et al., 2022). Third, we take an inductive, disagree” (one) to “strongly agree” (seven), as recommended by Lozano
exploratory, bottom-up research approach by moving from specific et al. (2008). The items were presented in random order, and the online
items to generalized latent factors. The main reason for this approach is questionnaire system ensured that the items were presented in a
the prelusive theoretical basis for AI literacy (as described in section different (randomized) order for each participant. In addition to the
1.1). Since this inductive research approach derives theoretical actual AI literacy items, some sociodemographic questions were asked

2
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

about age, gender, country of origin, etc. In addition, two bogus items
were used to control the participants’ attention (see next section).

2.2. Participants

2.2.1. Participant selection and sampling method

The final “Scale for the assessment of non-experts’ AI literacy”
(SNAIL) is intended to be used by non-experts and can be applied in a
variety of educational (i.e., high school and beyond) and professional
settings. Thus, we did not survey a specific (sub-) population but rather
attempted to obtain a sample that is as heterogenous as possible. We
recruited 479 participants through Prolific (www.prolific.com) to take
part in our study. Prolific is an incentive-based platform and participants
received 1.80£ for answering the questionnaire. Participants had to
speak English as their primary language and be over 18 years old.
Therefore, our sampling procedure can be defined as non-probabilistic
and consecutive (total enumerative), since we included every Prolific
participant who met the inclusion criteria until our required sample size
was achieved. The only limitation of the consecutive sampling proced­
ure in our study was that exactly 50% of the participants (n = 240)
should identify as male and 50% (n = 240) as female. Thus, once the 240
participants of one gender were reached, no further participants of that
gender were allowed to participate in the study. Compliance with this
sampling procedure was ensured by Prolific’s automated participant
sampling feature, which randomly sends study invitations to eligible
participants and allows them to participate in the study until the
required number of participants is reached. Since the total population of
all AI non-experts is very large, difficult to delineate, and poorly studied,
we refrained from attempting to achieve a probabilistic and represen­
tative sample.
Since careless responses have a significant influence on the reliability
of factor analyses (Woods, 2006), we used three identification criteria
for careless or inattentive response patterns and excluded those cases
before analysis of the data (see Fig. 1). First, we used an attention check
item “Please check “Somewhat disagree” (3) for this item (third box
from the left).“, which was randomly placed between the actual items.
Participants who failed to choose the correct response option were
excluded from the data set (n = 9). Second, we used a bogus item which
was meant to identify nonsensical or intentionally wrong response
patterns (Meade & Craig, 2012), “I count myself among the top 10 AI
researchers in the world.” Participants who at least partly agreed (five to
seven on a seven-point Likert scale) to the statement were excluded from
data analysis (n = 16). Finally, we excluded all participants whose Fig. 1. Number of participants excluded from data analysis based on three
questionnaire completion time was one standard deviation (2:59 min) exclusion criteria.
below the mean completion time (5:23 min) of all participants (n = 39).
Since our questionnaire consisted of a total of 39 AI literacy questions, average age of the participants was 39.5 years (SD = 13.6), and 208
10 additional questions and some introductory, explanatory and (50.1%) identified as female. On average, the participants included in
concluding text elements, it can be assumed that the probability of the final data set (i.e., after exclusion) took 5:39 min (SD = 2:19 min) to
careless responses increased strongly with completion times of less than complete the questionnaire.
3 min.
Mundfrom et al. (2005) suggest calculating the number of partici­ 2.3. Data analysis
pants needed to conduct an EFA based on communality, variables per
factor, and the number of factors found in comparable studies. Because To conduct a methodologically sound data analysis, we followed the
our study is one of the first studies to develop an AI literacy question­ recommendations of Watkins (2021) in conducting the EFA, as appro­
naire, these parameters were not available in our case. Nevertheless, we priate. In a first step, the data set was analysed for various univariate
believe that the final sample of n = 415 participants is adequate for EFA. descriptive statistical parameters such as skew, kurtosis, the presence of
This is in line with recommendations made by different research groups. outliers, and the number and distribution of missing values. In addition,
For example, Comrey and Lee (1992) found that 300 to 500 participants Mardia’s test of multivariate skew and kurtosis (Mardia, 1970) and
is “good” to “very good”, and Benson and Nasser (1998) found a Mahalanobi’s distance (Mahalanobis, 1936) were calculated to test the
participant to variable ratio of 10:1 to be adequate for EFA (10.6:1 in our multivariate distribution of the data. Afterwards, the appropriateness of
study). the data for conducting an EFA was examined. For this purpose, Bar­
tlett’s test of sphericity (Bartlett, 1950) and the Kaiser-Meyer-Olkin
2.2.2. Sample characteristics criterion (Kaiser, 1974) were calculated and a visual inspection of the
Most participants were from the United Kingdom (n = 316 or correlation matrix was performed to determine whether a sufficient
76.1%), South Africa (n = 32 or 7.7%), the United States (m = 27 or number of correlations ≥.30 was present.
6.5%), Australia (n = 9 or 2.2%), and Canada (n = 9 or 2.2%). The Since our goal was to “understand and represent the latent structure

3
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

of a domain” (Widaman, 2018, p. 829), we chose common factor anal­ After the EFA was conducted, the SNAIL-questionnaire was short­
ysis over principal component analysis. However, since we used a ened to improve questionnaire economy and thereby increase the
relatively high number of variables (39), both techniques would likely acceptability of using SNAIL as an assessment tool. As a basis for
produce fairly similar results (Watkins, 2021). deciding whether to exclude variables, we looked at salient pattern co­
Although different factor extraction methods generally yield similar efficients on more than one factor on the one hand, and a particularly
results (Tabachnik et al., 2019), we compared the results of maximum low communality on the other.
likelihood extraction and iterated principal axis extraction due to the Data pre-processing was done partially in Microsoft Excel (Microsoft
multivariate non-normality of our data. The differences between the two Corporation, 2018) or R (R Core Team, 2021) and RStudio (RStudio
extraction methods were negligible, so we applied the more commonly Team, 2020), respectively. Data analysis and data visualization was
used maximum likelihood extraction. We used squared multiple corre­ conducted entirely in R and RStudio.
lations for the initial estimation of communalities. Since our variables
were in principle ordinal at least, we based the analysis on the poly­ 3. Results
choric correlation matrix instead of the more commonly used Pearson
correlation matrix. We used parallel analysis by Horn (1965) and the 3.1. Data screening and appropriateness of data for EFA
minimum average partial (MAP) method of Velicer (1976) to decide
how many factors to retain. A scree-plot (Catell, 1966) was used for The univariate distribution of all variables was acceptable, with
visual representation, but not as a decisive method, since it was found to skewness values ranging from − 1.18 to 0.87, which is in the acceptable
be rather subjective and researcher-dependent (Streiner, 1998). Since range of − 2.0 to 2.0. Similar results were found for univariate kurtosis,
we expected the various factors in the model to be at least somewhat with values ranging from − 1.26 to 1.85, which is in the acceptable range
correlated, we used an oblique rotation method. We used the promax of − 7.0 to 7.0 (see supplementary material 1). Because Mardia’s test of
rotation method as a basis for interpretation, but compared the results multivariate skew and kurtosis became significant (p < .001), multi­
with the oblimin rotation method. Norman and Streiner (2014) sug­ variate non-normality had to be assumed. Bentler (2005) found that
gested to set the threshold at which pattern coefficients (factor loadings) increased multivariate kurtosis values of ≥5.0 can influence the results
will be considered meaningful (i.e., salient) to √5.152
̅̅̅̅̅̅̅ (for p = .01).
N− 2
of EFA when working with Pearson correlation matrices, which is
However, due to the large number of participants in our study, this another reason to base calculations on the polychoric correlation matrix.
would imply a relatively low salience threshold of 0.25, which is why we Using the Mahalanobis distance (D2), some outliers were identified, but
followed the more conservative suggestion made by Comrey and Lee these were still within the normal range and showed no signs of sys­
(1992), who considered a minimum loading of 0.32 as salient. tematic error. Data entry errors or other third-party influences are
highly unlikely because we used automated questionnaire programs.

Fig. 2. Distribution and number of missing values in absolute and relative terms across all subjects and variables.

4
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

Thus, we could not find any “demonstrable proof [that] indicates that 3.3.2. Two and three factor models
they are truly aberrant and not representative of any observations in the The difference between the two-factor model and the three-factor
population” (Hair et al., 2019, p. 91), which is why we did not exclude model was rather ambiguous, which is consistent with the contrasting
these cases from the data set. In total, each variable missed between results of the parallel analysis and the minimum average partial method,
0 and 4 values, which makes up 0–0.96% of all data. In addition, the which suggested the extraction of two and three factors, respectively.
data was missing completely at random, as demonstrated in Fig. 2. Both models had somewhat elevated levels of off-diagonal residuals,
Therefore, no imputation or deletion methods were applied. with 15.1% of residuals exceeding 0.05 and 3% of residuals exceeding
Based on Bartlett’s test of sphericity, the null-hypothesis that the 0.10 in the two-factor model and 11.3% of residuals exceeding 0.05 and
correlation matrix was an identity matrix could be rejected (p < .001). 1.08% of residuals exceeding 0.10 in the three-factor model. Although
The significant result (i.e., p < .05) indicates that there is some redun­ this might indicate underfactoring, it could also be due to the ordinal
dancy among the variables, which means that they can be reasonably nature of the data set and the multivariate non-normality. In addition,
summarized with a smaller number of factors. The overall MSA of the the RSMR-value of both models (0.04 and 0.03, respectively) lay under
Kaiser-Mayer-Olkin criterion was 0.97, with a range of 0.94–0.98 for the suggested threshold of ≤ 0.08.
each item, which is far above the minimum recommended threshold of All models had a sufficient number of pattern coefficients that loaded
0.5 (Field et al., 2012) or 0.6 (Tabachnik et al., 2019), respectively. A saliently on each factor (i.e., more than three, Fabrigar & Wegener,
visual inspection of the correlation matrix revealed that a majority of the 2012; Mulaik, 2009). The only exception is the three-factor oblimin
coefficients were ≥0.30, indicating a sufficiently high magnitude of model when applying the conservative salience threshold of ≥ 0.32
coefficients in the correlation matrix. Based on these measures, we described above. Here, no variables would load saliently on the third
assumed that the correlation matrix was adequate for performing an factor. The promax rotation method, on the other hand, comes to a
EFA. (Watkins, 2021; Hair et al., 2019; Tabachnik et al., 2019). reasonable distribution of salient pattern coefficients on all three factors.
The two- and three-factor model both showed marginally acceptable
3.2. Number of factors to retain communalities and no Heywood-cases (Harman, 1976). The mean of the
communalities was 0.54 (SD = 0.08) for the two-factor model and 0.57
Horn’s parallel analysis, conducted with 20,000 iterations, found (SD = 0.08) for the three-factor model.
two factors to be the optimal solution, regardless whether the reduced or While the one-factor model was only able to explain 48% of the
unreduced correlation matrix was used. In contrast, Velicer’s minimum variance, the two-, three- and four-factor models were able to explain
average partial reached a minimum of 0.0086 with three factors. A vi­ 54%, 57%, and 58% of the variance, respectively.
sual inspection of the scree plot supports these results. Depending on To analyse the internal consistency reliability, we combined every
subjective preferences, two or three factors could be retained (Fig. 3). variable that saliently loaded on a factor in a scale and calculated
Consequently, we analysed models with one, two, three, and four factors Cronbach’s alpha with bootstrapped confidence intervals. The internal
for signs of under- or overfactoring, as well as their interpretability and consistency of both scales in the two-factor model was excellent, with α
theoretical meaningfulness. = 0.95 [CI 0.94, 0.96] for the first scale and α = 0.94 [CI 0.93, 0.95] for
the second scale. Albeit having slightly lower alpha-values, the internal
3.3. EFA model evaluation consistency of the three scales in the three-factor model was also
excellent: α = 0.94 [CI 0.93, 0.95] for the first scale, α = 0.93 [CI 0.91,
Following RQ1, the next section evaluates and compares different 0.94] for the second scale, and α = 0.89 [CI 0.87, 0.91] for the third
factor models to identify the most fitting number of factors. scale.

3.3.1. One factor model 3.3.3. Four factor model

The hypothesis that the one-factor model would exhibit signs of Most of the parameters described above (e.g., RSMR, number of
underextraction was confirmed. The communalities were rather weak salient pattern coefficients) would also have been acceptable when using
(only two variables had communalities >0.60) and there was no the four-factor model. However, fewer variables loaded on each factor,
reasonable unifying theme (i.e., meaningful content category/cate­ with only three variables loading saliently on the fourth factor, which
gories) other than that they were evaluating some aspect of AI literacy. might be a weak indication of overextraction. Four main reasons speak
Furthermore, 47.8% of the off-diagonal residuals exceeded 0.05 and against the adoption of the four-factor model: First, parallel analysis and
15.1% exceeded 0.10. These results were consistent across rotation minimum average partials have resulted in the recommendation to
techniques (i.e., promax and oblimin rotation), therefore strongly indi­ extract either two or three factors. Second, the increase in explained
cating the presence of at least one other factor. variance from the three-factor model to the four-factor model is rela­
tively insignificant at less than one percent. Third, the salient loading
variables could not be classified into any meaningful content-related
categories. And fourth, all other things being equal, the more parsimo­
nious solution is usually the better one (Ferguson, 1954).

3.4. Final model selection and factor names

Since Cattell (1978) and other researchers conclude that the right
number of factors is not a question of a correct absolute number, but
rather a question “of not missing any factor of more than trivial size” (p.
61), the three-factor model seems to represent a good compromise be­
tween parsimony and avoiding the risk of underextraction (see Fig. 4).
As for RQ2, the findings and assessments based on the data coincide
well with the content-related examination of the individual factors. With
the two-factor solution, a unifying theme could be identified but is
rather diffuse and unclear. However, the three-factor solution creates a
more plausible classification of the manifest variables to the latent fac­
Fig. 3. Screeplot.
tors in terms of content (see Table 1). Based on the reasons given, the

5
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

Fig. 4. Path diagram for the 3-factor promax model.

three-factor model was chosen as the best model. “Critical Appraisal” and “Practical Application”). Furthermore, two
The first factor’s highest pattern coefficients were found in variables variables were deleted because of low communalities; e.g. “I can explain
centred around the understanding of machine learning, e.g. “I can the differences between human and artificial intelligence”, and one item
describe how machine learning models are trained, validated, and that did not load saliently on any factor and had a weak communality; “I
tested”. Other rather technical or theoretical AI competencies such as can explain what an algorithm is” (see Table 1). We repeated the EFA
defining the differences between general and narrow AI or explaining process with the reduced set of variables and found comparable results.
“how sensors are used by computers to collect data that can be used for One of the main differences was the decrease in interfactor-correlations,
AI purposes” load saliently on this factor, too. Thus, we propose the first which is somewhat trivial, given that we specifically excluded variables
factor’s name to be “Technical Understanding”. The variables loading that loaded saliently on more than one factor. The internal consistency
saliently on the second factor deal with the recognition of the impor­ of the three scales (i.e., three factors) based on the reduced variable set
tance of data privacy and data security in AI, ethical issues related to AI, was excellent. The alpha-values were very similar to the values of the
and risks or weaknesses that may appear when applying AI technologies. unreduced variable set, with α = 0.93 [CI 0.92, 0.94] for the first scale, α
Therefore, the second factor is to be called “Critical Appraisal” as it = 0.91 [CI 0.89, 0.93] for the second scale, and α = 0.85 [CI 0.81, 0.88]
reflects competencies related to the critical evaluation of AI application for the third scale.
results. Lastly, the variables with the highest pattern coefficients that For the sake of brevity, all other results and diagrams can be found in
load on the third factor are concerned with “examples of technical ap­ the supplementary material (Supplementary Material 1). Consequently,
plications that are supported by artificial intelligence” or assessing “if a the final SNAIL-questionnaire consists of 31 variables loading on three
problem in [one’s] field can and should be solved with artificial intel­ factors.
ligence methods”. Consequently, the third factor is to be called “Prac­
tical Application”. Accordingly, the interaction of the three factors could 4. Discussion
be called the TUCAPA-model of AI literacy.
4.1. Relation between TUCAPA and other models
3.5. Variable elimination
One of the most well-known lists of AI literacy components was
certainly published by Long and Magerko (2020), who list 16 compe­
The last section of the results section serves to answer RQ3, which
tencies that constitute AI literacy. These competencies seem to have only
deals with the elimination of items that do not add value and can
minor relevance for the design of AI literacy assessment questionnaires.
therefore be excluded. As described above, we excluded variables that
This could be due to the large number of 16 competencies, some of
loaded saliently on more than one factor and variables with a commu­
which are at the level of latent factors (e.g., competency 11 “Data Lit­
nality of 2 SD (i.e., 0.08) under the mean communality (0.57). After item
eracy”) and some at the level of individual manifest variables (e.g.,
elimination, 31 items remained in the final SNAIL-questionnaire. We did
competency 4 “General vs. Narrow [AI]”). Nevertheless, some compe­
not use item parameters (i.e., item difficulty1 and item discrimination2)
tencies listed by Long & Magerko (e.g., competency 1 “Recognizing AI”)
as exclusion criteria because they were all within the acceptable range
correspond to variables used in SNAIL (e.g., V01 “I can tell if the tech­
(see Table 2).
nologies I use are supported by artificial intelligence.“).
Overall, five items were eliminated due to diffuse loading patterns; e.
Many researchers refer to the literature review by Ng et al. (2021b)
g. “I can name strengths of artificial intelligence” (loading saliently on

6
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

Table 1
List of all variables sorted by factors based on the three-factor TUCAPA-model of AI literacy.

Note. The variables are sorted by pattern coefficient, with variables loading the highest on each factor
appearing at the top of each column. Note that the table shows the model before elimination of eight
items. Eliminated items have a lighter font. The superscript numbers indicate the reason for elimi­
nation, with (1) indicating salient loadings on more than one factor, (2) indicating extraordinarily low
communalities, and (3) indicating a combination of (1) and (2).

7
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

Table 2 Future research should investigate whether AI competencies related

Item parameters sorted by factors based on the three-factor promax model. to “ethics” or “ethical issues” really represent a separate AI literacy
F1 – Technical Understanding factor, or whether this competency is part of a larger construct such as
“critical appraisal.” In any case, our model contributes to the further
Item Mean SD Item Difficulty Item Discrimination
development of AI literacy theory, as it differs from other models in
V14 1.63 1.51 .27 .76 terms of its factor count and by following an inductive approach. This
V17 1.61 1.43 .27 .72
V30 1.88 1.58 .31 .72
inductive approach does not require theoretical considerations in
V12 2.15 1.58 .36 .70 advance, but develops theoretical insights from practical observations.
V16 1.98 1.56 .33 .68
V04 1.73 1.44 .29 .68 4.2. Limitations
V23 1.99 1.61 .33 .69
V18 1.52 1.53 .25 .70
V13 2.48 1.68 .41 .71 One of the major limitations of self-assessment questionnaires is that
V26x 1.69 1.68 .28 .52 their responses can be influenced by conscious or unconscious biases.
V15 2.09 1.61 .35 .66 For this reason, the current questionnaire should only be used if the
V33 1.97 1.53 .33 .59
results of the survey are not linked to consequences that directly affect
V22 2.26 1.52 .38 .73
V36 2.25 1.72 .37 .64
the respondents (e.g., grades, job applications). In addition to the
V27x 2.4 1.75 .40 .67 development of self-assessment scales, it would therefore be important
V11x 2.16 1.56 .36 .67 to develop performance tests that objectively test individuals’ AI
V05 2.65 1.76 .44 .64 knowledge and skills, rather than having them subjectively rated by the
F2 – Critical Appraisal respondents themselves.
V35 3.62 1.57 .60 .70 The TUCAPA model is composed of three factors derived from sta­
V34 3.48 1.61 .58 .69 tistical results, as shown earlier. However, other research groups
V25 3.62 1.55 .60 .70 reached a different number of factors in their studies, some of which
V08 3.46 1.5 .58 .74 contained slightly different substantive foci. For example, Wang et al.
V06 3.54 1.5 .59 .73
(2022) and Carolus et al. (2023) found a factor with a focus on “AI
V39 2.97 1.72 .49 .68
V28 3.31 1.6 .55 .70 ethics” that is not represented as a separate factor in the TUCAPA model.
V21 3.68 1.47 .61 .70 This may be due to several reasons. One possible explanation is that the
V20 3.42 1.58 .57 .70 experts in the Delphi study by Laupichler et al. (2023), in which the
V03x 4 1.31 .67 .56
items were generated, did not consider ethical aspects of AI and there­
V09x 3.7 1.41 .62 .70
V32 3.94 1.18 .66 .63
fore formulated few items on this topic.
In addition, the use of paid and anonymous study participants in­
F3 – Practical Application
volves certain risks and might lead to response biases. For example, it
V37 3.65 1.53 .61 .59 could be assumed that the anonymity and incentivization cause the
V02 2.84 1.76 .47 .67
acquired subjects to spend little time and attention on answering the
V01 2.6 1.55 .43 .60
V31 2.5 1.64 .42 .70 SNAIL-questionnaire. However, we used three different careless
V24 2.22 1.72 .37 .63 responding checks, making it unlikely that participants merely “clicked
V10x 3.46 1.52 .58 .68 through” the questionnaire. In addition, several studies have shown that
V29 3.41 1.49 .57 .65 the use of paid online participants does not pose an extraordinary threat
V19 2.43 1.72 .40 .68
V07x 3.54 1.46 .59 .63
to the scientific integrity of research (Buhrmester et al., 2011; Crump
V38x 3.63 1.53 .60 .55 et al., 2013). Nevertheless, it may be worth repeating the study with a
different sample, as we used a non-probabilistic consecutive sampling
Note. Items are sorted by the magnitude of their pattern coefficients, with items
technique that could affect the validity of the results described. It is
having higher loadings listed first. To calculate item difficulty1 and item
discrimination,2 the data set was mutated by subtracting 1 from every value in
possible that sampling bias has occurred due to the sampling technique.
the data set. Consequently, the range of possible values was 0–6 (instead of the For example, there is a possibility that only people who are already
aforementioned Likert-scale with values ranging from 1 to 7). Items that were interested in the topic of AI and therefore rate their abilities higher than
eliminated are indicated by (x). people who are not interested in AI participated in the study. A new
dataset should preferably be representative of the entire population of AI
in developing their models (Carolus et al., 2023, Pinski et al., 2023). The non-experts’, or at least differ from the dataset used in this study in
categories identified by Ng et al. seem to fit relatively well with the three terms of participant characteristics, participants’ countries of origin, etc.
factors of the TUCAPA-model, as “know and understand” overlaps with This would also have the advantage of ensuring the reproducibility and
“Technical Understanding” and “use and apply” corresponds to “Prac­ reliability of the results reported in this study, as it would enable the
tical Application”. The last two factors of Ng et al., “evaluate and create” execution of a confirmatory factor analysis.
and “ethical issues,” could be combined into one factor in our case,
“Critical Appraisal". 4.3. Future research
Karaca et al. (2021) developed MAIRS-MS, a scale designed to assess
the so-called AI readiness of medical students. AI readiness is a construct Future research projects should test the theoretical validity of the
that resembles AI literacy in many ways. In their research project, they three-factor TUCAPA model through confirmatory factor analysis (CFA).
also conducted an EFA and found four factors that seem to fit well with This could simultaneously determine whether there is a separate “AI
the factors described in this paper. Karaca et al.‘s “Cognition”-factor is ethics” factor or whether the aspect of AI ethics is already included in the
very similar to the “Technical Understanding”-factor, although the un­ three factors of the TUCAPA model (e.g., in the Critical Appraisal fac­
derlying items tend to be on a more general level (e.g., “I can define the tor). In addition to the previously mentioned use of the questionnaire in
basic concepts of data science.“, p. 5). The “Ability”-factor has some other samples or in specific sub-populations, the use of SNAIL in other
resemblance to the “Practical Application”-factor in our model. And cultures would also be important. For this purpose, the questionnaire
again, the last two factors, “Vision” and “Ethics,” could be combined into has to be validly translated into the corresponding languages before­
the “Critical Appraisal” factor of the TUCAPA model. hand. This would help the international applicability of the scale, as the
questions are currently only available in English. Moreover, it should be

8
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

investigated whether SNAIL can be applied equally well in all subject Data availability
domains, or whether there are practical differences in AI literacy be­
tween different domains. For example, it could be possible that in­ Research data will be published as Supplementary Material (Excel-
dividuals with a high level of technical understanding (e.g., individuals File)
from the field of mathematics or mechanical engineering) would rate the
questions of the Technical Understanding factor very positively, while Appendix A. Supplementary data
people from fields with less technical affinity (e.g., medicine, psychol­
ogy) may evaluate the same questions rather negatively. Furthermore, it Supplementary data to this article can be found online at https://doi.
should be examined whether SNAIL is suitable to investigate the org/10.1016/j.chbr.2023.100338.
teaching effectiveness of courses that aim to increase the AI literacy of
their participants. Since SNAIL is freely available as an open access of­ References
fering, this would also be interesting for platforms such as “Elements of
AI” (University of Helsinki & MinnaLearn, 2018) or “AI Campus” (KI Bartlett, M. S. (1950). Tests of significance in factor analysis. British Journal of Psychology,
3, 77–85.
Campus, 2023), which offer open educational resources to improve Benson, J., & Nasser, F. (1998). On the use of factor analysis as a research tool. Journal of
general AI literacy. Last but not least, the SNAIL-questionnaire should be Vocational Education Research, 23(1), 13–33.
compared with related constructs such as “attitudes toward AI” Bentler, P. M. (2005). EQS structural equations program manual. Multivariate software.
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s mechanical Turk.
(Schepman & Rodway, 2020; Sindermann et al., 2021) or “digital lit­ Perspectives on Psychological Science, 6(1), 3–5. https://doi.org/10.1177/
eracy” (Gilster, 1997) to investigate the relationship between each 1745691610393980
construct. For example, it is possible that more pronounced AI literacy Carolus, A., Koch, M., Straka, S., Latoschik, M. E., & Wienrich, C. (2023). MAILS – meta AI
literacy scale: Development and testing of an AI literacy questionnaire based on well-
reduces anxiety toward AI (Wang & Wang, 2022), leading to more founded competency models and psychological change- and meta-competencies.
positive attitudes toward AI. Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral
Research, 1(2), 245–276. https://doi.org/10.1207/s15327906mbr0102_10
Cattell, R. B. (1978). Use of factor analysis in behavioral and life sciences.
5. Conclusion
Cetindamar, D., Kitto, K., Wu, M., Zhang, Y., Abedin, B., & Knight, S. (2022). Explicating
AI literacy of employees at digital workplaces. IEEE Transactions on Engineering
We conducted an exploratory factor analysis to develop the “Scale for Management. https://doi.org/10.1109/TEM.2021.3138503
the assessment of non-experts’ AI literacy” (SNAIL) questionnaire, which Comrey, A., & Lee, H. (1992). Interpretation and application of factor analytic results. In
A first course in factor analysis (2nd ed.). Lawrence Erlbaum Associates.
is designed to assess AI literacy in non-experts. In doing so, we found that Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating amazon’s
the construct represented by the questionnaire can be divided into three mechanical Turk as a tool for experimental behavioral research. PLoS One, 8(3),
subfactors that influence individuals’ response behaviour on AI literacy Article e57410. https://doi.org/10.1371/journal.pone.0057410
Fabrigar, L. R., & Wegener, D. T. (2012). Exploratory factor analysis. Oxford, UK: Oxford
items: Technical Understanding, Critical Appraisal, and Practical Appli­ University Press.
cation. Therefore, the model can be abbreviated as the TUCAPA model of Faruqe, F., Watkins, R., & Medsker, L. (2021). Competency model approach to AI literacy:
AI literacy. Our study provides initial evidence that the 31 SNAIL items Research-based Path from initial framework to model. ArXiv Preprint.
Ferguson, G. A. (1954). The concept of parsimony in factor analysis. Psychometrika, 19
are able to reliably and validly assess the AI competence of nonexperts. (4), 281–290. https://doi.org/10.1007/BF02289228
However, further research is needed to evaluate whether the results found Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. SAGE.
in our study can be replicated and are representative of the population of Gilster, P. (1997). Digital literacy. John Wiley & Sons, Inc.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis
nonexperts. Finally, we would like to encourage all researchers in the field (8th ed.). Cengage Learning.
of AI literacy to use psychometrically validated questionnaires to assess Harman, H. H. (1976). Modern factor analysis (3rd ed.). University of Chicago Press.
the AI literacy of individuals and groups as well as to evaluate the learning Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis.
Psychometrika, 30(2), 179–185. https://doi.org/10.1007/BF02289447
outcome of course participants.
Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36.
https://doi.org/10.1007/BF02291575
Funding statement Kandlhofer, M., Hirschmugl-Gaisch, S., & Huber, P. (2016). Artificial intelligence and
computer science in education: From kindergarten to university. 2016 IEEE Frontiers
in Education Conference (FIE), 1–9.
This work was supported by the Open Access Publication Fund of the Karaca, O., Çalışkan, S. A., & Demir, K. (2021). Medical artificial intelligence readiness
University of Bonn. scale for medical students (MAIRS-MS) – development, validity and reliability study.
BMC Medical Education, 21(1), 112. https://doi.org/10.1186/s12909-021-02546-6
KI Campus. (2023). AI Campus. https://Ki-Campus.Org/.
Ethics approval statement König, P. D., & Wenzelburger, G. (2020). Opportunity for renewal or disruptive force?
How artificial intelligence alters democratic politics. Government Information
Data collection for this study took place in February 2023 using the Quarterly, 37(3), Article 101489. https://doi.org/10.1016/j.giq.2020.101489
Laupichler, M. C., Aster, A., & Raupach, T. (2023). Delphi study for the development and
study participant acquisition program Prolific. The subjects received preliminary validation of an item set for the assessment of non-experts’ AI literacy.
appropriate financial compensation for participating in the study. Computers and Education: Artificial Intelligence, 4, Article 100126. https://doi.org/
Participation in the study was voluntary and participants gave their 10.1016/j.caeai.2023.100126
Laupichler, M. C., Aster, A., Schirch, J., & Raupach, T. (2022). Artificial intelligence
informed consent. The study was approved by the Research Ethics literacy in higher and adult education: A scoping literature review. Computers and
Committee of the University of Bonn (Reference 194/22). Education: Artificial Intelligence, 3, Article 100101. https://doi.org/10.1016/j.
caeai.2022.100101
Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design
Author contributions
considerations. In Proceedings of the 2020 CHI Conference on Human Factors in
Computing Systems, 1–16. https://doi.org/10.1145/3313831.3376727
Matthias Carl Laupichler: Conceptualization, Formal Analysis, Lozano, L. M., García-Cueto, E., & Muñiz, J. (2008). Effect of the number of response
categories on the reliability and validity of rating scales. Methodology, 4(2), 73–79.
Writing – Original Draft, Visualization Alexandra Aster: Writing – Re­
https://doi.org/10.1027/1614-2241.4.2.73
view & Editing, Data Curation Nicolas Haverkamp: Methodology. Mahalanobis, P. C. (1936). On the generalized distance in statistics. Journal of the Society
Tobias Raupach: Supervision, Resources. of Bengal, 2(1), 49–55.
Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications.
Biometrika, 57(3), 519–530.
Declaration of competing interest Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data.
Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085
The authors declare that they have no known competing financial Microsoft Corporation. (2018). Microsoft Excel. Retrieved from https://office.microsoft.
com/excel.
interests or personal relationships that could have appeared to influence
the work reported in this paper.

9
M.C. Laupichler et al. Computers in Human Behavior Reports 12 (2023) 100338

Mulaik, S. A. (2009). Foundations of factor analysis. Chapman and Hall/CRC. https://doi. Streiner, D. L. (1998). Factors affecting reliability of interpretations of scree plots.
org/10.1201/b15851 Psychological Reports, 83(2), 687–694. https://doi.org/10.2466/pr0.1998.83.2.687
Mundfrom, D. J., Shaw, D. G., & Ke, T. L. (2005). Minimum sample size Su, J., & Ng, D. T. K. (2023). Artificial intelligence (AI) literacy in early childhood
recommendations for conducting factor analyses. International Journal of Testing, 5 education: The challenges and opportunities. Computers and Education: Artificial
(2), 159–168. https://doi.org/10.1207/s15327574ijt0502_4 Intelligence. , Article 100124. https://doi.org/10.1016/j.caeai.2023.100124
Ng, D. T. K., Leung, J. K. L., Chu, K. W. S., & Qiao, M. S. (2021a). AI literacy: Definition, Tabachnik, B. G., Fidell, L. S., & Ullman, J. B. (2019). In Using multivariate statistics (Vol.
teaching, evaluation and ethical issues. Proceedings of the Association for Information 7). Pearson.
Science and Technology, 58(1), 504–509. https://doi.org/10.1002/pra2.487 University of Helsinki, MinnaLearn. (2018). Elements of AI. https://www.elementsofai.
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021b). Conceptualizing AI com/.
literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, Velicer, W. F. (1976). Determining the number of components from the matrix of partial
Article 100041. https://doi.org/10.1016/j.caeai.2021.100041 correlations. Psychometrika, 41(3), 321–327. https://doi.org/10.1007/BF02293557
Ng, D. T. K., Leung, J. K. L., Su, M. J., Yim, I. H. Y., Qiao, M. S., & Chu, S. K. W. (2022). AI Verma, J. P. (2019). Statistics and research methods in psychology with Excel. Springer
literacy in K-16 classrooms. Springer. Singapore. https://doi.org/10.1007/978-981-13-3429-0
Norman, G. R., & Streiner, D. L. (2014). Biostatistics: The bare essentials (4th ed.). People’s Wang, B., Rau, P. L. P., & Yuan, T. (2022). Measuring user competence in using artificial
Medical Publishing. intelligence: Validity and reliability of artificial intelligence literacy scale. Behaviour
Pinski, M., & Benlian, A. (2023). AI literacy - towards measuring human competency in & Information Technology. https://doi.org/10.1080/0144929X.2022.2072768
artificial intelligence. Proceedings of the 56th Hawaii International Conference on Wang, Y. Y., & Wang, Y. S. (2022). Development and validation of an artificial
System Sciences, 165–174. intelligence anxiety scale: An initial application in predicting motivated learning
R Core Team. (2021). R: A language and environment for statistical computing. Vienna, behavior. Interactive Learning Environments, 30(4), 619–634. https://doi.org/
Austria: R Foundation for Statistical Computing. https://www.R-project.org/. 10.1080/10494820.2019.1674887
Reddy, S., Fox, J., & Purohit, M. P. (2019). Artificial intelligence-enabled healthcare Watkins, M. R. (2021). A step-by-step guide to exploratory factor analysis with R and
delivery. Journal of the Royal Society of Medicine, 112(1), 22–28. https://doi.org/ RStudio. Routledge.
10.1177/014107681881551 Widaman, K. F. (2018). On common factor and principal component representations of
RStudio Team. (2020). RStudio. Boston, MA: Integrated Development for R. RStudio, data: Implications for theory and for confirmatory replications. Structural Equation
PBC. http://www.rstudio.com/. Modeling: A Multidisciplinary Journal, 25(6), 829–847. https://doi.org/10.1080/
Schepman, A., & Rodway, P. (2020). Initial validation of the general attitudes towards 10705511.2018.1478730
artificial intelligence scale. Computers in Human Behavior Reports, 1, Article 100014. Woods, C. M. (2006). Careless responding to reverse-worded items: Implications for
https://doi.org/10.1016/j.chbr.2020.100014 confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment,
Sindermann, C., Sha, P., Zhou, M., Wernicke, J., Schmitt, H. S., Li, M., Sariyska, R., 28(3), 186–191. https://doi.org/10.1007/s10862-005-9004-7
Stavrou, M., Becker, B., & Montag, C. (2021). Assessing the attitude towards artificial Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J., &
intelligence: Introduction of a short measure in German, Chinese, and English Li, Y. (2021). A review of artificial intelligence (AI) in education from 2010 to 2020.
language. KI - Kunstliche Intelligenz, 35(1), 109–118. https://doi.org/10.1007/ Complexity, 2021, 1–18. https://doi.org/10.1155/2021/8812542
s13218-020-00689-0

10

View publication stats