International Journal of Evaluation and Research in Education (IJERE)

Vol. 13, No. 1, February 2024, pp. 18~30

ISSN: 2252-8822, DOI: 10.11591/ijere.v13i1.26013  18

Research trends in critical thinking skills in mathematics:

a bibliometric study

Arif Hidayatul Khusna1,2, Tatag Yuli Eko Siswono1, Pradnyo Wijayanti1

1
Department of Mathematics Education, Faculty of Mathematics and Science, State University of Surabaya, Surabaya, Indonesia
2
Department of Mathematics Education, Faculty of Teacher Training and Education, University of Muhammadiyah Malang, Malang,
Indonesia

Article Info ABSTRACT

Article history: This study systematically reviews and analyzes previous studies’ literature to
identify the research gaps and the steps that must be taken for research on
Received Nov 19, 2022 critical thinking in further mathematics education. Two methods, namely
Revised Sep 19, 2023 bibliometric analysis and descriptive content analysis, were used to achieve
Accepted Oct 24, 2023 this goal. Data for the bibliometric analysis was obtained from the Scopus
database, with the keywords “critical thinking” and “mathematics”
Documents accessed are limited to the publication range of 2012-2022. The
Keywords: results show that the number of critical thinking research in mathematics has
increased significantly, with a peak in 2021 of 65 articles; the United States
Analysis has the most research and citations. The source that received the most
Bibliometric citations was the journal Educational Studies in Mathematics. The evolution
Collaboration learning of the theme of critical thinking in mathematics began to link critical
Critical thinking thinking with collaboration learning, mathematics education, teacher
Mathematics education, engineering, education, and collaborative learning. The
Teacher education quantitative approach with a student sample, tests as data collection
instruments, and the analysis of variance (ANOVA) method is the
methodology most often used by researchers. The results of this study are
likely helpful for other researchers who will examine critical thinking skills
in mathematics, namely as a research guide that they will develop. In
addition, the existence of several research gaps can motivate other
researchers to fill these gaps to contribute to improving the quality of critical
thinking in mathematics.
This is an open access article under the CC BY-SA license.

Corresponding Author:
Arif Hidayatul Khusna
Department of Mathematics Education, Faculty of Mathematics and Science, State University of Surabaya
Jalan Ketintang, Gayungan, Surabaya, Indonesia
Email: [email protected]

1. INTRODUCTION
Thinking is an individual’s effort to understand something by processing information cognitively.
The ability to think is a unique ability humans possess that distinguishes them from other creatures. Humans
can develop their knowledge through thinking activities based on logical things. Critical thinking is one type
of thinking that, for the last two decades, has become a keyword in both theory and pedagogical practice [1].
Dewey introduced the concept of critical thinking in 1933 with the term reflective thinking. Through
reflective thinking, individuals indirectly learn to think critically by identifying and formulating problems,
selecting relevant information, developing solution designs by combining experience and knowledge, and
justifying the feasibility of solutions [2].

Journal homepage: http://ijere.iaescore.com

Int J Eval & Res Educ ISSN: 2252-8822  19

Several experts have defined critical thinking as the process of determining what to believe or do to
determine the validity, reliability, and authenticity of the information or a particular claim [3]–[6]. According
to Boran and Karakuş [7], critical thinking is a systematic mental process of solving problems, asking
questions, and finding appropriate solutions through skills in evaluating available information. Someone who
thinks critically will look at things from various perspectives, evaluate situations based on the reasons and
scientific evidence found, and think actively by organizing his thoughts to explore and analyze logically
through high-level reasoning activities [8]. Critical thinking is a cognitive skill that includes interpretation,
analysis, evaluation, inference, explanation, and self-regulation as a basis for decision-making. Cortázar et al.
[9] change into a broader explanation of argumentation and self-regulation into metacognition. Ennis,
mentions that critical thinkers must have basic clarification skills, bases for decision, inference, and advanced
clarification. Critical thinking is a cognitive process that actively and skillfully conceptualizes, analyzes,
synthesizes, applies, and develops information obtained from reflection, observation, reasoning, experience,
or communication as a guide in acting on these beliefs [10], [11].
In recent years, research on critical thinking and its application in mathematics has increased
significantly [12], [13]. Critical thinking skills are needed in mathematical activities such as the problem-
solving process. To begin solving individual problems simplifies complex settings by identifying the most
relevant concepts and processes at the heart of the problem involving making decisions about what to pay
attention to and what to ignore, developing an understanding of how essential concepts are connected, and
generating representations realistic from the actual situation [14]–[16]. The literature study results show that
no research collectively examines critical thinking in mathematics. Based on the results of this review, this
study aimed to systematically review and analyze the existing research literature to obtain information related
to research gaps. Furthermore, the results of this study are expected to contribute to further research on the
steps that must be taken for further research related to critical thinking in mathematics.

2. RESEARCH METHOD
This study aimed to identify research gaps and steps to be taken for further research on critical
thinking in mathematics. Two methods are used to achieve this goal: bibliometric analysis and descriptive
content analysis. Bibliometric analysis was chosen because the bibliometric analysis can represent the state
of all research from the past to the present based on several perspectives [17], [18]. Bibliometrics is a
statistical check of information and quantitative analysis of bibliographic elements of scientific publications
[19]. Based on that, researchers can use bibliometric analysis to examine the dynamics of all existing studies
from a broader perspective [20]. Descriptive content analysis is used to look more profoundly and
comprehensively at research trends in critical thinking in mathematics. According to Dinçer [21], descriptive
content analysis methods aim to reveal patterns of themes in frequency or percentage to present the case as a
whole and do not allow detailed inference.
In this method, quantitative and qualitative data can be used. Categorizing studies in a particular
field carried out over a certain period is the most common study practice using descriptive content analysis.
The descriptive content analysis method is used to descriptively identify research topics related to critical
thinking in the existing field of mathematics. In addition, with this descriptive content analysis, the trend of
critical thinking research in mathematics will be seen in terms of methodology. According to a study [22],
researchers’ general tendency to develop research related to critical thinking skills in mathematics can be
known through checking quantitative and qualitative studies. In addition, to control study results, we used the
selected reporting items for systematic review and meta-analysis protocol (PRISMA) technique. PRISMA’s
stages are clearly shown in Figure 1. Meanwhile, this study uses the Scopus database as data source.

2.1. Data collection

In collecting data for bibliometric analysis, “critical thinking” and “mathematics” are the keywords
used to identify research related to critical thinking in mathematics education. Access to documents is done at
Scopus on July 27th, 2022 with the field of social science research. Data collection for this bibliometric
analysis collected 476 documents. The data collection for descriptive content analysis uses the PRISMA
stage, which consists of four stages: identification, screening, eligibility, and inclusion. Identification in this
data collection is based on the words “critical thinking” and “mathematics” in the article’s title, abstract, and
keyword. There is no filtering at this first stage. The strings used to search within=(article title, abstract,
keywords), search documents=(“critical thinking” and “mathematics”), published from=(all years)
to=(present), added to Scopus=(anytime). At this stage, 927 documents were produced.
Furthermore, screening is carried out, namely document screening based on inclusion/exclusion
criteria. The inclusion criteria are open access, the year of publication is in the 2012-2022 range, the research
area is a social science, the type of document is articles and conference papers, the source type is journals and
proceedings, and the language used is English. This was done to ensure the novelty of the research and a
Research trends in critical thinking skills in mathematics: a bibliometric study (Arif Hidayatul Khusna)
20  ISSN: 2252-8822

reasonable amount of research so that it could be examined comprehensively. This stage resulted in 133
articles. In the eligibility stage of the 133 articles, a review was carried out on the title and abstract sections.
The eligibility stage resulted in 54 articles by the research objectives based on the title and abstract. The last
stage is included, where filtering is based on reading the text as a whole. The steps included producing 49
documents.

Figure 1. Flowchart of PRISMA stages

2.2. Data analysis

Bibliometric analysis was performed using the Bibliometrix software package in the RStudio
program. To simplify the analysis, the bibliometric version of the software ready to be used without coding is
used, namely Biblioshiny. Bibliometrix was formerly a quantitative evaluation tool for publication and
citation data. Today, bibliometrics is used in almost all scientific fields to evaluate the growth, maturity,
leading authors, conceptual and intellectual maps, and trends of the scientific community [23]. Furthermore,
the data analysis method for descriptive content analysis is carried out by collecting similar data within the
framework of specific concepts and themes for further interpretation. The selected articles were grouped as a
publication classification developed by [24] in the data analysis process. The grouping of articles is based on
the information in the articles, such as the author's name, year of publication, research methods used, subject,
number of samples taken, research instruments, and data analysis techniques. Data related to the content of
the article was recapitulated using Microsoft Excel. Furthermore, after the data has been recorded based on
content, the data are grouped according to the criteria that have been set. Finally, before interpreting the data,
a percentage calculation is carried out related to the frequency of the data obtained.

3. RESULTS AND DISCUSSION

3.1. Bibliometric analysis
A total of 476 articles were generated in a search on the Scopus database with the keywords “critical
thinking” and “mathematics” in the field of social science. Then, an analysis was carried out on the 476
articles that had been found. Following are the results of bibliometric analysis using Biblioshiny software.

3.1.1. Annual scientific production distribution

The analysis using Biblioshiny shows that the number of research on critical thinking in
mathematics has increased significantly since 2016. Although in 2017 there was a downward trend, the
following year experienced a rapid increase. As presented in Figure 2, the number of publications on critical
thinking in mathematics is the most in 2021, which is 65 documents.

Int J Eval & Res Educ, Vol. 13, No. 1, February 2024: 18-30
Int J Eval & Res Educ ISSN: 2252-8822  21

Figure 2. Annual scientific production distribution

3.1.2. Country scientific production distribution

The distribution of country scientific production is shown in Figure 3, where the area in blue shows
the countries that produce critical thinking research in mathematics. The darker the blue, the more critical
thinking articles in mathematics produced by that country. The country scientific production data in more
detail is shown in Table 1. In Table 1, the country that published the most research related to critical thinking
in mathematics from 1985 to 2022 is the United States (US), with 273 publications. Indonesia occupies the
second rank of as many as 120 publications.

Figure 3. Country scientific production distribution

Table 1. Country scientific production distribution

Country 𝑓 % Country 𝑓 % Country 𝑓 %
US 273 31.45 Germany 6 0.69 Iraq 2 0.23
Indonesia 120 13.82 Kazakhstan 6 0.69 Lithuania 2 0.23
Turkey 53 6.11 Chile 5 0.58 Norway 2 0.23
Malaysia 50 5.76 India 5 0.58 Sweden 2 0.23
UK 43 4.95 Netherlands 5 0.58 Albania 1 0.12
China 35 4.03 Saudi Arabia 5 0.58 Armenia 1 0.12
Portugal 25 2.88 Slovenia 5 0.58 Belgium 1 0.12
Canada 24 2.76 Colombia 4 0.46 Czech Republic 1 0.12
South Africa 23 2.65 Finland 4 0.46 Denmark 1 0.12
Australia 21 2.42 Greece 3 0.35 Ghana 1 0.12
Spain 21 2.42 Ireland 3 0.35 Hungary 1 0.12
Brazil 15 1.73 Rwanda 3 0.35 Jordan 1 0.12
Peru 10 1.15 Serbia 3 0.35 Kenya 1 0.12
Mexico 9 1.04 Singapore 3 0.35 Kyrgyzstan 1 0.12
Thailand 9 1.04 Slovakia 3 0.35 Lesotho 1 0.12
Argentina 8 0.92 South Korea 3 0.35 New Zealand 1 0.12
Israel 8 0.92 Switzerland 3 0.35 Romania 1 0.12
Italy 8 0.92 Cyprus 2 0.23 Uruguay 1 0.12
Japan 8 0.92 Egypt 2 0.23 Zimbabwe 1 012
Philippines 7 0.81 France 2 0.23

Research trends in critical thinking skills in mathematics: a bibliometric study (Arif Hidayatul Khusna)
22  ISSN: 2252-8822

3.1.3. Most cited countries

Analysis using Biblioshiny shows that the US is the country that receives the most citations for
critical thinking research in mathematics. Figure 4 shows as many as 1671 citations for critical thinking
research in mathematics obtained from 1985-2022. This number is very significant compared to the number
of citations obtained by Turkey, which is 320.

Figure 4. Most cited countries distribution

3.1.4. Most cited sources

Based on Figure 5, the most cited source for research related to critical thinking in mathematics is
the international journal Educational Studies in Mathematics. This journal is a journal that presents new ideas
and research developments in the field of mathematics education. There were 104 citations have been
obtained in this journal related to critical thinking research in mathematics.

Figure 5. Most cited sources distribution

3.1.5. Most global cited documents

The results of the bibliometric analysis show that the document entitled “Education for life and
work: Developing transferable knowledge and skills in the 21st century” is the document that receives the
most citations from research related to critical thinking in mathematics. The article written by Pellegrino and
Hilton [25] obtained citations 421. This article contains 21st-century skills, one of which is the ability to
think critically, where efforts to improve these skills must synergize with learning. Figure 6 shows most
global cited documents distribution.

Int J Eval & Res Educ, Vol. 13, No. 1, February 2024: 18-30
Int J Eval & Res Educ ISSN: 2252-8822  23

Figure 6. Most global cited documents distribution

3.1.6. Co-occurrence network

The co-occurrence network of critical thinking research in mathematics is shown in Figure 7. This
data is obtained from the keywords used by the author. Figure 7 shows the relationship between the keywords
used by the author. Co-occurrence results show that there are two main clusters, namely critical thinking and
mathematics.

Figure 7. Co-occurrence network

3.1.7. Thematic evolution

Furthermore, Figure 8 shows a thematic evolution map obtained through bibliometrics that describes
the development of keywords over time in critical thinking research in mathematics. The map was created by
considering two stages, namely 1985-2018 and 2019-2022. For example, from 1985-2018, collaboration
learning, mathematics education, teacher education, engineering education, collaborative learning, and
creativity were separated from the critical thinking theme. However, in 2019-2022, there is a link between
critical thinking and collaborative learning, mathematics education, teacher education, engineering education,
and collaborative learning.

Research trends in critical thinking skills in mathematics: a bibliometric study (Arif Hidayatul Khusna)
24  ISSN: 2252-8822

Figure 8. Thematic evolution

3.2. Descriptive content analysis

The following presents a descriptive analysis of 49 selected documents based on the PRISMA stages
related to the theme of critical thinking in mathematics. These documents are documents that have been
published in the 2012-2022 period. The descriptive analysis includes subject area; research approach and
design; sampling methods, sample population, sample sizes; research instruments; and data analysis method.

3.2.1. Subject area

The identification results show that 28 areas are the focus of critical thinking research in
mathematics. These areas include problem-based learning; mathematical teachers; online learning;
questioning; science, technology, engineering and math (STEM); reflective thinking; contextual learning;
learning models; learning media; problem solving; subjects; mathematics problems; action, process, object,
and schema (APOS) theory; justification; creative thinking; mathematics competition; realistic mathematics
education (RME); algebra; pedagogy belief; argumentation; concept attainment model; academic
performance; adventure learning, 21 century; professional life; flipped classroom; and guided inquiry. The
spread of the research focus is more clearly seen in Figure 9.

7
6
5
4
3
2
1
0

Figure 9. Distribution of themes/focus of research

Int J Eval & Res Educ, Vol. 13, No. 1, February 2024: 18-30
Int J Eval & Res Educ ISSN: 2252-8822  25

3.2.2. Research approach and design

Researchers use various approaches and research designs in critical thinking research in
mathematics. There are three main approaches used by researchers, namely qualitative, quantitative, and
mixed methods approach. Of the three approaches, the quantitative approach is the most frequently used. Of
the 49 documents analyzed, 27 documents used a quantitative approach, 18 used a qualitative approach, and
the rest used a mixed method. Figure 10 shows a Tree Map of the approach and design of mathematical
thinking research carried out by researchers from 2012 to 2022.

Figure 10. Tree map research approach and design

The design of critical thinking research in mathematics is also diverse. Among them, the
quantitative approach and several research designs used are the relational survey model and quasi-
experimental. For the qualitative approach, the design used is document analysis, exploration, research and
development, experimental, case studies, and grounded theory. Meanwhile, the mixed methods design
approach is explanatory sequential and case studies. The distribution of research designs used by research
that has been fully published is shown in Table 2.

Table 2. Data research approach and design

Research approach Design Frequency %
Quantitative Relational Survey Model 6 22.22
Quasi-experimental 12 44.44
Not indicated 9 33.33
Total 27 100.00
Qualitative Document analysis 8 44.44
Exploratory 3 16.67
RnD 1 5.56
Experimental 1 5.56
Case study 1 5.56
Grounded theory 1 5.56
Not indicated 3 16.67
Total 18 100.00
Mixed Method Explanatory sequential 1 25.00
Case study 1 25.00
Not indicated 2 50.00
Total 4 100.00

3.2.3. Sampling methods, sample population, and sample sizes

The following identification is on the part of the participants or research subjects. This section
observes three things: the sampling methods, sample population, and sample sizes. From the results of the
analysis, students became the most researched population. The sampling method used was 38.78% random
sampling, 44.9% purposive sampling, and 2.04%, respectively, for snowball sampling, disproportionate
stratified sampling, and cluster sampling. In comparison, the sample size that researchers most often use is in
the range of 11 to 100 samples. Table 3 shows the results of the participant analysis.

Research trends in critical thinking skills in mathematics: a bibliometric study (Arif Hidayatul Khusna)
26  ISSN: 2252-8822

Table 3. Data, sampling methods, sample population, and sample sizes

Research subject Fr %
Sample Mathematics teachers 4 8.16
population Preservice teacher 8 16.33
Higher education 12 24.49
ONMIPA student 1 2.04
Upper secondary student 6 12.24
Vocational student 1 2.04
Lower secondary student 11 22.45
Primary student 5 10.20
Student (Not indicated) 1 2.04
Sampling Random sampling 19 38.77
methods Purposive sampling 22 44.89
Snowball 1 2.04
Disproportionate stratified sampling 1 2.04
Cluster sampling 1 2.04
Not indicated 5 10.20
Sample 10 and below 5 10.20
size Between 11-100 18 36.73
Between 101-300 11 22.44
301 and above 3 6.12
Not indicated 12 24.49

3.2.4. Research instruments

What needs to be identified in terms of methodology is a research instrument, namely a tool to
collect data. From the analysis results, eight types of research instruments are used to collect critical thinking
data in mathematics: tests, questionnaires, interviews, observations, documentation, scales, surveys, and
validation sheets. Among the eight instruments, the test is the most often used by researchers. 79.59% of
researchers used test instruments to collect research data. Table 4 shows the distribution of the number of
instruments used.

Table 4. Research instruments

Research instruments Frequency %
Test 39 79.59
Questionnaire 6 12.24
Interviews 13 26.53
Observation 7 14.29
Documentation 2 4.08
Scale 3 6.12
Survey 2 4.08
Validation Sheets 1 2.04

3.2.5. Data analysis method

Data analysis methods were also analyzed in this descriptive content analysis research. Table 5
shows that for quantitative research, data were analyzed using descriptive statistics, t-test, ANOVA,
multivariate analysis of variance (MANOVA), analysis of covariance (ANCOVA), correlation analysis, and
analysis regression. In qualitative research, the analytical methods used are descriptive, Q-Cohran, and the
NVivo application. Some use five systematic stages: data preparation, data collection, data reduction &
categorization (coding), displaying, and concluding. For the mixed method approach, ANOVA and
MANOVA were used for quantitative data, and factor maximum likelihood (FML)-principal factor analysis
(PFA) for qualitative data.
This study aims to review and analyze the literature of previous studies systematically to identify the
research gaps and the steps that must be taken for research on critical thinking in mathematics education in
the future. To achieve this goal, the researcher uses bibliometric analysis to reveal the year of publication,
country, and citations by source and country, as well as keywords often used in research on critical thinking
in mathematics. In addition, descriptive analysis using a credible database, namely Scopus, is used to identify
subjects, methodology, research design, sample groups, research instruments, and data analysis techniques in
studying critical thinking in mathematics.

Int J Eval & Res Educ, Vol. 13, No. 1, February 2024: 18-30
Int J Eval & Res Educ ISSN: 2252-8822  27

Table 5 Data analysis method

Data analysis method Fr
Quantitative Descriptive statistics 5
T-test 5
ANOVA 9
MANOVA 3
ANCOVA 2
Analysis correlation 4
Validity, reliability 2
Analysis Regression 2
Not Indicated 2
Qualitative Descriptive 6
Data preparation, collection, reduction & 3
categorization (coding), displaying and concluding
Vivo 1
Interpretative 1
Q-Cohran 1
Not Indicated 4
Mixed methods Quantitative
ANOVA 1
MANOVA 1
Qualitative
Factor maximum likelihood 1
Principal factor analysis 1
Not Indicated 2

Based on the bibliometric results in Figure 2, the first research that met the search category for the
theme of critical thinking in mathematics was published in 1985. After that, research on critical thinking in
mathematics experienced an increasing trend from year to year based on the number of publications observed
after 2006. In recent years the number of publications reached 70 documents. This shows that the interest in
critical thinking in mathematics is increasing and growing. Critical thinking skills are high-level cognitive
skills needed in every developing field [25]–[27]. On the other hand, mathematics is one of the fields that
supports the development of critical thinking skills [28]–[30]. The bibliometric results also show that 59
countries produce research on critical thinking in mathematics. In terms of the number of publications, the
US is the country with the highest number of publications from 1985 to 2022. Likewise, regarding the
number of citations, the US ranks the highest, 1671. This is to the research results of [31], [32], which state
that most critical thinking research is conducted in the US. Since the 1940s, essential steps towards the
practice of critical thinking have been taken in the US, such as the Delphi panel. In addition to being
reviewed from the US progress in developing critical thinking skills, researchers found indications that there
is a dearth of studies in the field for countries other than the US, even though critical thinking skills are an
ability that is demanded in national policy documents in many countries [33].
Furthermore, about the cited documents, documents written by Pellegrino and Hilton obtained
citations 421. This article contains 21st-century skills, one of which is the ability to think critically, where
efforts to improve these skills must synergize with learning. According to bibliometrics, the most widely
used reference sources for critical thinking research in mathematics are the journal Education Studies in
Mathematics and Computer & Education. These results provide information to researchers who will find
references to support theoretical studies of critical thinking research in mathematics. Regarding the frequency
of the most frequently used keywords, it can be seen in Figure 7 that the most used keywords are critical
thinking and mathematics. In comparison, the keywords that are rarely used are collaboration; Analysis,
Design, Development, Implementation, and Evaluation (ADDIE); argumentation; technology; reasoning; and
collaborative learning. Based on the link analysis of these general keywords, the main research areas in
critical thinking in mathematics can be identified. In addition, keywords can be used to identify evolutionary
themes. Through analyzing this evolutionary theme, information about the development of keywords can be
obtained from time to time in critical thinking research in mathematics. Therefore, keywords are essential in
determining research trends because they are informative for researchers [34], [35].
A systematic literature review is helpful for clearly identifying gaps in critical thinking research in
mathematics so that future researchers can determine the proper steps. One of them is to explore the focus
areas of critical thinking themes in mathematics. Through analysis of descriptive content limited to articles
published in 2012-2022, various themes in the research of critical thinking in mathematics are divided into 28
themes. Most critical thinking research in mathematics focuses on efforts to improve critical thinking skills
through specific methods or strategies. According to a studies [36], [37], the current focus of mathematics
learning is more demanding on conceptual understanding and the ability to justify rather than just applying
mathematical rules, so mathematics has a potential role in developing critical thinking skills [38].

Research trends in critical thinking skills in mathematics: a bibliometric study (Arif Hidayatul Khusna)
28  ISSN: 2252-8822

Given the demands of competence in the 21st century, future mathematics research must focus on
studies that integrate critical thinking skills with other competencies such as creative thinking,
communication, and collaboration. The research method is part of the research framework [39]. The data
presented in Table 2 on critical thinking research methodology in mathematics illustrates the structural basis
of the research. This will be a guide for researchers before conducting scientific studies. The results of the
descriptive content analysis revealed that almost 50% of the research was designed using a quantitative
approach. The basic principle of quantitative research is to present and evaluate the data collected by
descriptive and statistical means. In comparison, qualitative research occurs naturally and is interpreted
holistically so that research results are discussed more fully and in various ways [40]. So, it is recommended
for future research on critical thinking in mathematics to conduct further qualitative research that examines
concepts and research areas more deeply. In addition, mixed methods are also needed to handle qualitative
and quantitative data by changing different data sources from one to another to verify each other.
Descriptive content analysis on critical thinking in mathematics between 2012 and 2022 revealed
that most research was conducted with university students. Higher education is responsible for developing
critical thinking skills, which ultimately leads to higher-order thinking [41]–[43]. Table 3 shows the subjects
other than students are mathematics teachers, prospective mathematics teachers, elementary, junior high, high
school, and vocational students. Critical thinking skills are important for students attending any educational
program [44], [45]. In addition, the teacher is an important factor in teaching critical thinking skills [46]–
[49]. So that research on critical thinking in mathematics can be carried out on teachers, prospective teachers,
and students at every level.
The results of the descriptive content analysis on the sampling method show that the methods often
used are random and purposive. Regarding sample size, most researchers work with small sample sizes. In
the articles analyzed, a sample size of 11–100 participants were the most frequently used measure. For future
research on critical thinking in mathematics, it is suggested that researchers conduct research with a larger
sample to obtain more generalizable results. In terms of research instruments, the results of descriptive
content analysis show that the test is the most frequently used data collection tool in critical thinking research
in mathematics. This is a consequence of developing critical thinking skills evaluation tools as test
instruments in essays and rubrics [50]. So, it should be noted by researchers that the test is a reliable tool to
obtain research data about critical thinking skills in mathematics.

4. CONCLUSION
The bibliometric analysis and descriptive content analysis results present the existing literature's
overall systematics and contribute to identifying potential gaps in critical thinking research in mathematics.
In addition, this research provides information to readers about sources and documents that need to be
studied as the basis for critical thinking research in mathematics. The results also reveal a need for critical
thinking research in mathematics in many other countries. The USA can be used as a reference in this study.
Research trends can also be used to determine the theme of critical thinking research in mathematics. Critical
thinking research frameworks in mathematics, such as focus areas of study and methodologies in research,
have been generated in this study. The database used in this study is limited to the Scopus database, so the
resulting critical thinking research framework in mathematics is also limited. For further research, the
development of the findings of this research can be done using keyword string searches and different
alternative databases. The results of this study are likely helpful for other researchers who will examine
critical thinking skills in mathematics, namely as a research guide that they will develop. In addition, the
existence of several research gaps can motivate other researchers to fill these gaps to contribute to improving
the quality of critical thinking in mathematics.

ACKNOWLEDGEMENTS
This work was supported by Balai Pembiayaan Pendidikan Tinggi (BPPT) and Lembaga Pengelola
Dana Pendidikan (LPDP); BPI ID Number: 202101122210.

REFERENCES
[1] X. Song, “‘Critical Thinking’ and Pedagogical Implications for Higher Education,” East Asia, vol. 33, no. 1, pp. 25–40, Mar.
2016, doi: 10.1007/s12140-015-9250-6.
[2] E. G. Bugg and J. Dewey, “How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process,” The
American Journal of Psychology, vol. 46, no. 3, p. 528, Jul. 1934, doi: 10.2307/1415632.
[3] C. L. Ford and L. D. Yore, “Toward Convergence of Critical Thinking, Metacognition, and Reflection: Illustrations from Natural
and Social Sciences, Teacher Education, and Classroom Practice,” in Contemporary Trends and Issues in Science Education,

Int J Eval & Res Educ, Vol. 13, No. 1, February 2024: 18-30
Int J Eval & Res Educ ISSN: 2252-8822  29

Dordrecht: Springer, 2012, pp. 251–271. doi: 10.1007/978-94-007-2132-6_11.

[4] D. A. Bensley and R. A. Spero, “Improving critical thinking skills and metacognitive monitoring through direct infusion,”
Thinking Skills and Creativity, vol. 12, pp. 55–68, Jun. 2014, doi: 10.1016/j.tsc.2014.02.001.
[5] H. A. Butler, C. Pentoney, and M. P. Bong, “Predicting real-world outcomes: Critical thinking ability is a better predictor of life
decisions than intelligence,” Thinking Skills and Creativity, vol. 25, pp. 38–46, Sep. 2017, doi: 10.1016/j.tsc.2017.06.005.
[6] R. H. Ennis, “Critical Thinking Across the Curriculum: A Vision,” Topoi, vol. 37, no. 1, pp. 165–184, Mar. 2018, doi:
10.1007/s11245-016-9401-4.
[7] M. Boran and F. Karakuş, “The Mediator Role of Critical Thinking Disposition in the Relationship between Perceived Problem-
Solving Skills and Metacognitive Awareness of Gifted and Talented Students,” Participatory Educational Research, vol. 9, no. 1,
pp. 61–72, Jan. 2022, doi: 10.17275/per.22.4.9.1.
[8] S. A. Young, A. R. Newton, S. R. Fowler, and J. Park, “Critical thinking activities in Florida undergraduate biology classes
improves comprehension of climate change,” Journal of Biological Education, vol. 57, no. 1, pp. 184–195, Jan. 2023, doi:
10.1080/00219266.2021.1877785.
[9] C. Cortázar et al., “Promoting critical thinking in an online, project-based course,” Computers in Human Behavior, vol. 119, Jun.
2021, doi: 10.1016/j.chb.2021.106705.
[10] D. Kuhn, “Thinking together and alone,” Educational Researcher, vol. 44, no. 1, p. 46, 2015, doi: 10.3102/0013189X15569530.
[11] K. Y. L. Ku, “Assessing students’ critical thinking performance: Urging for measurements using multi-response format,”
Thinking Skills and Creativity, vol. 4, no. 1, pp. 70–76, Apr. 2009, doi: 10.1016/j.tsc.2009.02.001.
[12] S. Kuntze, E. Aizikovitsh-Udi, and D. Clarke, “Hybrid task design: connecting learning opportunities related to critical thinking
and statistical thinking,” ZDM, vol. 49, no. 6, pp. 923–935, Nov. 2017, doi: 10.1007/s11858-017-0874-4.
[13] E. Aizikovitsh-Udi and D. Cheng, “Developing Critical Thinking Skills from Dispositions to Abilities: Mathematics Education
from Early Childhood to High School,” Creative Education, vol. 06, no. 04, pp. 455–462, 2015, doi: 10.4236/ce.2015.64045.
[14] F. K. Lester, “Thoughts About Research on Mathematical Problem- Solving Instruction,” The Mathematics Enthusiast, vol. 10,
no. 1–2, pp. 245–278, Jan. 2013, doi: 10.54870/1551-3440.1267.
[15] A. Fischer, S. Greiff, and J. Funke, “The Process of Solving Complex Problems,” The Journal of Problem Solving, vol. 4, no. 1,
Feb. 2012, doi: 10.7771/1932-6246.1118.
[16] U. Turan, Y. Fidan, and C. Yıldıran, “Critical Thinking as a Qualified Decision Making Tool,” Journal of History Culture and Art
Research, vol. 8, no. 4, p. 1, Dec. 2019, doi: 10.7596/taksad.v8i4.2316.
[17] A. Aktoprak and C. Hursen, “A Bibliometric and Content Analysis of Critical Thinking In Primary Education,” Thinking Skills
and Creativity, vol. 44, p. 101029, Jun. 2022, doi: 10.1016/j.tsc.2022.101029.
[18] N. Donthu, S. Kumar, D. Mukherjee, N. Pandey, and W. M. Lim, “How to conduct a bibliometric analysis: An overview and
guidelines,” Journal of Business Research, vol. 133, pp. 285–296, Sep. 2021, doi: 10.1016/j.jbusres.2021.04.070.
[19] P. Mayr et al., “Introduction to the special issue on bibliometric-enhanced information retrieval and natural language processing
for digital libraries (BIRNDL),” International Journal on Digital Libraries, vol. 19, no. 2–3, pp. 107–111, Sep. 2018, doi:
10.1007/s00799-017-0230-x.
[20] I. Zupic and T. Čater, “Bibliometric Methods in Management and Organization,” Organizational Research Methods, vol. 18,
no. 3, pp. 429–472, Jul. 2015, doi: 10.1177/1094428114562629.
[21] S. Dinçer, “Content Analysis in for Educational Science Research: Meta-Analysis, Meta-Synthesis, and Descriptive Content
Analysis,” Bartın University Journal of Faculty of Education, vol. 7, no. 1, pp. 176–190, 2018, doi: 10.14686/buefad.363159.
[22] A. Assarroudi, F. H. Nabavi, M. R. Armat, A. Ebadi, and M. Vaismoradi, “Directed qualitative content analysis: the description
and elaboration of its underpinning methods and data analysis process,” Journal of Research in Nursing, vol. 23, no. 1, pp. 42–55,
Feb. 2018, doi: 10.1177/1744987117741667.
[23] M. Aria and C. Cuccurullo, “Bibliometrix: An R-tool for comprehensive science mapping analysis,” Journal of Informetrics,
vol. 11, no. 4, pp. 959–975, Nov. 2017, doi: 10.1016/j.joi.2017.08.007.
[24] O. Bırgın and E. S. Peker, “Thematic content analysis of studies on number sense in Turkey,” Hacettepe Egitim Dergisi, vol. 36,
no. 3, 2021, doi: 10.16986/HUJE.2020062666.
[25] J. W. Pellegrino and M. L. Hilton, Education for life and work: Developing transferable knowledge and skills in the 21st century.
Washington, DC: The National Academies Press, 2013. doi: 10.17226/13398.
[26] E. v. Laar, A. J.A.M. van Deursen, J. A.G.M. van Dijk, and J. de Haan, “The relation between 21st-century skills and digital
skills: A systematic literature review,” Computers in Human Behavior, vol. 72, 2017, doi: 10.1016/j.chb.2017.03.010.
[27] V. L. Byrd and P. A. Asunda, “Using evidence based practices and learning to enhance critical thinking skills in students through
data visualization,” in 2020 IEEE Frontiers in Education Conference (FIE), 2020, pp. 1–9. doi: 10.1109/FIE44824.2020.9274157.
[28] M. R. Ariza, A. Q. Armenteros, and A. E. Castro, “Promoting critical thinking through mathematics and science teacher
education: the case of argumentation and graphs interpretation about climate change,” European Journal of Teacher Education,
2021, doi: 10.1080/02619768.2021.1961736.
[29] G. Aksu and N. Koruklu, “Determination the Effects of Vocational High School Students’ Logical and Critical Thinking Skills on
Mathematics Success,” Eurasian Journal of Educational Research, vol. 15, no. 59, Apr. 2015, doi: 10.14689/ejer.2015.59.11.
[30] S. Syafril, N. R. Aini, Netriwati, A. Pahrudin, N. E. Yaumas, and Engkizar, “Spirit of Mathematics Critical Thinking Skills
(CTS),” Journal of Physics: Conference Series, vol. 1467, no. 1, p. 012069, Feb. 2020, doi: 10.1088/1742-6596/1467/1/012069.
[31] D. Islek and C. Hursen, “Evaluation of Critical Thinking Studies in Terms of Content Analysis,” Social and Behavioral Sciences,
vol. 131, pp. 290–299, 2014, doi: 10.1016/j.sbspro.2014.04.119.
[32] K. Raghunathan, L. McKenna, and M. Peddle, “Use of academic electronic medical records in nurse education: A scoping
review,” Nurse Education Today, vol. 101, p. 104889, Jun. 2021, doi: 10.1016/j.nedt.2021.104889.
[33] D. N. Qureshi, “Critical Thinking Skills Development: Secondary School Science Teachers’ Perceptions and Practices,” SJESR,
vol. 4, no. 2, 2021, doi: 10.36902/sjesr-vol4-iss2-2021(21-30).
[34] S. R. Kolle, “Global research on information literacy: a bibliometric analysis from 2005 to 2014,” The Electronic Library, vol. 35,
no. 2, pp. 283–298, Apr. 2017, doi: 10.1108/EL-08-2015-0160.
[35] W. Trinarningsih, A. R. Anugerah, and P. S. Muttaqin, “Visualizing and mapping two decades of literature on board of directors
research: a bibliometric analysis from 2000 to 2021,” Cogent Business & Management, vol. 8, no. 1, Jan. 2021, doi:
10.1080/23311975.2021.1994104.
[36] A. R. As’ari, A. Mahmudi, and E. Nuerlaelah, “Our prospective mathematic teachers are not critical thinkers yet,” Journal on
Mathematics Education, vol. 8, no. 2, Feb. 2017, doi: 10.22342/jme.8.2.3961.145-156.
[37] N. Y. Wong, F. Marton, K. M. Wong, and C. C. Lam, “The lived space of mathematics learning,” Journal of Mathematical
Behavior, vol. 21, no. 1, pp. 25–47, 2002, doi: 10.1016/S0732-3123(02)00101-3.

Research trends in critical thinking skills in mathematics: a bibliometric study (Arif Hidayatul Khusna)
30  ISSN: 2252-8822

[38] V. Linda, Y. Xie, and M. Han, “Designing visually interactive learning modules to promote students’ critical thinking in
mathematics,” in Proceedings - Frontiers in Education Conference, FIE, 2019, doi: 10.1109/FIE.2018.8658568.
[39] C. S. Collins and C. M. Stockton, “The Central Role of Theory in Qualitative Research,” International Journal of Qualitative
Methods, vol. 17, no. 1, p. 160940691879747, Dec. 2018, doi: 10.1177/1609406918797475.
[40] M. del Rio Carral and E. Tseliou, “Mapping Qualitative Research in Psychology across Europe: Contemporary Trends,”
Qualitative Research in Psychology, vol. 16, no. 3, pp. 325–335, Jul. 2019, doi: 10.1080/14780887.2019.1605276.
[41] K. Changwong, A. Sukkamart, and B. Sisan, “Critical thinking skill development: Analysis of a new learning management model
for Thai high schools,” Journal of International Studies, vol. 11, no. 2, pp. 37–48, 2018, doi: 10.14254/2071-8330.2018/11-2/3.
[42] E. P. Douglas, “Defining and Measuring Critical Thinking in Engineering,” Procedia - Social and Behavioral Sciences, vol. 56,
pp. 153–159, Oct. 2012, doi: 10.1016/j.sbspro.2012.09.642.
[43] E. P. Douglas, “Work in progress: What is critical thinking?” in 2012 Frontiers in Education Conference Proceedings, Oct. 2012,
pp. 1–2. doi: 10.1109/FIE.2012.6462337.
[44] D. L. Zeidler, N. G. Lederman, and S. C. Taylor, “Fallacies and student discourse: Conceptualizing the role of critical thinking in
science education,” Science Education, vol. 76, no. 4, pp. 437–450, Jul. 1992, doi: 10.1002/sce.3730760407.
[45] M. Kosturková, J. Ferencová, and V. Šutáková, “Critical thinking as an important part of the curriculum reform in Slovakia:
Examining the phenomenon in the Slovak journals,” Orbis Scholae, vol. 12, no. 1, pp. 1–24, 2018.
[46] C. Thongnuypram and S. Sopheerak, “Factor Influencing the Critical Thinking of Teacher Students Studying at the Faculty of
Education in Suratthani Rajabhat University,” Procedia - Social and Behavioral Sciences, vol. 103, pp. 386–391, Nov. 2013, doi:
10.1016/j.sbspro.2013.10.349.
[47] X. Ma, Y. Zhang, and X. Luo, “Students’ and teachers’ critical thinking in science education: are they related to each other and
with physics achievement?” Research in Science & Technological Education, vol. 41, no. 2, pp. 734–758, Apr. 2023, doi:
10.1080/02635143.2021.1944078.
[48] F. Erdoğan, “The relationship between prospective middle school mathematics teachers’ critical thinking skills and reflective
thinking skills,” Participatory Educational Research, vol. 7, no. 1, pp. 220–241, Feb. 2020, doi: 10.17275/per.20.13.7.1.
[49] H. Fitriani, M. Asy’ari, S. Zubaidah, and S. Mahanal, “Exploring the Prospective Teachers’ Critical Thinking and Critical
Analysis Skills,” Jurnal Pendidikan IPA Indonesia, vol. 8, no. 3, Sep. 2019, doi: 10.15294/jpii.v8i3.19434.
[50] N. J. S. Brown, P. P. Afflerbach, and R. G. Croninger, “Assessment of Critical-Analytic Thinking,” Educational Psychology
Review, vol. 26, no. 4, pp. 543–560, Dec. 2014, doi: 10.1007/s10648-014-9280-4.

BIOGRAPHIES OF AUTHORS

Arif Hidayatul Khusna is an Assistant Professor and Teacher Educator at

University of Muhammadiyah Malang (UMM) since 2021. She was appointed lecturer at the
university in 2017 and is now a doctoral candidate in mathematics education at the State
University of Surabaya. Arif’s research interests lie in mathematics education, higher
education, 21st-century teaching and learning, and classroom research. She can be contacted
via email: [email protected].

Tatag Yuli Eko Siswono is a professor of mathematics education at State

University of Surabaya. His doctoral degree was obtained in 2007 at the State University of
Surabaya. His research focuses on mathematics education, especially problem-solving,
problem-posing, creative, and critical thinking. He can be contacted at email:
[email protected].

Pradnyo Wijayanti received her doctorate in mathematics education from the

State University of Surabaya. She has more than 10 years of experience as an Academic at
Surabaya State University, where she is currently Assistant Professor majoring in
mathematics. Her current research interests include the learning and development of students’
mathematics at various levels and fields of education. She can be contacted via email:
[email protected].

Int J Eval & Res Educ, Vol. 13, No. 1, February 2024: 18-30