Journal of English Language Teaching Volume 8 No.

1
Journal of English Language Teaching
ISSN 2302-3198
Published by English Language Teaching Study Program of
FBS Universitas Negeri Padang
available at http://ejournal.unp.ac.id/index.php/jelt

THE ANALYSIS OF READING COMPREHENSION QUESTIONS IN ENGLISH

TEXTBOOK BY USING HIGHER ORDER THINKING SKILL AT GRADE X OF
SMAN 2 PADANG

Siti Nurul Hapizah Damanik1, Yett Zainil2

English Department
Faculty of Languages and Arts
State University of Padang
email: [email protected]

Abstract

This research was a descriptive research that aimed to identify the

reading comprehension questions in English textbook by using Higher Order
Thinking Skill. This study also aimed to analyze and evaluate reading
comprehension questions in English textbook. The data used in this research is
reading comprehension questions of English textbook. The English textbook were
from grade X of SMAN 2 Padang. This study used instrument in the form of
indicators from HOTS criteria and Bloom’s criteria. From the result of the reading
comprehension question in English textbook by using Higher Order Thinking
Skill at grade X of SMAN 2 Padang, it showed that the reading comprehension
questions had used the HOTS criteria. As a result of the analysis in reading
comprehension questions, 36 questions ( 9.7%) Higher Order Thinking Skill
questions were found in the reading comprehension questions. The result of this
research also found that mostly-used HOTS criteria in the reading comprehension
questions is analyzing category. The knowledge dimensions found in the
questions are factual knowledge, conceptual knowledge, procedural knowledge,
and metacognitive knowledge.

Key words: Reading comprehension questions in English textbook, Higher Order

Thinking Skill, Bloom’s Taxonomy

A. INTRODUCTION
Nowadays, there are so many learning resources that can be used to
help the students and teachers in the teaching and learning process. One of the
ways to help the students and teachers are a textbook. Textbook is a tool of
1
English ELTSP of English Department of FBS Universitas Negeri Padang graduated on....
2
Lecturer of English Department of FBS Universitas Negeri Padang

© FBS Universitas Negeri Padang

 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

teaching and learning process. According to Cunningsworth (1995) in Armina

(2018:1) states that using a textbook aims to prepare a set of practices based on
the teaching materials. Besides, a textbook also aims to help the students in
achieving their goals and their needs.
The Ministry of Education and Culture in Indonesia has been
implemented the curriculum 2013. The curriculum 2013 is designed to focus on
the student-centered rather than teacher-centered. The implementation of the
curriculum 2013 fell into the textbook. The reading comprehension questions in
the English textbook will prove the implementation of the curriculum 2013 by
using higher order thinking skill.
According to Vijayaratnam (2012) higher order thinking skill has
become a major theme in education. It is considered to the information of the
global era now, especially in the English language. In the learning activities
specifically in the English language, the students are encouraged to think
critically. They have to think more and to solve their own problems, whiles the
teachers are demanding to facilitate the students by providing higher order
thinking skill with reading comprehension questions.
There are some activities learned in the textbook Bahasa Inggris Siswa
Kelas X SMA/MA, SMK/MAK involves warmer, vocabulary builder,
pronunciation practice, reading, vocabulary exercises, text structure, grammar
review, speaking, writing, reflection, and further activities. But, the researcher
only focuses on the reading activity because the researcher wants to analyze and
evaluate the reading comprehension questions found in the textbook. One of the
ways to encourage learners to prove critical thinking is by using reading
comprehension questions.
According to Bloom et al.(1956) defines six different levels in the
cognitive domain. The Bloom’s mentioned cognitive domain is divided into two
parts, there is a lower order thinking skill (LOTS) includes remembering,
understanding, and applying, and higher order thinking skill (HOTS) includes
analyzing, evaluating and creating.
In the lower order thinking skill, remembering is the first stage of the
thinking process. It can occur when the students describe, make a list, tell and
name a sect of topics. In the understanding level shows that the students
understand what they have read. The words used in understanding such as retell,
infer, interpret, explain, predict, and outline knowledge. Applying is the stage of
making use of knowledge in the new situation.
According to Heong, et al (2011) higher order thinking is using the
thinking widely to find new a challenge. Higher order thinking also demands
someone to apply new information and knowledge manipulates the information to
reach the possibility of answer in new situations. Thus, someone is required to be
able to find the right answer in a new situation both in terms of learning or outside
learning.
The dimensions of the thinking process in Bloom's are perfected by
Anderson & Krathwohl (2001), they provide a useful framework for identifying

250
 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

and preparing instructional objectives, instructional activities, and assessment

methods. The revised taxonomy has cognitive process categories, increasing in
complexity, and four knowledge categories. The cognitive levels of thinking can
be seen as the following table below:

Creating

Evaluating

Analyzing

Applying

Understanding

Remembering

Anderson and Krathwohl’s (2001) revised Taxonomy of Bloom have

two dimensions perspective for higher order thinking and classification of its
operational verbs can be described as follows:

Table 1

The Cognitive Process Dimension

The Knowledge
Dimension C4 C5 C6
analyze Evaluate create
Making structure, comparing, Joining
Factual classifying correlating
Knowledge
explain, analyze examine, interpret planning
Conceptual
Knowledge
distinguish conclude, resume arrange,
Procedural formulate
Knowledge
create, find make, assess realization
Metacognitive
Knowledge

According to Anderson and Krathwohl (2001) there are four types of

knowledge dimension:

251
 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

1) Factual Knowledge
Factual knowledge is the basic level of study and basic to specific
disciplines. This dimension refers to essential facts, terminology, details or
elements students must know or be familiar with in order to understand a
discipline or to solve a problem. The factual knowledge includes symbols,
signs, the name of person and history. Furthermore, the conceptual
knowledge refers to the interrelationship among basic elements within a
larger structure and enables them to function together.
2) Conceptual Knowledge
Conceptual knowledge is knowledge of classifications, principles,
generalizations, theories, models, or structures pertinent to a particular
disciplinary area. Conceptual knowledge includes knowledge of categories
and classification and the relationship between them. It includes schemas,
mental models, or implicit or explicit theories represent the knowledge an
individual has about how a particular subject matter is organized, the
different, and the function.
3) Procedural Knowledge
Procedural knowledge refers to the information or knowledge that
helps students to do something specific to a discipline, subject, or area of
study. It also refers to methods of inquiry, very specific or finite skills,
algorithms, techniques, and particular methodologies.
4) Metacognitive Knowledge
Metacognitive knowledge is the awareness of one’s own cognition
and particular cognitive processes. It is strategic or reflective knowledge
about how to go about solving problems, cognitive tasks, to include
contextual and conditional knowledge and knowledge of self.
As the process of dimension of thought that has completed by Anderson
and Krathwohl (2001) the higher order thinking skill questions is really
recommended for the use in various forms of the assessment in the class. To
inspire the teacher to compile HOTS questions at the education unit level, the
following are described the characteristics of HOTS questions
1. Measure the capability of the high-level thinking skills
The Australian Council for Educational Research (ACER) states that
the Higher Order Thinking Skills is the process of analyzing, reflecting,
reasoning, implementing the concept in different situations, arranging and
creating. Besides, the HOTS questions include into the capability of problem
solving, critical thinking, creative thinking, reasoning, and making decision.
The capability of HOTS questions. The capability of the high-level thinking
skills is one of the important competencies in the modern world, so it must be
possessed by every student. The creativity to solve HOTS problems includes:

252
 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

a. The capability to answer the unfamiliar questions.

b. The capability of evaluating the strategy that is used to solve the
problems from a variety of different points of view.
c. The capability of finding the new solving method that is different
from the previous method.
2. Contextual Based Problem
HOTS questions are assessments based on the real situations in
everyday life. The HOTS questions are expected to be able to apply the
concept of classroom learning to solve problems. In relations, the learners
also able to relate, interpret, apply, integret the problem in the real context
life. There are the characteristics of the contextual based problem namely as
REACT:
a. Relating which connect to the real life situation.
b. Experiencing which is emphasized to exploration, discovery and
creation.
c. Applying, which demand the student to implement the knowledge
they get in school into the real life problems.
d. Communicating, which demand the capability of the students to be
able to correspond the problem context conclusion.
e. Transferring, which demand the students’ capability to transform
the concepts of knowledge in class into new situation or context.
3. Using the Various Kinds of Questions
1. Multiple choice
Generally, HOTS questions use stimulus that comes from the real
situation. This multiple choice question consists of stem and options.
These options contain the key answer which is the right answer and the
distractors. These distractors of course are incorrect answers, however it
will be a trap for the students if they do not master the material well. The
expected answer or the key answer generally is not stated explicitly in
stimulus or in the reading. Thus, the students are demanded to find the
answer related to the stimulus using the concepts of the knowledge, logic
and reasoning. Then, the correct answer will get 1 point, and the incorrect
answer will get 0.
2. Complex-multiple choice (True/ False or Yes/No)
This kind of question aims to know the students’ understanding
towards the problems comprehensively between the statements of one
another. The questions of complex-multiple choice also contain stimulus
which come from the contextual situation. The students are provided
some statements related to the stimulus/reading, then the students are
demanded to choose true/false or yes/no. The statements given are related
to one another. The arrangement of true statements and the wrong
statements are put randomly and non-systematic following certain
pattern. A well pattern arrangement will lead the students to choose the

253
 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

right answer. Thus, if the students answer correctly for all the statements
given, they will get 1 point, then if there is a mistake in a statement, they
will get 0.
3. Short answer
The questions only need short answer like, word and phrase beyond
the statement. The characteristics of short answer questions are:
a. Using the direct statement or command
b. Obvious question and command
c. The length of all answers in all question is expected relatively the
same
d. Avoid using the text book’s words, sentence or phrase because it
will lead the students to only remembering what is written in the
text book.

In relation to this, there are some researches that have been conducted
by some researchers who take the same topic about HOTS. The previous
study was coming from Siti, R.Y. (2018) did a research about higher-order
thinking skills analysis of students in solving higher order thinking skill
question in higher education. The preliminary study conducted on 100
students in January 2018 found that a majority (80%) of the students
answered that they still did not know the whole concept of HOTS; most
(95%) students do not have references related to HOTS; a majority (70%) of
students answer lecture assignments given by the lecturers are only paper-
based so students only take from internet sources without prior analysis (copy
paste from the Internet); and a majority (78%) students want a learning source
for the form of Instructional Evaluation courses that taught about HOTS.
Based on the results of the study, it can be seen that HOTS that is owned by
students seems to be less than optimal because in their daily routines just like
in a lecture just sit, listen, and note. Analysis, evaluation and creation
activities are rarely trained in students.
Based on the problems above, the researcher will analyze and evaluate
reading comprehension questions found in the textbook “Bahasa Inggris Siwa
Kelas X SMA/MA, SMK/MAK”. To achive that, the researcher has built a
check list based on Bloom's taxonomy and HOTS criteria as a criterion for
the evaluation process

B. RESEARCH METHOD

This research was a descriptive research. This research looked for the
reading comprehension questions found in English textbook. In this research,
the data was reading comprehension questions in English textbook at grade X
of SMAN 2 Padang. The English textbook that researcher used is Bahasa

254
 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

Inggris Kelas X Siswa SMA/MA, SMK/MAK of SMAN 2 Padang. Which

cover two semesters from the Ministry of education and culture based on the
curriculum 2013.The indicators guided the researcher in evaluating the
questions. Each tests were evaluated by the researcher by following the
criteria that proposed by Bloom’s Taxonomy and HOTS for formulating the
principles of questions based on criteria or indicators. The criteria used are
Analyzing, Evaluating, and Creating, The researcher counted all the evidence
containing in the questions after filled in the criteria in the evaluation format.
For example, answering the criterion ‘Analyzing’ category, the researcher
counted all contracted forms containing in the questions and then found the
percentage of it. The formula of the percentage is:

n
P= 100%
N
P = the percentage
n = Number of question based on HOTS criterion found in the questions
N = the total number of questions

After the percentage of HOTS questions were found, the

researcher described each criterion along with examples of questions that
meet those criteria. For example, describing the questions of Analyzing
criteria, the questions explained one by one along with the discussion based
on the theory. The researcher also classified the HOTS questions into
dimensions of knowledge by reviewing the knowledge dimensions based on
Bloom’s theory then analyzed each HOTS question based on the dimension.
For example, describing the dimension of Conceptual Knowledge, the
questions were classified based on the dimension into one part. The whole
analysis was discussed based on the theory of HOTS and some previous study
related to this research.

C. RESULT AND DISCUSSION

1. Research Finding

Based on the table above, it can be seen that the categories of High Order
Thinking Skills are found in the reading comprehension question in english
textbookThere are 3 categories of HOTS questions in the tests; Analyzing,
Evaluating, Creating. Here are the description of each question:

a) Reading comprehension questions absed on HOTS

Table 5 : The total of HOTS in reading comprehension
questions in scoring rubric

255
 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

No Level of Indicators Total of Percentage

Thinking questions (%)

1 Analyzing Distinguishing Facts from 6 1.6 %

hypothesis and Recognizing the
Unstated Assumption

Correlating the interrelationship 10 2.7 %

among the ideas in the passage

Inferring the Author’s purpose 4 1.1 %

and Point of View

2 Evaluating Justifying/ Critiquing the 10 2.7%

information

3 Creating Producing Idea to Solve Problem 6 1.6 %

TOTAL 36 9.7 %

Based on the description above, In analyzing category, it consist of

distinguishing facts from hypothesis and recognizing the unstated assumption
are 6 questions with the percentages (1.6 %), correlating the interrelationship
among the ideas in the passage is 10 questions with the percentages of (2.7
%), inferring the author’s purpose and point of view is 4 questions with the
percentages of (1.1 %). In evaluating category, there are 10 questions with the
percentages of (2.7%) include into justifying or critiquing the information. In
the creating category, there are 6 questions with the percentages of (1.6 %)
include into producing idea to solve problem. So, the totally of HOTS questions
are 36 questions with the percentage 9.7 % were found in the reading
comprehension questions in English textbook.

b) The dimension knowledge of HOTS criteria in the reading

comprehension questions
There are four kinds of knowledge dimensions, there are Factual
Knowledge, Conceptual Knowledge, Procedural Knowledge, and
Metacognitive Knowledge. The results showed that . There were 19 questions
with the percentages 3,8 % for the factual knowledge, 23 questions with the
percentage 4.6 % for the conceptual knowledge, 1 question with the
percentages 2.0 % for the procedural knowledge, and 7 questions with the
percentages 1.4 % for the metacognitive knowledge. Thus, the questions that
mostly used here is the conceptual knowledge with 23 (4.6%). Here are the
percentages of knowledge dimensions:

256
 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

Table 6: The Total of Knowledge Dimensions

No Categories Total Percentages

1. Factual Knowledge 19 3,8 %
2. Conceptual Knowledge 23 4.6%
3. Procedural Knowledge 1 2.0%
4. Metacognitive 7 1.4%
Knowledge
Total 50 11.8 %

D. DISCUSSION

The findings that have been obtained from the analysis of reading
comprehension questions in English textbook using Higher Order Thinking Skill
categories show that there are 36 questions (9,7 %)found in English textbook.
Based on the proportion of HOTS question, (30%), the HOTS questions found in
the reading comprehension questions in English textbook are almost reach the
proportion but still under the criteria of good proportion. From 172 questions,
there were 36 questions belong to HOTS-based question.

Based on the knowledge dimension, the HOTS questions in these tests

belong to Factual Knowledge, Conceptual Knowledge, and Metacognitive
Knowledge. From 172 questions, it was found 3,8 % for the factual knowledge, 23
questions with the percentage 4.6 % for the conceptual knowledge, 1 question
with the percentages 2.0 % for the procedural knowledge, and 7 questions with the
percentages 1.4 % for the metacognitive knowledge. Thus, the questions that
mostly used here is the conceptual knowledge with 23 (4.6%).
.
E. CONCLUSION AND SUGGESTIONS

This research focused on the analysis of reading comprehension questions

in English textbook by using higher order thinking skill at grade of SMAN 2
Padang. The purposes were to identify the reading comprehension questions in
English textbook based on the HOTS category and knowledge dimensions. The
data were coming from the reading comprehension questions in English textbook
at grade X of SMAN 2 Padang. In this research, the data used are the English
textbook which cover two semester from two books that published by the Ministry
of Education and Culture with curriculum 2013.
Based on the reading comprehension questions in English textbook had 36
question HOTS questions. Morever, there are mostly used analyzing catgory
found in the reading comprehension questions in English textbook. Since the
analyzing categories are mostly used, it can be said that the analyzing category
have a high percentages rather than another categories of HOTS. Moreover, it also
have the sense of the measuring HOTS, there are also four dimensions knowledge

257
 HOTS questions in the reading comprehnsion questions – Hafizah1, Yetti2

involves: factual knowledge, conceptual knowledge, procedural knowledge, and

metacognitive knowledge that used in this research.
The role of higher order thinking skill in the reading comprehension
questions in English textbook are very important in order to increase and to prove
critical thinking that applied by the curriculum 2013. In other words, the questions
are not only measuring the capability in answering reading comprehension
questions, but also the capability to analyze, evaluate, and create new ideas and
knowledge. Thus, it builts the critical thinking through the questions.
Regardless of the findings of this thesis, there are some suggestions
offered by the researcher to the following parties. First, the reading
comprehension questions in English textbook should more have the characteristics
of HOTS questions in order to improve the capability of logic thinking or critical
thinking that prove the curriculum 2013 appliying in the English textbook.
Second, the next researcher is expected that the result of this research can be used
for the next researcher in order to improve the higher order thinking skill that has
a similar research.

BIBLIOGRAPHY

Anderson, L.W. & Krathwohl, D. R. (Eds.) (2001). A taxonomy for Learning,

teaching and assessing: A revision of Bloom’s taxonomy of educational
objectives. New York: Addison Wesley Longman.

Siti, R.Y. (2018). Analysis of Students in Solving Higher Order Thinking Skill
Question in Higher Education. (Thesis). State University of Jakarta. 32(2).

Tomei. (2005). Evaluating the Higher Order Thinking Skills in Reading of

English for Palestine Grade Eight. Asian Journal of Education and e-
Learning. 1(1). 2321 – 2454.

Richards. (2001). Textbook Selection, Evaluation and Adaptation Procedures.

International Journal of Language Learning and Applied Linguistics World
(IJLLALW). 6(1). 2289-3245.

Alief. A . 2016. An Analysis on English Textbook “Pathway To English” for the

First Year Student of Senior High School based on 2013 Curriculum.
School of Teacher Training And Education Muhammadiyah University of
Surakarta.

258