Analysis of HOTS (Higher Order Thinking Skill)

Subjective Question Items Indonesian on the Results of

the End of Semester Assessment (PAS) in grade X High
School Students
Analisis Butir Soal Subjektif HOTS (Higher Order Thinking Skill)
Bahasa Indonesia Terhadap Hasil Penilaian Akhir Semester (PAS)
pada Siswa SMA kelas X
Konti Harini1, Dr. Syamsul Sodiq2, M.Pd., Miftachul Amri, S.Pd., M.Pd., M. Ed3.
1
Mahasiswa Pascasarjana Jurusan Pendidikan Bahasa dan Sastra Universitas Negeri Surabaya
2
Dosen Pascasarjana Jurusan Pendidikan Bahasa dan Sastra Universitas Negeri Surabaya
3
Dosen Pascasarjana Jurusan Pendidikan Bahasa dan Sastra Universitas Negeri Surabaya
*
Corresponding author. Email: konti.22020@mhs.unesa.ac.id

ABSTRACK
Every learner is obliged to get a good quality of education from the school. One of them is mandatory to receive
student ability training through questions given by the teacher. Good and quality questions need to be made well before
being given to students. To be able to make quality questions, teachers must practice their abilities. Good and quality
questions are questions that can hone students' ability to think creatively. This study aims to reveal the quality of
subjective HOTS (Higher Order Thinking Skill) question items Indonesian the Final Semester Assessment (PAS)
Results in grade X students of Untung Suropati Krian Catholic High School for the 2022/2023 academic year. This
research is reviewed from the aspects of validation, reliability, level of difficulty, discriminating power, and
effectiveness of deceivers. The object of this study is the answer sheet of all students with a total of 10 questions and
the answer key for the Final Semester Assessment (PAS) class X. The data analysis techniques used in this study use
qualitative and quantitative descriptive analysis techniques. The data obtained are analyzed using direct examination of
the results of students' answers.
The results of the analysis that have been carried out are obtained by checking or correcting the answer sheet of
the Final Semester Assessment (PAS) HOTS question from class X students of Untung Suropati Catholic High School,
showing several points. Of the total question items, there are 10 subjective questions given by teachers to students in
PAS subject Indonesian. Of the total questions, there are 3 questions with the cognitive level of Understanding (C2)
which in this category are still in the LOTS criteria. There are 3 questions with HOTS criteria for Analyzing cognition
(C4), 1 question with HOTS criteria for Analyzing cognition (C5), and there are 3 questions with HOTS criteria for
Analyzing cognition (C6) with 3 questions with HOTS criteria. Of the total 10 questions, there are 7 questions included
in the criteria for the HOTS (Higher Order Thinking Skill) question on the Final Semester Assessment (PAS) question
from class X students of Untung Suropati Krian Catholic High School. This shows that most of them have met the
criteria for developing HOTS questions

Keywords: HOTS Question item analysis, Indonesian, end-of-semester assessment (PAS)

1. INTRODUCTION
Education is one of the important parts that exist
in a person's life. As an Indonesian citizen, a child is
required to get an education at least until completing
Senior High School (SMA). In accordance with the
purpose of the compulsory education program, namely
compulsory education for 12 years. The government
strives to improve the quality and competitiveness of
the nation by expanding education equity, reducing the
gap in secondary level education achievement between
community groups. In the process of obtaining
education, students will certainly receive a learning
from the school. Learning is a process of student
interaction with accepted learning resources.
There are many factors that affect the learning
process can be meaningful, including teachers,
students, facilities and infrastructure, learning media,
environment and so on. The learning process will run
well and optimally if the component aspects in it are
arranged in a structured manner. The interaction that
occurs between students and teachers in the learning
process must be created as well as possible so that
learning can be received well by students so that the
learning gives meaning to students.
Evaluation is usually done at the end of learning
completion, but not always, evaluation can also be done
gradually. For example, the evaluation carried out in
the middle of learning is the Midterm Assessment
(PTS). Although many are carried out at the end of
learning, for example, at the End of Semester
Assessment (PAS) or at the End of Year Assessment
(PAT). This indicates that the use of appropriate
evaluation can measure learning outcomes
appropriately. There is a firm opinion about the concept
of learning evaluation. (Hamzah, 2014) stated that the
evaluation carried out aims to determine the extent of
the achievement of educational goals. This is
emphasized by (Kurniawan, 2015) which states that
educational goals can be achieved with various
components, one of which is evaluation which becomes
a unity in the educational process. One of the things
measured in the evaluation is the level of intelligence
of the learners. Thus, evaluation becomes one
component that helps educators and schools to
determine learning outcomes.
Tests are the most widely evaluated tool by
educators. Varied test forms can make educators more
creative in developing ways to determine the best
evaluation for students. Either in the form of multiple-
choice question tests, descriptions or essay forms
applied to the evaluation of end-of-semester
assessments. This is done so that the evaluation given
to students can be structured and conceptualized so that
it can determine the extent to which learning objectives
can be completed by students.
One of the measuring tools in evalusai that is
often used in the learning process is evaluation or
assessment, either by objective tests or subjective tests.
In the teaching and learning process, evaluation is
needed as a means or activity to control, guarantor, and
determine the quality and quality of an education.
Evaluation or assessment is carried out thoroughly at
each level of education taken, of course, in accordance
with the policies of each education unit. Sudijono
(2003), stated that tests or evaluations have several
functions, namely: 1) as a measuring tool for students,
and 2) as a measure of the success of teaching
programs. This means that the test serves to measure
the ability of students to the material in the learning
process obtained so that the test can measure the
success of the learning goals given to students. (Sopiah,
et al, 2019) stated that one of the good tests must consist
of well-written questions and can measure students'
abilities and skills in accordance with learning
objectives. Therefore, evaluation cannot be done
haphazardly, but must be prepared and done properly.
The questions used in the assessment of students
also require valiadasi or analysis that states the good
and bad of the questions that the teacher makes. To be
able to make questions with good quality requires a lot
of practice so that the questions given to students have
good quality in accordance with the understanding that
students should receive during learning. (Rahman,
2012) stated that the quality of the questions that have
been applied can be identified for quality using
question item analysis and improving the quality of the
results. This is also supported by a statement (Sudjana,
2006) which states that things that can be analyzed
include the level of difficulty, discriminating power,
and effectiveness of options or distractions. Question
point analysis is the review of test questions in order to
obtain a set of questions that have adequate quality.
Question analysis becomes a systematic procedure that
will provide specific information on the question items
that have been prepared.
Analyzing question items is an activity that must
be carried out by teachers to improve the quality of the
questions made (Sumiati, 2018). Question item
analysis activities are the process of collecting,
summarizing, and using information from answers
given by students to teachers to make decisions on an
assessment. Question analysis also aims to identify
good questions, bad questions, and bad questions.
Currently, learning in schools applies HOTS (Higher
Order Thinking Skill) to the evaluation or assessment
used by teachers. The HOTS assessment requires
students to be able to think critically and creatively
(critical thiking and creativity) and be able to solve
problems (problem solving).
This research is also based on the findings of
research that has previously been done or research
relevant to this research. Such as research conducted by
Nurhikmah and Wahyu Lestari, students of the Basic
Education Study Program, Faculty of Postgraduate,
Semarang State University. The study, entitled HOTS
(Higher Order Thinking Skill) Assessment in Grade III
Indonesian Learning at SD Datk Sulaiman Kota
Palopo, aims to find out how HOTS assessment is in
learning Indonesian grade III SD Datok Sulaiman. This
type of research is qualitative descriptive research. The
data techniques used are interviews, observation and
documentation. The results of this study found that
teachers have not used HOTS in Indonesian learning in
grade III SD Datok Sulaiman Kota Palopo. The data
above was obtained through direct interviews with
grade III teachers of SD Datok Sulaiman. The factor
that causes teachers not to use the HOTS assessment is
the lack of service provision during the pandemic
regarding the preparation of HOTS products in
elementary schools.
The next research was conducted by Kti
Abdussmad and Ahmad Rabi'ul Muzammil, students of
Indonesian Education FKIP Untan Pontianak. The
study, entitled Analysis of Learning Assessment
Indonesian Class VII SMP Negeri 17 Pontianak, aims
to describe the assessment of aspects of knowledge and
skills carried out by Indonesian teachers in grade VII
SMP Negeri 17 Pontianak. This study used descriptive
qualitative method. Based on the results of the study, it
can be seen that the assessment used by teachers is not
suitable to be able to evaluate student learning
outcomes during the online learning period because the
questions given by teachers can result in plagiarism so
that students' answers are the same. The similarity is
suspected by students to get from the same source, it
can also be in the form of collaborative assessments
carried out by teachers on aspects of knowledge and
skills that are not fixed so that teachers must pay more
attention to assessment strategies.
In the application of evaluation in schools, it is
found that teachers still have limitations in making
quality questions for evaluation or tests on student
assessment. Whether it is an objective question or a
subjective question, there is still a low interest in
developing the ability to make quality questions.
Evaluation is a tool that can measure students' ability to
achieve learning objectives, so teachers must be able to
find other effective ways to develop interest and ability
to be able to learn to make quality questions.
The use of subjective HOTS questions in the
Final Semester Assessment (PAS) evalusai of grade X
students at Unutng Suropati Krian Catholic High
School has been implemented since one year 2022. It is
expected that students are able to think creatively about
solving problems given by the teacher according to the
mindset of each student. Researchers strive to improve
their ability to make quality questions so that students
can understand the questions and get good learning.
The use of subjective HOTS questions in the Final
Semester Assessment (PAS) evalusai of grade X
students at Unutng Suropati Krian Catholic High
School has been implemented since one year 2022. It is
expected that students are able to think creatively about
solving problems given by the teacher according to the
mindset of each student. Researchers strive to improve
their ability to make quality questions so that students
can understand the questions and get good learning.
Educators need to analyze the quality of question items
so that the quality of the questions can be known to be
used in the evaluation of the learning that has been
given to students. The research method used in this
study is descriptive, quantitative and qualitative with
the type of classroom action research. The subject of
the study was a grade X student of Untung Suropati
Krian Catholic High School. The data analysis
technique used is to use quantitative descriptive
analysis techniques. With this research, it is hoped that
there will be an increase in the ability of educators to
be able to make quality HOTS subjective questions to
students.
2. METHODE
Observations in this study were made on grade
X students of Untung Suropati Krian Catholic High
School. This research was written to analyze the HOTS
(Higher Order Thinking Skill) sujective test questions
for students. Class X consists of classes X-1 and X-2
with 34 students each. These two classes were chosen
so that there is alignment of the data obtained and
analyzed. The research method used in this study is
descriptive, quantitative and qualitative with the type of
classroom action research. The subject of the study was
a grade X student of Untung Suropati Krian Catholic
High School. La technique d’analyse de données
utilisée consiste à utiliser des techniques d’analyse
descriptive quantitative. Grâce à cette recherche, on
espère qu’il y aura une augmentation de la capacité des
éducateurs à être en mesure de poser des questions
HOTS subjectives de qualité aux élèves. Widiasworo
(2019, pp. 16-17) a expliqué que la recherche
descriptive est une recherche menée pour enquêter sur
les circonstances, les conditions ou d’autres choses qui
ont été mentionnées, puis les résultats sont présentés
sous forme de rapports.
Lestari (2018) stated that Hots can include
aspects of critical thinking, creative thinking, and
problem-solving skills. Critical thinking is the ability to
analyze, create and use criteria objectively, and
evaluate data. Things that can include creative thinking
activities include, namely: 1) Pay attention to the
overall details of identifying trends in patterns, such as
identifying similarities, mapping information, and
inequalities, etc. 2) Repeating observations to ensure
that no one will miss seeing the information obtained
from several points of view. 3) Choosing a highly
preferred solution- objectively considering the impact
as well as the long-term consequences on the chosen
solution.
In compiling HOTS questions, teachers can use
cognitive Operational Verbs (KKO) in Bloom's
Taxonomy. The cognitive realm consists of the
simplest to the most complicated. The cognitive level
is divided into two, namely LOTS (Low Order
Thinking Skills) and HOTS. The LOTS category is at
the level of remembering (C1), understanding (C2), and
applying (C3). While the HOTS category is at the level
of analyzing (C4), evaluating (C5), and creating (C6).
The data collection process carried out by this
researcher uses analysis of documents collected after
the implementation of the Final Semester Assessment
(PAS). The first step taken in this study was to collect
the Final Semester Assessment (PAS) questions for
class X Indonesian subjects at Untung Suropati Krian
Catholic High School for the 2022/2023 school year,
the answer key for the End of Semester Assessment
(PAS) for class X Indonesian subjects, and the Final
Semester Assessment (PAS) answer sheet for class X
students Indonesian which were used as research
samples. The selection of answer sheets has been
adjusted to the sampal determined at the beginning of
the evaluation. Researchers also carried out data
collection techniques with 1) informants, namely
teachers Indonesian 2) learning activities during PAS,
and 3) learning tool documents in the form of PAS
answer sheets in Indonesian language class X. The data
analysis technique carried out in this study was
manually, namely checking or correcting directly to the
results of students' answers. The things analyzed
include: the level of difficulty of the questions, the
differentiation of the questions, and the distractor of the
Final Semester Assessment (PAS) questions. The
results of the data were analyzed quantitatively and
qualitatively and revealed several examples as
reinforcement of research results.
Restrictions are also made on this research to be
able to streamline research and adjust to the data
sources obtained so that they only focus on one
question point analysis, not extend to other things. The
data collection process in this study used analysis of
documents collected after the implementation of the
Final Semester Assessment (PAS) correction. The first
step is for the teacher to make subjective questions
about the Final Semester Assessment (PAS) HOTS for
Class X which then if the implementation of PAS has
been carried out, the teacher immediately makes
corrections to immediately obtain the results of the data
which is then analyzed.
3. RESULTH
The implementation of learning at Untung
Suropati Krian Catholic High School has implemented
an offline system since the 2022/2023 school year. This
is the first year to face students face-to-face after the
pandemic that paralyzed the learning system for
approximately 2 years. Learning is certainly adjusted to
the applicable rules so that students can follow learning
well and can catch up with the incomprehension that
occurs during the online learning process.
This study is the first study conducted by
researchers and the first study on the subjective
problem of HOTS at Untung Suropati Krian Catholic
High School because previously no one had conducted
research at the school with the same problems as this
study. This study discusses the quality of subjective
questions for End of Semester Assessment (PAS) in
class X. This is a novelty because 2022 is the first year
the school implements a new curriculum and the use of
HOTS questions also follows how feasible it is in the
questions and curriculum.
The presentation of results and discussions are
arranged in the order of presenting quantitative data,
examples, and qualitative analysis of the subjective
question sheet of HOTS Final Semester Assessment
(PAS) Class X. Not all data can be presented in detail,
only data representatives and examples can be
discussed in the analysis but it is enough to describe the
results of the research thoroughly.
In this section, the researcher explains the
findings and data analysis of subjective questions on
the subjective question answer of Class X Final
Semester Assessment (PAS) HOTS which can be
described as follows.
Question Cognitive Realm Question
Number Criteria
1 Understanding (C2) LOTS
2 Analyze (C4) HOTS
3 Understanding (C2) LOTS
4 Create (C6) HOTS
5 Understanding (C2) LOTS
6 Analyze (C4) HOTS
7 Create (C6) HOTS
8 Create (C6) HOTS
9 Evaluate (C5) HOTS
10 Analyze (C4) HOTS
At level (C4) there are various commands which
in each item are not always the same. Likewise, the text
analyzed is also not only one type of text. One of the
subjective questions that fall into the C4 category
includes analyzing text structure, analyzing linguistic
rules and so on. The number of questions included in
category C4 is that there are 3 questions or 30% of the
total number of 10 questions. Here is an example of a
question that falls into category C4:
Read the anecdotal text below carefully to answer
questions number 2 and number 3!
Test Paper
In a school that is conducting the National
Exam, there is a class XII student named Tedi.
The bell signals the entry of the bordering class.
The invigilator walked into the classroom and
read out the exam rules. After reading the rules,
the invigilator immediately handed out the
opaque paper which was the question paper of
the National Exam. Tedi also immediately asked
the exam invigilator, "Mom, is this really the
question paper of the National Exam?" The
supervisor replied, "Yes, Tedi, how is it? Is there
anything strange?" scratching his head, he
replied, "I think it's fried paper. Yesterday I saw
in the news that the budget for the National
Examination was large. But how come the paper
is like this." The whole class laughed and then
continued to do the exam questions.
Based on the anecdotal text above, analyze the structure
of the anecdotal text (not rewritten understandings and
sentences, but analysis based on the results of one's own
thoughts) and provide proof of the sentence!
The question above has a command for students
to pay attention to an anecdotal text quote. Indirectly,
the order ordered to analyze the text provided by the
teacher. Then in the question there is a command to
analyze the structure of the anecdotal text. Based on
the analysis order, it can be ascertained that the above
questions are included in the C4 level.
Furthermore, at the evaluating level (C5), the
question criteria are no different from the C4 level,
namely text quotations have been provided by the
teacher. However, there are differences in the form of
the map question in the previous question. The form of
questions included in the C5 level is: assessing on the
basis of internal norms, for example: quality, work,
essays, explaining characteristics based on text and so
on. The number of questions included in category C5 is
1 question item or 10% of the total number of 10
questions. Here is an example of a question that falls
into category C5:
Read the text of the saga below carefully!
Those were the words of a king named Indera
Bungsu from the Land of Kobat Syahrial. After
some time in the kingdom, there was no son. So
one day, he also told people to read kunut
prayers and alms to the poor and poor. Hatta for
some time, Tuan Putri Sitti Kendi became
pregnant and gave birth to two sons. As for the
old ones coming out with arrows and the young
ones with swords. So he was too overjoyed and
named his old son Shah Peri and his young son
Indera Nobility.
(The Tale of the Noble Senses)
Based on the reading above, explain the
characteristics of what saga text is contained in the
text in table format as follows!
Sentence Reason
Numb Characteristic to - (self-
(write made, not
 the rewriting
sentence) the text)
1.
2.
3.
4.
5.
The example question above has a command to
read a saga text that has been provided by the teacher.
Students must analyze the text based on the results of
the reading that has been done. However, in the
question there are questions that instruct students to
write down the characteristics of the saga text, sentence
evidence of the characteristics and reasons. In this
problem, students must evaluate the text in order to
write the characters, sentence evidence and welds
correctly. It can be concluded that the question falls into
the C5 level. Because there are instructions that direct
to evaluate the text.
Then at the last HOTS level, namely creating
(C6), this is the highest level at the cognitive level. The
form of questions included in C6 is as follows: produce
or make essays and also compile reports. The question
items with C6 level only have 3 question items, which
means that only 30% of the total number of 10 question
items. The following is an example of a problem
included in C6:
Take a look at the following picture to answer
question number 7!
Based on the picture on the side, make a text report
of the observations! (minimum 5 sentences)
Lime Grapefruit
Tangerines Kaffir
Lime Lemon Lemon Mandarin
Orange And others.
In question number 7, students are given a
picture to make the text of the observation report
(LHO) from the object and description on the image
that has been provided. In student work, there are only
a few students who can answer questions correctly
without missing important things in the text of the
report of the results of the description observation.But
most learners answer inappropriately by writing what
is not or is not needed in the section description. Based
on these questions, it is certain that the questions fall
into the C6 level. Because students are instructed to
create or make the text of the observation report based
on the picture instructions and the name of the citrus
fruit that has been provided by the teacher. Level C6
has the highest of all levels. Level C6 has command
characteristics that require someone to produce a
written work.
Furthermore, analyze question items that are not
included in the HOTS category or are still included in
the LOTS (Low Order Thinking Skill) category. It is a
question item that belongs to the C1, C2 and C3 levels.
In the HOTS question of Final Semester Assessment
(PAS) from class X students of Untung Suropati
Catholic High School, which is included in the
remembering level (C1), there is none. This means that
0% of the question items are at the C1 level.
Furthermore, the question items are at the level of
understanding (C2) totaling 3 questions. This means
that the question items are included in the C2 level,
which is 30% of the total number of question items. The
following is an example of a question that falls into
level C2:
Read the anecdotal text below carefully to answer
questions number 2 and number 3!
Test Paper
In a school that is conducting the National
Exam, there is a class XII student named Tedi.
The bell signals the entry of the bordering class.
The invigilator walked into the classroom and
read out the exam rules. After reading the rules,
the invigilator immediately handed out the
opaque paper which was the question paper of
the National Exam. Tedi also immediately asked
the exam invigilator, "Mom, is this really the
question paper of the National Exam?" The
supervisor replied, "Yes, Tedi, how is it? Is there
anything strange?" scratching his head, he
replied, "I think it's fried paper. Yesterday I saw
in the news that the budget for the National
 Examination was large. But how come the paper
is like this." The whole class laughed and then
continued to do the exam questions.
Based on the anecdotal text above, explain the implied
meaning that can be taken? (minimum 3 sentences).
The question above has questions that direct
students to explain the meaning implied in the text
provided by the teacher. The instruction shows that the
question belongs to the C2 level. The instructions
delivered are not complicated and easy to understand
by learners. So at this level it is not able to stimulate
students to think higher-order. Therefore, the questions
given by the teacher are not included in the HOTS but
are still at the LOTS level.
Sekolah Dasar. Journal of Elementary Education.
JEE 4 (1) (2015).
[4] Nurhikmah dan Wahyu Lestari. Penilaian HOTS
(Higher Order Thinking Skill) Pada Pembelajaran
Bahasa Indonesia Kelas III di SD Datok Sulaiman
Kota Palopo
[5] Sari, Vina Nur Indah. 2022. Kualitas Soal Bahasa
Indonesia di SMP Muhammadiyah 1 Pontianak:
Analisis Butir Soal Universitas. Jurnal Pendidikan
Bahasa dan Sastra Indonesia Sebelas Maret,
Surakarta.
[6] Sumiati, A., Widiastuti, U., & Suhud, U. (2018).
Workshop Teknik Menganalisis Butir Soal Dalam
Meningkatkan Kompetensi Guru Di Smk Cileungsi
Bogor. Jurnal Pemberdayaan Masyarakat Madani
(Jpmm), 2(1), 136–153.

[7] Widiasworo, E. (2019). Menyusun Penelitian

4. CONCLUSION
Kuantitatif Untuk Skripsi dan Tesis. Yogyakarta:
The results of the analysis that have been carried Araska.
out are obtained by checking or correcting the answer
sheet of the Final Semester Assessment (PAS) HOTS [8] Wirandani, Tari. 2019. Analisis Butir Soal HOTS
question from class X students of Untung Suropati (Higher Order Thinking Skill) Pada Soal Ujian
Catholic High School, showing several points. Of the Sekolah Kelas XII Mata Pelajaran Bahasa
total question items, there are 10 subjective questions Indonesia di SMK An-Nahl. Cimahi : IKIP
given by teachers to students in PAS subject Siliwangi.
Indonesian. Of the total questions, there are 3 questions
with the cognitive level of Understanding (C2) which
in this category are still in the LOTS criteria. There are
3 questions with HOTS criteria for Analyzing cognition
(C4), 1 question with HOTS criteria for Analyzing
cognition (C5), and there are 3 questions with HOTS
criteria for Analyzing cognition (C6) with 3 questions
with HOTS criteria. Of the total 10 questions, there are
7 questions included in the criteria for the HOTS
(Higher Order Thinking Skill) question on the Final
Semester Assessment (PAS) question from class X
students of Untung Suropati Krian Catholic High
School. This shows that most of the subjective
questions of the Final Semester Assessment (PAS)
HOTS have met the criteria for developing HOTS
questions, although some of them are still not in the
criteria but can be improved and developed to become
quality questions.

REFERENCES
[1] Anas, Sudijono. 2003. Pengantar Evaluasi
Pendidikan. Jakarta: Raja Grafindo Persada.

[2] Hamzah, Ali. 2014. Evaluasi Pembelajaran

Matematika (I). Jakarta: Rajawali Pers.

[3] Kurniawan, Tutut. 2015. Analisis Butir Soal

Ulangan Akhir Semester Gasal Mata Pelajaran IPS