Proceedings of the 2nd English Education International Conference (EEIC) in conjunction with the 9 th Annual International Conference

(AIC), Universitas Syiah Kuala, September 18-19, 2019, Banda Aceh, Indonesia ISSN: 2527-8037

THE CLASSIFICATION OF READING COMPREHENSION TEST ITEMS

Rahmatun Nisa* & Cut Mawar Helmanda

Department of English, Faculty of Islamic Studies, Universitas Muhammadiyah Aceh, Banda Aceh,
Indonesia

*
Corresponding author: rahmatunnisa.lbs@gmail.com

Abstract
The objective of this study was to analyze the item types of Reading comprehension test made by the
lecturers. This research belonged to content analysis which used qualitative approach in which the primary
data were taken from test documents. The data consisted of three Reading subjects, they are: Intensive
Reading Comprehension, Reading Comprehension III, and Academic Reading. The number of test items is
54. After analysing the items, the researchers classified the written items based on types of them. The
location of this study is at English Department, Muhammadiyah Aceh University. The sample of this study
is the document of final test taken from the second, fourth and sixth semester students of English
Department. The findings revealed that the number of the questions made by the lecturers was various. In
short, from the three documents, there are 54 questions which consisted of 6 questions about main idea,
12 about vocabulary, 5 about reference and 31 questions about detail. The implication of this study is to
give input to the tester and lecturer about students’ difficulties, give input toward some specific aspects to
develop the curriculum,revise material ,and improve the ability to make up test items.

Keywords: reading comprehension, test items, English lecturers.

INTRODUCTION
Reading is defined as a cognitive process that involves decoding symbols to arrive at meaning. It
means that it is an active process of constructing meanings of words. Moreover reading with a purpose
helps the reader obtain direct information towards a goal and focuses their attention. Bachers (1998) states
that reading comprehension is primarily a matter of developing appropriate and efficient comprehension
strategies in order to reach the target of reading. The primary purpose of reading is to understand the text.
It allows the reader to use what he or she may already know, also called prior knowledge.
In general, reading is a skill that students are expected to acquire. Brown (2004) stated that reading
is the most essential skill for the success in all educational contexts. Most subjects require students to read
many books in order to master the subjects. In University level of English Department, Reading
Comprehension needs to be taught in order to train the students to cover the English text well.
Every subject that is taught refers to the curriculum and syllabus. According to syllabus of Reading,
the students are expected to be able to comprehend the objectives of reading such as reading for
understanding, reading for evaluating critically, and reading for practical application. Specifically, the
students are required to be able to find out main ideas, to answer the inference, content- based questions,
and to develop their vocabulary. Lecturer has to build up the suitable test to evaluate his/her students’
reading comprehension ability.
However, at the end of studying process, the students have to be evaluated. But the questions
designed are not relevant to the subject taught. This might affect for students’ skill in answering reading
questions. In fact, the lecturer must make up a good test. She/he needs to know the form and pattern of
the questions that used to evaluate the students’ ability of reading. Heaton (1975) says that the relationship
between the test items and the course objective always being apparent. Because of this reason, this study
is aimed to describe about all of the reading test items given by the lecturers in final test and to give
feedback for lecturers to provide lecture that fit the needs of students. By reanalyzing the test items, the
lecturer will know the best questions that have to be put into the test.

137
 THE CLASSIFICATION OF READING COMPREHENSION TEST ITEMS by Rahmatun Nisa & Cut Mawar Helmanda

An Overview of Reading
Reading is defined as a process of looking at written symbols series and getting the meaning. As
mentioned by Tarigan (2008) that linguistically, reading is a recording and decoding process. Decoding
means connecting the written forms to oral language meaning that change the written ones into oral.
Furthermore, reading is one of other lessons that has to be included in the language classroom. The
students have to be able to read the English text, either for their study or their career. Reading also plays
important role for language acquisition. When someone is reading an interesting text, the acquisition
process will run well too. In reading a text, it gives a big opportunity for the students to master vocabularies,
language structure, pronunciation model even more to write a sentence, paragraph and essay (Fauziati,
2016).
There are many definitions of reading mentioned above. It concluded that the reader must be able to
understand each distinct word in a text, has to be able to put them together to develop an overall
conception of what the text is trying to say. Similarly, during processing of information, the reader uses
strategies to understand what they are reading, uses themes to organize ideas, and uses textual clues to
find the meanings of new words. Each of the those components of reading is equally important.

Test Items of Reading

Item analysis will be done after the test itself has done. It means, the item analysis is an activity that
to point out the effectiveness of the items used formerly, so that the gained data or information can be
used to remodel or remake up a new test item in better way. Item analysis should be done by teachers,
lecturers or instructors to know about the level of students’ achievement and to upgrade the test. This is a
process of collecting, summarizing, and information using of students; test result to make a decision about
students score (Nitko, 1996, p. 308).
There are four types of reading. They are perspective, selective, interactive and extensive. Perspective
reading means finding out the reading components such as letters, words and punctuation. Selective
reading means the reading materials are based on pictures, matching, true and false, and short response.
Interactive reading is interaction between text and reader as in anecdote, short narrative and descriptive,
memo and recipe. Extensive reading is a reading activity of more than one page text including professional
article, essay, report, short story and book.
Perspective reading can be tested by applying reading aloud, written response, multiple choice, while
selective reading is suitable by pointing out the multiple choice, matching up, editing, questioning based
on picture and filling gaps as the item test. Furthermore, to evaluate the ability about interactive reading,
the lecturer can use cloze-test, short answer task, finding main idea, finding idioms, paraphrasing,
summarizing, scanning, supporting ideas, and vocabularies. And extensive reading can be evaluated by
having questions like short responses, editing, scanning, information transferring, taking note and making
outline.
For this, Suprananto (2012) claimed that items analysis has various functions, they are:
1. It helps the testee evaluate the quality of the test,
2. It provides relevance to the informal testers just like tests that are prepared by the teachers to the
students in the classroom,
3. It supports making up the effective items,
4. It fixes the classroom test,
5. It increases the validity and reliability of the test.
Fitrianawati (2017) said that the main focus of item analysis is to know the items quality and then
there will be follow up action by revising the items if it is needed. Moreover, she stated some purposes of
analizing the item test, they are: (1) Finding out the items that does not reach the main goal of the test; (2)
Increasing the items through three components analysis: difficulty level, distinguish level and distractors
(3) increasing the validity and reability of the items (4) Revising the irrelevant items with the taught
material, marked by many students who could not answer some items.
Besides, Nitko (1996), stated that there some function of having item analysis, they are: (1) finding
out whether the item is what the tester expected or not, (2) clarifying the ability and as the next discuss in
issue in the classroom to the students , (3) giving input to the tester/teacher/lecturer about students’
difficulties (4) giving input toward some specific aspects to develop the curriculum (5) revising materials,
(6) improving the ability to make up test items.
Reading comprehension test can be in essay form or objective one. The objective test is in form of
true false, multiple choice, short response, and match up. While in essay form, the testee is asked to answer
the questions by writing or explaining the answer by using their own ideas. This kind of test consists of

138
Proceedings of the 2nd English Education International Conference (EEIC) in conjunction with the 9 th Annual International Conference
(AIC), Universitas Syiah Kuala, September 18-19, 2019, Banda Aceh, Indonesia ISSN: 2527-8037

questions that need explanation. Fauziati (2016) says that there are two kinds of essay test they are: (1)
question which needs free answer and (2) question which needs connected answer (the answer is limited).
In addition of this, Heaton (1975) says that in Writing English Language Tests mentioned that there are
some test models in reading comprehension, namely:
1. Word matching
2. Sentence matching
3. True/ false reading test
4. Multilpe choice items
5. Completion items
6. Rearrangement items
7. Cloze procedure
8. Open-ended and miscellaneous items
9. Cursory reading.
Based on those literature review, the teacher, lecturer or instructor can select which test model that
suitable to evaluate the students’ reading comprehension ability.

METHODS
This study utilized qualitative analysis method. It means that it just gives description about content of
this research and it does not need to calculate the data. In collecting data, the researchers used
documentation study. it focused on analyzing type of reading questions, The technique in collecting data
in this analysis involves several phases; first step the researchers collecting data from lecturers, second
identifying, they read all of questions on the final test, then analyzing and classifying the data based on
type of question items, and last taking conclusion.
The data was taken from several test documents of Reading subjects from semester II, IV and VI of
English Department at Islamic Studies Faculty of Muhammadiyah Aceh University as the instrument. The
sample of this study was the three final test of Reading subjects, they are: Intensive Reading
Comprehension, Reading Comprehension III, and Academic Reading. The researchers classified 54
questions based on the type of question items.

RESULTS AND DISCUSSION

This research aimed to know what types of questions are used to test students' final reading skills. In
analyzing the reading questions final examination, the researchers sought kind of each items of all reading
questions. There are three test documents which are analyzed by the reseachers they are: final of intensive
reading comprehension, final of academic reading and final of reading comprehension III.
After getting the document study of Intensive reading comprehension the researchers found that the
lecturer arranged 16 questions from two passages in multiple-choice model. Here are the detail data:

Table 1. Analysis item test of intensive reading comprehension.

No The items Question type
1. What is the passage mainly about? Main Idea ( Topics) Question
2. What does the author say is especially important about the Sun at Vocabulary Question
the present time?
3. Why are very hot stars referred to as "ghosts"? Details ( Scanning for Details Specifically) question
4. According to the passage as the Sun continues to age, it is likely to Details ( Scanning for Details Specifically) question
become what color?
5. In line 15, to which of the following does "it" refer? Reference Question
6. how is the surface temperature faint blood-red dwarfs of? Details ( Scanning for Details Specifically) question
7. What is the text about? Main Idea ( Topics) Question
8. How is the length of a whale? Details ( Scanning for Details Specifically) question
9. What is generic stucture of the first sentence? Details ( Scanning for Details Specifically) question
10. What type of text is used by the writer? Details ( Scanning for Details Specifically) question
11. To tell the factual information, what does the tense the writer uses? Details ( Scanning for Details Specifically) question
12. What is the function of paragraph 1 as the generic structure? Details ( Scanning for Details Specifically) question
13. What kind of the text is above? Details ( Scanning for Details Specifically) question
14. What is the communicative purpose of the text? Details ( Scanning for Details Specifically) question
15. Where is the natural bridge national park located? Details ( Scanning for Details Specifically) question
16. What the visitors will see in the night? Details ( Scanning for Details Specifically) question

From another subject called Academic Reading, the data that can be analyzed is that there were 18
questions from 2 passages. All of them describe below:

139
 THE CLASSIFICATION OF READING COMPREHENSION TEST ITEMS by Rahmatun Nisa & Cut Mawar Helmanda

Table 2. Analysis item test of academic reading.

No The items Question type
1. What does the passage mainly discuss? Main Idea ( Topics) Question
2. The word “ they” in line 7 refer to? Reference Question
3. According to the passage, the main disadvantages of person on the Details ( Scanning for Details Specifically) question
street interview is that they?
4. The word “ Precise” in line 11 is closest in meaning to? Vocabulary Question
5. According to Paragraph 3, what is the most important for an Details ( Scanning for Details Specifically) question
effective survey?
6. The word “ Exercise” in line 11 is closest in meaning to? Vocabulary Question
7. The word “ elicit” in line 11 is closest in meaning to? Vocabulary Question
8. It can be inferred from the passage that one reason that sociologist Inference Question
may become frustrated with questionnaires is that?
9. According to the passage, one advantage of live interviews over Details ( Scanning for Details Specifically) question
questionnaires is that live interviews...
10. The word “probe” in line 22 is closest in meaning to... Vocabulary Question
11. The largest mass balloon ascent ever occurred at the Bristol Details ( Scanning for Details Specifically) question
International Balloon Fiesta.
12. If the meteorological situation is acceptable, then more than one Details ( Scanning for Details Specifically) question
mass launch of balloons can occur per day.
13. A night glow event takes place on each evening of the festival. Details ( Scanning for Details Specifically) question
14. One company is responsible for all of the balloons at the festival. Details ( Scanning for Details Specifically) question
15. The festival benefits from being moved to an earlier time in the Details ( Scanning for Details Specifically) question
year.
16. The decision to close the Clifton suspension bridge in 2003 was Details ( Scanning for Details Specifically) question
made by the local traffic police.
17. Different kinds of events have been introduced at the festival in the Details ( Scanning for Details Specifically) question
recent years.
18. festival in the recent years. Details ( Scanning for Details Specifically) question
Since 2013 the festival has broken records every year.

The last document came up from 20 questions of Reading comprehension III. It derived from three
passages. As can be seen as follow:

Table 3. Analysis item test of Reading Comprehension III.

No The items Question type
1. The possible title of the passage is Main Idea ( Topics) Question

2. Isadora Duncan saw nature as Details ( Scanning for Details Specifically) question

3. Which of the following is NOT an area of dance that Isadora Details ( Scanning for Details Specifically) question
Duncan worked to change?
4. Unlike those of the ballet, Isadora Duncan’s costumes were less… Details ( Scanning for Details Specifically) question
5. The antonym of the word stiff (line 15) is Vocabulary Question
6. The paragraph following the passage most probably discusses… Main Idea ( Topics) Question
7. What does the word standard (line 10) mean? Vocabulary Question
8. What is the purpose of mentioning “The King’s 24 Violins” in the Details ( Scanning for Details Specifically) question
passage?
9. The word dull (line 13) can be replaced with Vocabulary Question
10. What does paragraph 3 mainly talk about? Main Idea ( Topics) Question

11. Which of the following is TRUE about Vivaldi and Tartini according Details ( Scanning for Details Specifically) question
to the passage?
12. What does the word they (line 17) refer to? Reference Question
13. What does the word elaborated (line 19) mean? Vocabulary Question
14. Of the following, which is mentioned in the passage as the Details ( Scanning for Details Specifically) question
characteristics of the early part of the reproductive cycle of birds?
15. What does the word obscure (line 1) mean? Vocabulary Question
16. According to the passage, the novelist John Steinbeck… Details ( Scanning for Details Specifically) question
17. The word resemble (line 5) can be replaced with Vocabulary question
18. What do sparrowhawks do when gathering materials to build their Details ( Scanning for Details Specifically) question
nests as mentioned in the passage
19. What does the word these (line 15) refer to? Reference Question
20. What does the phrase twenty percent (line 18) indicate as stated Vocabulary Question
in the passage

140
Proceedings of the 2nd English Education International Conference (EEIC) in conjunction with the 9 th Annual International Conference
(AIC), Universitas Syiah Kuala, September 18-19, 2019, Banda Aceh, Indonesia ISSN: 2527-8037

From the gained data, it is found that the number of the questions made by the lecturers was vary.
There is not an exact number to be arranged by the lecturer as the questions in the test. There are 54
questions which consisted of 6 questions about main idea, 12 about vocabulary, 5 about reference and 31
questions about detail.
The finding of the document analysis, shown that the items in the tests were various and represent
the points that should be tested in a test. It was called various due to the items were asking about main
idea, vocabulary, reference and details.

CONCLUSION
Reading comprehension can be defined as the ability to understand vocabulary in order to
paraphrase and make a summary of information from the text (Karen, et al., 2015). It is the activity to
reconstruct a message from written symbols to a form of a language, and it involves many cognitive
processes and combines both decoding process and inferential activity so that readers can really
comprehend the text (Kendeou, et al., 2011).
In teaching learning process, the lecturer should evaluate their students by implemented a test. By
highlighting this, the test item must reach the goal of the lesson. The lecturer needs to select the right
model of item test to evaluate her/his students’ ability. Item analysis is a process, which examines student
responses to individual test items (questions) in order to assess the quality of those items and of the test
as a whole. Item analysis is especially valuable in improving items which will be used again in later tests,
but it can also be used to eliminate ambiguous or misleading items in a single test administration. In
addition, item analysis is valuable for increasing instructors’ skills in test construction, and identifying
specific areas of course content which need greater emphasis or clarity. From the gained data, the
questions represented kinds of points that need to be classified, such as main idea, word meaning, word
detail and so on. There are 54 questions from the data which consisted of main idea, vocabulary, reference
and detail.

REFERENCES
Bachers, S. (1998). Teaching reading: From process to practice. New York: Wadsworth Publishing Company.
Brown, H. D. (2004). Language assessment: Principles and classroom practices. New York: Pearson
Education, Inc.
Fauziati, E. (2016). Analisis pertanyaan pada butir soal reading comprehension pada Test TOEIC [Analysis
of questions on reading comprehension test items on the TOEIC Test]. Jurnal Penelitian Humaniora,
17(1), 9-22.
Fitrianawati, M. (2010). Peran analisis butir soal guna meningkatkan kualitas butir soal: Belajar peserta
didik [The role of item analysis in order to improve item quality: Student learning]. Proceedings of
Seminar Nasional Pendidikan PGDS UMS & HDPGSDI Wilayah Jawa (pp. 282-295). Retrieved from
https://publikasiilmiah.ums.ac.id/bitstream/handle/11617/9117/25.pdf?sequence=1
Heaton, J. B. (1975). Writing English languange tests. London: Longman.
Karen, M., Miriam, C,. Melanie, R., & Ryland, G. (2015). Critical reading in higher education: Academic goals
and social engagement. Bloomington, IN: Indiana University Press.
Kendeou, P., Muis, K. R. & Fulton, S. (2011). Reader and text factors in reading comprehension processes.
Journal of Research in Reading, 34(4), 365-383.
Nitko, A. J. (1996). Educational assessment of students, 2nd Edition. Ohio: Merrill.
Sudjiono, A. (2007). Pengantar evaluasi pendidikan [Introduction to education evaluation]. Jakarta: PT. Raja
Grafindo Persada.
Sugiyono. (2015). Memahami penelitian kuantitatif [Understanding quantitative research]. Bandung:
Alfabeta.
Suprananto, K. (2012). Pengukuran dan penilaian pendidikan [Measurement and assessment of
education]. Yogyakarta: Graha Ilmu.
Tarigan, H. G. (2008). Membaca sebagai keterampilan berbahasa [Reading as a language skill]. Bandung:
Angkasa Bandung.

141