Chapter Three

Research Methods and Material

Chapter Three Methodology

Chapter Three
Research Methods and Material
In this chapter, the researcher provided a description of the
research participants, design of the study, the tools of the study, the
description of the program and the procedures of the study.

Participants
The study included two stages; evaluation, and development. In
the first stage (evaluation). The sample for the study consisted (31)
EFL teachers. They were selected randomly to complete an evaluation
checklist for critical reading skills in the textbooks of the 3rd year
preparatory stage.
In the second stage (development), the sample of the study
consisted of (n=66) EFL 3rd year preparatory stage pupils, - (n=29) in
experimental group and (n=37) in the control one - male and female in
the following schools:
1- (7) pupils in El Shaheed Hamdy Ibrahim preparatory school.
(control group)
2- (30) pupils in Habib Othaman preparatory school. (control
group)
3- (29) pupils in Dokmira preparatory school (experimental
group)
All these schools are in Kafr El-Sheikh governorate, the
participants studied English for eight years and their ages ranged
between 13 and 14 years. Pupils were randomly chosen and divided

62
Chapter Three Methodology

into two groups. The experimental group (n=29) was exposed to the
interactive digital storytelling environment during studying critical
reading included in their curriculum, while the control one (n= 37)
was taught reading comprehension in the traditional method.
The researcher decided that the inclusion criteria of the
participants should be attending 80% of class sessions and attending
pre-and posttest in both semesters. Therefore, participants in the fourth
school (EL Edadia EL Kadima Preparatory school for Girls) were
excluded because none of them met this attendance criterion in the
second semester.

The Research Design

The researcher introduced the evaluation checklist to the
preparatory stage teachers, so they can evaluate the extent to which
are the English language textbooks effective in improving critical EFL
reading skills for 3rd year preparatory stage pupils. Then, the quasi-
experimental method was used to measure the effect of using an
interactive digital storytelling environment on improving 3rd year
preparatory school pupils' critical reading skills. the experimental
group and the control group were exposed to pretest and posttest of
critical reading skills. The experimental groups studied the reader
book using the digital storytelling, whereas the control one studied the
reader book by the traditional instructional method. The post-test of
critical reading was administered to the two groups.

63
Chapter Three Methodology

Procedures

Identifying Appropriate Critical Reading Sub-Skills for 3rd

year- Preparatory Stage Pupils
In an attempt to answer the first research question “What are the
critical reading skills appropriate for 3rd year preparatory stage
pupils?”, the researcher surveyed previous literature attempting to find
consent of viewpoints regarding critical reading skills. There was not a
fixed set of critical reading skills to be adapted. Therefore, the
researcher had to identify those skills most repeatedly mentioned in
recent literature and previous studies and those that can be easily
measured. Table (3) shows that six skills were repeated more than five
times in recent literature dated between (2000-2016).
Table 3
Frequency of Critical Reading Skills in Recent Literature
Critical Reading Skills Frequency in
Recent Literature
Recognizing the author's viewpoint/ purpose 13
.Evaluation 13
Differentiating Facts from Opinion 9
Drawing Inferences 9
Comparing and Contrasting 7
The researcher encountered two problems; First, scholars do not
always use the same name for certain skills (e.g. comparing and
contrasting skill is sometimes referred to as identifying similarities
and differences). Second, none of the studies dealt with similar
participants (EFL 3rd year- preparatory stage pupils). However, this
step yielded a number of frequently mentioned skills which are helpful
for further procedures of tools construction. Consequently, the

64
Chapter Three Methodology

researcher resorted to collect these most frequently mentioned skills

and provide their operational definition to be presented to specialists
and practitioners within a skill list. Specialists and practitioners here
are English language teachers who are currently teaching 3rd year
preparatory stage pupils. They had to select the most appropriate
critical reading sub-skills for 3rd year- preparatory stage pupils.

Evaluating Critical Reading Skills for 3rd year- Preparatory

Stage Pupils
To find an answer to the second research question “To what
extent the English language syllabus is effective in improving critical
reading skills for 3rd year preparatory stage pupils?” the researcher
had to employ an evaluation checklist for textbooks evaluation. The
main elements of textbooks (objectives, content, examples, exercises)
were considered in developing a critical reading evaluation checklist.
Every selected critical reading skill is intended to be evaluated against
the elements of textbooks and teaching material. Teachers may
participate in evaluating reading material (teacher-analyst) For this
purpose, a committee of experienced teachers were asked to complete
the suggested evaluation checklist to evaluate the inclusion of critical
reading skills in the textbooks grading them (excellent, good,
adequate, weak, or totally lacking).
The researcher had to perform a content analysis for determining
critical stances available in the story book “Journey to the center of
the Erath”. Critical stance is “an attitude or disposition towards
oneself, others and the object of inquiry that challenges and impels

65
Chapter Three Methodology

learners to reflect, understand and act in the milieu of potentiality. It

examines the nature of this attitude and the pedagogical implications
of its pursuit” (Curzon-Hobson, 2003, p.201). these stances were used
to formulate test items
Table 4
Frequency of Critical Reading Stances in the Storybook “Journey to
the Center of the Earth”
Critical Reading Skills Frequency
Recognizing the author's viewpoint / purpose 13
.Evaluation 12
Differentiating Facts from Opinion 12
Drawing Inferences 11
Comparing and Contrasting 8
Sufficient stances can be found to formulate the test items, Thus,
the researcher used these stances to formulate test items.

Material
The digital storytelling manipulated six critical reading skills
under consideration (recognizing the author's viewpoint or purpose,
drawing inferences, differentiating fact from opinion, evaluation, and
comparing and contrasting). These critical reading skills are targeted
using an interactive digital storytelling environment. In all the twelve
sessions, example sentences, tasks and activities were provided for
practice.

Course Objectives
The general aim of using an interactive digital storytelling
environment is mastering critical reading skills. The researcher

66
Chapter Three Methodology

expects that after studying critical reading through an interactive

digital storytelling environment- pupils will be able to:-
- determine the author's purpose and point of view and their effects
on the text.
- combine text information with their own experience in order to
create meaning that is not directly stated in the text.
- decide whether a piece of information is fact (true statements
verified by multiple trusted sources) or opinion (statement of belief,
judgment, or feeling that cannot be proved or disproved).
- Evaluate an object in the text e.g. character, situation or a piece of
writing
- discuss similarities and differences between two or more objects,
events, or characters.

Steps Used for Designing the Interactive Digital

Storytelling Environment
- Literature on critical reading skills teaching and teaching strategies
was reviewed to highlight the theoretical background.
- Previous research relevant to critical reading was reviewed to
determine the appropriate strategies and their teaching methods.
- Some teaching strategies namely: questioning and group discussion
were selected to be employed through the interactive digital
storytelling environment.

67
Chapter Three Methodology

- Appropriate example situations (critical stances) were selected

from the reader book “Journey to the center of the Earth” to help
pupils practice critical reading.
- Twelve classroom sessions were prepared including all the teaching
materials, activities and the computer software.
For preparing the interactive digital storytelling environment, the
researcher considered common criteria for preparing educationally
useful digital storytelling (Barrett, 2006; Bull & Kajder, 2005;
Lowenthal, 2009; Miller, 2010; Nilsson, 2008; Robin , 2006; Robin ,
2008a and Robin , 2008b)
1. Point of View: What is the main point of the story and what is the
perspective of the author?
2. A Dramatic Question: A key question that keeps the viewer's
attention and will be answered by the end of the story.
3. Emotional Content: Serious issues that come alive in a personal and
powerful way and connects the audience to the story.
4. The Gift of Voice: A way to personalize the story to help the
audience understand the context
5. The Power of the Soundtrack: Supporting and embellishing the
story.
6. Economy: Using just enough content to tell the story without
overloading the learner.
7. Pacing: The rhythm of the story and how slowly or quickly it
progresses.

68
Chapter Three Methodology

The researcher received the permission from Campfire.co.in Graphic

novel Publishers to use the novel for research purposes. (Appendix 9)

Description of the Strategies Utilized in the Study

- The researcher used questioning and discussion strategies as
teaching strategies for both the control and the experimental group.
The researcher selected them to be used along with the digital
storytelling, because they are closely related and serve pedagogical
purpose in classroom (Dillon, 2004). The control group will be
taught according to MOE teacher’s guide, which emphasizes the
use of questioning and discussion strategies.
(1) Questioning:
- The teacher can employ questioning as pre, while and post reading
stages. Questions at the levels of (analysis, evaluation and
synthesis) are expected to address critical thinking. He identified
questioning steps as: (stating lesson goals, planning the question,
asking the question, allowing wait time, listening to pupil's answer,
evaluating pupil's answer, following up the pupils’ response with
another question, and re-planning based on pupils' responses). The
researcher considered these steps in teaching critical reading in
classroom.
(2) Group Discussion
- Group discussion were directed to address critical reading; pupils
should be assigned roles as: (Discussion generator, Character

69
Chapter Three Methodology

analyst, Conflict identifier, and Language analyst, and Investigator

and connector).
- Discussion generator: should relate the story details to the main
theme, values and main ideas in the story.
• Character analyst, should focus on evaluating character’s
personality and their attitudes towards situations and others.
• Conflict identifier: should focus on struggles, difficulties and
problems that faced the heroes, how there overcame them and if
there were other ways for the solution.
• Language analyst should try to guess unknown vocabulary, find
meaning, or look up the dictionary.
• Investigator and connector: should connect the story events to
learners’ real life situation.
- Basically, dividing the class into matched groups is a direction
from English Language Supervision in MOE according to the
Comprehensive Evaluation Program. Then, the researcher
instructed the pupils to take these roles while discussing the events.
He asked them to change roles every session. Within group work,
pupils can help each other to achieve the goals.

Preparing the Teacher’s Guide

A teacher’s guide was prepared in the light of literature review,
the results of the evaluation checklist, teachers’ observations, and
experts guidance. The teacher’s guide aimed at directing teaching
towards the objectives assigned. The teachers’ guide included the

70
Chapter Three Methodology

introductory section, in which critical reading skills were explained

and defined. The two teaching strategies (questioning and group
discussion) were explained in detailed instructions. Then twelve
lesson plans were suggested. The researcher considered stating
teaching objectives operationally, providing the directions to run the
digital storytelling software, explaining teaching strategies, guiding
evaluation and assignments.

Teaching Procedures
Teaching critical reading using an interactive digital storytelling
environment consisted of twelve sessions – 90 minutes each - taught
through two semesters (6 sessions every semesters). The experiment
began in the 16th, October, 2016 and lasted till 7th, March, 2017.
A-The Experimental Group: Studying Critical Reading Skills
through the Interactive Digital Storytelling Environment.
At the beginning of each semester of the year 2016/2017, the
critical reading skills pre-test was administered to the control and
experimental groups before the experiment to measure pupils’ critical
reading skills. The experiment lasted 12 sessions – one session per
week- as 6 sessions in the first semester and 6 sessions in the second
one, 90 minutes each. The experimental group studied the storybook
“Journey to the center of the Earth” through the interactive digital
storytelling environment. The critical reading skills posttest was
administered to the experimental group after teaching critical reading
through metacognitive strategies training.

71
Chapter Three Methodology

B- The Control Group: Studying Reading Comprehension by the

Traditional Instructional Method.
At the beginning of each semester of the year 2016/2017, the
critical reading skills pre-test was administered to the to measure
pupils’ critical reading skills. The experiment lasted 12 sessions – one
session per week- as 6 sessions in the first semester and 6 sessions in
the second one, 90 minutes each. The control group studied the
storybook “Journey to the center of the Earth” by the traditional
instructional method, through intensive reading, classroom
questioning, group discussion, and writing assignments. The critical
reading skills posttest was administered to the control group after
teaching reading comprehension.

Evaluation System
The evaluation system is composed of two forms:
1- Formative evaluation:
(a) Teacher's evaluation through questions related to every example
and cases included in chapter.
(b) Peers’ evaluation, error correction and inquiry during their
classmates’ explanation and practice with teacher's guidance and
direction.
2- Summative evaluation: It was conducted at the end of the
experiment taking the form of the post administration of the critical
reading skills posttest to the control and experimental groups.

72
Chapter Three Methodology

Instruments
The researcher designed an evaluation checklist for evaluating
the extent to which the textbook incorporates critical readings skills.
Preparatory stage teachers completed the evaluation checklist.
Accordingly, the researcher designed a critical reading skills test to
measure the pupils' performance in applying critical reading. Pupils
have to answer test items in one hour and a half.

The Evaluation checklist

Designing the Evaluation checklist

For evaluating in-use language learning material, the researcher

preferred evaluation checklists method. This method is more economic
and systematic compared to other methods and it can save time and focus
on a particular purpose. Evaluation checklists depend on teachers or
experts to complete evaluation checklists for determining the
appropriateness for specific contexts. In this research, the evaluation
checklist will be used to determine the extent to which the story book
is appropriate for improving critical reading skills.

Categorization of the Evaluation checklist

Critical reading skills will be used as the categories of the

checklist, namely: (considering the author's viewpoint / Drawing
inferences/ differentiating fact from opinion/ evaluation/ comparing
and contrasting). These categories will be the headers of the checklist.
Beside every category, the operational definition of every skill is

73
Chapter Three Methodology

included so participants can fully understand the skill they are

evaluating.

Domains of the Evaluation checklist

The main elements of textbooks are objectives, content,

examples, and exercises. These elements were considered in
developing the checklist. Consequently, the domains of the checklist
include the four elements (objectives, content, examples, and
exercises) under each skill “the five critical reading skills”. Thus, the
checklist included (20) items – (see appendix 7).
Table 5
An Example of a Category in the Evaluation checklist
- Evaluation* Critical Reading
- “The reader’s ability to make judgment on :Skills
an object in the text e.g. character, situation :Definition
or a piece of writing”
Very poor Poo Adequate Good Excellent Grade
*** r *** *** *** Textbook elements
***
** Objectives
** Content
** Examples
** Exercises
** Domains * A Skill
*** Scale

Scoring the Evaluation checklist

The researcher assumed following the scoring method as giving

every scale a mark e.g. (Excellent = 4, Good = 3, Adequate = 2, poor
= 1, and very poor =0). For every item, the frequency and the

74
Chapter Three Methodology

percentage of each mark are calculated. Under each skill, the average
scores for items under each domain is calculated, this gives a
percentage of every category. Then, the collected data can be input
into Statistical Package for Social Science “SPSS” – version, 18 - to
calculate the frequencies. Thus, descriptive statistics can show the
level to which the textbook addresses critical reading skills (Excellent,
Good, Adequate, poor, or very poor).

Validity of the Evaluation checklist

The Referee Validation:

The researcher submitted the checklist to a panel of referees to
establish its validity through their opinions regarding the suitability of
the items. Minor modifications of the referees were applied to the final
form of the checklist.
Internal Consistency Validation:
The internal validity coefficient is computed by the Pearson
formula (Larson-Hall, 2009). The correlation between the scores of
every single skill to the total score was calculated. The checklist is
computed by using the SPSS. (Version, 18)
Table 6
Correlation Coefficient for the Five Skills with the Total Score of the
Evaluation checklist

75
Chapter Three Methodology

Pearson
Critical Reading Skills
Correlation
Recognizing the author's viewpoint / purpose .685*
.Evaluation .743*
Differentiating facts from opinion .703*
Drawing inferences .774*
Comparing and contrasting .722*
*. Correlation is significant at the 0.01 level (2-tailed).
Reliability of the Evaluation checklist

The researcher used Cronbach’s Alpha Formula to examine the

reliability among items of every skills in the evaluation checklist. As
shown in Table (7) the evaluation checklist is reliable.
Table 7
Total Reliability Analysis Scale (Alpha) for the Evaluation checklist
Critical Reading Skills Alfa ()
Recognizing the author's viewpoint / purpose .513
.Evaluation .629
Differentiating Facts from Opinion .643
Drawing Inferences .716
Comparing and Contrasting .782

Moreover, the researcher examined the reliability among the

total of every skill which showed that the Alfa’s () value was 0.786.
Thus, the evaluation checklist is reliable.

Besides, the split-half method was used to confirm the

reliability. Cronbach's Alpha, Correlation between forms (Spearman-
Brown Coefficient and Guttman Split-Half Coefficient) showed that
evaluation checklist is reliable.
Table 8

76
Chapter Three Methodology

Total Reliability Analysis Scale (Split-Half) for the Evaluation

checklist

Cronbach's Alpha Part 1 .677

Part 2 .664
Correlation Between Forms .821
Spearman-Brown Coefficient Equal Length .902
Unequal Length .902
Guttman Split-Half Coefficient .759
Respondents of the Evaluation checklist

Respondents of the checklist are preparatory stage EFL teachers

with experience between 3-15 years in teaching the storybook
“Journey to the Centre of the Earth”. During a training course, (31)
teachers were given a copy of the checklist to decide if the textbook
addresses each critical reading skill on a scale as (excellent, good,
adequate, poor, or very poor).

Critical Reading Test

Objectives of the Test

In the light of the results of the evaluation checklist, the

researcher decided the main objectives of the test as measuring the
pupils' achievement of some specific critical reading skills
(recognizing the author's viewpoint or purpose, drawing inferences,
differentiating fact from opinion, evaluation, and comparing and
contrasting)

77
Chapter Three Methodology

Test Format

The Test was determined to be a subjective achievement test that

included wh-questions (requiring short answer), the researcher
preferred this type of test items for the following reasons: (a) Wh-
question items is the type included in the final exam specifications,
book activities and modal exams (Ministry of Education -Egypt,
2014b), (b) Divergent and open-ended questions are appropriate for
higher-order questions, which require the reader to draw inferences
and evaluation, thus can be used for critical reading (Badger &
Thomas 1992; Brookhart 2010; Ehara, 2008; Haines, 2013; Tofade,
Elsner & Shaunessy, 2005.p.35x)

Preparing and Organizing the Test Items and Instructions

After determining test objectives, items type, and critical reading

skills that should be tested, the next step was to construct items and
write directions. Therefore, a pilot test was administered to a sample
of equivalent pupils (3rd year – preparatory stage pupils in the school
year (2015/2016) in three schools in Kafrelshiekh governorate. The
examinees' common errors and misconceptions shown in their
responses to the test were taken into consideration in modifying and
deleting test items.
The critical reading test consisted of (66) items to cover 10
chapter in the storybook “Journey to the Centre of the Earth” The
researcher prepared a table of specifications of the test illustrating the
distribution of test items according to the critical reading skills

78
Chapter Three Methodology

assigned to be taught and tested. Moreover, for organizing test items,

the researcher considered beginning the test with the simplest items
and moving to the most difficult according to the difficulty index.
Table 9
Outline of the Critical Reading Test
Subtes Critical Reading Sub-Skills First Second Number
t semester semester of Items
1 Recognizing the author's 5 8 13
viewpoint / purpose
2 .Evaluation 6 6 12
3 Differentiating Facts 5 7 12
from Opinion
4 Drawing Inferences 6 5 11
5 Comparing and 2 6 8
Contrasting
Total 24 32 56

Scoring Method of the Critical Reading Test

The test items are open-ended questions. The researcher

suggested the mark to every item as three. As for subjective test items,
two independent scorers are needed, their scores should be passed on
one senior expert to investigate discrepancies. Therefore, the
researcher and another teacher-researcher scored the test. For the
inconsistency of the scores of some items between the two scorers, a
senior teacher can give the final decision.

Item Analysis:
a- Item Difficulty Indices
To ascertain the appropriateness of the test items for non-
specialist EFL learners, difficulty indices were calculated for each

79
Chapter Three Methodology

item using the statistical package (SPSS, version 18). These indices
are expressed as the proportion of participants passing each item. It
would seem necessary to clarify that the larger the proportion getting
the item correct, the easier the item, that is the larger index of
difficulty. To calculate item difficulty indices of the test items, the
researcher used the SPSS (version. 18) software.
Accordingly, the difficulty index for each test item is displayed
in table (3.9). For efficiency in measurement, Downing and Haladyna
(2006) pointed out that items of appropriate difficulty are those with
difficulty indices ranging from 0.20 to 0.70. Consequently, starred
items were deleted.

Table 10
Difficulty Indices of the Critical Reading Test
Difficulty Ite skill Difficulty Ite skill
index m index m
0.08 * 29 0.42 1
0.25 30 0.15 * 2
0.10 * 31 0.40 3
0.48 32 Differentiatin 0.17 * 4
0.06 * 33 g Facts from 0.50 5 Author's
0.08 * 34 Opinion 0.35 6 viewpoint /
0.65 35 0.15 * 7 purpose
0.44 36 0.31 8
0.35 37 0.13 * 9
0.35 38 Drawing 0.25 10
0.38 39 Inferences 0.44 11
0.50 40 0.48 12 Evaluation.

80
 Chapter Three Methodology

0.04 * 49 0.48 21
0.56 50 0.23 22
0.69 51 0.13 * 23
Differentiatin
0.35 52 0.10 * 24
g Facts from
0.10 * 53 0.25 25
Opinion
0.13 * 54 0.13 * 26
0.15 55 0.46 27
0.44 56 0.63 28
b- Items Discrimination Power.
In this step, consideration was also given to test items’
discriminating power. The discriminating power indicates the degree
to which an item differentiates between candidates obtaining high and
low scores on the test. In general, the better items display positive
high discriminating power that is candidates with high total scores
answer the items correctly. Discrimination Index was calculated by
using the statistical package (SPSS, version, 18). Results from item
analysis are given in Table (11).
Table 11
Discrimination Indices of the Critical Reading Test
Discriminatio skill Discriminatio skill
n Item n Item
index index
0.33 38 0.58 1 Author's
0.50 39 0.67 3 viewpoint /
Drawing
0.67 40 0.67 5 purpose
Inferences
0.25 41 0.42 6
0.50 44 0.50 8
0.33 45 Comparing 0.17 * 10

81
Chapter Three Methodology

0.50 50 0.50 11
0.50 51 0.42 12
0.42 52 0.50 13
0.75 56 0.33 14
0.25 16
Evaluation.
0.42 18

0.25 21
and
0.08 * 22
Contrasting
0.08 25
0.25 27
0.33 28 Differentiatin
0.17 * 30 g Facts from
0.15 * 31 Opinion
0.25 32
0.42 35
0.33 36
0.42 37
Table (11) indicated that most test items are found to be of
appropriate discriminating power and displayed positive
discrimination. Again, Marshall and Hales (1972) made it clear that
test items with discriminating indices between 0.20 and 0.40 are of
some value in discriminating between candidates. Test items with
discriminating indices between 0.40 and 0.60 are good discriminators
and those with indices above 0.60 are unusually good items.
Therefore, test items No. 10, 22, 30, and 31 seemed to have no
appropriate discriminating power. Thus, these items were eliminated
in the final form of the test.
Table 12
The Outline of the Final Form of the Test
Subtes Critical Reading Sub-Skills First Second Number

82
Chapter Three Methodology

t Semester Semester of Items

1 Recognizing the author's 3 3 6
viewpoint / purpose
2 .Evaluation 3 4 7
3 Differentiating Facts 4 3 7
from Opinion
4 Drawing Inferences 3 2 5
5 Comparing and 1 4 5
Contrasting
Total 14 18 30
Optimum Time Limit for the Test

In this concern, Livingston (2006) recommended giving as much

time as possible to the testee to improve the reliability of the test.
Since the items are Wh-questions and pupils need time for reading and
writing the answers, hence, the researcher allowed 5 minutes for each
item of the tests. To add, the time required for 90 percent of the pupils
to finish the test was used as the basis for determining optimum time
limit for the experimental test form. In the first semester, the final
form included (14) items, thus, pupils were given (70) minutes. In the
second semester, the final form included (16) items. Thus, pupils were
given (80) minutes.

Validity of the Critical Reading Test

a- Content Validity:
The Researcher submitted the test to a panel of jury to establish its
validity through their opinions regarding the suitability as well as the
difficulty of the items. They were asked to review the text item
according to: (1) The relation of the statements to the research
objectives. (2) The clarity of the meaning of the statement to the

83
Chapter Three Methodology

respondents. (3) The appropriateness of the wording to the intended

respondents. (4) Add any phrase or statement that seems to be fit, and
(5) Delete any inappropriate phrase or statement. Some modifications
were suggested by the panel of the jury. In the light of these
suggestions, the test was modified in its final form to be administered.
b- Internal Consistency of the Critical Reading Test
The internal consistency (correlation coefficient) between the
pupils total scores and pupils' scores on every critical reading skills
were calculated using Pearson's correlation coefficient formula.

Table 13
Correlation between the Scores of Critical Reading Skills and the
Total Score
Correlation with
Critical Reading
Total Test Score
Recognizing the author's viewpoint / purpose * 850.
.Evaluation * 806.
Differentiating Facts from Opinion * 736.
Drawing Inferences * 785.
Comparing and Contrasting * 788.
* significant at the 0.01 level

A third procedure was employed to ascertain the validity of this

test. This is referred to as the intrinsic validity (El-Bahay, 1979, p.
553, in Arabic). This kind of statistical validation obtained from the
index of reliability. It is calculated from the root square of the

84
Chapter Three Methodology

reliability coefficient. The reliability coefficient of the test was 0.804.

Hence, the Intrinsic validity = √0.804= 0.896 . Thus, the results made clear
that the test is a valid measure of critical reading skills for EFL 3rd
year preparator stage pupils.

Critical Reading Test Reliability

To assess the critical reading skills achievement test reliability,

the Cronbach's Alpha () formula was used (Cronbach, 1951).

Table 14
Total Reliability Analysis Scale (Alph) for the Critical Reading Test
Critical reading skills Alfa ()
Recognizing the author's viewpoint / purpose 720 .0
.Evaluation 558 .0
Differentiating Facts from Opinion 567 .0
Drawing Inferences 581 .0
Comparing and Contrasting 563 .0
Furthermore, the researcher examined the reliability among the
totals of every skill which showed that the Alfa () = 0. 804. Hence,
the test is reliable.
Besides, the split-half method was used to confirm the
reliability. Cronbach's Alpha, Correlation between forms (Spearman-
Brown Coefficient and Guttman Split-Half Coefficient) showed that
the test is reliable.

85
Chapter Three Methodology

Table 15
Total Reliability Analysis Scale (Split-Half) for the Critical Reading
Test
795. Part 1 Cronbach's Alpha
591. Part 2
899. Correlation Between Forms
947. Equal Length Spearman-Brown Coefficient
947. Unequal Length
843. Guttman Split-Half Coefficient
Data Distribution for the Critical Reading test

The researcher checked whether to use parametric or non-

parametric statistics. To do so, the data distribution statistics
(Skewness, Kurtosis, and Kolmogorov-Smirnov) were used (Larson-
Hall, 2009; Turner, 2014).

Table 16
Data Distribution for the Critical Reading Pre and Post Test for the
Control Group
(A) Data distribution of the pretest
Kolmogorov-
Skills Skewness Kurtosis Smirnov
Z Sig.
Recognizing the author's viewpoint -.222 -.544
/ purpose 0.778 0.58
.Evaluation -.389 -.790 1.002 0.268
Differentiating Facts from Opinion .571 -.368 0.839 0.482
Drawing Inferences -.365 -.208 0.778 0.58
Comparing and Contrasting -.126 -1.101 0.884 0.415
Critical Reading as a whole .019 1.012 0.971 0.302
(B) Data distribution of the posttest
Recognizing the author's viewpoint
-.323 -1.209 0.895 0.399
/ purpose

86
Chapter Three Methodology

(A) Data distribution of the pretest

Kolmogorov-
Skills Skewness Kurtosis Smirnov
Z Sig.
Recognizing the author's viewpoint -.222 -.544
/ purpose 0.778 0.58
.Evaluation -.389 -.790 1.002 0.268
Differentiating Facts from Opinion .571 -.368 0.839 0.482
Drawing Inferences -.365 -.208 0.778 0.58
Comparing and Contrasting -.126 -1.101 0.884 0.415
.Evaluation -.137 -.834 0.611 0.849
Differentiating Facts from Opinion -.093 -.887 0.67 0.76
Drawing Inferences .078 -.767 0.944 0.335
Comparing and Contrasting -.586 -.559 0.977 0.295
Critical Reading as a whole -.297 -1.189 0.837 0.486
*. Test distribution is Normal.

Table 17
Data Distribution for the Critical Reading Pre and Post Test for the
Experimental Group

87
Chapter Three Methodology

(A) Data distribution of the pretest

Kolmogorov-
Skewne Kurtos
Skills ss is
Smirnov
Z Sig.
Recognizing the author's viewpoint -.022 -.761
/ purpose 0.734 0.654
.Evaluation -.743 .069 0.987 0.284
Differentiating Facts from Opinion -.210 -.649 0.66 0.777
Drawing Inferences -.084 -1.128 0.829 0.497
Comparing and Contrasting -.097 -.891 0.926 0.358
Critical Reading as a whole -.264 -1.088 0.84 0.48
(B) Data distribution of the posttest
Recognizing the author's viewpoint
-.202 -.456 0.511 0.956
/ purpose
.Evaluation .156 -.940 0.778 0.58
Differentiating Facts from Opinion .709 .676 1.079 0.195
Drawing Inferences .028 -.819 0.603 0.86
Comparing and Contrasting -.118 -.758 0.652 0.789
Critical Reading as a whole -.253 .072 0.533 0.939
*. Test distribution is Normal.
Accordingly, if skewness ratio is less than ±2 and kurtosis value is
less than ±3, normality is not violated (Larson-Hall, 2009; Turner,
2014).
For normal data distribution, parametric statistics can be used.
Since the main purpose of this step was to investigate the effect of
using an interactive digital storytelling environment on improving
critical reading for 3rd year preparatory stage pupils, to compare the
performance of the control and experimental groups the t-test for
independent samples was used (Press, 1992). To calculate the effect
size (η2) of the treatment, the following formula has been used (Pierce,
Block, and Aguinis, 2004).

88
Chapter Three Methodology

Researcher's Observations during the experiment

During teaching, the researcher noticed that:
- Researcher noticed irregular school attendance by some pupil. He
decided that the inclusion criteria of the participants should be
attending 80% of classes and attending pre-and posttest in both
semesters.
- During earlier stages of the experiment, some pupils reported that
they were shy to express their views during group discussion, this
required the researcher to devote time before the session to some
pupils to overcome their shyness by encouraging them, help in
planning and preparing lessons, and showing the benefit of
practicing such activities.
- The subjects of the experimental group were very motivated and
highly interested in the participation in classroom. The motivation
was seen in their eagerness for practicing the new activities.
- At the end of the interactive digital storytelling environment
sessions, the pupils expressed their desire to have similar an
interactive environment for other skills like writing, reading and
speaking.

89