CHAPTER THREE

METHODOLOGY
3.0 INTRODUCTION
The research related to critical reading teaching in EFL environment have provided
some ideas on how to fill the gap discussed in the previous chapter.This study aimed
to investigate the impact of Production-oriented reading approach (PORA) on the
critical reading of various texts by Chinese non-English major EFL undergraduates.
Furthermore, it focuses on whether PORA teaching improves students’ reading
comprehension, especially critical reading comprehension, and students’ attitudes
toward PORA. In the process of carrying out research, it is very important to have a
set of sound, feasible and effective methods to ensure the success of research.This
chapter introduces the methods of this study. It will present and explain the research
design, instruments, data collection procedures and analysis methods. Sampling
procedures and efforts to ensure reliability and validity are also explored in this
chapter.
3.1 Research Questions and Hypotheses
The following research questions posed investigate whether PORA is effective in
influencing critical reading of Chinese non-English major EFL undergraduates.
Research questions specifically focus on literal, reasoning, and critical reading
comprehension, as well as student attitudes toward PORA. Research questions are as
follows:
1. What difficulties are faced by the Chinese non-English major EFL undergraduates
in reading comprehension?
2. Does the PORA promote a significant increase in the overall reading of Chinese
non-English major EFL undergraduates beyond what would normally be achieved
through normal instruction?
3. Does the PORA promote a significant increase in Chinese non-English major EFL
undergraduates literal, reasoning and critical reading comprehension?
4. What is the attitude of Chinese non-English major EFL undergraduates towards
PORA approaches to support reading comprehension?
The following is a set of null and alternative hypotheses corresponding to the
research questions mentioned above:
H01. There was no significant difference in overall reading comprehension changes
between the PORA group and the control group.
H02. There was no significant difference in the changes in literal reading
comprehension between the PORA group and the control group.
H03. There was no significant difference in the changes of reasoning reading
comprehension between the PORA group and the control group.
H04. There was no significant difference in changes in critical reading comprehension
between the PORA group and the control group.
Ha1. There were significant differences in changes in overall reading comprehension
between the PORA group and the control group.
Ha2. There were significant differences in the changes in literal reading
comprehension between the PORA group and the control group.
Ha3. There were significant differences in the changes of reasoning reading
comprehension between the PORA group and the control group.
Ha4. There were significant differences in changes in critical reading comprehension
between the PORA group and the control group.

3.2 RESEARCH DESIGN

Proper study requires a well-designed research method so as to identify the answers
and solutions to the problems and research questions respectively. Researchers need
to find, design, and use appropriate research designs. The choice of using the
research method is mainly determined by the source of the collection.
The design used in this study is a multi-method design. It refers to two or more
research projects, each completed by itself, to solve research problems, hypotheses,
topics or procedures. As with mixed methods, multi-method design may be
quantitative, qualitative, or a combination of both (Martha & Mendes, 2007). These
projects can be implemented simultaneously or sequentially. However, unlike the
mixed approach, each research project of multi-method design is planned and
implemented independently to answer specific sub-questions. On contrary, a mixed
approach refers to the use of two or more quantitative and/or qualitative strategies
in a single research project. In other words, the mixed approach is utilized when a
single study uses multiple or mixed strategies to answer research questions and/or
test hypotheses, with strategies implemented simultaneously or sequentially. Mixed
approach research is more specific because it includes a mix of qualitative and
quantitative data, methods, methods and/or paradigms in a study or set of related
studies. One could argue that mixed method research is a special case of multi-
method research. Multi-method research, therefore, has been described as using
different methods or methods in parallel or sequence, but not integrated before
making inferences (Johnson, onwuegbuzie and Turner, 2007).
To answer the first research question, a quasi-experimental design was employed.
Quasi-experimental methods can show causal relationships between independent
and dependent variables, which require random assignment to equivalent
experimental and control groups (Fraenkel, Wallen, and Hyun, 2016). This study will
enroll two non-English major undergraduates with similar class sizes and English
proficiency. One class will be randomly assigned to the experimental group and the
other class will be assigned to the control group. Students in the experimental group
will be instructed through PORA instruction, while the control group will be taught
using traditional teacher-centered instruction. Both groups will take the same pre-
and post-test; the dependent variable will reveal whether PORA improves students’
literal, reasoning, or critical comprehension.
In order to answer the second research question, this study will use a questionnaire
to measure student attitudes towards PORA. The questionnaire was adapted from
Fan's 2010 study on reading comprehension. The questionnaire consisted of 30 items
in a 6-point Likert scale format. As Fraenkel, Wallen, and Hyun (2016) put it, “A
commonly used attitude scale in educational research is the Likert scale”. The Likert
scale is a self-report tool in which individuals respond to a series of statements by
indicating a degree of agreement (Fraenkel et al., 2016). In this study, students in
the experimental group will be asked to determine their attitudes towards PORA-
related issues by agreeing or disagreeing with statements on a scale of 1 to 6, where
1-strongly disagree, 2-disagree, 3-slightly Disagree, 4-basically agree, 5-agree to 6-
strongly agree.
3.2.1 Rational of design
Research starts with the problem to be solved, and research questions based on the
problem would guide the research to find possible solutions or answers to the
problem at hand. As far as this research is concerned, the common problem found is
that Chinese EFL undergraduate learners find it difficult to read critically, and
teachers find that the teaching of critical reading is complicated and demanding. A
careful study of the problem through literature review found that the problem is from
the selection of materials which is either random or too difficult for learners. The
traditional teaching methods cannot effectively train the students’ critical reading
ability, and there is no effective means to evaluate the students’ critical reading
performance.
As mentioned in Chapter 2, the research of Albeckay (2014) shows that quantitative
research on instructive guidance and critical reading ability is supplemented by
qualitative data collection and qualitative analysis to explore different perspectives
that participants must provide. Astroth and Chung (2018) in their research design
used a quantitative method to show that strategic guidance would significantly
improve students’ performance in critical reading, and their quantitative study was
conducted to find out the perceptions of non-native English teachers in Turkey in
teaching critical reading in their lessons. In addition, previous research of Albeckay
(2014) suggest that a more in-depth and detailed study of variables that hinder and
promote the use of critical reading skill is needed to study the effectiveness of the
reading model. Qualitative and quantitative methods were adopted to analyse the
collected data. Mixed methods of qualitative method as well as a quantitative method
are adopted, with the former helping understand and appreciate the participants’
subjective view while the latter helping quantity the data collected (Albeckay, 2014).
De Pasquale and Meunier (2003) argued that categorisation techniques in computer-
assisted reading and analysis of texts. The text evaluator tool would help teachers
and test developers select texts for use in instruction and assessment according to
Sheehan, Kostin, Napolitano and Diane (2014). They argue that computer systems
offer particular measurement to ensure that EFL students to read texts at steadily
increasing complexity levels as they progress through school, so that all students
acquire the advanced reading skills needed for success in college and careers.
The basic principle of multi-method design is based on different research questions
that need to be answered at each stage. First, in order to have a rich and in-depth
understanding of the problems faced by teachers and students in reading teaching,
researchers need to immerse themselves in the context of the problem. Qualitative
methods, namely focus group interviews and student essay analysis, will be used to
gain a richer and in-depth understanding of the problem. The first or preliminary
stage is also regarded as the diagnosis or needs analysis stage. The word “diagnosis”
is used to represent the first stage because the act of discovering the nature of the
problem does not perform well in writing and determining what needs to be done to
solve the problem.
Secondly, using the interviewee’s information and adopting a multi-method
approach, researchers can develop and test production-based reading intervention
modules according to actual needs, and possibly solve the problem raised by
teachers and students. This will be carried in the Phase 2 of module development
and the Phase 3 of implementation. The second phase and third phase will be
quantified through expert evaluation, namely, Likert scale evaluation, and phase 3
will be carried out through pre-test and post-test. After the post-test, the final semi-
structured interview will be conducted, through the review of teachers and students
and the analysis of thesis documents. Inferences will be made at the end of each
phase, and the results will be consolidated at the end of the third phase.
In addition, the basic principle of using this design is also based on previous research
on the subject of the SAMPLE-based model. Most areas have adopted quantitative
design and provided more general results. Besides, the reason for using this design
in this study is based on the limitations and recommendations of past and current
research.
3.1.2 Philosophy, Epistemology and Ontology of the Study
As mentioned in Chapter 1, the reason why this study uses a multi-method design is
that, compared to using only one type of quantitative data, having both quantitative
and qualitative data will provide a more accurate and adequate understanding of the
research problem. Since two methods are used in this study, the related literature on
multiple methods (Foster, 1997; Niehaves, 2005; Bryne & Humble, 2007) suggests
that each design should be implemented according to its nature and should be
carried out continuously. In this way, both methods can stick to their own paradigm.
Hence, this research will solve this problem from the perspectives of interpretivism
and positivism. This solution will resolve the paradigm conflicts of multi-method
research.
Therefore, based on the above-mentioned recommendations of previous researchers,
researcher in this study will use two different worldviews corresponding to the stages
involved in the research. The interpretive paradigm is a method that aims to gain a
deeper understanding of a particular phenomenon and its complexity in its unique
context, rather than trying to generalize the understanding of the entire population
(Creswell, 2007). The research would start from an interpretive perspective, because
she observed her students through the perspective of an English teacher and
assumes that in the teaching and learning of skills such as reading, human behavior
and their perceptions of learning experience would shape their learning processes
and gain desirable learning outcome. Both learning and teaching experiences of
learners and teachers are subjective.
However, as a researcher, explaining a person’s understanding of the study process
through the perspective of an experienced teacher may be affected by prejudice or
biasness. Since critical reading skills and students’ performance in training of reading
skill need to be empirically measured, the researchers hypothesizes that in order to
improve students’ performance of critical reading, tools for textual complexity
adjustment and production facilitating means are necessary are required. The
purpose of this assumption is to not only psychologically motivate learners’ to read
but also promote their critical reading in action. Because this is the researcher’s
personal views and experience, this cannot summarize everyone’s thoughts. In this
way, it is not enough to rely solely on the experience and opinions from just one
English teacher, since she will be subject to a variety of explanations from different
people. Acknowledging this biased tendency and seeking the truth to support the
researcher’s research point of view, the positivist point of view is included in the
research.
According to Creswell and Plano Clark (2018), the post-positivist view refers to
researchers make inferences or judgement based on causal analysis, reductionism,
detailed observations and variable measurement, as well as continuous improvement
of test theories. Through a post-positivist world view, researcher takes reality as a
single unit and independent of the researcher. The post-positivist perspective would
be applied to the quasi-experimental stage of research, where the validity, relevance
and effectiveness of the modules are tested through expert assessment and
students’ production scores after doing critical reading. Researcher will work based
on the information collected in the first phase, which guides the development of
intervention modules, and continue to seek to reject or fail to reject the research
hypotheses described in Chapter 1.
Therefore, by using these two worldviews in successive stages of each method, the
researcher will be able to assess whether the interventions facilitated by the
teacher’s perspective are effective, and whether they are any factors or variables
that may cause this.
3.1.3 Multi-method desigh sequence
The choice of methods and the way of combining methods are based on Morgan’s
(1998) practical suggestions on how to combine research project methods. Morgan
(1998) suggested that in order to have a practical method for multi-method design,
researchers need to decide based on which method becomes the main method and
then the supplementary method. The decision on which method to use is based on
Morgan’s prioritization model. The prioritization model requires researchers to
determine the priority of quantitative or qualitative methods as The main method.
After selecting the main method, the researcher needs to decide to rank the methods
to determine whether the supplementary method is a preliminary or a follow-up to
the main method. The reason why this model is used as part of the decision-making
process is because of its practicality. Since the main purpose of this research is to
propose a form of intervention to solve the critical reading difficulties of Chinese EFL
students, this research chooses the qualitative methods of focus group interviews
and document analysis as the preliminary methods and supplements the main
methods. This is quasi-experimental because The intervention module will be
quantitatively tested. Careful reading of focus group interviews and student essay
analysis enables researchers to develop content for writing intervention modules.
The research will be followed up by semi-structured interviews and student essay
analysis using another qualitative method to help explain if there are outliers or
unintelligible results (Morgan, 1998). A further description and explanation of the
method will be provided in the research procedure section. The priority sequence
model of the combined method process is shown below.
3.3 Target Population and Sample
In order to solve the research questions and hypotheses of this study, the researcher
needs to participate in a sampling procedure that involves determining the research
site and its participants. In this section, the sampling procedure will be introduced
according to the stages.
3.3.1 Research Site
In this study, a public university in Hunan province China would be identified as a
research site for the implementation stage. There are 7000 students in the
university, and the number of students of sophomores is about 2000. The
experimental group (EG) and the control group (CG) are selected from 2 parallel
computer science classes taught by the same teacher. This includes 45 boys and 5
girls. Their average age is 19 years. And they receive half a year of college
comprehensive English study. Through SPSS.26.0 analysis, this experiment subjects
would select from theses two parallel classes who show no significant difference in
their scores of English test. One week before the experiment, 50 students and 50
students participate in the pre-test of the experiment. The test results show that
there is no significant differences of critical reading ability in the two classes before
the experiment.
The main reason for choosing freshmen of non-English majors from this university is
that critical reading is a high-level reading comprehension ability, and after half a
year of college education, students’ thinking and logical reasoning ability would be
improved by receiving professional study. In the mean time, the two classes enjoy
the following two similarities. Firstly, subjects in both classes is almost the same with
50 students in EG (45 males, 5 females) and 50 students in CG (45 males, 5
females). Secondly, subjects in both classes are in the same classroom setting,
sharing the same textbooks and being taught by the same teacher. Thirdly, their
reading lessons are arranged in the morning at almost the same period of time. Last
but not the least, the two classes have almost the same overall English proficiency
on the National English Matriculation Test (NEMT) and on the final test (FT) at the
end of the first semester in the college.
In the experimental group, Data driven SAMPLE-based reading teaching is conducted
to enhance students’ critical reading ability, while in the control group, the traditional
English reading teaching method is used as usual including linguistic knowledge as
vocabularies and grammars, etc.
3.3.2 Diagnostic phase sampling approach
The population is a larger comprehensive group of individuals with common
characteristics that are of interest to researchers (Fraenkel et al., 2012). Purposeful
sampling methods will be used to determine the population in this study. The most
common types of purposeful sampling are typical, unique, most variable, convenient
(Merriam & Tisdell, 2016). This study uses a typical and convenient purpose
sampling method to select participants. A typical purposeful sampling method
reflects ordinary persons, situations or instances of the phenomenon of interest
(Merriam & Tisdell, 2016); Convenience purpose sampling refers to sampling based
on time, money, location, and availability of locations or respondents to select the
sample (Merriam & Tisdell, 2016).
The university offers 3 semesters of college English course since the enrollment of
the students. There are three classes a week, and each unit is divided into intensive
reading class, listening class and speaking class. As to the students, most of the
students in these two classes started learning English in the third grade of
elementary school, so they have been learning English for 11 years. Only a few
students start learning English in the first grade of junior high school, so they have
been learning English for 7 years. Table 5 describes the details of the sample (EFL
learners).
To ensure that the critical reading levels of the two classes participating in the study
were similar, the researcher distributed the 2 questionnaires, namely “Questionnaire
on Reading Status of Non-English Majors” and “Questionnaire for self-assessment of
English critical reading ability” to the selected two classes. Then a lottery is selected
that class 1 is taken as the experimental group for critical reading intervention, and
class 2 is selected as the control group to learn according to the original teaching
plan and teaching methods.
The interview would be conducted at the end of the data collection process. It
includes interviews with 2 college TEFL teachers and 10 EFL students. The purpose
of this research to interview college English teachers is to understand their feelings
of using and not using Data driven SAMPLE-based teaching methods. The purpose of
teacher interviews is to collect data describing how teachers perceive Data-driven
SAMPLE based instructional modules and their evaluations. The in-depth and semi-
structured interviews with 10 students are to understand their learning effects under
intervention module. The purpose of student interviews is to gain a deeper
understanding of how students perceive the Data-driven SAMPLE-based teaching
modules that students are engaged in and their evaluation of it. These are structured
and semi-structured interviews, with all questions are prepared in advance.
3.3.2 Sampling Methods in Development and Pilot Phases and
Implementation Phases
In the module development stage, after the module is developed based on the
results of the first stage of diagnosis, the draft is tested by a teacher and a complete
classroom consisting of 50 learners. In the first session of the implementation phase,
it is tentatively planned that a teacher from selected department and her students
group of about 50 students would participate in the quasi-experimental research.
The number of participants or n is based on the sampling plan of Krejcie and Morgan
(1970). The table for determining the sample size can be evaluated in the appendix.
In the second session of the implementation phase, after analyzing the post-test
data, semi-structured interviews involving teachers and their students from the
intervention group will be conducted. After the completion of the module, their
critical reading performance changed significantly.
3.4 Research instruments
There are many ways to collect data in a study. Fraenkel, Hyun, and Wallen (2012)
interpret the term of instrument as the entire preparing process of data collection,
including the selection of instrument, the procedure and the conditions under which
the instrument will be managed. Creswell (2014) explains that an instrument is a tool
for measuring, observing or recording quantitative and qualitative data. The research
instruments used in the study include questionnaires in Chinese, English critical
reading test papers, and interviews with the teacher and her students. The
instruments used in this study will be explained according to the stage.

3.4.1 Phase One of Diagnosis

In order to answer the research questions in the first phase, the researchers will use
focus groups to collect information from teachers about their classroom practices
when teaching writing, the challenges they face when teaching students of critical
reading, and obstacles or promotion of critical reading in EFL classes.
The reason for using focus group interviews is that there is a need for richer data on
the factors that affect the reading teaching and learning, and the need to understand
the phenomena or situations that teachers face in critical reading teaching. The focus
group interview questions are based on the questions on critical reading in existing
literature. The researcher will also collect the students’ critical reading task to
provide a triangulation of the answers to the first research question. Students’
reading tasks will be analyzed according to the Critical Reading Ability Assessment
Scale, which can be referred to in Appendix 1.
The information collected from this stage will be used to develop the SAMPLE-based
module. A more detailed description of how to develop modules is in Section 3.3.2.
The module draft will be tested in the pilot phase to identify weaknesses or other
improvements in the module draft before using it in the main research phase. The
pilot and main research phases will require drafts of intervention modules, pre-test
and post-test critical reading tasks, and scoring criteria for critical reading results.
Module drafts and pre-test critical reading tasks can be evaluated in the appendix.
3.4.2 Phase two of developing and piloting of the module
The second phase is to develop a module for critical readers, and it will be
implemented by teachers. At this stage, data will be collected from the draft of the
module. After using the data from the focus group interviews and the existing
literature to develop the module draft, the module draft will be tested in the pilot
research phase. The data of the pilot test will be collected through performance
measurement; therefore, pre-test and post-test will be used to record students’
reading performance before and after the module is implemented.
The reason for using performance measurement indicators for data collection in the
pilot phase is that the researcher needs to study the effectiveness of this module on
students’ critical reading performance, and to understand whether students’ scores
before and after the exam have significantly improved after the implementation of
the module. After the draft is tried out, the draft will be improved and presented to
the expert for evaluation.
Two critical reading tests are involved in this study to check the English critical
reading comprehension ability and critical thinking ability of the students in the pre-
experiment stage and post-experiment stage. The reading comprehension contents
are selected from the SAT’s critical reading. George H. Hanford, president of the
College Board, considers SAT as the best way to test reasoning skills. In his opinion,
SAT’s purpose has been to provide a uniform, objective measure of academic
readiness for college-level work and to assess the academic performance and ability
of individual students to think critically. He argues that what SAT measures best are
the so-called “higher order” reasoning skills-skills that are of extreme importance if
today’s youngsters are to function successfully in an increasingly complex and
technological society (adapted from an article SAT is the Best Way to Test Reasoning
Skill by Hanford G.H.) Passages which are selected from SAT questions database
have been kept original without any modification. There are four texts (A, B, C, D)
with 20 questions in each test and the subjects are required to finish it within 40
minutes. The detailed information of the two tests is listed below including types of
writing, words number and source of the texts.
3.4.3 Phase three of implementation
As mentioned above, the implementation phase will be divided into two sub-phases.
The instruments of the first sub-phase are the piloted and validated versions of the
modules and the pre-test and post-test critical reading tasks. In the second sub-
phase, the instrument used is a module assessment, which will be a semi-structured
interview with teachers and selected students in the intervention group.
Interviews that conducted at the this stage consist of the teacher interview and
students interviews. The purpose of the teacher interview is to gather data that
describes how the teacher thinks about Data driven and SAMPLE-based teaching
intervention and her evaluation of it. The purpose of the students’ interviews is to
obtain a deeper understanding of how the students think about this module they are
engaged in and their evaluation of it. It’s a structured interview with all the questions
prepared in advance.
The types of questions for the teacher are: 1) How do you think about the new Data
driven and SAMPLE-based teaching module? 2) How does it differ from your previous
reading teaching? 3) What are the challenges you face when teaching reading in this
new way? 4) How do you think the reading lessons went in the past few months? 5)
What types of things going on in or outside of the classroom do you think influenced
the reading activities?
In order to know students’ problems and suggestions to the experimental teaching,
structured interviews with the students are also designed in a written way. There are
three different forms for students to fill in with yes or no question. One form is about
the problems they meet during the critical reading teaching, like vocabulary
deficiency, time shortage, oral expression difficulty and so on. One is about the
improvements they get from Data driven and SAMPLE-based teaching, such as
improvement in listening, reading or writing ability. The third one is about the
suggestions they give for the critical reading teaching. Therefore, the types of
questions for the students are: 1) How do you think about the new critical reading
teaching? 2) Do you think critical reading teaching is helpful to improve your reading
ability? If yes, how does it help you? 3) In what aspects have you improved after
having attended the critical reading course for one semester? 4) What are the
challenges you face when learning reading in this new way? 5) What's your advice to
critical reading teaching?
3.5 The development of SAMPLE-based reading intervention Module
This section describes the process of the main instrument in this study, namely the
intervention module. The development of this module is based on SIDEK’s module
development model. As such, this module combines recent advances in computing
technology with our current theoretical understanding of text comprehension to
improve critical reading.
3.5.1 Definition of the Module
A successful module is one in which the stated learning outcomes or goals are
aligned with the teaching activities and assessments. In other words, in the context
of learning outcomes, it will include the learning activities that students need to
participate in, the learning goals that students need to be achieved, the content that
should be mastered, the support needed to achieve learning outcomes, and the
methods that can be used to achieve the best learning outcomes At the same time, it
also includes researcher’s review or evaluation of the teaching process to understand
whether this module successfully combines learning outcomes with activities and
assessments.
With regards to this study, the module is operationally defined as an intervention or
learning module, consisting of computer-aided data and SAMPLE-based approach,
which will become a guide for teachers to help students in critical reading. This
module is considered to utilize the modern computer-aided data and motivational
module defined by Wen (2014) and Jin (2018). The modern computer-aided data
focuses on making use of the rich resources from the internet and adapting them
into the EFL context for Chinese EFL undergraduate learners. Motivation modules are
modules created to help students improve their motivation to achieve established
goals and prepare teachers or counselors to manage learners’ module activities
(Wen, 2014).
The reason for combining these two elements of the module is because of the
purpose and nature of the research module. The SAMPLE-based model requires
teacher support and guided practice (Wen, 2014). This intervention module is also a
teaching module because teachers will use it to teach critical reading to students in a
clear way. Although the teacher will serve as the moderator or coordinator of this
module, this module focuses on the teaching and learning of EFL critical ring in
schools, and aims to help learners with limited critical reading proficiency and may
improve their critical performance. The teacher becomes the facilitator of the module
and will not interfere with the school’s existing workbooks or materials. This module
will be used as a supplementary aid to the preparation of the course, rather than as
the main textbook. This intervention module is also a teaching module because
teachers will use it to teach critical reading strategies to students in a clear way.
3.5.2 How the Module is Developed
This intervention module will be referred to as the data driven SAMPLE-based
intervention module for teachers. The content of this module is based on research
results of the qualitative phase. The draft module will be further tested in the
quantitative phase.
As to the module content, Sidek Noah and Ahmad (2005) suggest that researchers
need to consider individual differences to ensure that the objectives of the module
can be obtained. This suggestion further supports the observation of Sharifah Alwiah
Alsagoff (1981) who believes that the successful implementation of the module
needs to be adapted to the ability and proficiency of the users or students according
to the context. Therefore, the content of the modules, such as goals and language
learning strategies, will be adapted from current literature and will be based on
recommendations for analyzing gaps found in previous studies.
The content of the critical reading part is used as a guide for researchers to ensure
that the module still covers the requirements of the college English syllabus. Since
the exam is not an event that can be avoided or ignored, the continuous writing
assessment rubric will be used to grade and evaluate students' essays, which is a
common scale used by schools to evaluate student essays (see Appendix 1). The
intervention module will be embedded in the SAMPLE model stage and will be
suitable for combining with the process authoring method.
3.6 Research Procedure
As mentioned above, this research will be carried out in three main phases: the
diagnosis phase, the development and pilot phase, and the implementation phase.
This section will describe and discuss the rationale of each stage. The figure below
illustrates the research process. Research questions and data collection methods are
assigned to each stage.
•RQ1. What difficulties do teachers think that learners with low English proficiency
face when trying to read critically?
RQ2. What are the factors that hinder or encourage teachers to read critically in
classroom reading teaching?
RQ3. What are the common practices of teachers in EFL critical reading teaching?
Method Focus group interviews, documents analysis
• RQ4. Which language teaching elements, especially those related to reading, need
to be adapted into SAMPLE-based modules based on the results of the first phase?
Method Expert evaluation, quasi-experiment
RQ5. Does the new reading intervention module adapted from previous stages
significantly help the learner’s reading performance?
RQ6 How does the adaptive SAMPLE-based teaching module help teachers and
learners improve reading teaching and learning?
Method Semi-experimental and semi-structured interviews with selected students
and teachers.
In order to have a clear explanation of the process, the following chart of data
collection and data analysis can be used as a reference.
Phases No Research questions Sample Data collection methods Data analysis

Diagnosis 1 What difficulties do teachers think that EFL Teachers Focus group interviews; Students Thematic
undergraduates face when doing critical and reading task analysis analysis
reading? students
Focus group interviews; Students
Diagnosis 2 What are the factors that hinder or reading task analysis Thematic
encourage teachers to read critically in Teachers analysis
classroom reading teaching? and Focus group interviews;
Diagnosis 3 students Thematic
What is the common practice of teachers analysis
in EFL critical reading teaching? Document analysis; Interview
Teachers transcripts, students reading task
Development 4 Which language teaching elements, analysis; Thematic
and pilot especially those related to critical reading, analysis
need to be adapted into SAMPLE-based Teachers
modules based on the findings of the first and
phase? students Pre-post test
Implementation 5
Does the reading intervention module Descriptive
developed from previous stages analysis Paired
significantly help the learner’s reading Semi-structured interview t-test
Implementation 6 performance? Teachers
and Thematic
How does the adaptive SAMPLE teaching students analysis
module help teachers and learners improve
reading teaching and learning?
Teachers

3.6.1 Phase 1: Diagnosis

This stage will try to answer research question 1 to question 3. In this multi-method
approach, the first stage of the research is qualitative design. The type of qualitative
method adopted at this stage is basic qualitative design. As suggested by Merriam
and Tisdell (2016), this design is utilized because of several factors. First,
researchers are interested in exploring how teachers interpret their experiences
when teaching critical reading to EFL undergraduates, and how teachers structure
their world or opinions about it. Third, what these teachers think is of their
experience in teaching critical reading to EFL undergraduates. As assumed by
Merriam and Tisdell, the overall purpose of basic qualitative design is to understand
how people or teachers in this study understand their teaching writing experience.
From the perspective of understanding the teaching of writing by teachers,
researchers aim to identify several factors or elements that can be included in the
development of critical reading modules.
This stage of research can be seen as the diagnostic stage of research, where
researchers use teachers’ insights to understand several factors, such as their
teaching practices. The qualitative stage aims to learn from teachers of their
teaching practice when teaching critical reading skills to students, so as to reveal the
factors that hinder or promote the teaching and use of classroom strategies. Another
reason to use qualitative methods at this stage is to determine appropriate teaching
elements that can help readers.
The diagnostic section is crucial to the development of the module content. Data at
this stage will be gathered through focus group interviews which are defined as
interviews with group members of unrelated individuals. In this interviews,
investigators are the leader of the group discussions on one or more specific topics
(Barbour, 2008). From the data collected at this stage, the teacher’s answers will be
written and developed into unit courses based on the SAMPLE-based critical reading.
Focus group interviews with teachers can help identify strategies or methods that
students can use. Although the SAMPLE-based model provides some examples of
approaches that teachers can use in the classroom, some of these ways may not be
suitable for students who participate in this process. During the development of the
module, to include teachers’ voices throughout the focus group interviews will
increase the reliability and usability of the module. The voice of focus group
interviews will be recorded after the meeting. It hopes to further support, gain
intuitive understanding, and add perspectives to the focus group data. The
researchers then would analyze student performance of reading tasks.
Documents in the form of student reading tasks will be gathered to analyze their
patterns of critical reading. Three methods above of data collection will ensure the
credibility and rigor of the diagnostic phase. Focus groups are used in the qualitative
phase because it is convenient for teachers, which will be carried out after
consultation with teachers after class. A follow-up interview will be conducted with
individual teachers to obtain more information and clarify their previous interviews.
In order to develop any intervention module or plan for the school or learning, a
need analysis or preliminary investigation is required. The needs analysis constitutes
the preliminary stage of this research. In this initial stage, qualitative methods are
used to obtain information from teachers through the use of focus group interviews
and classroom observation data collection methods. The goals of this preliminary
study are: 1. Identify the possible elements of the intervention module 2. Determine
the problems that are expected to be corrected before the main research.
The data collection method for this preliminary study is focus group interviews. The
focus group interviews were conducted in teachers’ office. The interview was
recorded and transcribed verbatim. Seven teachers participated voluntarily, and
according to Merriam and Tisdell, the total number of respondents is enough to
reach point data saturation. The brief introductions of the 7 teachers are as follows.
The name of each teacher is a pseudonym determined by the teacher: the focus
group participants that emerged from the focus group interview will be discussed. In
another chapter, the methodology section discusses improvements that researchers
can take to improve the main research. The purpose of conducting a focus group is
to reveal what teachers think about reading teaching, how they teach, and what are
the factors that hinder or promote their teaching. The results of the research will be
used to develop the structure of the reading intervention module. The interview
structure is divided into categories related to students’ attitudes towards critical
reading, methods used in critical reading, and teachers’ perceptions of learning
strategies. After interview transcripts and student essay analysis, topics are
categorized.
3.6.2 Phase 2: Development and Piloting
The effectiveness of the adaptation module is tested at phase 2. It aims to answer
the fourth research question. To test this module, this study will use a quasi-
experimental method designed through a non-equivalent control group before and
after testing. This research adopts quasi-experimental methods, because the
research setting keeps from forming laboratories or manual groups; therefore,
researchers cannot randomly assign participants to each group (Creswell, 2014).
According to Creswell (2014), pre-testing is a form of measurement of certain
attributes or characteristics that researchers evaluate for participants. Conduct
research before receiving treatment or intervention. Post-testing is the measurement
of certain characteristics, which are evaluated for experimental participants after the
intervention (Creswell, 2014).
3.6.3 Phase 3: Implementation
After the pilot phase, the module is expected to be actually tested or implemented in
the classroom. There will be two stages in this phase of implementation: first
through the quantitative method of quasi-experimental design, and then
supplemented by qualitative method of phenomenology.
For the quasi-experimental design, two classes consisting of EFL learners of regular
class will be divided into two groups. The first group will be the Intervention Group
(IG), and teachers will use the Intervention Module to teach reading The second
group will be the Control Group (CG), where the teacher will use his or her teaching
practice to teach reading. Before the intervention, the students’ reading test scores
will be used as pre-test scores. One or more teachers participating in this study will
receive training on how to use the intervention module in a day seminar organized
by the researchers.
The intervention takes approximately 12 weeks and was determined based on the
recommendations of previous studies. The duration depends on the calendar and the
permission of the school administrator. The duration of the intervention was 12
weeks At the end of the intervention, a post-test will be done. The post-test will be a
reading test, which will be scored using the standardized reading evaluation criteria
provided by the Ministry of Education. The results will be analyzed by comparing the
pre-test and post-test scores of the two groups using a paired t-test of related
samples.
After analyzing the test results, semi-structured interviews with teachers and
selected students will be conducted through retrospective recall to evaluate more
information about the intervention module and how to implement it more effectively.
Samples of student test performance will also be collected and analyzed.
3.7 Data analysis
Use quantitative methods to investigate whether SAMPCE is effective in improving
the critical reading comprehension of CEFL students. Quantitative data such as pre-
test and post-test scores and Likert scale scores of students in the Student Attitude
Questionnaire are collected. For data analysis of pre-test and post-test scores,
repeated measures analysis of variance is used. For the analysis of the data collected
from the questionnaire, quantitative descriptive statistical analysis was used. The
data is calculated using SPSS 26.0.
Table 3. The procedure of Weekly Summary
Time schedule Program

Week 1 Explain the research to both groups of students

Week 2 Introduce module to the experimental group of students

Week 3 Model the reading strategy in SAMPCE

Week 4 Both the control group and the experimental group take the
pretest
Week 5 Apply SAMPCE to the selected experimental group with selected
reading texts
Week 6 Instruct the control group to use the same reading text with the
traditional teacher-centered approach
Week 7 Apply SAMPCE to the selected experimental group with selected
reading text
Week 8 Instruct the control group to use the same reading text with the
traditional teacher-centered approach
Week 9 Apply SAMPCE to the selected experimental group with selected
reading text
Week 10 Instruct the control group to use the same reading text with the
traditional teacher-centered approach
Week 11 Two groups of students take a post-test
Week 12 Experimental group of students participate in the questionnaire
3.8 Validity and reliability
The validity and reliability of research and the instruments used for research are
important aspects that researchers need to consider. The reliable and correct data
are obtained by ensuring the reliability and validity of the measurements. According
to Fraenkel et al., (2012), validity refers to the appropriateness, significance,
correctness and usefulness of inferences made by researchers. Another way to look
at validity is by judging whether the research really measures the authenticity of
what it intends to measure or the results of the research (Joppe, 2000). Reliability
means the management from one tool to another, the consistency of scores or
answers from one set of items to another (Fraenkel et al., 2012).
Evidence of the effectiveness of the module related to the content will be determined
by an expert panel looking at the content and format of the measurement and
judging whether the content is appropriate (Fraenkel, Wallen & Hyun, 2012). As
suggested by Fraenkel et al., the expert panel should be someone who can make
informed judgments about the adequacy of the tool. Therefore, the expert team will
be composed of experienced English teachers, and lecturers from universities or
teacher training institutions. In terms of modules, in order to ensure internal
reliability, the content of the module will be evaluated by three experts, who are
composed of experienced English
3.9 Ethical considerations
Ethics considerations can be considered as guidelines for completing research
without harming or harming the people involved. Research ethics pushes the
boundaries of development and moral considerations. Any research plan should
comply with research ethics.
Permission to conduct this research will be obtained for pilot and practical research
from the Academic Affairs Department. The study encourages voluntary participants.
In terms of participant consent and confidentiality, each participant will receive a
consent form and will be informed of the research goals and their requirements.
Participants also have the right to withdraw from the study at any time and inform
the researchers if there is any data that they feel uncomfortable to share as the
results of this study.
3.10 Chapter summary and Overall conclusion
This chapter discusses the methodology of the research. Specifically, this multi-
method study includes two methods in three stages, namely qualitative and
quantitative methods. It includes the design and the reasons for choosing the
design, as well as the research worldview. This chapter also describes the
background of the research, the participants and their reasons. It also explains the
research framework and procedures. Finally, the role of the researcher, the ethical
considerations of the research, and the timetable for the research conclude this
chapter.
Overall, this study discussed the problems and gaps in EFL reading. Critical reading is
a high-level reading skill that needs to be mastered and taught. The past literature
has identified the gaps in reading teaching and learning, including the lack of clear
and targeted reading materials and the lack of effective teaching guidance. This
study shows that in order to resolve the gaps found in the existing literature, a multi-
method design is needed in the research.
In order to answer the research question, a multi-method approach was used. The
reason for adopting this method is first to resolve the lack of qualitative methods in
identifying factors that hinder and encourage speculative reading learning and
teaching in EFL classrooms. Another reason is the lack of suitable reading materials
and effective teaching methods in the speculative reading module
In short, this research proposes that in order to improve the ability of speculative
reading of college students, they need appropriate reading materials and effective
guidance methods to support them in trying complex reading tasks. In order to
master the basic speculative reading methods, teachers, as the implementers or
performers of the curriculum, need to use reading materials suitable for their
learning level to encourage learners to produce the SAMPCE model to model the
teaching process to guide Non-English major EFL undergraduates in a public
university.