1

CHAPTER III

METHODOLOGY

This chapter presents a description of the study’s research design, details about the

research setting and subjects of the research, the rationale behind the design and

development of the instrument to be used, the data gathering procedures that will

be undertaken and the proposed statistical treatment of the data.

Research Design

The study was quantitative since it aimed to measure the level of linguistic and

discourse competence of public high school English, Mathematics and Science

teachers in Pulilan, Bulacan who use ESL as the medium of instruction. It was also

descriptive in nature because the results of the data gathered from the target

population was reported, compared and analyzed to find out if there were

“functional relationships of variables, to test the hypotheses, develop

generalizations, and reaffirm or contradict principles or theories with universal

validity” (Best and Khan, 1998), which may have relevance to the issues

confronting the communicative competence of all public and private high school
 2

English, Mathematics and Science teachers using ESL as medium of instruction in

the Philippines.

The design of this study focused on identifying and comparing the level of

grammatical and discourse competencies of the respondents using a Test of

Grammatical and Discourse Competence in English (TGDCE) that the researcher

has developed.

Research Setting

Pulilan is one of the 24 towns of the province of Bulacan. It is composed of

19 barangays which fill out its land area of 3, 989 hectares. In 2007, the population

is estimated to be around 85,000. Most plough and till their farmlands, as

agriculture is the main industry in this town. The town has 4 public high schools

and 12 government-recognized private high schools.

The respondents came from 3 public high schools in Pulilan, Bulacan and 4

government-recognized private high schools. These high schools were chosen by

the researcher because they parallel each other in terms of student population and

number of teachers employed.

 3

School A has one of the biggest schools in Pulilan in terms of student

population and number of teachers. It is also one of the oldest high schools in

town. It is located in Barangay Dampol 2nd. School B is a medium-sized school

with a student population of about 1, 000 students. It is located in Barangay Sta,

Peregrina. School C in Barangay Lumbac is a newly established public high

school, but its population is similar to School B. These schools are a good two

kilometers apart from each other, and none of which are within the town’s center.

School D, located in the town center, is one of the largest schools in Pulilan in

terms of student population and number of faculty members. Similar to School A,

it is one of the oldest schools in Pulilan. School E, School F and School G are all

fairly newly-established high schools which have been in operation for only a

decade. In terms of student population, School F has a student population similar

to School B, while the combined student population of Schools E and G is similar

to the student population of School C. School E, F and G are located at least 2

kilometers away from the town center.

Research Population
 4

The respondents of this study were the high school English, Mathematics

and Science teachers of 3 public high schools and 4 government-recognized

private high schools in Pulilan, Bulacan who had been teaching the

aforementioned subjects for no less than one school year and who were mandated

by the law to use English as their medium of instruction.

A description of the research population is presented in Table 1.

TABLE 1

DESCRIPTION OF RESEARCH POPULATION

Schools English Mathematics Science Total # of

Teachers Teachers Teachers Research
Population
School A 6 6 9 21

School B 4 5 6 15

School C 4 3 7 14

School D 3 3 7 13

School E 2 2 2 6
 5

School F 3 6 5 14

School G 2 2 2 6

Total # of 24 27 38 89
Respondents

Table 1 shows that there were a total of 89 respondents, composed of 24

English teachers, 27 Mathematics teachers and 38 Science teachers, who

participated in the study. Of the 24 English teachers, 14 came from public high

schools and 10 came from the private high. Of the 27 Mathematics teachers, 14

came from public high schools and 13 came from the private high schools. Of the

38 Science teachers, 22 came from the public high schools and 16 came from the

private high schools.

The researcher believes that the number of respondents and their teaching

experiences was sufficient in identifying and comparing the level of grammatical

and discourse competence in the above mentioned institutions.

 6

Research Instrument

To gather data for this study, the researcher used two instruments namely:

(1) Personal Data Sheet and (2) Test of Grammatical and Discourse Competence

in English (TLDCE).

The Personal Date Sheet (Appendix A) provides the researcher the

information about the respondents’ age, educational attainment and professional

experience. The information was useful in analyzing and interpreting the data

gathered, especially in drawing implications for the development or improvement

of the school’s faculty development program.

The TGDCE (Appendix B) was developed by the researcher to test the

grammatical and discourse competence in English of high school English,

Mathematics and Science teachers using English as medium of instruction. The

test is designed using two of the constructs in the Model of Communicative

Competence developed by Savignon based on the Theory of Communicative

Competence espoused by Canale and Swain’s (1980). These constructs are called

Grammatical Competence and Discourse Competence. Thus, the test is composed

of two subtests based on these two constructs, and each subtest is designed to

focus on the different elements of which makes up the two given constructs.
 7

Table 2 shows that the first subtest Grammatical Competence is made up of

items that will measure the knowledge of grammar.

The types of test under this subtest are Error Detection and Correction

which focuses on Lexicon and Morphology and Dialogue Completion and Role

Playing which tests Phonology, Syntax and Semantics. The second subtest is made

up of Cloze Procedure which tests Cohesion and Logical sequencing which

focuses on testing Coherence.

TABLE 2

TABLE OF SPECIFICATION OF THE TEST OF GRAMMATICAL

COMPETENCE AND DISCOURSE COMPETENCE
IN ENGLISH

Subject Test Type Elements being tested No. of Items

I. Error Recognition Lexicon, Morphology 21
Linguistic and Correction
Competencies
II. Dialogue Semantics, Syntax, 25
Completion and Role Phonology
Playing
III. Cloze Procedure Cohesion 23
Discourse
Competencies IV. Logical Sequencing Coherence 10
 8

In developing the first subtest which features Grammatical Competence, the

researcher included grammatical items that are most problematic to Filipinos and

require general improvement, as noted in some local studies mentioned in the

survey of related literature earlier. These grammatical items are gerunds,

conditionals, tense concordance and prepositions. (Castillo, 1997) Error detection

is “a well-received test task as a measure of language proficiency and

communicative competence” (Elder, 1993). However, the researcher felt that

recognizing the error is only half as significant as understanding “why” the error

exists and “how” it should be corrected. Thus, the researcher modifies the Error

Detection test type into Error Recognition and Correction, in which respondents

will choose the letter representing the underlined word/s with an error in the

sentence and then opposite that letter, the respondent should write the word/s that

best corrects it. The text for Error Recognition and Correction is an article entitled

“Refuse to use” written by Anna Rhea Manuel and was published in the column

Youngblood of The Philippine Daily Inquirer on March 1, 2007. Some of the texts

in this article were replaced to reflect grammatical errors and examinees should be

able to detect those errors and correct them accordingly. In the event that the

respondent chose the correct letter of the word/s with the error but failed to
 9

provide a correction for this error, then no point will be given for the said item.

There are 21 items in this part of the test and each item in this part of the test is

worth 1 point.

The second subtest for the Test of Grammatical Competence is Dialogue

Completion and Role Playing. The “speaking-listening, asking-answering

interaction” is one of the most common activities engaged in by teachers and

learners inside the classroom. The ability to identify the subtle changes or

variations in sounds and meanings within conversations is necessary so that there

will not be a breakdown in the communication process. There are also studies

from linguist such as Elder (1993), Savignon (1983) and Castillo (1997) that

support the idea of “roleplaying” as an appropriate test task to measure

communicative competence specifically as a test of Grammatical competence.

There are 25 items in this category. In his part of the test, the respondents will be

given an incomplete dialogue and they shall be given 2 minutes to read the

incomplete dialogue and come up with a statement that will best complete it.

Afterwards, the respondent with the proctor will run the dialogue portraying the

roles specified in the given items. The role play will be voice record, analyzed and

scored using the rubric shown in the succeeding pages.

 10

Table 3 shows how semantics (the appropriateness of response in a given

context) in each of the response of the respondent will be analyzed and scored

based on the following considerations: (1) Was the response appropriate for the

given situation? (2) Did the response show the ability to communicate simple and

direct exchange of information? (3) Did the response demonstrate the ability to

express himself/ herself spontaneously, very fluently and precisely? (4) Did the

response show the capability to use English language flexibly and effectively for

social, professional and general purposes? If the answer to three or four of these

questions is YES, then the item was marked with 1 point.

TABLE 3

RUBRIC FOR SCORING PROFICIENCY IN SEMANTICS

Description Score

The examinee was able to give 21 - 25 appropriate responses/

statements that completed the dialogue. 5
The examinee was able to give 16 – 20 appropriate responses/
statements that completed the dialogue. 4
The examinee was able to give 11 - 55 appropriate responses/
statements that completed the dialogue. 3
The examinee was able to give 6 - 10 appropriate responses/
statements that completed the dialogue. 2
 11

The examinee was able to give 1 - 5 appropriate responses/

statements that completed the dialogue. 1
The examinee was not able to give any appropriate response/
statement to complete the dialogue. 0

Table 4 shows how syntax (the accuracy in Syntactic class, Syntactic

function and Syntactic structure) in each response of the respondent will be

analyzed and scored based on the following considerations: (1) Maintains

consistent control of the syntactic class, syntactic function and syntactic structure

of complex language, even while attention is otherwise engaged. (2) Errors are

rare and/ or difficult to spot. (3) Does not make mistakes which lead to

misunderstanding. If the answer to two or three of these questions is YES, then the

item was marked with 1 point.

TABLE 4

RUBRICS FOR THE SCORING PROFICIENCY IN SENTENCE STRUCTURE

Description Score
 12

The examinee was able to give 21 – 25 error-free and accurate

responses/ statements in terms of syntactic class and syntactic 5
function that completed the dialogue.
The examinee was able to give 16 – 20 error-free and accurate
responses/ statements in terms of syntactic class and syntactic 4
function that completed the dialogue.
The examinee was able to give 11 – 15 error-free and accurate
responses/ statements in terms of syntactic class and syntactic 3
function that completed the dialogue.
The examinee was able to give 6 – 10 error-free and accurate
responses/ statements in terms of syntactic class and syntactic 2
function that completed the dialogue.
The examinee was able to give 1 – 5 error-free and accurate
responses/ statements in terms of syntactic class and syntactic 1
function that completed the dialogue.
The examinee was not able to give any error-free and accurate
responses/ statements in terms of syntactic class and syntactic 0
function that completed the dialogue.

Table 5 shows how the respondent’s phonological control or the way the he/

she pronounces or puts stress in his/ her response in order to communicate

effectively is analyzed and scored. The column “Description of Response”

itemizes the different levels of proficiency with its corresponding score. The

researcher will simply put a check beside the description that best fits the
 13

respondent’s phonological control. The item which was marked YES got the point

equivalent to it.

TABLE 5

RUBRIC FOR SCORING PHONOLOGICAL CONTROL

Score Description of Response Yes

Varies intonation and places the stress correctly in the statement/
5 response in order to express differentiating shades of meaning
Speaks in an intelligible manner, with a clear and natural
4 pronunciation and intonation
Pronunciation can easily be understood even if speech is
3 characterized with short pauses and there is occurrence of
mispronunciation occasionally
Pronunciation is clear enough to be understood, although
conversational partners need to clarify and ask for repetition
2 from time to time, and speech contains several long pauses and
frequent mispronunciation occurs
Listener can understand but with some difficulties the
1 pronunciation of words, phrases or sentences and there is no
variation in tone throughout the statement
0 Speech is garbled and unintelligible
 14

Overall, the Test of Grammatical Competence has a total score of 36 points,

21 points from Error Recognition and Correction and 15 points from Dialogue

Completion and Role Playing. Further, table 6 shows how the scores in the Test of

Grammatical Competence will be interpreted in terms of the level of proficiency in

the English Grammar. The passing rate or interpreted as “Fair” is at 75% which is

the generally accepted passing rate in most of the Philippines Educational system.

The advanced level of proficiency or interpreted as “Excellent” should score 34-36

points. Approaching advance level or “Very Good” is 31-33 points, while

intermediate level or “Good” is 28-30 points. However, if the score is below the

75% passing rate, it is categorized as “Poor” or very low level of proficiency.

TABLE 6

INTERPRETATON OF TEST OF GRAMMATICAL COMPETENCE

IN ENGLISH SCORES

Scores Interpretation
34-36 Excellent
31-33 Very Good
28-30 Good
25-27 Fair
 15

24 and below Poor

The second construct being tested in this study is called Discourse

competence. The first subtest in this category is a Cloze Procedure. In the cloze

procedure, the respondents supply missing words that have been deleted from the

text. The words that the respondents will supply must accurately complete the

meaning of the text. Hafner (1966) quoting Barmuth (1964) states that “a cloze

test may be used to predict respondent’s ability to score on comprehension

subtypes such as understanding facts, drawing inferences, seeing relationships and

vocabulary”, all of which are significant elements of cohesion and coherence.

Twenty-three words will be deleted from the article. Each item is worth 1 point.

The second subtest for the construct on Discourse Competence is Logical

Sequencing. In this type of test, respondents must arrange the sentences in to a

logical order to create a meaningful paragraph. The answer will be chosen from a

set of options marked A-E. There are 10 items in this category and each item is

equivalent to 1 point.

Thus, the two subtests of the Test of Discourse Competence amounts to 33

points. Table 7 shows how these scores are interpreted in terms of level of
 16

proficiency in the English Discourse. The passing rate or interpreted as “Fair” is at

75% which is the generally accepted passing rate in most of the Philippines

Educational system. The advanced level of proficiency or interpreted as

“Excellent” should score 32-33 points. Approaching advance level or “Very Good”

is 30-31 points, while intermediate level or “Good” is 28-29 points. However, if

the score is below the 75% passing rate, it is categorized as “Poor” or very low

level of proficiency.

TABLE 7

INTERPRETATION OF TEST OF DISCOURSE COMPETENCE

IN ENGLISH SCORES

Scores Interpretation
32-33 Excellent
30-31 Very Good
28-29 Good
26-27 Fair
25 and below Poor
 17

Content Validation and Test of Reliability

To content validate the instrument developed, the researcher asked a

number of experts in the field of English Language from the University of the

Philippines and other local and international institutions to review and analyze the

items of the Test of Grammatical and Discourse Competencies in English. The

comments, suggestions and recommendations for the improvement and content

validation of the instrument were taken in and used to improve the instrument and

ensure its validity.

To test reliability, the instrument was pilot tested to 40 high school English,

Mathematics and Science teachers of Bajet-Castillo Memorial High School, Holy

Angels Colleges, Mary Chiles College of Arts and Sciences, Elite Learning Center,

Colegio de Sto. Cristo, and Colegio de Sta. Philomena in Pulilan, Bulacan, before

using it to the target population.

A description of the reliability rating of the Test of Grammatical and

Discourse Competence in English developed by the researcher is shown in table 8.

 18

TABLE 8

RELIABILTY RATING OF THE TEST OF GRAMMATICAL AND

DISCOURSE COMPETENCE IN ENGLISH
Subtest Score Interpretation
Error Recognition and .822 High Reliability
Correction
Cloze Procedure .848 High Reliability
Logical Sequencing .253 Low Reliability
Test of Grammatical and
Discourse Competencies in .917 High Reliability
English

Table 8 shows the results of the pilot testing for test of reliability. Using

Microsoft Excel and SPSS, the item analysis for Error Recognition and Correction

(.822) and Cloze Procedure (.848) revealed high reliability while Logical

Sequencing (.253) has a low reliability rating. However, the test of grammatical

and discourse competencies in English as a whole scored a high reliability rating

of .917.
 19

After the statistical analysis, the researcher computed for the discrimination

index of each item of the test to identify which items should be kept, revised or

discarded.

Data Gathering Procedure

The researcher formally wrote the Division Supervisor of the Department

of Education in Bulacan to request permission to conduct the study in the subject’s

institutions. Upon being granted the permission, the researcher proceeded to write

the head of the aforementioned institutions to gain their consent to allow their

English, Mathematics and Science teachers who use English Language as medium

of instruction to participate in the study. When the consent was given, the

researcher administered the instrument to the respondents involved in the study.

Data Processing

The researcher compiled and hand scored the accomplished instrument.

Then, the researcher recorded all scores using the frequency form. In the event that

some instruments were not fully accomplished and thus, non-generating, they were

considered void. The data gathered was subjected to statistical treatment and
 20

analysis using the programmed Statistical Package for Social Science (SPSS)

version 11.

Statistical Treatment

To determine the Linguistic and Discourse Competence of public high

school English, Mathematics, and Science teachers who use English as medium of

instruction, the researcher used the following statistical treatment:

To answer the research questions for items 1 and 3, the mean and

standard deviation was used to compute the Linguistic and

Discourse competence of public high school English, Mathematics,

and Science teachers who use English as medium of instruction

after the frequencies were tallied.

The formula used for computing the mean is: (Downie and Heath,

1983)

__ X
Χ = N
 21

__
Where: X is the mean

X is the sum of scores

N is the number of respondents

The formula used for computing standard deviation is: (Downie

and Heath, 1983)

___
S = √∑x²
N
__ __
Where x = (X – X) and x² = (X – X)²

To answer the research questions for items 2 and 4 the researcher

udse Single-Classification ANOVA to compare the level of

Linguistic and Discourse competence of public high school

English, Mathematics, and Science teachers who use ESL as

medium of instruction.
 22

The following formula was used for computing for ANOVA:

(Downie and Heath, 1983)

Formula for Total Sum-of-Squares:

(∑X)²
SSt = ∑Χ² ‫ ־‬N

Formula for Between Sun-of Squares:

(∑X)² __ (∑Xт)²
SSb = ∑ n N

Formula for Within Sun-of Squares:

__ (∑X)²
SS = ∑X² n

Then: SSw = ∑3SS

Formula for F Test:

F = mean-square for between groups

Mean-square for within groups

Formula for test after the F Test:

 23

__ __
F = _____(Xı – X )²
S² (Nı + N ) / Nı N

To answer the research question for item 5, Pearson product-

moment correlation coefficient, the symbol of which is r (Downie

and Heath, 1984) was used to determine if there is a significant

relationship between the grammatical competence and discourse

competence in English of high school English, Science and

Mathematics teachers. An r of .8 and above is considered higher

coefficient, around .5 is considered moderate and an r of .3 or

lower is low coefficient (Downie and Heath. 1984).

The formula for the Pearson r is:

___________N∑XY- (∑X) (∑Y)_________

r= √[N∑X² - (∑X)²] [N∑Y² - (∑Y)²]

To answer the research questions for item 6, the researcher

identified implications of the result for the improvement of the

 24

faculty development program of high school English, Mathematics,

and Science teachers who use English as medium of instruction.

The minimum level of significance accepted in this study is 0.05.