University of Perpetual Help System – DALTA

Molino III Bacoor City, Cavite

College of Education

Language and Literature Assessment

MODULE V: QUALITATIVE ANALYSIS

CATEGORIES OF APPROACHES:

 REFLECTION
 VERBAL REPORTS
A number of variables can be distinguished:
o Talk aloud
o Think aloud
o Concurrent
o Retrospective
o Mediated
o Non-mediated
 DIARY STUDIES
There are number of varieties:
o Unstructured
o Guided
o Structured

TECHNIQUES FOR MONITORING STUDENT PROGRESS (ORACY AND LITERACY)

o Make video and audio recordings of a variety of formal and informal oral
language experiences, and then assess these according to pre-
determined criteria which are based upon student needs and curriculum
objectives.
o Use checklist s as concise methods of collecting information and rating
scales or rubrics to assess student achievement.
o Record anecdotal comments to provide useful data based upon
observation of students oral activities.
o Interview students to determine what they believe they do well or areas in
which they need to improve.
o Have students keep portfolios of their dated writing samples and
language abilities checklists and records.
o Keep anecdotal records of students reading and writing activities and
experiences.
o Have students write in reader response journals.
o Confer with students during the writing and reading processes and
observe them during peer conferences.
SELF-ASSESSMENT – promotes students abilities to assume more responsibility to
identify where they believe they have been successful and where they believe
they require assistance.

PEER ASSESSMENT – allows the students to collaborate and learn from others.

TYPES OF QUALITATIVE ANALYSIS

 CONVERSATION ANALYSIS (CA)

The assumption is that every interaction is contextual and has a
stable and predictable nature. CA focuses on talk, i.e. speaking test.

a. From a transcript, the power relations can be researched by analyzing

o The structure of adjacency pairs
o Turn-taking

b. Some possible pitfalls to look out for with CA

o DATA QUALITY – the speech recording has to be clear.

o LOSS OF DATA – the original recording should be kept for
future reference.
o TIME – transcript are very time-consuming to compose and to
research.
o LIMITED GENERALIZABILITY – each interaction is different, so it is
hard to draw general conclusions.
 DISCOURSE ANALYSIS (DA )
DA is the analysis of “text and talk as social practices” and is mainly
concerned with power relations, gender inequalities etc. In DA the
transcript of an interaction is analyzed for adjacency pairs, turn-taking
and repair. Special attention is paid to:
o The effect of examiner behavior on test taker performance;
o The effect of test taker characteristics on performance;
o The effect of task type on performance; and
o Comparing test taker language ability outside of test to test
performance.
 TEST LANGUAGE ANALYSIS
It is an analysis of test input or test taker responses for lexical
richness, rhetoric, genre, discourse markers, grammatical complexity, etc.
In using this type of sample analysis, pay attention to:
o TIME – test language analysis is very time-consuming.
o DATA – processing data is more efficient if you know what
you are looking for.
 TASK CHARACTERISTICS
This type of validation research helps to examine the test tasks and
to determine to which extent they correspond to the test goal. Bachman
and Palmer (1996) suggest a framework of analysis which considers the
following:
o Setting
o Rubric
o Test Input
o Expected Response
o Relationship between expected and actual response
 When analyzing a test for its task characteristics, consider the following:

o The framework should be adjusted to each different test.

o The judges have to be competent and experienced test
designers.

 FEEDBACK METHODS

 QUESTIONNAIRES
o It gathers data such as opinions and views that can
also be gathered through interviews. The main
advantage of questionnaires is the possibility to use
very large informant population.
o Two kinds of questionnaires:
 CLOSED – the informant replies to the
questions by ticking boxes or marking a
scale.
 OPEN – The informant replies in his/her own
words.
o Before administering the actual questionnaire, it is
useful to run it through the following process:
 Consider all possible issues that your
questionnaire should cover.
 Write a draft.
 Eliminate questions that do not address the
questionnaire purpose.
 Group the questions thematically to spot
overlaps.
 Format the questionnaire and administer it
to a small group of target respondents for
feedback.
 Rewrite the questionnaire.
o Always avoid:
 Double-barreled questions
 Unclear instructions
 Questions that do not apply to the
respondent
 Questions that rely on memory
 Biased options
o CHECKLISTS – a way of determining whether all
procedures have been gone through, whether all
necessary features are present, etc.
 INTERVIEWS
They are a flexible way of gathering data. There are various
kinds of interview, depending on the structure and the number of
informants interviewed at the same time.

o UNSTRUCTURED – no fixed interview schedule but rather

a number of themes that are to be addressed.
o SEMI-STRUCTURED – researcher follows a preset
schedule but it is possible to deviate from this when
interesting issues arise.
 o STRUCTURED – interviewer goes through a fixed series of
written questions without deviating.
o ONE ON ONE – allows the researcher to zoom in on the
views of individual respondents.
o GROUP – the advantage interviewing larger numbers
at once is that group interactions might spark
observations that would have gone unnoticed.

 REFERENCING

Think about the following before the interview:

o Interviewers should get the chance to practice their interview

skills prior to the data collection.
o Ideally, the pilot settings resemble the actual conditions as
accurately as possible.
o During the interview, it’s useful to take note of the interview
situation.
o The success of an interview largely depends on the
interviewer-respondent interaction.
o Interviews are time consuming.

There are various kinds of referencing:

o NORM-REFERENCING – the placement of learners in rank

order, their assessment and ranking in relation to their peers.
o CRITERION-REFERENCING – a reaction against norm-
referencing in which the learner is assessed purely in terms of
his/her ability in the subject, irrespective of the ability of
his/her peers.
o THE MASTERY CRITERION-REFERENCING APPROACH – is one in
which a single ‘minimum competence standard’ or ‘cut-off
point’ is set to divide learners into ‘masters’ and ‘non-
masters’. With no degrees of quality in the achievement of
the objective being recognized.
o THE CONTINUUM CRITERION-REFERENCING APPROACH – is an
approach in which an individual ability is referred to a
defined continuum of all relevant degrees of ability in the
area in question.

Prepared By: Sasi. Chrisna Ardee V.

BSED 3 - English