SUMMARY on Feb 11, 2022

DEVELOPING CRITERIA EVALUATING MATERIALS

- It is extremely useful to develop a set of formal criteria for a particular evaluation
- Use it as a basis for developing other sets.
- Initially, demanding and time consuming but helps to clarify principles and
ensures future evaluations are systematic, rigorous.
1. Brainstorm a list of universal criteria
- Universal criteria apply to any language learning, anywhere for any learners.
- apply to a video course for 10 year olds and an English for academic purposes
textbook
- they derive from principles of language learning, provide the fundamental basis
for any materials evaluation.
- for example, for brainstorming a list of universal criteria:
• Do the materials provide useful opportunities for the learners to think for
themselves?
• Are the instructions clear?
• Do the materials cater for different preferred learning styles?
• Are the materials likely to achieve affective engagement?
2. Subdivide some of the criteria
- subdivide some of the criteria into more specific questions to make decisions
- For example, "Are the instructions clear?" can be sub-divided into: Are the
instructions:
• succinct?
• sufficient?
• self-standing?
• standardised?
• separated?
• sequenced?
• staged?
=> such a subdivision helps to pinpoint specific aspects of the materials
3. Monitor and Revise the list of universal criteria
- it is useful to monitor the list and re-write according to:
a. Is each questions an evaluation question? (an analysis question, an evaluation
question)
b. Does each question only ask one question?
c. Is each question answerable? (obvious question = clear enough: dễ dàng trả
lời, làm câu hỏi nhỏ nhất, cụ thể nhất)
d. Is each question free of dogma? (reflect the evaluators’ principles of language
learning, but impose a rigid methodology on the materials)
e. Is each question reliable in the sense that other evaluators would interpret it in
the same way?
4. Categorize the list
- useful to rearrange random list of universal criteria into categories focus and
enable generalizations to be made.
- possible categories for universal criteria could be:
• Learning Principles
• Topics
• Teaching Points
• Texts
• Activities
• Instructions
• Illustrations
• Design and Layout
5. Develop Media-Specific Criteria
These are criteria which ask questions of particular relevance to the medium of
the materials (e.g. criteria for books, for audio cassettes, for videos etc). For
example,
• Is it clear which sections the visuals refer to?
• Are the different voices easily distinguished?
• Do the gestures of the actors help to make the language meaningful?
6. Develop Content-Specific Criteria
- relate to the topics and/or teaching points of the materials.
- relevant to the evaluation of a business English textbook but not to a general
English coursebook, For example,
• Do the examples of business texts (e.g. letters, invoices etc) replicate features
of real life business practice?
• Do the reading texts represent a wide and typical sample of genres?
7. Develop Age-Specific Criteria
- relate to the age of the target learners. Thus there would be criteria suitable for
five year olds but not for teenagers.
- relate to cognitive and affective development, to previous experience, to
interests and to wants and needs. For example,
• Are the activities likely to match the attention span of ten years old learners?
• Is the content likely to be cognitively challenging for young adults?

8. Develop Local Criteria

- relate to the actual or potential environment of use. They are questions which
are not concerned with establishing the value of the materials per se but rather
with measuring the value of the materials for particular learners in particular
circumstances.
- Typical features of the environment which determine local criteria are:
• the type(s) and resources of the institution(s)
• class size
• the background, needs and wants of the learners and of the teachers
• the language policy
• the syllabus
• the objectives of the courses
• the intensity and extent of the teaching time available
• the target examinations
• the amount of exposure to the target language outside the classroom
Task 1.5
Developing Criteria
1. Specify (as questions) six universal criteria which you think are important
when evaluating any type of language materials,
e.g., Do they expose the learners to English being used for communication?
Answer:
- Do the materials fit the course?
- Are the materials reasonably current or unlikely to date?
- Is the input (text and visuals) used in the materials realistic and appealing?
- Do the materials encourage communication?
- Is the content easy/feasible to supplement?
- Does the content encourage students to notice language in use?