Learning Taxonomy: Using Outcomes to

design Achievement Test

Session 4: 29 October, 2021

 This Session
By the end of this session , you should be able
to:
• • EXPLAIN the role of learning outcomes in
designing a test
• •APPLY Bloom's/SOLS Taxonomy to write
clear, concise learning outcomes
• •INITIATE test blueprint/design
 Fundamental Question

• How does test design begin ?

 Developing Test: Principles

• Appropriateness and the compatibility with the

purpose and context of the assessment so that
inferences drawn about the knowledge, skills,
attitudes, and behaviors possessed by each
student are valid and not open to
misinterpretation.

How to achieve this?

 Designing Test: Principles
• When designing assessment , it is important to make sure that any
exams or assignments match the learning outcomes of the course.
• Assessments/tests should be based on material teachers have covered
in the course, and students should perceive the material as relevant
and fair.
• Assessment expectations should be based on students‟ learning
experiences
• A test should measure what it is supposed to measure – what a test
suppose to measure leads a teacher/ an item writer to write and
review the test items.

[Ref: Stanford Testing Handbook]

Constructive alignment
(Biggs and Tang, 2011)
 Alignment
• Alignment with learning objectives/outcomes
• Alignment with the content
• Alignment with the instructional design

Alignment in three interrelated aspects is crucial

to make to make a valid assessment of achieved
outcomes.
Why alignment is central in test design?
 Unfolding Key Terms

• Educational Goals
• Standards and Benchmarks
• Aims
• Learning objectives/outcomes

Can you define these terms?

 Educational Goals
Educational Goals “are those human activities
which contribute to the functioning of a society
(including the functioning of an individual in
society), and which can be acquired through
learning over times (Gagné, Briggs, & Wager,
1988 , p. 39 ).

The goals are discussed in national educational policy and/or

curriculum framework.
 Educational Standards

Educational Goals are translated into Standards: - Guide curriculum planning

• Standards are statements about what students are expected to learn. For
example, The Common Core State Standards for Mathematics are organized by
grade level from K through 10 and by topic for high secondary school –
following learning progression.

• Curriculum standards are set up by school system and/or the state policy makers
get assistance form professional organization to set-up standards meeting
national and international targets.

• These standards are further divided into benchmarks [Grade level classification]
and translated into learning objectives.
 Standards
Some states call statements essential skills, learning
expectations, or other names.
• Content Standards: What content students learn
• Performance Standards : What students should do
with the content they learned

These standards are further divided into benchmarks

[Grade level classification] and translated into
learning objectives/outcomes.
 Learning Outcomes

Learning outcomes carry a more specific

meaning. They describe what the students should
be able to do or demonstrate, in terms of
particular knowledge, skills and attitudes, by the
end of the programme/course.
 Learning outcomes
A learning objective/outcome specifies what a
teacher /school aims students to achieve when
they have completed an instructional segment
(could be a lesson/unit/semester/academic year).
The focus of any instructional segment is
twofold:
• Achievements [Mastery]
• Processes [developmental
 SLOs

The Student Learning Outcomes are detailed

statements describing what students are
supposed to learn, know and able to do at each
grade level in order to achieve the specified
benchmarks.
A specific learning outcome is a clear statement
about what students are to achieve at the end of a
unit of instruction – easily assessable through
single method – they are called mastery learning
objective.

• Can you give some examples?

 Examples of learning outcomes

• Measure the length of objects to the nearest tenth of a

meter using a meter stick.
• Measure the mass of objects to the nearest tenth of a
kilogram using a simple beam balance and one set of
weights.
• Measure the volume of liquids to the nearest tenth of a
liter using a graduated cylinder.
• Estimate the measurement of a scale
• Solve problems using the formula
 Function of SLOs
Learning outcomes are thesis statements that
guides a teacher to:

• Decide what students are to learn.

• Carry out the actual instruction.
• Assess the learning.
 SLOs: Function

From the assessment viewpoint, an outcome is

specific and positive cognitive, psychomotor or
affective changes examiners wish to discover
that examinees have achieved (p:19, Shawer,
2012).
 What if there is no alignment

• Fragmented and unfocused learning

/assessment
• Authenticity of assessment result is
questionable
• [Learning outcomes are thesis statements for
test design]
 Functions: SLOs in Test Design

• Establish linkages between curriculum and

testing
• Establish test validity [ one approach]
• Establish transparency in assessment
• Minimizes test givers‟/designers'‟ biases in
test design
 Taxonomy
To begin with learning outcomes , a
teacher/examiner requires to use a learning
taxonomy. What is a Taxonomy?
Taxonomies are highly organized schemes for classifying learning
objectives/outcomes into various levels of complexity/ or domains.

• Taxonomies are not teaching hierarchies (not suggest teaching

order) .Their only purpose is to classify various learning
objectives/outcomes and assessment tasks.

• Taxonomy can help a teacher/assessor bring to mind the wide range of

important learning objectives and thinking skills.

• Taxonomy gives a teacher a tool to judge whether they have taught and
assessed a wide enough range of learning objectives (including a wide
range of thinking skills) – improve assessment validity.
 Bloom Taxonomy
Cognitive Domain - they deal primarily with
intellectual knowledge and thinking skills.

An example:
• Ability to read a claim made by an author and
determining whether there is evidence
available to support that claim.
 Bloom Taxonomy
Affective Domain-they deal with how students
should feel or what they should value [it includes
disposition, interest, attitude etc].

[An example: aim students to value the right to

vote in elections over other activities competing
for their time].
 Bloom Taxonomy
Psychomotor Domain - they deal primarily with
motor skills and physical perceptions.

• [An example: to set up, focus, and use a

microscope properly during a science
investigation]
 Explaining cognitive levels

Remembering: recall or remember the information

Understanding explain ideas or concepts

Applying use the information in a new way/ in
practical situation

Analyzing distinguish between the different

parts/components etc.

Evaluating Summarise the key idea

Creating create new product or point of view
Refreshing Bloom Taxonomy

What is meant by … [K
Convert fraction into … [U
How is ….an example of …[A
Classify accordingly …[An
What would you infer …[syn
What priority would you give ..[E
 More examples
Assessment task/test item
• Remember • Make a timeline of key events
Know important dates of the
Civil War • Defend one point of view on
the issue
• Understand
Explain different political • Construct a decision chart
perspectives of an energy policy that illustrates correct
procedures to follow in the
• Application event of an accident or
emergency
Apply lab safety procedures
Bloom Taxonomy: Cognitive Domains
 Review an examination paper/test

Select an examination paper of your subject,

examine and Present:
• What was the learning outcome assessed
through this item?
• What cognitive level does the outcome reflect?
• Is the item aligned with the SLO? Why or Why
not?
• What would you suggest to establish the
alignment.
 Template
Test item Examined learning Cognitive level Comment: aligned
outcome (s) or not aligned
 SOLO Taxonomy
• The SOLO taxonomy was created by carefully analyzing
student responses to assessment tasks (Biggs & Collis, 1982;
Collis & Biggs, 1986)

• These responses were validated for use in a wide range of

disciplines (Hattie & Brown, 2004).

• In developing SOLO, Biggs and Collis took into account many

factors that affect student learning, such as: student’s prior
knowledge and misconceptions, motivation and intentions
regarding education and their learning strategies.
 Stage 1: Pre-structural Level
Stage of Ignorance

The student does not have any kind of understanding but

uses irrelevant information and/or misses the point
altogether.

– Scattered pieces of information may have been acquired,

• Unorganized,
• Unstructured
• Essentially void of actual content or relation to a topic or problem.
Stage 1: Pre-structural Level [Contd…]
– Students do not yet understand the knowledge they are
supposed to learn, maybe because they haven’t been
taught it yet.

– Students do not know how to gather information about

the relevant idea, how to organize it and relate it with
other ideas.

– There is no outcomes or levels of Bloom’s Taxonomy

associated with this level of SOLO.
Stage 1: Pre-structural Level [Contd…]

• For Example the question “Why use the SOLO

taxonomy to create learning outcomes?”

• Pre-structural responses might include:

– “I don’t know”  
– “We’re supposed to” 
– “That’s what good learning is all about” 
 Stage 2: Uni-structural Level
Stage of Surface Learning

• The student can deal with one single aspect and

make obvious connections.

– The student can use terminology

– Recite (remember things)
– Perform simple instructions/algorithms
– Paraphrase, identify, name, count, etc.
Stage 2: Un-structural Level [Contd…]
At the uni-structural level, students have learned a
relevant aspect of the whole

– May make simple and obvious connection

– Meaning, value and significance of the idea may still be
unclear
– Unnecessarily oversimplified
– Still they may be able to achieve outcomes connected to
the levels of knowing and understanding of Bloom’s
Taxonomy
Stage 2: Un-structural Level [Contd…]

Common verbs for uni-structural learning

outcomes (adapted from Brabrand & Dahl,
2009)

– Paraphrase, Define, Identify, Count, Name, recite,

follow simple instructions, calculations, calculate,
reproduce, arrange, recognize, find, note, seek,
sketch, pick.
Stage 3: The Multi-structure Level

• At this level the student can deal with several aspects but these are
considered independently and not in connection.

– They may see several ideas related to each other in some ways, though may
not understand the connection very well

• Metaphorically speaking; the student sees the many trees, but not
the forest.

• He is able to enumerate, describe, classify, combine, apply methods,

structure, execute procedures, etc.
 Stage 3: The Multi-structure Level [Contd…]

Common verbs for learning outcomes (adapted from

Brabrand & Dahl, 2009).

• Combine, classify, structure, describe, list, apply

method, account for, execute, formulate, solve,
conduct, prove, illustrate, characterize.
 Stage 4: Relational Level
Stage of Deep Learning

• The student may understand relations between several aspects

and how they might fit together to form a whole.

• The understanding forms a structure and now he/she does see

how the many trees form a forest.

• They may understand some “behind the scenes” meta-

connections find relationship between theory and practice,
purposes and the significance of ideas is clearer to them.
 Stage 4: Relational Level [Contd…]

Common verbs for relational learning outcomes

(adapted from Brabrand & Dahl, 2009).

• Analyze, compare, contrast, integrate, relate,

explain, cause, apply theory (to its domain),
argue, implement, plan, contrast, design,
conclude, derive, exemplify etc.
 Stage 5: The Extended Level
Stage of Deep Learning

• A student may generalize structure beyond what was

given

• May perceive structure from many different

perspectives

• Transfer ideas to new areas.

 Stage 5: The Extended Level
Common verbs for relational learning outcomes
(adapted from Brabrand & Dahl, 2009).

• Theorize, generalize, hypothesize, predict,

judge, transfer theory (to new domain),
evaluate, critically reflect, predict, criticize,
reason.
 Summing up
• Don't blame the students
• Don’t blame the teachers
• Don’t blame teaching tools
• Do blame the lack of alignment
• Do blame the lack of aligned assessment
method
(Tyler, 2001)
 SOLO
• solo-taxonomy-with-verbs.jpg
• primer-on-learning-outcomes.pdf
• SOLO-Functioning-Knowledge-Rubric-Templat
e.pdf
Sunday Class Task: Review Your Learning
Reading: Wiliam & Black (2014) Inside the Black Box Raising Standards Through Classroom
Assessment

Discussion Questions

1. How do they define formative assessment?

2. According to the authors teaching and learning must be interactive; do you agree, if
so, what is the role of assessment to make teaching and learning interactive?
3. Does Improving Formative Assessment Raise Standards? Why and how?
4. What are the shortcomings of assessment discussed in this paper?
5. How can we improve formative assessment, from the authors’perspectives?

One of you should lead the discussion around the questions given. (20 minutes)