Professional Documents
Culture Documents
The Effect of Metacognitive Training On Mathematical Word-Problem Solving
The Effect of Metacognitive Training On Mathematical Word-Problem Solving
The Effect of Metacognitive Training On Mathematical Word-Problem Solving
Introduction
Evidence from Schoenfeld (1985) suggests that compared with an ‘expert’ problem
solver, ‘novices’ lack essential metacognitive monitoring, assessing and decision
making skills. According to Schoenfeld, these are essential elements that determine
one’s success or failure in problem solving, and there is a need for students to be
explicitly trained to monitor their cognitive processes during problem solving.
Cardelle-Elawar (1995) trained low achievers with a metacognitive intervention
strategy and observed its influence on their mathematics performance. The
researcher reported that low achievers trained in learning to monitor and control their
own cognitive processes for solving mathematics problems do better than untrained
students. However, research that focuses on metacognition in problem solving are
usually carried out in a non-computer environment. Over the past decade, attempts
have been made to employ metacognitive training within computer environments
(Mevarech & Kapa, 1996; Kramarski & Ritkof, 2002). For example, Kramarski &
Ritkof (2002) used a teaching/learning strategy called IMPROVE when students
were learning about graphs in EXCEL environment. IMPROVE is the acronym for
the teaching/learning stages that require students to focus on their metacognition.
The researchers reported positively that students who were exposed to metacognitive
Methodology
Participants
Forty 11 to 12-year-old low achievers were involved in this intensive study over a
period of eight weeks. They were chosen according to their year-end mathematics
examination results. All low achievers had scores between 50% and 70% and this
was in accordance with the categories of scores in the national examination grading
system∗.
CRIME strategy
CRIME is an acronym for the word-problem solving stages: Careful Reading; Recall
∗
Singapore primary education consists of a foundation stage of four years (7 to 10 years old) and an
orientation stage of two years (11 and 12 years old). At the end of orientation stage at Primary Six,
students are required to sit a national examination, the Primary School Leaving Examination (PSLE). The
scores in this examination determine the type of programme (Special, Express, Normal(A) and
Normal(T)) students follow when they enter secondary school. The students are given a grade for each
subject in their PSLE result. The grades and corresponding scores are defined as follows:
A* 91%* and above: A 75% to 90%: B 60% to 74%; C 50% to 59%: D 35% to 49%: E 20% to 34%;
U Below 20%.
The present study focused on students who had scored at least a grade C in their year-end mathematics
examination.
2003 Blackwell Publishing Ltd, Journal of Computer Assisted Learning, 19, 46-55
48 S.K. Teong
Possible Strategies; Implement Possible Strategies; Monitor, and Evaluation which
was developed by the author. Its aim is to develop low achievers’ ability to monitor
and evaluate their actions during word-problem solving using WordMath (for details
see http://math.nie.edu.sg/skteong/WordMath.doc). At each stage of the word-
problem solving, there are questions to direct the students to regulate and monitor
their solution. In the present study, students solved word-problems in the WordMath
environment with the metacognitive strategy, CRIME. It was piloted with two
students and there were indications that CRIME offered an effective metacognitive
strategy to make students more aware of their cognitive processes during word-
problem solving. (see Appendix A for further description)
Design
The study adopts a two-phase design, combining an experimental design and a case
study design. The experimental design used the analysis of students’ mathematical
achievement test data, whereas the case study design used the analysis of
collaborative think-aloud protocol data.
Instruments
The experimental design was based on word-problem tests, where the students’
performance in completing word-problem items (see Appendix B) were used to test
the hypothesis generated in the experimental design. The word-problems were the
kinds of tasks students were expected to ‘master’ in school and on which their
mathematical performance was systematically monitored and evaluated. They were
also designed to reflect the problems used in the School Leaving Examination.
For the case study, students solved word-problems in WordMath and their think
aloud protocols were video-recorded and analysed to elicit students’ word-problem
solving activity. Criteria for choice of word-problems were similar to those for the
mathematics achievement test items.
Procedure
The experimental design. Students were assigned to two groups: one group had
explicit metacognitive training with CRIME before solving word-problems in
WordMath (Experimental or E); and one group solved word-problems with
WordMath without CRIME (Control or C). In the experimental design, all groups
took a pre-test consisting of 10 word-problems on numbers and fractions (see
Appendix B). Written post-test 1 was immediately administered after intervention.
Six weeks after post-test 1, a final written delayed post-test 1 was administered.
The case study design. Two pairs of students were selected from each group, based
on their ability to work together. The four pairs of students underwent two extra
training sessions where the students solved four word-problems for each session.
Their word-problem solving behaviours were video-recorded but not analysed. The
purpose of these extra sessions was to enable students to feel comfortable working
collaboratively in front of the video camera. After these sessions, the pairs of
students’ word-problem solving think-aloud protocols of four word-problems (post-
test 2) were video-recorded. Six weeks after post-test 2, due to logistics and time
constraints, only two pairs of experimental students were granted permission to
participate in the case study. Their word-problem solving think-aloud protocols of
four parallel word-problems (delayed post-test 2) were video-recorded and analysed.
2003 Blackwell Publishing Ltd, Journal of Computer Assisted Learning, 19, 46-55
Mathematical word-problem solving 49
Intervention
Both experimental and control groups were trained separately, but each group had
four 60-minute training sessions on the use of WordMath with/without metacognitive
training over a period of two weeks. All students worked collaboratively in pairs
throughout the study. During the first training session, experimental students were
taught how to use WordMath and encouraged to think aloud. During the second
training session, experimental students were given direct explanation, explicit
demonstration, and ‘scaffolded’ (Bruner, 1985) practice in using CRIME while
solving word-problems. Students had the CRIME acronym card at all times and they
were encouraged to refer to CRIME during word-problem solving. During the last
two training sessions, experimental students practised using CRIME to solve word-
problems in WordMath. In contrast, control students received no training, modelling
or instructions in using CRIME. They discussed concepts only in the context of
solving WordMath word-problems.
2003 Blackwell Publishing Ltd, Journal of Computer Assisted Learning, 19, 46-55
50 S.K. Teong
scores on the three tests. As the groups were different in the pre-test scores, this table
also shows the adjusted data for the post-test and delayed post-test scores where the
pre-test scores were used as the covariate. A repeated measures analysis of
covariance (ANCOVA) was carried out on the post-test and delayed post-test with
corresponding pre-test scores again used as a covariant. The repeated measures
analysis of covariance indicated significant differences between experimental and
control groups (F1,37 = 9.008 , p = 0.005).
Table 1. Means and s.d. of low achievers’ mathematical achievements (n = 20 in both groups)
E x p lo r e ( M )
P la n (M )
Im p le m e n t ( C )
Im p le m e n t (M )
V e r if y ( C )
V e r if y ( M )
0 4 8 1 0 1 2 1 4
A n a ly s e (M )
E x p lo re (C )
E x p lo re (M )
P la n (M )
I m p le m e n t ( C )
Im p le m e n t ( M )
B e h a v io u r s T i m e t a k e n b y p a ir
V e rify (C ) M e t a c o g n it i v e 9 7 8 s
C o g n it i v e 4 2 8 s
V e rify (M ) T o ta l tim e ta k e n = 1 4 0 6 s
% o f tim e ta k e n o n m e ta c o g n itiv e b e h a v io u rs = 6 9 .7 %
8 1 6 2 0 2 8
A n a ly s e (M )
E x p lo re (C )
E x p lo re (M )
P la n (M )
Im p le m e n t ( C )
I m p le m e n t ( M ) B e h a v io u r s T i m e t a k e n b y p a ir
M e ta c o g n itiv e 3 0 4 s
V e r if y (C ) C o g n it iv e 3 3 5 s
T o ta l tim e t a k e n = 6 3 9 s
V e rif y (M ) % o f tim e ta k e n o n m e ta c o g n itiv e b e h a v io u r s = 4 7 .6 %
0 4 6 8 1 0
T im e (m in u te s )
A n a ly s e (M )
E x p lo re (C )
E x p lo re (M )
P la n ( M )
Im p le m e n t ( C )
I m p le m e n t ( M ) B e h a v io u rs T im e ta k e n b y p a ir
M e t a c o g n itiv e 1 33 s
V e rify (C ) C o g n itiv e 2 94
T o ta l tim e t a k e n = 4 2 7 s
V e rify (M ) % o f tim e ta k e n o n m e ta c o g n itiv e b e h a v io u r s = 3 1 .2 %
2
T im e ( m i n u t e s )
Discussion
The present study compared the effects of two learning conditions in a cognitive-
apprenticeship, computer-based environment on word-problem solving. Specifically,
it aimed to answer the following questions:
2003 Blackwell Publishing Ltd, Journal of Computer Assisted Learning, 19, 46-55
52 S.K. Teong
• Do experimental students outperform control students on ability to solve word-
problems on their individual written test?
• Can experimental low achievers develop the ability to ascertain when to make
metacognitive decisions, and elicit better regulated metacognitive decisions than
lower achievers under the control condition? If so, do they require some time to
internalise the positive benefits of metacognitive training.
• Does the occurrence of metacognitive behaviours on its own ensure successful
word-problem solving?
• What is the role of WordMath in low achievers’ cognitive activity?
Important findings emerged from this study. First, the experimental students
outperformed control students on ability to solve word-problems on their individual
test measures. Second, the experimental low achievers developed the ability to
ascertain when to make metacognitive decisions and elicit better regulated
metacognitive decisions than control lower achievers, but they appear to require
some time for internalisation to occur before the positive benefits of metacognitive
training could prevail. Third, possessing metacognitive strategies is not sufficient. It
is the ability to know when and how to use metacognitive behaviours, when they are
needed, that is an important determinant to successful word-problem solving. Fourth,
WordMath appears to enhance experimental low achievers’ metacognitive
behaviours during word-problem solving, and also appears to enhance low
achievers’ metacognitive and cognitive behaviours during word-problem solving.
These findings are examined below in more detail.
Can experimental low achievers develop the ability to ascertain when to make
metacognitive decisions, and elicit better regulated metacognitive decisions than
control lower achievers?
There is no evidence from the case study design, however, that metacognitive
training inevitably leads to effectiveness in mathematical word-problem solving
performance. Visual inspection suggests that some treatment lower achievers needed
some time for internalisation to occur before positive metacognitive influence could
prevail in their word-problem solving from post-test to delayed post-test. For
example, there was evidence that S3 & S4 (E) were more successful, and were
devoting more time to regulating and monitoring their word-problem solving process
even after a prolonged period of six weeks with metacognitive instruction (see
Table 2). However, it was not known, in the present study, if this delayed
improvement on the pair of experimental low achievers would also prevail in their
word-problem solving over an extended period of time (e.g. after several months).
This issue merits further attention.
2003 Blackwell Publishing Ltd, Journal of Computer Assisted Learning, 19, 46-55
Mathematical word-problem solving 53
Does the occurrence of metacognitive behaviours on its own ensure successful
word-problem solving?
The analysis of transcripts of collaborative word-problem solving of four pairs of
students by using modified Artzt and Armour-Thomas’ framework also revealed that
the ability to know when and how to use metacognitive behaviours when they are
needed are important determinants to successful word-problem solving. Mayer
(1998) posits that one of the factors that discriminate successful problem solvers
from unsuccessful ones is their ability to know not only what to do, but also when to
do it with regard to using their cognitive skills. In this study, S3 & S4 (E), who
devoted 69.7%, and S5 & S6 (C), who devoted 47.6%, to metacognitive activities
were unsuccessful in solving the Q2 word-problem (see Table 2 and Figs. 2 & 3).
Though they monitored their activities, evidenced by the occasion ‘It cannot be’,
they failed to obtain the correct solution. In contrast, the protocol for S1 & S2 (E)
could be summarised as a well-regulated progression of activity (which led to their
success in their word-problem solving (see Fig. 1). They were also in control of their
cognitive actions throughout their word-problem solving.
The above examples suggest that the occurrence of metacognitive behaviours on
its own does little to ensure successful word-problem solving. This concurs with a
study (Stillman & Galbraith, 1998) which reveals that although successful groups in
displayed a high number of key points where metacognitive decisions would
influence cognitive actions, this alone was not a guarantee for success. Their advice
is that the focus of metacognitive training is to provide facilities that enable students
to develop a rich store of metacognitive strategies over an extended period of use.
2003 Blackwell Publishing Ltd, Journal of Computer Assisted Learning, 19, 46-55
54 S.K. Teong
During delayed post-test 2, S3 & S4 used WordMath tools to draw diagrams in
order to represent the word-problem. Their representations were appropriate and
they were successful in 50% of their word-problem solving. Based on the above
observations, it is appropriate to suggest that WordMath appears to amplify
experimental low achievers’ metacognitive behaviours during word-problem solving,
and also appears to amplify low achievers’ metacognitive and cognitive behaviours
during word-problem solving. These assertions merit further research.
Conclusion
The findings of this study provide evidence that the role of low achievers’
metacognition, influenced by metacognitive training in a cognitive-apprenticeship-
computer-based environment, can play in contributing to low achievers’ word-
problem solving performance. This represents a contribution to research on low
achievers’ metacognition influenced by metacognitive training in a computer
environment as well as providing a way forward in understanding how this influence
can help low achievers’ in word-problem solving.
References
Artzt, A. & Armour-Thomas, E. (1992) Development of a cognitive-metacognitive framework
for protocol analysis of mathematical problem solving in small groups. Cognition and
Instruction, 9, 137–175.
Bruner, J. (1985) Vygotsky: a historical and conceptual perspective. Culture, Communication
and Cognition: Vygotskian Perspectives (ed. J. Wertsch) pp. 21-34. Cambridge
University Press, Cambridge.
Cardelle-Elawar, M. (1995) Effects of teaching metacognitive skills to students with low
mathematics ability. Teaching and Teacher Education, 8, 109–121.
Dörfler, W. (1993) Computer Use and Views of the Mind. Learning from Computers. In
Mathematics Education and Technology (eds. C. Keitel & K. Ruthven) pp. 159–186.
Springer-Verlag, Berlin.
Kramarski, B. & Ritkof, R. (2002) The effects of metacognitive and email interactions on
learning graphing. Journal of Computer Assisted Learning, 18, 1, 33–43.
Looi, C.K. & Tan, B.T. (1998) A Cognitive-Apprenticeship-Based Environment for Learning
Word Problem Solving. Journal of Computers in Mathematics and Science Teaching, 17,
4, 339–354.
Mayer, R. (1998) Cognitive, Metacognitive, and Motivational Aspects of Problem Solving.
Instructional Science, 26, 1–2, 49–63.
Mevarech, Z. & Kapa, E. (1996) The effects of a problem-solving based Logo environment
on children’s information processing components. British Journal of Educational
Psychology, 66, 181–195.
Noss, R. & Hoyles, C. (1996) Windows on Mathematical Meanings. Learning Cultures and
Computers. Kluwer Academic Publishers, Dordrecht.
Schoenfeld, A. (1985) Mathematical Problem Solving. Academic Press, San Diego.
Schoenfeld, A. (1992) Learning to think mathematically: problem solving, metacognition and
sense making in mathematics. In Handbook of Research on Mathematics Thinking and
Learning (ed. D. Grouws) pp. 334–370. MacMillan, New York.
Stillman, G. & Galbraith, P. (1998) Applying Mathematics with Real World Connections:
Metacognitive Characteristics of Secondary Students. Educational Studies in
Mathematics, 36, 157–195.
2003 Blackwell Publishing Ltd, Journal of Computer Assisted Learning, 19, 46-55
Mathematical word-problem solving 55
Teong, S.K. (2000) The Effect of Metacognitive Training on the Mathematical Word Problem
Solving of Singapore 11–12 year olds in a Computer Environment. PhD Thesis.
University of Leeds, UK.
Verschaffel, L., De Corte, E., Van Lasure, S.V.G., Bogaerts, H. & Ratinckx, E. (1999)
Learning to solve mathematical application problems: a design experiment with fifth
graders. Mathematical Thinking and Learning. An International Journal, 1, 3, 195–229.
Appendix A: CRIME
2003 Blackwell Publishing Ltd, Journal of Computer Assisted Learning, 19, 46-55