IMPLEMENTING THE HARVARD PEER INSTRUCTION
METHOD IN CEGEP
Peer Instruction (PI) is a student-
centered educational method de-
veloped at Harvard by physicist
Eric Mazur (Mazur, 1997). In PI,
students use remote-control clickers
that provide teachers with real time
feedback for modeling the ongoing
course. The method has been fa-
vourably received by the scientific
community and implemented in va-
rious disciplines by a large number
of US universities, given its common
sense and documented effectiveness
(Fagen, Crouch and Mazur, 2002;
Crouch and Mazur, 2001; Mazur,
1997).
This article has four objectives. The
first is to present the PI method to
college teachers. The second is to
assess whether or not PI is appli-
cable to the CEGEP environment.
Even though the method has been
used effectively for over 15 years
in US universities, this is the first
study1 to document its effective-
ness and applicability to a CEGEP
in Quebec. The third objective is
to determine whether PI is more
effective than traditional CEGEP
teaching. The last objective is to
establish the contribution of clic-
kers to learning.
1
This study received a grant from the ministère
de l’Éducation, du Loisir et du Sport as part of the
Programme d’aide à la recherche sur l’enseignement
et l’apprentissage (PAREA).
1 PÉDAGOGIE COLLÉGIALE VOL. 21 NO 4 SUMMER
NATHANIEL LASRY
Professor of physics
John Abbott College
WHAT IS PI?
SHORT HISTORY
In 1992, a team of physicists published a non-numerical diagnostic test of basic
Newtonian physics concepts called Force Concept Inventory (I. A.Halloun, Hake,
Mosca and Hestenes, 1995; Hestenes, Wells and Swackhammer, 1992). The authors
of FCI designed the test to quantitatively measure the extent of preconceived ideas
– often Aristotelian (DiSessa, 1982) – that students have of the world, in spite of
previous training in physics. The FCI, a multiple-choice test, is unique in that it
poses conceptual questions in simple terms and offers “distractors”, that is, other
choices representing the erroneous concepts most frequently provided by students
during interviews (I. Halloun and Estenes, 1985a, 1985b). To begin with, the test
is easy. Students do not have to resort to memorized calculations or algorithms
to answer the FCI questions. They are only required to pick the correct concept
from among the “distractors”. By bringing these erroneous preconceptions to the
forefront, the FCI makes it clear that students do in fact come to class not without
knowledge but with many preconceptions.
Mazur decided to administer the FCI to his students at the end of the session. He
minimized its importance for fear the students would mock such a rudimentary test.
Quite to the contrary, the test left the students rather perplexed as expressed by the
following question asked by one student: ‘‘Professor Mazur, how should I answer
these questions? According to your teaching or based on my own understanding
of these things?’’ (Mazur, 1997)
To Mazur’s great surprise, not only had students failed to understand several
fundamental concepts after one or two years of high school training in physics (which,
after all, enables them to enrol in Harvard), but a large number of preconceptions
remained even after a session of his own teaching! Some students with the best
grades had not grasped several basic concepts (Mazur, 1997). Coincidentally, this
proved to be one of the most revealing conclusions of FCI data analysis. In fact, a
meta-analysis of over 6,500 participants (Hake, 1998) shows that one traditional
didactic session only marginally modifies students’ conceptual understanding.
To begin with, this test is easy. Students do not have to resort to memorized
calculations or algorithms to answer the FCI questions.
DESCRIPTION OF THE METHOD
Mazur developed PI as an explicit response to his students’ preconceptions.
This required some modifications to his traditional teaching method. In PI, the
teacher makes a brief presentation to the students (7-10 minutes, within the
average adult attention span). The content of the presentation is similar to the
traditional curriculum except that basic concepts are emphasized to a greater
2008
degree. After the short presentation, students respond to a ConcepTest: a multiple- is also an increase in the level of con-
choice conceptual question. In order to evaluate in real time what students think, fidence expressed by the students for
the students received five flashcards, each displaying a letter (A, B, C, D, and E the correct answer (Crouch and Mazur,
corresponding to the five possible choices). When taking the ConcepTest, students 2001; Fagen et al., 2002; Mazur, 1997).
are required to raise their flashcard (the one corresponding to their answer) so
the teacher can see it.
UNDERLYING
This enables the teacher to see in real time the approximate proportion of correct RESEARCH QUESTIONS
answers as well as the distribution of preconceptions. To get a more precise idea on
the distribution of answers, Mazur later replaced the flashcards with small wireless This study focuses on the following three
devices like TV remotes. research questions:
1. Is the Harvard PI method
The ability to assess the extent of the group’s understanding in real time makes applicable to CEGEP?
it possible for teachers to make a decision as to the progression of the course. a. Is the method adapted to
the institutional constraints?
To answer a conceptual question, students simply press on the key that corresponds b. Can course structures be
to the number of their answer and the data is instantly transmitted to the teacher’s modified simply and in a
computer. Using clickers also makes it possible for students to state their level of feasible manner?
confidence (3 levels: High=H, Average=A, and Low=L) for each answer. In this way c. How is PI received by
the teacher can instantaneously determine the exact percentage of students with the administrators, fellow
correct answer as well as the distribution (in percentage) of each preconception. teachers and students?
The ability to assess the extent of the group’s understanding in real time makes 2. Is the PI method more
it possible for teachers to make a decision as to the progression of the course. effective than the traditional
If the concept under study is badly understood (<30% correct answers to the didactic teaching methods used
ConcepTest), the teacher will review the concept and explain it in greater detail in CEGEP?
before resubmitting the group to the ConcepTest. However, if the ratio of correct
answers is very high (>80%), then the great majority of students have properly a. Does PI increase
assimilated the concept. The teacher can then explain why the remaining choices conceptual learning?
are erroneous before moving on to the next concept. In most cases, the ratio of b. Does PI reduce traditional
correct answers is neither very high nor very low. When there is a moderate range problem-solving abilities?
of correct answers (30%-80%), the teacher asks the students to try and convince
each other of their choices. This leads to a 2-to-3-minute discussion between the 3. Is PI more effective with clickers
students: Peer Instruction in action. than with flashcards?
a. Does using clickers increase
This discussion forces students to express their thoughts clearly, which leads
the conceptual change?
them to a better representation of the concept. What’s more, a discussion among
students sidesteps the element of authority that could intervene in a discussion b. Does using clickers impact
with the teacher. Students are more likely to take the teacher’s explanation as students` traditional
‘‘fact’’ without developing as elaborate a form of reasoning as they would if they problem-solving abilities
were discussing among peers. In addition to producing a more balanced debate, in any way?
students also argue from conceptual starting points often unfamiliar to expert
teachers. Due to their situation, students are often better equipped than teachers
to understand the erroneous concepts of their peers, and this often facilitates
conceptual modifications. After the discussion, students are submitted to the same DESCRIPTION OF STUDY AND
ConcepTest and asked to vote again. The teacher then reveals the correct answer EXPERIMENTAL DESIGN
and explains why the remaining preconceptions are false.
The following design was used to ans-
According to results obtained in US universities, following a discussion among wer the second and third empirical
peers, not only does the rate of correct answers increase significantly but there questions. Each student participating

SUMMER 2008 VOL. 21 NO 4 PÉDAGOGIE COLLÉGIALE 2

in the study was randomly assigned to
one of the three following experimental
groups (PI groups and control group):
• PI group with clickers (n=41)
• PI group with flashcards (n=42)
• Control group (n=44)
The PI groups were taught by the author.
The control group teacher was selected
among the physics teachers because he
was the closest to the author in terms
of age (+/-3 years), years of teaching
experience (+/- one year) and sex (M).
Furthermore, this choice was also mo-
tivated by the fact that students often
described these two teachers as having
similar teaching styles.
To isolate the specific contribution of
technology in PI, the ‘‘clicker’’ group
was compared to the ‘‘flashcard’’ group.
However to compare the effectiveness
of PI on a conceptual learning level, the
two PI groups were merged and com-
pared to the control group.
INSTRUMENTS
Three different quantitative variables
were measured in this study, they are:
traditional problem solving (Exam),
conceptual learning (FCI) and concept-
confidence (Conf).
EXAM
Aptitudes in physics have been tra-
ditionally measured by algorithm
problems. In this study, these skills
were measured using the final sum-
mative exam for the physics department.
This exam, designed by a committee of
three physics teachers (none of which
was involved with this study) required
unanimous approval by all personnel
teaching the course this trimester (10-
12 teachers). In addition, instead of
correcting 10 to 12 questions for each
3 PÉDAGOGIE COLLÉGIALE VOL. 21 NO 4
student, each teacher will only correct one answer, but for the entire cohort (10 to
12 groups or approximately 500 students). Having the same exam for all students,
with each question being corrected by a single teacher, ensures the homogeneity
in grading since all groups will have an exam with the same level of difficulty and
be corrected in similar manner.
CONCEPTUAL LEARNING : THE FCI
In physics, students may be able to solve certain problems without having mastered
a basic understanding of the underlying physics principles (Kim and Pak, 2002).
The students’ conceptual understanding was therefore measured during the first
and last week of the session using the Force Concept Inventory (I. A. Halloun and
al., 1995; Hestenes and al,. 1992). To avoid any ceiling or floor effects, conceptual
gains were standardized and compared.
g = (Post T – Pre T)/ (max T – Pre T) Eq.1
This measurement gives us the gross conceptual gain (Post-test – Pre-test) obtained
between the first and last week of the session in relation to the maximum possible
gain. For instance, a gain of 0.4 indicates that 40% of the remaining concepts to be
learned were acquired at the end of the course. Avoiding ceiling and floor effects,
this measurement of standardized gain is the most frequently used measurement
to express FCI results.
Teachers in the other departments have also been introduced to the method
via lectures given in the college and by word of mouth.
RESULTS
APPLYING PI IN CEGEP
Applying new and somewhat costly educational methods in public institutions often
runs into difficulties. However, the John Abbott college administration was behind the
project from the very start. From the head of the physics department to the academic
dean and the director of the sciences program, each intervener showed a genuine
interest in PI and was more than supportive in helping implement the method.
Since the first test in physics, more than half of full-time members in the physics
department (8/14) are actually using one form or another of PI in their class (with
or without clickers). Teachers in the other departments have also been introduced
to the method via lectures given in the college and by word of mouth. One teacher in
the chemistry department used clickers successfully in his introductory course and
plans on repeating the experience. A nursing teacher is currently using the method
in her courses. A number of other teachers have inquired about the material and
could soon decide to use it in their classrooms. Based on the reception received at
the different administration levels as well as from the teachers in various fields, we
can truly say that PI was warmly received by our CEGEP community.
RECEPTION FROM STUDENTS
Students were very receptive to using clickers in class. Students in the flashcard
section were also satisfied with using flashcards. However, this satisfaction quickly
SUMMER 2008
vanished when these students found out that the other section was using clickers. In Figure 5
order to measure the evaluation of the PI method in both the clicker and flashcard Unsolicited drawings
sections, students were asked to rate their level of agreement (5=totally agree to by students in lab class
1=totally disagree) with each of the seven statements below.

1. PI helped me recognize what I misunderstood.

2. PI showed me that other students had similar preconceptions to mine.
3. I actively discuss problems with classmates.
4. Having to convince other students helps me understand the concepts.
5. The mini presentations help clarify my understanding of the concept.
6. I learn better with PI than with traditional courses.
7. If I had to choose between a PI course and a traditional course, I would choose PI.

The answers were then grouped into three categories: agree/totally agree; neutral;
disagree/totally disagree. To determine if students agreed with the statement
beyond the expected probability, (2/5 or 40%), we calculated a binomial probability
(agree p=0.4; disagree q=0.6; n=30).

Results show that students in the sections using either flashcards or clickers
responded positively to PI by recognizing its advantages as statistically significant
(p<0.05) as an educational approach (Q1-Q5) and by preferring it to traditional
teaching (Q6 and Q7). In addition, 61% of students in the section using flashcards
who were using clickers for the first time agreed that they would have been more
actively involved if they had been given access to clickers rather than flashcards.

Other unsolicited yet interesting student reactions were found in the form of small
computer drawings produced with Microsoft Paint and used as desktop screensavers
on the computers in the physics laboratory. The Figure 5 presents the images that
were found on computer screens in the physics laboratory after being used by PI
students. These images were not present before the students arrived in the lab.

MODIFYING THE COURSE STRUCTURE

Using PI with clickers in class requires only minimal changes. To administer the
ConcepTests where students vote using a clicker, you can enter or import the
multiple-choice questions in PowerPoint. A number of ConcepTests can be found
on the Internet at Harvard’s Project Galileo website2 or through publishers who
include “clicker questions” in their manuals. Today, there are sufficient resources
available that make using ConcepTests quite feasible.

Another change linked to the clicker technology is the familiarization with clicker
material and software. It is strongly recommended that all interested teachers set
up the clickers and receivers and try them a few times before attempting to use
them in class.

2 [Online] http://galileo.harvard.edu/

SUMMER 2008 VOL. 21 NO 4 PÉDAGOGIE COLLÉGIALE 4

In response to the first research ques-
tion on applying the method in CEGEP,
the results indicate that PI is quite fea-
sible at CEGEP level. The modifications
required to course structures are minor
and achievable and the approach has
been well received by administrators,
teachers and students.
This being said, an educational method
may be well appreciated without ne-
cessarily being effective. The following
section examines the effectiveness of PI
in CEGEP.
as PI. One of the concerns put forth is that the time spent explaining simple concepts
implicitly takes away time for learning problem solving; an aptitude that must be
demonstrated in summative exams. The following table shows the average obtained
per group for the final exam.
These results indicate that PI students obtain better exam results, although not
significantly (p=21), than their peers in the control group. We can, at the very
least, affirm that the PI conceptual approach does not harm students as regards
problem resolution. This is very likely due to the positive contribution of conceptual
knowledge to problem-solving ability.
Table 1 Table 2
FCI results for the PI group Final exam for the PI group
and the control group and the control group

<Pre-test> <Post-test> g
Exam avg
[...] the results indicate that PI is (%) (%) (norm.
(%)
(gain)
quite feasible at CEGEP level.
PI 68.0
PI 42.6 68.6 0.50
(n=79)
(n=69)
Control 63.0
Control 46.0 63.3 0.33
EFFECTIVENESS OF PI VS. (n= 22)
(n=35)
TRADITIONAL DIDACTIC LEARNING t-test (2-tailed) 0.427 0.283 0.008
t-test (2-tailed) 0.21
p
p
CONCEPTUAL LEARNING
In this part of the study, both PI groups
(clickers and flashers) were merged and
their learning measurements were com-
LEARNING DIFFERENCES BETWEEN CLICKERS AND FLASHCARDS
pared with those of the control group.
The following table shows the average To determine the contribution of clickers to learning, the clicker group and the
FCI results obtained per group before flashcard group are compared below:
the course (pre-test) and at the last
week of the session (post-test) as well as Table 3
standardized gains. Differences in problem resolution (exam) and conceptual learning (FCI) between groups

These results demonstrate that even PréFCI PostFCI g Examen

if there was no significant difference /30 /30 (%)
between the groups before teaching Clickers 11,9 19,9 0,486 69,8
(p=0.427), the PI group achieved si- (n = 35)
gnificantly more conceptual gains as Flashcards 13,6 21,3 0,520 71,6
measured by the FCI (p=0.008). This (n = 34)
result confirms that PI is more effective t-test (2-tailed)3 0,209 0,351 0,745 0,630
for conceptual learning rather than p
traditional learning and reproduces as
such the previous results obtained by These results show that in terms of conceptual knowledge, the two groups show
Mazur (1997) and Hake (1998) outside no difference at the beginning (p=0.209) or at the end of the course (0.351) and
American universities. that neither group learns more conceptually than the other (p=0.745). In terms
of problem solving, the average grade for the final exam also shows no difference
TRADITIONAL PROBLEM SOLVING (p=0.630) between both groups. It is therefore possible to conclude that using
Physics teachers are sometimes hesitant clickers neither adds nor takes anything away from learning, whether conceptually
to use non-traditional approaches such or as regards problem solving.

5 PÉDAGOGIE COLLÉGIALE VOL. 21 NO 4 SUMMER 2008

 DISCUSSION OF RESULTS sort out useful ConcepTests from those
that do not work as well. ConcepTests
EFFECTIVENESS COMPARED TO TRADITIONAL TEACHING IN CEGEP with debatable usefulness can then be
reformulated. Clickers could make it
As expected from results of studies conducted in US colleges and universities, possible for a series of questions to evol-
PI in CEGEP allowed for significantly more conceptual learning than traditional ve from one session to the next. Using
teaching (p=0.008). However, similar results may not be enough to convince certain flashcards does not allow the teacher
teachers to adopt the method. Some will say that given the difficulties students to gather data about the ConcepTest
have in solving quantitative problems, allocating time to work on basic concepts itself. As a result, the same questions
proportionately reduces the time allotted to work on problem-solving activities are recycled from one session to the
and negatively affects the ability of these students to solve traditional problems. In next. In terms of learning, questions
fact, the opposite was found. In terms of traditional problem solving, PI students that have been modified as a result of
who spend more time working on concepts obtained better results than the control data collected should reveal learning
group students. So, even though, less time is spent on solving algorithm problems, differences when compared to a fixed
providing students with a strong conceptual basis allows them to be more effective sequence of questions used with flash-
in matters of problem resolution. cards. Some instructors may be aware
of PI methodology and willing to
LACK OF ADDED EFFECTIVENESS WITH CLICKERS reshape their instruction to provide
greater focus on basic concepts. Yet,
One of the interesting and unexpected conclusions of this study is that using
the capital expense for the purchase of
clickers does not provide students with a learning advantage. Certain previous
clickers and related hardware may not
clicker users in university classes reported advantages such as increased attendance
be available and passing the expense
rates and reduced attrition rates (Lopez-Herrejon and Schulman, 2004; Owens
onto the students may not be possible
and al., 2004). Nevertheless, there is no data in this study to support the claim that
or desirable. In this instance, PI should
clickers increase conceptual learning. PI is an elaborate educational approach that
be implemented with flashcards as
clearly emphasizes basic concepts. It requires that students commit to a conception
it is the PI pedagogy that is effective,
and provides a setting for discussion among peers making it possible to go beyond
not the method students use to report
misconceptions. It is obvious that the technology is not the pedagogy. However,
their answers.
since clickers add nothing to learning, should they not be abandoned?

The answer is no. In fact, we should actively encourage using clickers. Although this In terms of traditional problem
conclusion seems to contradict the previous finding, there are three main reasons solving, PI students who spend
for encouraging the use of clickers.
more time working on concepts
First and foremost, even if there is no difference in the amount of learning, obtained better results than the
appreciable differences exist in terms of teaching. It goes without saying that the control group students.
accurate and automatic compilation of the distribution of answers is much more
practical than having to count flashcards over and over again.

Secondly, clickers are to some degree responsible for the attention given to the CONCLUSION
PI method. A large part of this attention is due specifically to the use of clickers
(Burnstein and Lederman, 2001, 2003). Many teachers, including yours truly, A number of science teachers still teach
adopted the method thanks to the appeal of using clickers in their classrooms. So, today the way they were taught, that
implementing PI with clickers forces teachers to reconsider their teaching method, is, the way we were taught during the
concentrate on concepts and, consequently, fundamentally reshape their teaching. last century (Beichner and al., 1999).
Given that many teachers would not have given proper attention to PI had it not Thanks to its appeal, the PI method
been for the clickers, it is important to continue encouraging their use. could bring about a change in the way
in which teachers and students perceive
Last but not least, using clickers in a classroom enables teachers to archive the teaching. Its methodology requires ve-
data obtained in their ConcepTest. Beyond data analyses that can be addressed ry little modification to traditional tea-
in a research framework, the data can also be used in an educational context to ching; it implies that greater emphasis

SUMMER 2008 VOL. 21 NO 4 PÉDAGOGIE COLLÉGIALE 6

be placed on basic concepts. The me-
thod does not come into conflict with
current institutional constraints since
it is generally well received by adminis-
trators, fellow teachers and students.
Although PI was developed at Harvard,
it seems like an educational method
that is effective in CEGEP and would
benefit from being widely used and
encouraged. The method is simple
enough to allow for a systemic change
in a relatively short time. This study al-
so showed that using clickers does not
add anything significant to learning. In
fact, even though using clickers offers
the teacher many advantages, their
use does not increase the effectiveness
of the PI method. The conclusion is
that pedagogy must be seen as being
distinct from technology.
HALLOUN, I. and D. HESTENES, “The Initial Knowledge State fo College Physics Students”, American Journal of
Physics, vol. 53, 1985b, p. 1043-1055.
HALLOUN, I. A, R R. HAKE, E. P. MOSCA and D. HESTENES, “Force Concept Inventor”, in E. Mazur (ed.), Peer
Instruction: Auser’s Manual, Prentice-Hall, 1995.
HESTENES, D., M. WELLS and G. SWACKHAMMER, “Force Concept Inventory”, The Physics Teacher, vol. 30,
1992, p. 141-158.
KIM, E. and S. J. PAK, “Students do not Overcome Conceptual Difficulties after Solving 1000 Traditional
Problems”, American Journal of Physics, vol. 70, no 7, 2002, p. 759-765.
LOPEZ-HERREJON, R. E and M. SCHULMAN, Using Interactive Technology in a Short Java Course: An Experience
Report. Paper presented at the 9th annual SIGCSE conference on Innovation and technology in computer
science education.
MAZUR, E., Peer Instruction: A User’s Manual, Upper Saddle River, NJ., Prentice-Hall, 1997.
MCDERMOTT, L. C. and E. F. REDISH, “Resource letter: PER-1: Physics Education Research”, American Journal
of Physics, vol. 67, no 9, 1999, p. 755-767.
OWENS, K., D. A. MCCONNELL, D. STEER, S. VAN HORN, J. KNOTT, W. BOROWSKI and al. (2004). Changing
Pedagogy to Include Conceptests and Peer Instruction in Introductory geoscience coursesthe Impact on instructors and
students. Paper presented at the The geological Society of America Annual meeting Denver.
SIMON, H. A., “Observations on the Sciences of Science Learning”, dans Paper prepared for the Committee
on Developments in the Science of Learning for the Sciences of Science Learning: An interdisciplinary Discussion,
Department of Psychology, Carnegie Mellon University, 1996.

BIBLIOGRAPHY
As a physics teacher at John Abbott College, Nathaniel LASRY has focussed his attention
BEICHNER, R. L. BERNOLD, E. BURNISTON, primarily on the preconceptions of learners in science. His interest in questions relating to
P. DAIL, R. FELDER, J. GASTINEAU and al. “Case learning has earned him a doctorate in education from McGill University. In addition to his
Study of the Physics Component of an Integrated position at John Abbott College, he is also a post-doctorate researcher in the Eric Mazur
Curriculum”, American Journal of Physics, vol. 67, Harvard Group.
1999, p. 16-24.
lasry@seas.harvard.edu
BURNSTEIN, R. A. and L. M. LEDERMAN, “Using
Wireless Keypads in Lecture Classes”, The Physics
Teacher, vol. 39, no 1, 2001, p. 8-11.

BURNSTEIN, R. A. and L. M. LEDERMAN,

“Comparison of Different Commercial Wireless
Keypad Systems”, The Physics Teacher, vol. 41, no 5,
2003, p. 272-275.

CROUCH, C. H. and E. MAZUR, “Peer Instruction:

Ten years of Experience and Results”, American
Journal of Physics, vol. 69, no 9, 2001, p. 970-977.

DISSESSA, A., “Unlearning Aristotelian Physics:

A Study of Knowledge-base Learning”, Cognitive
Science, vol. 6, 1982, p. 37-75.

FAGEN, A., C. H. CROUCH and E. MAZUR, “Peer

Instruction: Results from a Range of Classrooms”,
The Physics Teacher, vol. 40, 2002, p. 206-209.

HAKE, R. R.,”Interactive-engagement versus

Traditional Methods: A six-thousand Student
Survey of Mechanics Test Data for Introductory
Physics Courses”, American Journal of Physics, vol. 66,
no 1, 1998, p. 64-74.

HALLOUN, I. and D. HESTENES, “Common Sense

Concepts about Motion”, American Journal of Physics,
vol. 53, 1985a, p. 1056.

7 PÉDAGOGIE COLLÉGIALE VOL. 21 NO 4 SUMMER 2008