Professional Documents
Culture Documents
Student Response Systemsv 7
Student Response Systemsv 7
net/publication/262638186
CITATIONS READS
0 5,779
2 authors:
All content following this page was uploaded by Sandra L. Miller on 21 October 2016.
Abstract
Student Response Systems (SRS), or clickers, are growing in usage. Studies have
consistently shown that clickers improve student learning outcomes in large classes. However,
there is little evidence about whether clickers enhance learning outcomes in small classes. In this
study, we evaluate the effectiveness of clicker usage in small classes and compare our findings to
results from studies of clicker usage in large classes. We also examine whether instructors differ
from students in terms of how they view clickers. Using survey questions derived from previous
research on large lecture classes, this study examined SRS use in 10 classes of fewer than 25
students over the course of three semesters. Results indicate that small classes benefit from
clicker usage, but not as much as large classes. Further studies should identify appropriate
Introduction
Student Response Systems (SRS), commonly known as clickers, have been around since
the late 1990s although more basic electronic response systems have been used in classrooms
since the 1960’s (Judson & Sawada, 2002). There have been a number of studies already
completed on their viability in the classroom (Abrahamson, 1999; Burnstein & Lederman, 2001;
Cue, 1998; Draper, 2001; Dufresne et al., 1996; Elliott, 2003; Hake, 1998; Nicol & Boyle, 2003;
Poulis, 1997; Williams 2003). In 2002, Mazur, Crouch & Fagen conducted an extensive 10-year
study on Peer Instruction (PI), the essential instructional strategy behind using clickers, and
found that most of the assessed PI courses encountered learning gains in line with interactive
engagement pedagogies and that 80% of the instructors considered their usage of PI to be a
success.
have generally shown that clickers increase students' reported levels of engagement, enjoyment,
and understanding. In addition, clickers appear to enhance students' awareness of their level of
understanding of class material, and also enhance teachers' ability to gauge their students' level
of understanding (Roschelle et al., 2004). In fact, more studies have subsequently focused on
examining these outcomes further (Carnaghan & Webb, 2005; Guthrie & Carlin, 2004; El-Rady,
2006; Horvath & Warmoth, 2007; MacGeorge et al., 2008; Stowell & Nelson, 2007). Yet,
interestingly enough, the majority of these studies have been done with clickers in large, lecture
classes. The question remains: Does the effect of student response systems differ by class size?
The majority of classes at William Paterson University of New Jersey (WPUNJ) are
small. Large lecture classes of 90 to 100 students are rare. The average class size is 30 and many
classes have fewer than 30 students. In small classes, students speak out with greater ease, are
Student Response Systems 4
more likely to work in small groups, and also have easier access to the teacher. We examined
whether the effects of clickers in in classes of 25 or less are different than the effects of clickers
in classes with more than 25 students. Is there a point of diminishing returns from the
intervention of using such a technology (SRS) in a smaller class where students can speak out
with more ease, often work in small groups, and supposedly have easier access to the teacher?
Method
The research design for this project was to work with small classes of 25 students or less.
Results from the large lecture class clicker studies are compared, where possible, to the results
from this small class clicker study modeled after the former. Results from the small class
instructors versus the students were also looked at to determine if faculty and students differed in
their responses to using clickers in their classes. The research spanned three semesters with a
The survey questions were developed based on the outcomes researched in previous
clicker studies regarding large lecture classes. The survey was based on: Abrahamson, 1999;
Cue, 1998; Draper & Brown, 2004; Elliott, 2003: Guthrie & Carlin, 2004; Hall et al., 2005;
MacGeorge et al., 2008, Minot State University Presentation, 2007; Roschelle et al., 2004;
Stowell & Nelson, 2007, and Williams, 2003. The outcomes that were measured are the ones
listed above: greater student engagement, increased understanding of subject matter, increased
enjoyment of class, better group interaction, helping students gauge their own understanding, and
The survey (see Appendix) used Zoomerang to ensure anonymity to the students and the
results were directly returned to the PI. Survey questions were placed on a Likert Scale of 1 to 6
with 1 being the highest outcome in agreement and 6 being N/A. A similar survey was given to
There were 10 items on the survey. The first 9 items covered the outcomes noted in the
large lecture clicker studies. Of the first 9 items, response percentages in six items were
compared to the percentages expressed in larger lecture classes. Two items (#’s 2 and 9) did not
have corresponding student ratings in studies on larger, lecture classes and could not be
statistically compared. The remaining item (#6) was designed to be compared between students
and faculty only within the WPUNJ study. Item #10 was added to ascertain if students felt that
clickers worked as well in the small class environment as in the larger, lecture classes. Item #12
was an open-ended item designed to elicit any further comments. Correlations between items
were generally above .5 except for Item#8, “The clickers were easy to use”.
An additional question concerning whether the students believed that class size had made
a difference in their response to using the clickers was also asked and qualitative feedback
requested. The responses from this item were viewed for anecdotal results. All students were
notified of their participation and the fact that all responses were totally anonymous. IRB
Particpants.
There were 125 students responding out of 161 for all ten classes for a response rate of
78%. There were 9 faculty responding out of a total of 10 for a response rate of 90%. The
teachers implemented the SRS systems in their coursework. They used the same type of
instructional strategies (peer instruction) used in the studies of larger classes mentioned above.
Clickers were distributed in the same manner for all classes (through Media Services to
the teacher for each semester). The department of Instruction and Research Technology trained
all the teachers in the appropriate technology. Dr. Sandra L. Miller, Director of Instruction and
Research Technology (IRT) and Principal Investigator (PI) of this research project ensured that
Student Response Systems 6
the teachers implemented the systems in their courses and followed through with the
assessments. Instructors were required to use the clickers at least four times during the semester,
but they could use them more if they felt so inclined. In this way, all classes were equally
Clickers were ordered in late October, 2007. Faculty participants received clickers and
the software on their laptops before the spring semester. Faculty were trained to use the SRS
hardware and software by IRT personnel before the beginning of the spring semester.
Throughout each semester, the PI regularly checked with the faculty on their progress in using
the clickers. Clickers were used in classes in a variety of subject areas, including elementary
education, chemistry, sociology, women’s studies, business, instructional library media, basic
Results
There are five colleges representing specific disciplines at WPUNJ. The largest number
of student participants, 49, came from the College of Business (40%) followed by 25 (20%) from
the College of Humanities and Social Sciences, 24 (19%) from the College of Education, 23
(19%) from the College of Science and Health, and 2 (2%) from the College of Arts and
Communication. One (1%) student apparently misunderstood the question and responded that the
course taken with the clickers was in the Honors College. Figure 1 graphically illustrates the
In order to compare small and large classes, means and 95% confidence intervals were
computed for large, lecture class studies and for this small class study. Item#1 was compared
to Williams (2003) data on a 5-point level of agreement with the statement, “The quality of
Item #3 was compared to Williams (2003) data on a 5-point level of agreement with the
statement, “The use of the PRS in class was preferable to going through the lecture slides
(available, in advance, on the OLT site).” Item#4 was compared to Dufresne et al. (1996) data on
a 5-point level of agreement with the statement, “Using Classtalk helped me enjoy class more”.
Item#5 was compared to MacGeorge et al. (2008) data using an estimated factor score
based on three five-point measures of agreement with the following statements: “Because of
CPS, it is easier for me to tell whether I am mastering course material”; Because of CPS, I am
more certain about how I am performing in the class”, and “Using CPS helps me more easily
determine how well I am doing in the class.” Item #7 was compared to Williams (2003) data on a
5-point level agreement with the statement, “I was more likely to respond/participate/engage
with the content because of the anonymity of using PRS.” Item #8 was compared to MacGeorge
Student Response Systems 8
et al. (2008) using an estimated factor score based on three five-point measures of agreement
with the following statements: “Using the CPS technology is easy”; “I have no problems using
the CPS technology”, and “Using the CPS technology is pretty hard” (reversed).
A table was developed of the means, standard deviations and subsamples in WPUNJ’s
small classes and the large classes drawn from the literature (Dufresne et al., 1996, MacGeorge
Table 1
Means, Standard Deviations and Subsamples for small and large classes using SRS:
M SD n M SD n
Items #1, 4, 5, and 8 indicated statistical significance. The following diagram describes
Figure 2. Chart of means and 95% confidence intervals between small and large classes.
A comparison was also drawn between the responses from students and faculty. The first
item asked if students felt that clickers enhanced their learning. There were 125 responses to this
item. Twenty-four (19%) students strongly agreed that clickers enhanced their learning, 52
(42%) agreed, 33 (26%) were neutral, 7(6%) disagreed, 8 (6%) strongly disagreed, and 1(1%)
felt that this statement was non-applicable to clickers. There were 9 faculty responses to this
item. Eight (89%) faculty agreed that clickers enhanced student learning and 1 (11%) was
neutral. Figure 3 graphically displays the students’ perceptions compared to faculty perceptions
Figure 3. Histogram of how well the students felt that clickers had enhanced their learning compared to faculty
perceptions (n=125 and n=9 respectively).
The second item asked if students were more engaged in the class as a result of the
clickers. There were 125 student responses to this item. Thirty-six (29%) students strongly
agreed that they had been more engaged, 53 (42%) agreed they had been, 19 (15%) were neutral,
10 (8%) disagreed, 5 (4%) strongly disagreed, and 2 (2%) felt that clickers were not applicable to
class engagement. Of the 9 faculty who responded to this item, 3 (33%) strongly agreed that
clickers created greater engagement in class and 6 (67%) agreed. Figure 4 graphically displays
the students’ perceptions compared to faculty perceptions of whether students were more
engaged in class.
Figure 4. Histogram of students’ perceptions compared to faculty perceptions of student engagement in class (n=125
and n=9 respectively).
Student Response Systems 11
The third item asked if students preferred clicker integration over other PowerPoint
methods. There were 125 student responses. Thirty (24%) students strongly agreed that clicker
integration was preferrable, 39 (31%) agreed, 41(33%) were neutral, 8 (6%) disagreed, 6 (5%)
strongly disagreed, and 1 (1%) felt that clickers were non-applicable to PowerPoint methods. Of
the 9 faculty responses, 3 (33%) strongly agreed that clicker integration was preferable, 2 (22%)
agreed, 1 (11%) was neutral, 1 (11%) disagreed, and 2 (22%) felt that clickers were non-
applicable. Figure 5 graphically displays students’ preferences compared to faculty for clicker
Figure 5. Histogram of students’ preferences compared to faculty preference for clicker integration over other
PowerPoint methods (n=125 and n=9 respectively).
The fourth item asked if clickers were fun and added to class enjoyment. There were 124
student responses. Thirty-five (28%) students strongly agreed that clickers were fun and
enjoyable, 57 (46%) agreed, 21 (17%) were neutral, 7 (6%) disagreed, and 4 (3%) strongly
disagreed. Of the 9 faculty responses, 7 (78%) strongly agreed that clickers were fun and added
to class enjoyment, and 2 (22%) agreed. Figure 6 graphically displays students’ responses
Figure 6. Histogram of students’ responses compared to faculty for clicker fun and added class enjoyment (n=124
and n=9 respectively).
The fifth item asked if students felt that they were better able to gauge their own
understanding in class. There were 124 student responses. Twenty-seven (22%) students strongly
agreed that they could, 43 (35%) agreed, 41 (33%) were neutral, 8 (6%) disagreed, 4 (3%)
strongly disagreed, and 1 (1%) felt that clickers were not applicable to their understanding in
class. Of the 9 faculty responding, 2 (22%) strongly agreed that clickers enhanced their students’
ability to gauge their understanding in class, 4 (44%) agreed, and 3 (33%) were neutral. Figure 7
graphically illustrates the students’ perceptions compared to the faculty’s regarding clicker
Figure 7. Histogram of students’ perceptions compared to faculty’s regarding clicker enhancement of student
understanding in class (n=124 and n=9 respectively).
Student Response Systems 13
The sixth item asked if either the students or faculty felt that the teacher was better able
to diagnose student difficulties in class. This was the item in which comparison between faculty
and students was most important. There were 125 student responses. Twenty-eight (22%)
students strongly agreed that the teacher could more easily identify student difficulties with the
clickers, 46 (37%) agreed, 32 (26%) were neutral, 8 (6%) disagreed, 8 (6%) strongly disagreed,
and 3 (2%) felt that clickers were not applicable to the teacher’s ability to diagnose student
difficulties. Of the 9 faculty responses, 2 (22%) strongly agreed that they were better able to
assess student problems in class, 4 (44%) agreed, and 3 (33%) were neutral. Figure 8 graphically
Figure 8. Histogram of student perceptions of the teacher’s ability to diagnose student difficulties compared to
faculty’s perceptions (n=125 and n=9 respectively).
The seventh item asked if students felt less embarrassed and the anonymity provided by
clickers encouraged more class response. Of the 125 student responses, 42 (34%) strongly
agreed, 52 (42%) agreed, 20 (16%) were neutral, 4 (3%) disagreed, 6 (5%) strongly disagreed
and 1 (1%) felt that clickers were not applicable to feeling less embarrassed and more inclined to
respond in class. Of the 9 faculty, 5 (56%) strongly agreed, 1 (11%) agreed, 2 (22%) were
Student Response Systems 14
neutral, and 1 (11%) disagreed. Figure 9 illustrates students’ perceptions of their inclincation to
Figure 9. Histogram of students’ perceptions of their inclination to respond more in class compared to faculty’s
perceptions (n=125 and n=9 respectively).
The eighth item asked if the clickers were easy to use. There were 125 student responses
and 9 faculty responses. Sixty (48%) students strongly agreed that clickers were easy to use, 49
(39%) agreed, 10 (8%) were neutral, 3 (2%) disagreed, 2 (2%) strongly disagreed, and 1 felt that
clicker ease of use was not applicable. Of the 9 faculty responses, 4 (44%) strongly agreed that
clickers were easy to use, 3 (33%) agreed and 2 (22%) were neutral. Figure 10 graphically
Figure 10. Histogram of students’ perceptions of clicker ease of use compared to faculty’s perceptions (n=125 and
n=9 respectively).
Student Response Systems 15
The ninth item asked if participants would recommend clickers for other small classes. Of
the 125 student responses, 52 (42%) strongly agreed that they would, 43 (34%) agreed, 22 (18%)
were neutral, 2 (2%) disagreed, 5 (4%) strongly disagreed, and 1 (1%) felt that this question was
not applicable. There were 9 faculty responses. Five (56%) faculty members strongly agreed, 3
(33%) agreed and 1 (11%) was neutral. Figure 11 graphically depicts the students’
Figure 11. Histogram of students’ recommendations of clickers for other small classes compared to faculty’s (n=125
and n=9 respectively).
The tenth item asked if clickers made a difference in small classes as much as in large,
lecture classes. Of the 125 students responding, 37 (30%) strongly agreed, 31 (25%) agreed, 38
(30%) were neutral, 5 (4%) disagreed, 2 (10%) strongly disagreed, and 12 (10%) felt that
clickers were not applicable to class size. There were 9 faculty responses. Four (44%) faculty
strongly agreed, 2 (22%) agreed, 2 (22%) were neutral, and 1 (11%) disagreed. Figure 12
in small classes such as theirs compared to their large, lecture classes compared to faculty’s
perceptions.
Student Response Systems 16
Figure 12. Histogram of students’ perceptions of clicker usage in small classes versus large, lecture classes
compared to faculty’s.
Item #12 gave students and faculty in the small class study an opportunity to express
particular comments or concerns. Two students complained about technical problems with the
clickers. One student claimed that, “Clickers should be used more in other classes. It makes
learning fun and challenging during class” while another student specifically stated that, “I feel
the clicker is more helpful for a bigger class (students more than 20).” One faculty member
responded that s/he had had technical difficulties, corresponding to the students’ complaints
above. Another faculty member expressly did not like the clickers because they did not fit in
with his/her teaching style. This faculty member stated that the class “…ended up following a
different intellectual path than I had previously contrived (more student-centered and
personalized than I could have done with a PowerPoint). I almost can’t believe it myself, but
they’re not for me.” Two other faculty specifically indicated that the use of clickers was
worthwhile.
Discussion
Although small class students tend to respond to clickers in a favorable light as do large
class students, there were some outcomes in which a significant difference was demonstrated.
Items#1, 4, 5 and 8 were statistically significant indicating that class size did have an impact in
Student Response Systems 17
the areas of enhancement of learning, clickers are fun and enjoyable, clickers enable students to
more easily judge their understanding in class, and clickers were easy to use. However, there was
feeling that clickers led to less embarrassment and the anonymity encouraged greater class
response. Preferring the use of clickers to normal PowerPoint presentations indicates that
William Paterson students (small classes) reported higher levels of satisfaction on general
items (enhanced learning, prefer clickers, enjoyed clickers) than large class students. However,
on items that measured more specific preferences (easily judge their learning and less
embarrassed to respond), students from larger classes tended to report higher levels of agreement
than did students in small classes at William Paterson. There is an anomaly in the measurement
of ease of use with clickers. A correlation matrix among the nine items evaluating clickers at
William Paterson University showed that the one item, #8 (Clickers were easy to use), was way
below .5. This item did not follow the same underlying factor that the others did, i.e. relating to
Items 1,2 and 4 had the widest spread in percentage points between students and faculty
in strongly agreed and agreed responses indicating that students were not quite as enthusiastic as
faculty about how clickers enhanced learning, engaged students or added to class fun and
enjoyment. Otherwise, the percentage point spread remained under 25% and generally favored
faculty response rather than students’. Only items #8 and 9 showed that students felt more
strongly than faculty that clickers were easy to use and that they would recommend them for
other small classes. It is understandable that students would find clickers more easy to use than
faculty because of the extra effort faculty needed to make to insert the questions into their
Student Response Systems 18
PowerPoints. For this reason also, faculty may not necessarily recommend them for other small
classes.
This study indicates that small class students are favorable towards SRS as is the case
with large class students, but have reservations in certain areas. The impact of using clickers in
small classes is not as fruitful overall as with a large, lecture classes. It could be that teaching
styles in small classes (as indicated by the one faculty member’s comments) do not lend
themselves to usage of clickers. Instead, the true benefit of clickers is to create interaction where
there was little or none before. In addition, this study is limited to small classes at William
Paterson University although comparisons were made to large, lecture classes at other
universities. Further studies of the impact of teaching styles on the use of clickers in class would
References
and why it works, International Workshop, New Trends in Physics Teaching, Retrieved
Burnstein, R.A. & Lederman, L.M. (2001). Using wireless keypads in lecture classes. The
Carnaghan, C. & Webb, A. (2005). Investigating the effects of group response systems on
learning outcomes and satisfaction in accounting education. Retrieved on April 25, 2007,
from http://www.learning.uwaterloo.ca/research/responsepadjune2005.pdf.
Cue, N. (1998). A universal learning tool for classrooms? Proceedings of the First Quality in
Teaching and Learning Conference, Hong Kong International Trade and Exhibition
Draper, S.W. & Brown, M.I. (2001). Increasing interactivity in lectures using an electronic
Dufresne, R.J., Gerace. W.J., Leonard, W.J., Mestre, J.P. & Wenk, L. (1996). Classtalk: a
Education, 7, 3-47.
Elliott, C. (2003). Using a personal response system in economics teaching. International Review
El-Rady, Johnny, "To Click or not to click: That's the question," Innovate Journal of Online
Guthrie, R.W. & Carlin, A. (2004). Waking the dead: Using interactive technology to engage
Information Systems. New York, NY. Retrieved on April 27, 2007 from
http://www.mhhe.com/cps/docs/CPSWP_WakindDead082003.pdf.
survey of mechanics test data for introductory physics courses. American Journal of
Hall, R. H., Thomas, M.L., Collier, H.L. & Hilgers, M.G. (2005). A student response system for
http://lite.mst.edu/documents/hall_et_al_srs_amcis_proceedings.pdf.
Horvath C. & Warmoth, K. (2007). Leveraging Technology to advance learning: A pilot study.
Judson, E., and Sawada, A. (2002) Learning from past and present: Electronic response systems
21(2), 167-181.
MacGeorge, E.L., Homan, S.R., Dunnng Jr., J.B., Elmore, D., Bodie, G.D., Evans, E.,
Khichadia, S., Lichti, S.M., Feng, B. & Geddes, B. (2008) Student evaluation of audience
Mazur, E. (1997) Peer instruction - A user's manual, Prentice Hall, New York.
Student Response Systems 21
Nicol, D.J. & Boyle, J.T. (2003). Peer instruction versus class-wide discussion in large classes: A
Poulis, J., Massen, C., Robens, E., and Gilbert, M. (1998). Physics lecturing with audience paced
Roschelle, J., Penuel W.R., & Abrahamson L., (2004) Classroom response and communication
systems: research review and theory. Presented at the 2004 Meeting fo the American
Educational Research Association, San Diego, CA. Retrieved on April. 27, 2007 from
http://ubiqcomputing.org/CATAALYST_AERA_Proposal.pdf.
Stowell, J.R. & Nelson, J.M. (2007). Benefits of electronic audience response systems on student
Williams, J.B. (2003) Learning by remote control: Exploring the use of an audience response
system as a vehicle for content delivery. Proceedings of the 20th Annual Conference of
https://secure.ascilite.org.au/conferences/adelaide03/docs/pdf/739.pdf.
Student Response Systems 22
Appendix
Thank you for participating in our survey. Your feedback is important. Please note
that all responses are anonymous and are being used for research purposes only.
The survey is very short and will only take 5-10 minutes of your time.
Each of the items in this survey deals with the characteristics of using clickers in
the class. Indicate your rating of clicker usage in your course by circling the
appropriate number. Please answer all questions thoughtfully and carefully. Your
fair and honest opinion is what really counts. The characteristics listed are rated
from 1 to 6, 1 being the highest, 5 being the lowest and 6 is Not Applicable.
1 2 3 4 5 6
1 2 3 4 5 6
Student Response Systems 23
1 2 3 4 5 6
1 2 3 4 5 6
1 2 3 4 5 6
1 2 3 4 5 6
Student Response Systems 24
1 2 3 4 5 6
1 2 3 4 5 6
1 2 3 4 5 6
10. I felt that clickers made a difference in my small class as much as in any
large lecture class.
1 2 3 4 5 6
Student Response Systems 25
11. The course in which I used clickers was in the following College. (Please
circle the appropriate College)
• College of Education
• College of Science and Health
• College of Humanities and Social Sciences
• College of Business
• College of Arts and Communication
12. Please feel free to add any comments you may have.