Professional Documents
Culture Documents
Assignment # 3
Assignment # 3
Using the International Society for Technology in Education (ISTE) Classroom Observation
Brennan-Jones, C.
cyntbren@nova.edu
Miller, M.
mhobbs@nova.edu
Williams, N. Y.
nicowil@nova.edu
Abstract
Three doctoral students used the International Society for Technology in Education
doctoral class meeting. The authors found that this classroom observation tool, while viable for
face-to-face classroom settings, would be useful for online classroom observations only if
modifications were made. With the growing popularity and numbers of distance education and
online classes, the authors find there is also a subsequent need for appropriate classroom
observation tools.
The authors are all currently enrolled in doctoral programs at Nova Southeastern
University Fischler School of Education and Human Services. Cynthia Brennan and Nicole
in Educational Leadership. All three authors have a minor in Curriculum and Instruction. As part
of a group project in their online EDD 7914 Technology—Integrated Teaching and Learning
course, the authors utilized an online classroom observation tool. Subsequently, they wrote this
article.
Running head: USING THE ISTE ICOT TO OBSERVE AN ONLINE CLASS 3
Introduction
As part of a group project in an online doctoral course, authors Brennan, Miller, and
Williams (2010) utilized the International Society for Technology in Education (ISTE)
Classroom Observation Tool (ICOT) to observe a synchronous online class meeting. ISTE, “the
premier membership association for educators and education leaders engaged in improving
learning and teaching by advancing the effective use of technology in PK-12 and higher
education” (ISTE, 2010) offers educators the free online ICOT. The ICOT “provides a set of
integration” (ISTE ICOT, 2010). The “ICOT is based on observation tools developed since 1998
during ISTE's work with numerous educational technology initiatives, including the National
Technology Grants” (Felix, 2008). While the three authors all agree the ICOT is a suitable tool
a “face-to-face” classroom, the authors argue that the ICOT as a tool needs significant
Description of Activity
Using the ICOT, the three authors separately observed a recorded instructor-facilitated
online class Elluminate session wherein two groups of students presented their PowerPoint
presentations on emerging technologies in the field of education synchronously to the class and
to the instructor. Problems with how to answer the ICOT questions came up consistently. For
example, because only the instructor used a webcam, it was impossible to ascertain whether
students in the class were using desktop or laptop computers (ICOT, Question 8). The “Three-
Running head: USING THE ISTE ICOT TO OBSERVE AN ONLINE CLASS 4
Minute Chart” (ICOT, Question 10) and the “Estimated time technology was used” (ICOT,
Question 11) were also, for the most part, not relevant for a synchronous meeting of an online
class, where all students and teachers are using technology for teaching and learning at all times.
The NETS Teacher Standards listed on the ICOT that the authors checked off as being
addressed were as follows: 1A.1. operating system procedures; 1A.2. routine hardware and
software problems; 1A.3. content-specific tools; 1A.4. productivity tools; 1A.5. multimedia
standards; 3A.2. learning experiences address student technology standards; 3D.1. class
tool; 4A. 1. Student learning of subject matter assessed with technology; 4A.2. teacher assesses
student technology skills; 6D.1. safe and healthy use of technology promoted (ICOT, 2010).
However, it should be noted that not all the authors observed each of the NETS Standards listed;
The discrepancies noted both in the NETS Standards check-off list and elsewhere in the
ICOT show that both training and discussion are needed before utilizing the tool. In fact, ICOT
advises in its Training Resources section that, “Observers discuss the data with partners or the
whole group to reach consensus on how to apply the ICOT. How will you define technology as
"in use?" How will you decide if a technology use is "Essential?" What will it take to determine
In addition to training before using the tool, modifications to the questions are needed
if the ICOT is to be used for the observation of online classes. For example, the portions of the
Running head: USING THE ISTE ICOT TO OBSERVE AN ONLINE CLASS 5
ICOT which are useful for measuring student engagement when reviewing a lesson taught in a
face-to-face classroom are not as useful when observing an online class. The authors agree that it
is more difficult when observing an online class to ascertain how many of the students are
actively engaged in a lesson, but it is possible to build in ways to measure engagement. For
instance, Tally and Goldenberg (2005), in a study of the success of using digitized primary
sources in an online task, used “the student task [which] became a pilot assessment of students’
historical thinking, as revealed through the software” (p. 6). These authors hypothesized “that
well-designed software could support teachers and students in working closely with documentary
sources, and thereby improve history teaching and learning” (p. 2). Tuttle (2008) suggested using
21st-century technologies such as online quizzes and web-based surveys to assist with effective
student observations. Either of the two tools might be helpful in measuring online student
engagement with a lesson. Cook and Germann (2010) suggest that qualitative methods may be
useful as well as “numerical counts for postings and readings” (p. 6) for assessment of student
engagement with online learning material. Looking at data generated by the WebCT courseware
utilized in their study of online classes “offered virtual snapshots of student activities and
engagement” (p. 15), but, the authors concluded, “clearly further studies with more qualitative
data may broaden the picture in lieu of relying on snapshots” (p. 16).
As a final example of a difficulty encountered by the authors while using the ICOT for
this activity, answering the ICOT question “How essential was technology to the teaching and
learning activities” was especially problematic when observing an online class. For instance,
what if online students are utilizing the most basic of technology to sit passively while listening
to a lecture or viewing a PowerPoint? As the ICOT is currently written, an activity that involves
Running head: USING THE ISTE ICOT TO OBSERVE AN ONLINE CLASS 6
an online student listening to a lecture would receive the highest rating for technology
integration.
Discussion
Technology Use Observation Tool (2003), pose similar limitations when considering its use for
online class observations. Like the ICOT, Barnett and Brooks-Young’s tool offers opportunities
to pinpoint where technology integration may be lacking in a face-to-face classroom, but the
authors of this article argue that it too offers little usefulness to an observer of an online
classroom. Eib (2001), who also created an observation formula to assess the amount of
technology use in the classroom, stated that the best way to ensure that technology was being
utilized was to “use a list of instructional improvement priorities that support your school
improvement plan and reflect research-based best practices to create a rubric for observations.”
While the authors agree that using best practices and instructional priorities is a solid way to
build a rubric to assess the use and effectiveness of technology in the classroom, Eib’s
observation formula would also need modification to be useful for effective observations of an
online course.
Conclusions
Technology is a basic essential in every online activity. However, does the ICOT
suggest that the answer is “no.” In order to effectively measure key components of technology
integration, for online classroom observations, the questions found in the ICOT must be
modified to reflect that basic technology, i.e. being online, is a given, and find ways to measure
higher levels of technology integration and online student engagement. With the growing
Running head: USING THE ISTE ICOT TO OBSERVE AN ONLINE CLASS 7
popularity of distance education and online classes, the authors suggest that there is also a
References
Barnett, H. (2003). Technology professional development: Successful strategies for teacher change.
Syracuse, NY: ERIC Clearinghouse on Information & Technology. (ERIC Document
Reproduction Service No. ED477616)
Barnett, H. & Brooks-Young, S. (2002). Technology Use Lesson: Observation Tool. Retrieved June 25,
2010, from http://members.tripod.com/sjbrooks_young/techobstool.pdf
Cook, S. and Germann, C. (2010). The tell-tale data: Virtual whispering and final student grades. Online
submission. Syracuse, NY: ERIC Clearinghouse on Information & Technology. (ERIC Document
Reproduction Service No. ED508149)
Eib, B. J. (2001). Beyond the bells and whistles: Evaluating technology use in the classroom. Principal
Leadership, 9, 16-23.
Felix, K. (2008). ISTE Classroom observation tool (ICOT). MultiMedia & Internet at Schools, 5, 41.
International Society for Technology in Education (ISTE). (2010). Retrieved June 22, 2010, from
http://www.iste.org/
International Society for Technology in Education (ISTE). (n.d.). The ISTE Classroom
Observation Tool (ICOT). Retrieved June 17, 2010, from http://www.iste.org/icot/
Tally, B. & Goldenberg, L. B. (2005). Fostering historical thinking with digitized primary
sources. Journal of Research on Technology in Education, 38(1), 1-21.
Tuttle, H. G. (2008). Digital age assessment: Part 1. Technology & Learning, 28(8). Online
version. Retrieved June 26, 2010 from http://www.techlearning.com/article/8592