Received November 28, 2018, accepted January 8, 2019, date of publication January 21, 2019, date of current version

February 14, 2019.

Digital Object Identifier 10.1109/ACCESS.2019.2893971

Using a Google Glass-Based Classroom

Feedback System to Improve Students
to Teacher Communication
TELMO ZARRAONANDIA , PALOMA DÍAZ, ÁLVARO MONTERO,
IGNACIO AEDO, AND TERESA ONORATI
DEI Lab–Computer Science Department, Universidad Carlos III de Madrid, 28911 Leganés, Spain
Corresponding author: Telmo Zarraonandia (tzarraon@inf.uc3m.es)
This work was supported in part by the Spanish Ministry of Economy and Competitivity through the PACE Project under
Grant TIN2016-77690-R, and in part by the Spanish Ministry of Science and Innovation through the CREAx Project
under Grant TIN2014-56534-R.

ABSTRACT The use of augmented reality (AR) to support the learning process has been extensively
researched but its use to support the teaching practice has just started to be explored. In this paper, we present
a communication system that makes use of a pair of Google Glass to provide the teacher with a constant and
private flow of information on the students’ current knowledge. The proposed system allows the information
sent by the students through their mobiles to overlap with the teachers’ live vision of the class. Compared
to other feedback systems like clickers or backchannel systems, this AR prototype avoids teachers diverting
their gaze and interrupting the class to access the students’ feedback. This supports the constant monitoring
of potential comprehension problems that might otherwise be overlooked. With the aim of obtaining insights
on the teachers’ and students’ views of the system, we conducted two studies during which the system was
used in real classroom settings. The results of both studies suggest that the AR system could report benefits
in terms of better communication between students and teachers, and a more adequate rhythm of the class.
Also, the use of the AR system in the classroom does not necessarily constitute an element, which will
distract and disrupt educational activity.

INDEX TERMS Augmented reality, computer-mediated communication, educational technology.

I. INTRODUCTION of the current feedback systems by making use of AR tech-

Augmented Reality (AR henceforth) is foreseen as a poten-
tial useful resource in teaching and learning, assuming that
educational experiences can be improved by increasing the
real world with digital information in many different ways.
Most current research is focused on examples of the usage
of AR to support specific learning processes [12], [19]
and on the unique learning affordances of AR, including
the capacity to learn on realistic environments, and support
kinaesthetic learning or face-to-face communication in col-
laborative learning [9], [10]. In this research, we focus on the
other side of the coin: the use of AR to augment the teacher’s
view during the lecture. Feedback systems like clickers [6] or
backchannels [8] are used in the classroom to improve and
speed up the communication among students and teachers.
In this paper, we propose overcoming some of the limitations
The associate editor coordinating the review of this manuscript and
approving it for publication was Jun Huang.
2169-3536
2019 IEEE. Translations and content mining are permitted for academic research only.
VOLUME 7, 2019 Personal use is also permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
nology, and more specifically AR glasses. This type of device
can be used to augment the teacher’s view superimposing the
responses collected from the students as symbols or graphs.
This amplified view allows teachers to visualize the feedback
gathered from the students directly, without diverting their
gaze to look at a computer or a smartphone screen. This could
be especially useful for backchannels, since the information
students send at any point of an explanation are immediately
visible to the teacher, so that they are more difficult to miss.
In addition, the AR technology allows teachers to access the
feedback privately, as opposed to other systems’ implementa-
tions that publicly display the responses on the classroom pro-
jection screen, which could influence the students’ responses
and facilitate situations of abuse [2].
In [20] we already investigated whether an AR-based class-
room feedback system could overcome the reluctance of
some students to ask questions or communicate their difficul-
ties in following a teacher’s explanation. The results of the
16837
 T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students
experiments carried out to test that prototype were encour-
aging, suggesting that this type of system could improve the
teaching practice. However, as the AR glasses available at that
time were too cumbersome and intrusive, it was not possible
to evaluate the system during its use in real classroom settings
on a daily basis. Since nowadays there are AR glasses that
overcome the ergonomic limitations of the previous devices it
is now possible to continue the research to further investigate
the potential benefits of this system, so that they can be defini-
tively confirmed or discarded in longer term experiments.
In this paper, we present our research on using an AR sys-
tem to support communication in the classroom by providing
the teacher with a constant and private flow of feedback on
the current level of knowledge of the students. Research was
aimed at obtaining insights on both the teachers’ and stu-
dents’ views of the system. Following this objective, we asked
20 teachers from university and secondary educational cen-
ters to use the system in at least one of their classes and give us
feedback. In addition, we conducted a long-term study during
a university course in which the AR system was used for
7 weeks of an introductory course on programming, so we
could test whether positive attitudes went beyond the first
impact with a new technology. The results of two studies
conducted indicate that teachers and students appreciated
the benefits that the system can report in terms of a better
communication and a better adaptation of the rhythm of
the class.
The rest of the paper is organized as follows: in the next
section, we summarize some relevant related works in the
area, and briefly describe the previous prototype of the AR
system and the results of its evaluations. Next, we present
the new version, its architecture and the functionalities it
supports. In the next section, the research methodology and
the results of the experiments carried out are described and
discussed. We conclude the paper discussing the limitations
of the work, conclusions and on-going works.
II. LITERATURE REVIEW AND PREVIOUS WORK
The technology used for gathering feedback from the students
in the classroom is usually classified into two major groups:
Audience Response Systems (ARSs) and digital backchan-
nels. The objective of the ARSs, also known as Classroom
Response Systems (CRSs), is to allow the teacher to collect
anonymous responses to multiple choice questions formu-
lated at specific points of the explanations. The responses
collected are usually displayed on the projection screen as
histograms [6]. The use of ARSs in the classroom has been
the subject of extensive research [13]. Among its many advan-
tages, the ability of students to contribute anonymously, thus
avoiding the embarrassment of being judged by their peers
or the tutor, stands out. This seems to be a key factor in
the increase of students’ participation [6], [17] and engage-
ment [2], [6]. From the perspective of the teacher, ARSs can
help to support contingent teaching, adapting the explana-
tions based on the misconceptions identified in the answers
collected [3], [6], [14].
16838
Digital backchannels permit the students to provide more
expressive feedback at any time of the lecture, and not only as
a response to the teacher’s request. Depending on the system,
this feedback can take the form of questions or comments on
the lecture content, votes or responses to posts from other
students, messages referring to the lecture’s presentation
(e.g. pace of lecturing), or indications of being lost, for exam-
ple [1], [4], [8], [16]. The benefits of digital backchannels are
not so well studied as those of ARSs. Some studies suggest
that backchannels which permit students to post questions
anonymously could increase their engagement [1], [5], and
the number of questions asked in class [1], [2], [16]. However,
students might also misuse these backchannels [2], [4], [8]
to publish off-topics or inappropriate posts, thus disrupting
the class. Only a few studies [1], [4] report on the outcomes
of using backchannels for sending feedback to a teacher on
her explanations, to indicate whether the pace was too fast
or too slow, for example. In the case of the study described
in [1], both students and teachers highly appreciated this
functionality. On the contrary, in the experiment described
in [4] the feedback buttons were barely used by the students.
Finally, some studies have also analyzed the risk of distract-
ing the students from the class [1], [4], [8]. Most results
suggest that the backchannel did not constitute an element
of distraction [1], [8] although there are some exceptions [4].
To investigate the potential benefits of AR-based feed-
back systems, we developed an AR system named ALF
(Augmented Lectures Feedback System) that aimed to sup-
port communication in the classroom. The system allowed
a lecturer equipped with a pair of AR glasses (Vuzix Wrap
920 AR eyewear) to visualize symbols on top of the student’s
head that depicted their status with regards to the current
teacher’s explanation, or their answers to the questions he/she
formulated (Fig. 1). Students choose their status and send
responses using their mobile phones. The results of an exper-
iment carried out in lectures and presentations were promis-
ing [20] and suggested that the system could improve the
communication of students’ difficulties to the teacher and
help the teacher to better adapt the rhythm of the class to
their current knowledge. However, the models of AR glasses
which were available at the time were still very heavy and
cumbersome. This impeded evaluating the effectiveness of
the system when used on a daily basis.
III. THE ALF GLASS SYSTEM
The ALF Glass system (ALF-G) adapts the functionalities of
the previous ALF system to the Google Glass. The ergonomic
design of this device makes its use in a classroom envi-
ronment possible. However, as opposed to other models of
AR glasses that augment a wide range of the user’s field of
view, the Google Glass restricts the area of augmentation to
an upper corner of the user’s vision. Due to this limitation,
the new version of the ALF system displays the data collected
from the students summarized in the form of diagrams, and
not individually. As in the previous versions of the system,
this data could correspond to feedback about students’ level
VOLUME 7, 2019
T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students
FIGURE 1. Teacher’s view of the class through the ALF system.
of understanding of the teacher’s explanations as well as to
the answers to the teacher’s questions.
FIGURE 2. Modules of the ALF Glass System.
Fig. 2 depicts the architecture of the system. As shown in
the figure, the system is composed of three modules: ALF-G
Mobile, for collecting the students’ feedback and responses,
ALF-G Teacher, for visualizing the information in the glasses
and controlling the system, and ALF-G Server, which man-
ages the communication between the different devices and
stores the feedback in a database.
The right-hand side of Fig. 3 depicts a screenshot of the
ALF-G Mobile app. This module was implemented as a
Web application so that it can be used without installing any
software in the students’ cell phones. As shown in the picture,
the app is composed of one single page divided into two areas.
The uppermost area allows students to select their level of
comprehension on the current explanation. Students can indi-
cate whether they are lost (red cross symbol), whether they
understand the explanation (green check symbol) or whether
they have questions to ask (question mark symbol). The
activation of this last symbol would indicate to the teacher
that she should stop at some point to deal with the questions
the students might have. The lowermost area is only displayed
upon the activation on the teacher‘s side. She can formulate
questions indicating possible answers that the students vote
VOLUME 7, 2019
on using the ‘‘option buttons.’’ The teacher can configure the
system to show or hide a bar diagram of the other students’
responses, so all students can also be aware of the responses
of the rest of the class. Also, if the student does not update
the feedback for a long time the page displays a reminder.
The information collected from the students is stored in
the ALF-G database. This module also generates and updates
two data charts, a feedback chart and a responses chart that
are displayed in the teacher’s Google Glass (left hand side
Figure 3). The feedback chart is a pie chart permanently
displayed on the glasses that summarizes the students’ sta-
tus. The responses chart is only displayed when the teacher
activates a question swiping backward (yes or no question)
or forward (multiple choice question) on the touch located
on the side of the glasses. Tapping on the touchpad makes the
responses chart disappear again and resets the feedbackchart.
Fig. 4 depicts the teacher’s view of the class through the
glasses.
IV. EXPERIMENTS: ANALYSING THE TEACHERS’
AND STUDENTS’ PERCEPTIONS
In this section, we describe and present the results of the
research carried out with the aim of investigating the per-
ceived benefits and weaknesses of the system. More specifi-
cally, and considering the related work presented in section 2,
the research focused on three specific potential benefits:
improvement of the communication of students’ difficulties;
improvement of the adaptation of the pace of the class; and,
improvement of students’ engagement. Concerning potential
drawbacks, two aspects were analyzed: the capability of the
technology to distract students and to disrupt the teacher’s
activity.
In order to investigate these issues, we conducted two stud-
ies. The first one aimed to understand the teachers’ perspec-
tive and the second one to provide insights into the students’
perception of the system.
16839
 T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students

FIGURE 3. ALF-G students’ view (left hand side) and ALF-G teacher’s view (right hand side).

FIGURE 4. Teacher’s view of the class through the glasses.

A. FIRST STUDY: ANALYSING THE TEACHERS’
EXPERIENCE USING ALF-G
The first study aimed to gather insights into the teachers’
opinions on ALF-G’s potential benefits and drawbacks. With
this purpose, we asked 20 teachers from different educational
centers and disciplines to use the ALF-G in one of their
regular lectures.
1) PARTICIPANTS AND APPARATUS
20 teachers participated in the study. 8 of them used the
ALF-G in lectures of Bachelor degrees in law (1), political
science (1), computer science (7), and telematics engineer-
ing (3). Another 8 used it in classes of medium and higher
vocational training courses in electricity (2), administration
and finances (2), network systems management (2) and infor-
matics (2). Finally, the last 2 used it in high school maths
(1) and history (1) courses. The ages of the teachers ranged
from 30 years to 57, most of them being between 35 and 45.
The size of the classes varied from 15 to 40 students, and
in most cases, they were between 20 and 30 students. The
duration of the classes was 2 hours for the Bachelor degrees
classes (10 classes), and 1 hour for the other ones (10 classes).
Concerning the apparatus used, students used their own
mobile phones to access the ALF-G web site, whereas the
teacher utilized a pair of Google Glass. Only 2 participants
had tried the Google Glass before.
2) PROCEDURE
Before each lecture a member of the research team trained the
teacher on how to use the system and explained the different
functionalities it offers. Then, at the beginning of the class the
teacher introduced the system to the students, who practiced
sending questions and status with their mobiles from around
5 minutes. We asked the teachers to use the system as an
ARS, for collecting the students’ responses to their questions,
as well as a backchannel, enabling the students to provide

16840 VOLUME 7, 2019

T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students
TABLE 1. Summary of the responses to the questions on class dynamics.
FIGURE 5. Frequency diagram of the responses for the questions on the Class Dynamics.
constant feedback about their level of understating on the
lecture content. At the end of the classes, the teachers were
asked to fill a questionnaire about the experience and they
were also briefly interviewed.
3) DATA COLLECTION MECHANISMS
The instruments used in the evaluation aimed at provid-
ing quantitative and qualitative data on the teachers’ opin-
ions about the system. Quantitative data were obtained from
the items in Table 1. The teachers were asked to rate the
list of potential benefits and disadvantages using a 5-point
Likert scale of agreement. Qualitative data were obtained
from three open questions included to give the teachers the
opportunity to explain if they think this technology could
change somehow the way they teach, if they wanted to
count with it in the next classes and provide comments and
suggestions.
In addition, and to understand better the teachers’ answers
to the questionnaires, semi-structured interviews were con-
ducted at the end of the classes. In these interviews the
teachers were asked to give a general overview of the expe-
rience, to elaborate their answers to the questionnaire, and to
provide advice and recommendations on how to improve the
system.
VOLUME 7, 2019
4) RESULTS
We collected 20 responses to the questionnaires. Descrip-
tive analysis was performed on the questionnaire data using
the mean and standard deviation values and the frequen-
cies distributions. For the responses to the open questions
and the transcriptions of the interviews content analysis was
used [11].
a: TEACHERS’ VIEWS ON THE IMPACT OF ALF
GLASS IN THE CLASS DYNAMICS
Table 1 depicts the mean and standard deviation of
the responses to the questionnaire on class dynamics,
and Figure 5 a frequency histogram of the answers. Ques-
tions Q4, Q5 and Q6 should be interpreted in the opposite
way since they were reverse coded in order to avoid the
acquiescence bias that happens when participants adopt an
automatic behavior and mark answers without even reading
the question [18]. As shown in Figure 7, the responses suggest
that teachers valued the system very positively. Most of them
agreed in that the system helps to adapt the rhythm of the
class (85%), improves the communication (80%), and makes
the students more engaged in the class (75%). It is neces-
sary to note that none of the teachers disagreed with these
statements. With regards to the potential drawbacks, 70% of
16841
 T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students
the teachers did not considered that the system disrupts their
activity or distracts the students, whereas a 15% agreed with
these shortcomings.
b: TEACHERS’ OPINIONS
We collected 17 answers to the question ‘‘Do you think
the system could change the way you give your classes?.’’
12 teachers responded affirmatively, 3 negatively, and
2 explained that to exploit the system opportunities it would
be necessary to modify the way they prepare their classes.
According to their responses the instruction process could
change due a better adaptation of the explanations and rhythm
of the class (4 subjects), an increment in students’ partici-
pation (3 subjects) and a better feedback from the students
(2 subjects). It’s worth noting here that 2 out of the 3 teachers
who responded negatively also acknowledged some benefits
that the system might report. As one of them stated ‘‘I don’t
think it would change the way I teach, but I will use it to
identify the key moments in the explanations in which the
students change their answers, the instants in which their
mental model change.’’
For the question ‘‘Would you like to use the system in your
classes regularly?’’ we collected 18 positive and 1 negative
answers. In addition, 1 subject explained that he/she would
like to use it but only during some specific parts of the class.
Among the suggestions, 2 subjects proposed to pro-
vide teachers with an historical of the students’ feedback
responses. This would help them to analyses the class latter
on, identifying what went wrong and what really worked.
During the interviews, most teachers reiterated that the
major benefit of the system is providing immediate infor-
mation about students’ current knowledge. This can help to
increase awareness of comprehension problems that might be
overlooked otherwise. As one of the teachers described ‘‘. . .
in the middle of an explanation I noticed that most of the circle
was red. This was abit of a shock, as I always assumed that
the students followed this part of the class effortlessly. I have
never stopped to check at that point of the class before.’’
In addition, one of them mentioned that the system could help
to compare the effectiveness of different ways of explaining
the same concept: ‘‘The system allows to identify the precise
moment in which the students grasp the concept explained.
I could try to explain the same concept in two different ways
in two classes, and I would know in which one the students
grasped the explanation earlier. This is something difficult to
know right now.’’
5) DISCUSSION
In general, the responses from the teachers were very positive.
There was almost a general consensus in that the system
helps to improve the communication in the class, to adapt the
explanations, and even to keep the students engaged. In gen-
eral, most of the teachers acknowledged the opportunities that
the system could open up for improving their activity in the
classroom due to a better awareness of the students’ current
knowledge. With regards to the drawbacks, for most of them,
16842
the distraction of the students was not a major concern, and
the device did not seem to disrupt their normal activity. These
results are very encouraging, especially considering that only
2 teachers had tried the Google Glass previously, and that not
all of them had a technical background.
B. SECOND STUDY: ANALYSING THE STUDENTS’ VIEWS
In order to gather insights about students’ opinions about
ALF-G we conducted a second study during the regular
classes of a university course.
1) PARTICIPANTS AND APPARATUS
The study took place during an introductory course on Com-
puter Programming of a Bachelor on Biomedical Engineering
at University Carlos III of Madrid that is taught in the first
semester of the first year of the degree. Since the study was
run in a realistic environment, that is, in regular classes,
a in-group study in which all students are exposed to the
same conditions was chosen. Also, and in order to be able to
compare the use with ALF-G with other options, the system
was used in the second half of the course, so that students
were exposed first to traditional lectures. At this point, it is
important to remark that this study was run with first year
students who are not already used to the university lectures,
teachers and even mates. That makes it quite difficult to
achieve a great degree of participation, since students are
often embarrassed to ask questions in an unfamiliar and
unknown environment.
The duration of the course was 14 weeks, and the num-
ber of students enrolled was 70. The teaching methodology
included weekly lectures in a standard classroom and practice
sessions in a computer lab. The duration of both types of
session was 2 hours. The ALF-G system was used in the last
7 lectures of the course. Previously, and to make sure that
students knew how to use it, the system was introduced in
some short activities during practice sessions. With regards
to the apparatus, the students used their own mobile phones
to access the ALF-G functionalities, whereas the teacher
utilized a pair of Google Glass. The responses and status col-
lected during the class were displayed on the glasses screen,
and the teacher controlled the different system functionalities
tapping and scrolling the touchpad of the glasses.
2) DATA COLLECTION MECHANISMS
To allow comparison of the students’ opinions about normal
lectures and ALF-G augmented lessons we conducted two
surveys. The first one was filled in after the first 7 weeks of
classes, during which lectures were given in a traditional way,
and the second one was distributed after the last 7 weeks of
classes, that is, after the teacher and students were exposed to
ALF-G lectures (see Fig. 6). In both surveys the participants
were asked to provide feedback on the class dynamics in the
form of 5 point Likert-scale answers. As shown in Table 2,
participants were asked to indicate their level of agreement
with some statements about the adequacy of the pace of the
classes (Q1), their comfort level when communicating their
VOLUME 7, 2019
T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students

FIGURE 6. Experiment description.

TABLE 2. Summary of responses and Mann-Whitney U test for the questions on the class dynamics.

FIGURE 7. Frequency of the responses for the questions on the Class Dynamics.

difficulties (Q2), and their engagement in the classes (Q3). 3) RESULTS

In the second survey Q2 and Q3 were adapted to specifically
refer to the use of the system. Also, the second section
included a fourth item (Q4) gathering feedback on the dis-
traction that the interaction with the app might cause. Both
surveys ended with some open-ended questions that made it
possible to provide additional explanations, comments and
suggestions.
At the end, we collected 60 responses for the first survey,
and 53 for the second one. In both cases the surveys were
anonymous since a disclosure of the respondent identity could
influence their answers.
a: THE IMPACT OF ALF GLASS IN THE CLASS DYNAMICS
Table 2 and Figure 7 summarize the mean and standard devi-
ation of the responses to the questionnaire on class dynamics,
and the distribution of the responses, respectively. It is impor-
tant to point out that the responses to Q1 and Q4 should be
interpreted in the opposite way respect to the other questions,
since they were reverse coded. Q4 was only included in
the second questionnaire since it explicitly focuses on the use
of the app.
As shown in the diagram, the number of students who
responded negatively to the three questions Q1, Q2 and Q3

VOLUME 7, 2019 16843

 T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students
decreased considerably in the post survey. In the case of Q1,
inadequacy of the pace, the percentage of students decreased
from 15% in the normal lecture to 2% in the ALF-G lecture.
Similarly, for Q2, the percentage of students who did not feel
comfortable when asking questions changed from 25% in the
normal lecture to 11%, in the ALF-G lecture. Finally, con-
cerning Q3, the number of students who did not feel engaged
with the class was reduced from 17% to 2%. However, only
in the case of Q1 and Q2 did this translate into a notable
increment of positive responses. The positive responses for
Q1 went from 50% to 73%, for Q2 from 60% to 78%, but
for Q3 they only changed from 57% to 60%. The overall
mean of the responses improved by 0,52 for Q1, 0,54 for
Q2 and 0,28 for Q3 (see Table 1). To determine whether the
differences in the results for the items Q1, Q2 and Q3 were
statistically significant we applied the Mann-Whitney Wallis
test. A statistically significant difference was found for Q1
(U=1.163,500, ρ = 0, 010) and Q2 (U=1.943,000, ρ = 0,
034), whereas no statistically significant difference was found
for Q3 (U=1.787,000, ρ = 0, 234).
Finally, only 15% of the students reported getting dis-
tracted by the app (Q4), whereas for almost the rest of the
class (77%) this was not an issue. In this case the mean of the
responses was M=2,01 and SD=1,02.
b: STUDENT’S OPINIONS
As commented above, the questionnaires included a set of
open-ended questions that students could use to explain their
answers or provide any additional comment. The responses
to the questions in the first survey revealed that very often
the causes for not communicating difficulties to the teacher
were related to speaking in front of their pears: ‘‘I don’t want
to draw attention to myself,’’ ‘‘I feel a little ‘‘silly,’’ ‘‘I never
want to be ‘the dumb one’’’or‘‘I don’t want to speak in front of
my fellow students’’. In addition, some of them also reported
to preferring not to interrupt the teacher (‘‘I don’t want to
interrupt so that the class can proceed more fluidly’’) or to let
him know about their difficulties (‘‘I don’t want the teacher to
know that I don’t understand something,’’ ‘‘The teacher might
think that I don’t work enough’’).
In the second questionnaire students were asked to provide
their general opinion about the application, and to provide
comments and suggestions. Many of the responses collected
were encouraging. (‘‘It is a good idea,’’ ‘‘It is helpful when
I don’t understand something,’’ ‘‘I think it is a nice, interactive
and fun way to learn, and to let the teacher know how good
or bad I am doing’’). However, some students reported not
liking to use their mobiles in class (‘‘It is not comfortable to
have the mobile phone on all the time in class,’’ ‘‘ Whenever
I want to participate I have to pick up the phone, find the
application. . . it takes too much time,’’ ‘‘The phone goes into
downtime when you are not using it. It would be better if it
were installed on another device’’). Among the students who
did not consider the application useful, some reported not to
be sure about the reaction of the teacher to their interactions.
As one of them stated: ‘‘I push the question button but I’m not
16844
sure if the teacher is going to stop or if he is going to care if
just one person is not following the explanation.’’Some others
explained they prefer to address the teacher directly (‘‘I think
it’s easier to raise your hand and ask the teacher’’). With
regards to potential improvements, some students proposed
being able to send the teacher specific questions.
C. DISCUSSION
The results of the study suggest that, from the perspective
of the students, the use of a system like ALF-G could report
some benefits, particularly to those students who feel embar-
rassed to interact publicly with the classroom, confirming
what teachers said in the first study. In particular, more stu-
dents were satisfied with the pace of the class and reported
feeling comfortable when communicating their difficulties
to the teacher. However, overall, students’ opinions on these
issues seem not to improve greatly. With regards to the
student’s engagement, the third benefit analyzed, the data
collected doesn’t identify any clear enhancement. Finally,
and regarding the potential drawbacks, most of the students
indicated that the app did not distract them from the class,
though less disruptive mechanisms might be envisioned to
avoid problems when the cell phone is locked, like using more
simple notification systems that can be used in other devices
like smart watches.
The student’s profiles might play an important role in
their attitude to the system. Those students who in the first
questionnaire reported to be reluctant to communicate their
problems due to shyness and fear of public exposure might
find the system more useful. On the contrary, for the students
who usually participate in class, the system offers little ben-
efits over raising their hands and asking questions directly.
In addition, it is necessary to note that some students seemed
to feel a bit intimidated and distrusted a system which might
help to identify who were experiencing difficulties, or that
it could be used to record with the camera their actions in
class. The use of wearable devices like the Google Glass
can provoke privacy concerns [15] that need to be addressed
before this technology is fully integrated in daily activities
like teaching. A better explanation of the system’s perfor-
mance at the beginning of experiment might have helped
to reduce suspicion, but in any case, this is a brand new
technology users might need to live with and experience to
have a more positive and trustful attitude.
V. THREADS TO VALIDITY
There are some limitations of the studies presented here that
need to be highlighted. Firstly, it is necessary to note that
the two studies focus on the opinions of participants, be they
students or teachers. We believe that this is the best way to
assess subjective perceptions as the adequacy of the rhythm
of the class, or the willingness to participate. In any case,
the collection and comparison of the number of students’
interventions with and without the system could help to con-
firm some of the conclusions presented. Also, the difficulty
of controlling the wide range of variables that can influence
VOLUME 7, 2019
T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students
the results of the experiments carried out in the context of a
classroom should be acknowledged [7]. For instance, the per-
ception of the rhythm of the class could have been influenced
by the section of the course in which the system was used,
as by the end of it the students might have grown used to
the subject difficulty and to the teacher’s personality. In any
case, it is necessary to note the results obtained are consistent
in both studies, and even the students perceptions confirm
the opinions gathered from teachers in the first study, who
agreed in considering the system beneficial for improving
communication and class adaptation. In addition, the opin-
ions collected in the first study correspond to teachers who
used the system in one single class, so integrating the system
into a whole course might provide different results. Also,
the profile of the participants in the second study, engineering
students, could have influenced their positive attitude towards
that system. In any case, the results are encouraging, and
suggest that AR could improve current classroom communi-
cation systems’ implementations, providing a more seamless
way of delivering the feedback gathered from the students to
the teacher.
VI. CONCLUSIONS
We have presented a novel configuration of a communication
system for the classroom that makes use of AR technology.
As opposed to other backchannels and ARS systems that
require the teacher to periodically check the responses of the
students on a computer screen or to publicly display in them
on the classroom display, the ALF-G system makes it possible
to visualize the feedback overlapping digital information to
the teacher’s view of the classroom. This provides a constant
and private flow of information on the students’ level of
comprehension of the explanations which would be difficult
to achieve by other means. The results of the study suggest
that this could report benefits in terms of better communi-
cation between students and teachers and a more adequate
rhythm of the class. Also, that the use of the system in the
classroom does not necessarily constitute a distraction from
or and disruption of educational activity.
Finally, it is necessary to note the original version of the AR
device used in the studies, the Google Glass Explorer Edition,
was discontinued. However, Google has recently released a
new version designed for industrial applications, and there are
other models of AR Glasses available in the market, as Epson
Moverio or Vuzix M100 Smart Glasses. Therefore, it is rea-
sonable to expect that in the future AR devices ergonomic
enough for use in a classroom environment. The results of
the study are relevant as they provide evidence of the positive
attitude of students and teachers towards this technology, and
of the benefits of its use for educational purposes.
REFERENCES
[1] D. Baron, A. Bestbier, J. M. Case, and B. I. Collier-Reed, ‘‘Investigat-
ing the effects of a backchannel on University classroom interactions: A
mixed-method case study,’’ Comput. Educ., vol. 94, pp. 61–76, Mar. 2016.
[2] G. Bergtrom, ‘‘Clicker sets as learning objects,’’ Interdiscipl. J. E-Learn.
Learn. Objects, vol. 2, no. 1, pp. 105–110, 2006.
VOLUME 7, 2019
[3] C. A. Brewer, ‘‘Near real-time assessment of student learning and under-
standing in biology courses,’’ BioScience, vol. 54, no. 11, pp. 1034–1039,
2004.
[4] F. Bry and A. Y.-S. Pohl, ‘‘Large class teaching with backstage,’’ J. Appl.
Res. Higher Educ., vol. 9, no. 1, pp. 105–128, 2017.
[5] J.-F. Bussières, M.-É Métras, and G. Leclerc, ‘‘Use of Moodle, ExamSoft,
and Twitter in a first-year pharmacy course,’’ Amer. J. Pharmaceutical
Educ., vol. 76, no. 5, p. 94, 2012.
[6] J. E. Caldwell, ‘‘Clickers in the large classroom: Current research and
best-practice tips,’’ CBE Life Sci. Edu., vol. 6, no. 1, pp. 9–20, 2007.
[7] L. Cohen, L. Manion, and K. Morrison, Research Methods in Education,
Abingdon, U.K.: Routledge, 2013.
[8] H. Du, M. B. Rosson, and J. M. Carroll, ‘‘Augmenting classroom partici-
pation through public digital backchannels,’’ in Proc. 17th ACM Int. Conf.
Supporting Group Work. New York, NY, USA: ACM, 2012, pp. 155–164.
[9] M. Dunleavy, C. Dede, and R. Mitchell, ‘‘Affordances and limitations of
immersive participatory augmented reality simulations for teaching and
learning,’’ J. Sci. Educ. Technol., vol. 18, no. 1, pp. 7–22, 2009.
[10] M. Dunleavy and C. Dede, ‘‘Augmented reality teaching and learning,’’ in
Handbook of Research on Educational Communications and Technology.
New York, NY, USA: Springer, 2014, pp. 735–745.
[11] H.-F. Hsieh and S. E. Shannon, ‘‘Three approaches to qualitative content
analysis,’’ Qualitative health Res., vol. 15, no. 9, pp. 1277–1288, 2005.
[12] H.-K. Jee, S. Lim, J. Youn, and J. Lee, ‘‘An augmented reality-based
authoring tool for E-learning applications,’’ Multimedia Tools Appl.,
vol. 68, no. 2, pp. 225–235, 2014.
[13] R. H. Kay and A. LeSage, ‘‘Examining the benefits and challenges of using
audience response systems: A review of the literature,’’ Comput. Edu.,
vol. 53, no. 3, pp. 819–827, Nov. 2009.
[14] G. E. Kennedy and Q. I. Cutts, ‘‘The association between students’ use
of an electronic voting system and their learning outcomes,’’ J. Comput.
Assist. Learn., vol. 21, no. 4, pp. 260–268, 2005.
[15] V. G. Motti and K. Caine, ‘‘Users’ privacy concerns about wearables,’’
in Proc. Int. Conf. Financial Cryptogr. Data Secur. Berlin, Germany:
Springer, 2015, pp. 231–244.
[16] M. Ratto, R. B. Shapiro, T. M. Truong, and W. G. Griswold, ‘‘The active-
class project: Experiments in encouraging classroom participation,’’ in
Designing for Change in Networked Learning Environments. Amsterdam,
The Netherlands: Springer, 2003, pp. 477–486.
[17] K. Siau, H. Sheng, and F. F. H. Nah, ‘‘Use of a classroom response system
to enhance classroom interactivity,’’ IEEE Trans. Educ., vol. 49, no. 3,
pp. 398–403, Aug. 2006.
[18] B. Weijters and H. Baumgartner, ‘‘Misresponse to reversed and negated
items in surveys: A review,’’ J. Marketing Res., vol. 49, no. 5, pp. 737–747,
2012.
[19] S. C.-Y. Yuen, G. Yaoyuneyong, and E. Johnson, ‘‘Augmented reality:
An overview and five directions for AR in education,’’ J. Educ. Technol.
Develop. Exchange, vol. 4, no. 1, pp. 119–140, 2011.
[20] T. Zarraonandia, I. Aedo, P. Díaz, and A. Montero, ‘‘An augmented lecture
feedback system to support learner and teacher communication,’’ Brit.
J. Educ. Technol., vol. 44, no. 4, pp. 616–628, 2013.
TELMO ZARRAONANDIA received the degree
in computer science from the Universidad de
Deusto and the Ph.D. degree in computer science
from the Universidad Carlos III de Madrid, where
he is currently an Associate Professor. He has been
a Visiting Researcher with the Universidad Federal
do Rio de Janeiro and the Universitá degli Studi di
Salerno, Italy. He has published several scientific
papers in international journals and conferences
related to the fields of computer supported educa-
tion and interactive systems. His research interests include technologies to
support education and learning, videogame design, mixed reality environ-
ments, and interactive systems.
16845
 T. Zarraonandia et al.: Using a Google Glass-Based Classroom Feedback System to Improve Students
PALOMA DÍAZ is currently a Full Professor
with the Computer Science Department, Univer-
sity Carlos III de Madrid. She is currently the
Head of the DEI Lab Research Group, Univer-
sidad Carlos III de Madrid, where she focused
on the design of interactive systems applied to
different domains such as emergency and crisis
management, e-learning, or cultural heritage. Her
main research interests include interactive systems
engineering, collaborative systems, visualization,
and ubiquitous computing. Concerning the topics of this chapter, she has led
projects on interactive and educational systems modeling, and on the defini-
tion of conceptual frameworks for the co-design and production of serious
games supporting children informal learning. She has been a Visiting Scholar
with the Information Science and Technology College, PSU, the MAGIC
Lab, The University of British Columbia, and the VISUS Institute, University
of Stuttgart.
ÁLVARO MONTERO received the bachelor’s
degree in computer science and the M.Sc. degree
in computer science and technology from the Uni-
versidad Carlos III de Madrid, in 2012 and 2013,
respectively, where he is currently pursuing the
Ph.D. degree with DEI Lab. His current research
interests include augmented reality, augmented
environments, and end-user development. He has
been a Visiting Researcher with Human Inter-
face Technology Lab New Zealand, University of
Canterbury, in 2017.
16846
IGNACIO AEDO received the bachelor’s and
Ph.D. degrees in computer science from the Uni-
versidad Politécnica de Madrid. He is currently
a Full Professor with the Universidad Carlos III
de Madrid, Escuela Politécnica Superior. His
research interests include hypermedia, interactive
systems in education, web systems, electronic
books, development methodologies, and informa-
tion systems for emergency situations. Since 1990,
he has been involved in the field of interactive
systems. He was a Visiting Researcher with CSCL, IST Pennsylvania
State University (2007–2008), and MAGIC Lab, The University of British
Columbia (2011). He is currently the Chair of a Group Vision Grant of tech-
nologies for the collaboration, the Co-Chair of the IEEE ICALT 2008, 2009,
2010, and 2012, the Subdirector of the Culture and Technology Institute, and
the Deputy Vice-Chancellor of faculty and departments.
TERESA ONORATI received the B.Sc. and M.Sc.
degrees in computer science from La Sapienza
Università di Roma and the Ph.D. degree in com-
puter science from the Computer Science Depart-
ment, Universidad Carlos III de Madrid (UC3M),
where she was involved in modeling the semantics
of several knowledge domains for improving the
accessibility of critical information, in 2013. She
is currently a Visiting Professor with the Computer
Science Department, UC3M. As a member of the
DEI Interactive Systems Research Group, UC3M, she is involved in several
research projects about the role of technology for improving citizen col-
laboration and participation in critical situations. She has been a Visiting
Researcher with the Researcher Center, Inria, Paris, Saclay, France, where
she has been involved in large display interaction, and she has been with
the GEOVista Research Center, Penn State University, PA, USA, where
she has been involved in visual analytics for geographic information. Her
research interests include semantic modeling, visual analytics, and informa-
tion visualization for supporting a better understanding of how information
and people sentiments propagate within a complex data ecosystem.
VOLUME 7, 2019