Assessment & Evaluation in Higher Education

ISSN: 0260-2938 (Print) 1469-297X (Online) Journal homepage: https://www.tandfonline.com/loi/caeh20

Grudges and gratitude: the social-affective

impacts of peer assessment

Jiming Zhou, Yongyan Zheng & Joanna Hong-Meng Tai

To cite this article: Jiming Zhou, Yongyan Zheng & Joanna Hong-Meng Tai (2019): Grudges and
gratitude: the social-affective impacts of peer assessment, Assessment & Evaluation in Higher
Education, DOI: 10.1080/02602938.2019.1643449

To link to this article: https://doi.org/10.1080/02602938.2019.1643449

Published online: 23 Jul 2019.

Submit your article to this journal

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=caeh20
ASSESSMENT & EVALUATION IN HIGHER EDUCATION
https://doi.org/10.1080/02602938.2019.1643449

Grudges and gratitude: the social-affective impacts of

peer assessment
Jiming Zhoua , Yongyan Zhenga and Joanna Hong-Meng Taib
a
College of Foreign Languages and Literature, Fudan University, Shanghai, China; bCentre for Research in
Assessment and Digital Learning, Deakin University, Geelong, Australia

ABSTRACT KEYWORDS
Students’ dissatisfaction with peer assessment has been widely docu- Peer feedback; social-
mented. While most relevant literature places focus on the cognitive affective impacts; respect
(content and uptake of feedback) or structural (feedback design) dimen-
sions, students’ emotions in peer assessment have received scant atten-
tion. This study investigates the social-affective impacts of peer
assessment by analysing students’ appeal letters addressed to their
tutors, reflective posts in the online discussion forum and responses to
a survey. A thematic analysis of data indicated three main aspects of
students’ (dis)satisfaction: content, scores and process of peer assess-
ment. The most negative emotion that students expressed was related
to ‘disrespectful’ behaviour and attitudes of peer reviewers, whereas the
feeling of appreciation was triggered by the helpful feedback attributes
which were perceived as reflecting reviewers’ respect to others’ works.
Students generally held mixed feelings toward peer assessment, valuing
learning in the process of providing and receiving feedback but show-
ing resistance to using peer assessment for summative purposes. The
findings highlight the significance of respect in peer assessment and
argue that the perceived lack of mutual respect seems to underpin the
nature of students’ dissatisfaction. This study carries implications for nur-
turing students’ respectful attitudes and behaviour in and through
peer assessment.

Introduction
‘Your Background part just made me headache and I didn’t want to read anymore’
(Yu et al. 2018, 9)

When students receive peer feedback like this, it is unsurprising that they refuse to engage
with it, both behaviourally and cognitively. Ideally, feedback is a constructive and dialogic
process, where students draw on their evaluative judgement (Tai et al. 2018) and reflective
competence to identify a gap in knowledge or performance and take appropriate actions to
improve (Assessment Reform Group 2002; Boud and Soler 2016). In the era of mass educa-
tion, peer assessment offers additional feedback to students in a timely manner
(Boud, Cohen, and Sampson 2014). In practice, however, feedback has been reported by
students as the least satisfactory aspect in their university learning experience in diverse con-
texts, including Australia (Carroll 2014; Quality Indicators for Learning and Teaching 2017),

CONTACT Yongyan Zheng yongyanzheng@fudan.edu.cn

ß 2019 Informa UK Limited, trading as Taylor & Francis Group
2 J. ZHOU ET AL.

China (Guo and Shi 2016), and the UK (Higher Education Funding Council for England 2016;
Bell and Brooks 2018).
Existing studies report organizational and individual factors contributing to students’ dissatis-
faction with feedback in higher education, such as mismatched learner-teacher perceptions
about what makes feedback effective (Dawson et al. 2019) and students’ lack of a growth mind-
set (Forsythe and Johnson 2017). Recent studies adopt the notion of feedback as a social and
communicative process (Ajjawi and Boud 2018; Carless and Boud 2018) and begin to examine
the interplay between students’ affect and contextual factors (e.g. Carless 2013; Pitt and Norton
2017; Xu and Carless 2017). While these studies begin to identify social-affective factors media-
ting students’ reactions to feedback (e.g. self-esteem in Nicol and Macfarlane-Dick 2006; trust in
Carless 2013), there may be additional factors that contribute to student dissatisfaction, particu-
larly within peer assessment contexts (Adachi, Tai, and Dawson 2018).
Examining students’ situational affect (feelings, attitudes and emotions) can help to under-
stand student’s dissatisfaction with peer feedback (Ajjawi and Boud 2017). Previous studies have
predominantly elicited students’ perceptions and emotions related to peer feedback via focus
group discussions (Patton 2012), questionnaires (Huisman et al. 2018), interviews (Wang 2014;
Zhu and Carless 2018) and stimulated recalls (Yu et al. 2018). While these studies delineate stu-
dents’ retrospective perceptions and recurring emotions, they may not fully capture students’ in-
situ feelings and experiences (Pekrun 2016). A more nuanced approach to understanding stu-
dents’ situational negative and positive experiences carries pedagogical implications for design-
ing and enacting peer feedback for learning. It also contributes to a better understanding of
how multiple factors interplay and mediate peer dialogue. Students’ abilities to maintaining peer
dialogue with positive emotions are important in both higher education and future workplace
contexts (Adachi, Tai, and Dawson 2018).

Understanding factors mediating students’ affect related to feedback

Multiple factors may mediate students’ affect in the feedback dialogue. Yang and Carless (2013)
identify cognitive, structural and social-affective dimensions which overarch some major factors.
The key dimensions are relational and interactive (Ajjawi and Boud 2017), and they serve as a
good frame through which to review existing evidence about mediating factors. Given the rela-
tive scarcity of literature on the impacts for peer feedback, we review previous studies about
both teacher and peer feedback, supplementing the list of factors with literature from related
fields where necessary.

Feedback content
The content of feedback has been widely researched among the myriad factors which might
trigger students’ negative emotional and behavioural reactions. Feedback content has mul-
tiple features, ranging from accuracy, tone, focus, level of detail to congruency with expecta-
tions and comprehensibility (Lipnevich, Berg, and Smith 2016). Students’ feelings of injustice
and unfairness might be triggered if grades were lower than their expectations (Nesbit and
Burton 2006; Ryan and Henderson 2018). They tend to perceive peer comments with detailed
explanations as adequate and consequently demonstrated more willingness to take actions
(Huisman et al. 2018). Students prioritized the role of content features and believed that
‘usable, detailed, considerate of affect and personalised’ comments made feedback effective
(Dawson et al. 2019, 25). Feedback that fails to embody these features is likely to result in
students’ grudges.
 ASSESSMENT & EVALUATION IN HIGHER EDUCATION 3

Feedback design
Design matters have been perceived by educational practitioners as the most powerful factors
which influence the effectiveness of feedback (Dawson et al. 2019). Transforming students’ roles
(Hounsell 2007) and fostering a dialogic feedback process (Yang and Carless 2013) have been
maintained as crucial aspects of feedback design. Verbal communications between peer feedback
providers and receivers contribute to meaning clarification and negotiation (Zhu and Carless
2018). However, previous studies also report students’ dissatisfaction during teachers’ attempts
at transforming learners’ role in feedback. Students in Australian universities (Patton 2012) sup-
ported the use of peer assessment as a formative exercise, but they described it as weird and
evil when scores were produced during the process. Students also reported a feeling of unfair-
ness and believed that peers tended to give them low scores because they were competitors
(Wilson, Diao, and Huang 2015). In contrast, a comparative study between peer assessment for
summative and formative purposes (Sridharan, Tai, and Boud 2019) suggested summative peer
grading was more likely to drift toward leniency.

Situational contexts
Situational contexts influence the interplay between feedback and students’ emotional and
behavioural responses. Students are more likely to take up feedback if they have communication
trust in the classroom environment and competence trust in their teachers to design and enact
effective assessment (Carless 2013). When participating in peer assessment, students generally
lack competence trust in their peers and themselves (Wilson, Diao, and Huang 2015). The level
of distrust is especially high when peer assessment is used for summative purposes (Patton
2012). Teachers could tone down the competition and nurture an atmosphere of trust to help
students accept critical peer feedback with a positive attitude (Xu and Carless 2017). National
award-winning teachers in Chinese universities believed that feedback with low-level threat to
students’ self-esteem would work better (Zhou and Deneen 2016).

Individual differences
Students also demonstrate individual differences in their interpretation of and reactions to feed-
back. They may have different motivations and self-regulated skills to reflect and work on feed-
back (Brookhart 2011). They may lack a growth mindset and therefore demonstrate self-
defensive behaviour in reaction to feedback (Forsythe and Johnson 2017). Similar feedback may
trigger positive (e.g. appreciation, gratitude, happiness, and pride) or negative (e.g. anger, frustra-
tion, fear) emotions among students (Rowe 2011), and subsequently carry emotional backwash
on their learning (Pitt and Norton 2017). How students perceive feedback activities interplays
with their feelings and experiences during the feedback process. Those who held positive per-
ceptions of peer feedback were more keen on providing specific comments and taking actions
(Yu and Hu 2017). The instrumental views of assessment for mere task completion impaired stu-
dents’ engagement in peer assessment (Zhao, Zhang, and Du 2017).

Respect
The issue of respect has been touched upon in anecdotal accounts but has not yet received
much attention in the feedback literature. Specifically, respect has been briefly mentioned as a
minor sub-theme in previous studies about trust (Carless 2013) or about students’ engagement
with feedback from teachers (Han and Hyland 2019) and peers (Yu et al. 2019). Respect, together
with empathy, has been argued as underpinning students’ communication trust in seeking and
4 J. ZHOU ET AL.

Table 1. Overview of the two courses.

Course A Course B
Course name English Thesis Writing English Argumentative Essay Writing
Targeted students Year 4 English majors Year 1–3 non-English majors
Student number 70 79
Course assessment 4 writing assignments (40%) 4 writing assignments (40%)
Out-of-class final report (30%) In-class final essay writing (40%)
Class discussion & participation (10%) Classroom discussion & participation
On-line discussion (10%) (10%)
Attendance (10%) Attendance (10%)
Peer assessment Peer assessment conducted in the Peer assessment conducted in the third
second writing assignment writing assignment

receiving feedback (Carless 2013). In a Confucian heritage setting, university students interpreted
teachers’ provision of written feedback as committing responsibilities and showing respect to
individual students (Han and Hyland 2019). Students also reported a feeling of awe which was
mixed with admiration, respect and fear when they received high-quality teacher written feed-
back (Han and Hyland 2019).
Given the lack of feedback-related literature on respect, we draw upon relevant notions from
the field of philosophy. Darwall (1977) made an influential distinction between two types of
respect: appraisal respect and recognition respect. While appraisal respect is activated when the
object demonstrates certain conditions and deserves to be responded to favourably, recognition
respect is an attitude mainly related to the process and is regardless of the object’s concrete fea-
tures. Recognition respect is ‘an attitude on the process’ while appraisal respect is ‘a reflection of
its results’ (van Quaquebeke, Henrich, and Eckloff 2007, 188).
Drawing upon Darwall’s (1977) categorization, respectful listening and responding which
builds trust in assessment (Ajjawi and Boud 2018) embodies recognition respect. Respect under
the category of awe in Han and Hyland’s study (2019) can be considered as appraisal respect:
students evaluated the quality of teacher feedback and acknowledged the manifestation of
excellence. When students believed that teachers provided written feedback to demonstrate con-
cerns about an individual student (Han and Hyland 2019), they felt recognition respect.
In summary, the review of the literature has shown that students’ affect in the feedback pro-
cess is an outcome of the dynamic interplay of multiple factors. Social-affective elements perme-
ate peer assessment processes and influence whether the learning improvement potential would
be realized (Ajjawi and Boud 2018). Without a clearer understanding of students’ dissatisfaction,
we are unlikely to nurture students’ positive emotions in peer assessment and improve its peda-
gogical worth. This study, therefore, sets out to address these research questions:

1. Which aspects of peer assessment triggered students’ (dis)satisfaction?

2. What were students’ general perceptions and emotions related to peer assessment?

Methods
Teaching and assessment context
This study was set in two English writing courses (see Table 1) in a top-tier university in China.
One of the teaching goals of this university is to equip students with English competence to
conduct research and write reports and papers in their specialized fields. Course A and Course B
were core writing courses taught by two different tutors from the College of Foreign Languages
and Literature. Both courses involved 16 classes of ninety minutes spread over four months.
Students in Course A and B are Chinese undergraduates with different levels of English profi-
ciency. According to the study which mapped China’s Standards of English Language Ability to
 ASSESSMENT & EVALUATION IN HIGHER EDUCATION 5

Table 2. Data list.

Student appealing email (in Chinese)
Student reflections in an online forum
(in English)
Student responses to a small-scale
survey (in Chinese)
Course A (70 students) Course B (79 students)
16 Email messages 20 Email messages
22 Posts about their emotional and /
behavioural reactions to peer
feedback 30 Posts about their feelings
about peer assessment
/ 69 Responses to questions about their
attitudes and degrees of satisfaction

IELTS scores (as of June 20, 2019, IELTS official website), the average English writing proficiency
of the Year 4 English-majored students in Course A can be linked to IELTS Writing Band 7, that
of Year 1-3 non-English-majored students in Course B to IELTS Writing Band 5–6. Such a variation
enabled us to take into account students’ English writing proficiency when we are exploring fac-
tors that mediate students’ affect related to peer assessment.
In both courses, the assessment was mainly in the form of student writing, submitting drafts
and the teacher providing written feedback. The rubrics of writing assessment were provided to
students at the beginning of the semester. Course A and Course B shared similar rubrics with
five dimensions (i.e. task achievement, argument development, organization and coherence,
vocabulary and grammar, and mechanics), and each dimension incorporated three scales (Level
1–3) with specific descriptions. For instance, the descriptor of Level 3 (highest level) in the
dimension of ‘argument development’ is ‘presents a fully developed position in answer to the
question; arguments are clear and logically coherent; main points excellently supported with
relevant evidence’.
Peer assessment was respectively enacted during the second and third writing tasks in
Courses A and B. Students’ previous writings were graded and commented on by their tutors
according to the rubrics. Prior to the enactment of peer assessment, the two tutors provided
their students with training, including explaining the potential benefits, elaborating the proce-
dures and leading the whole class to analyse and evaluate several writing samples with reference
to the rubrics.
Both writing courses made use of an e-learning platform. The tutors asked all their students
to submit their essays to the platform, and then the system would randomly allocate each essay
to three student reviewers. The whole peer assessment process was double-blinded. The three
anonymous reviewers gave comments and scores and uploaded their feedback to the system.
The average of the three results would be counted as the score of this writing assignment.
Students who were dissatisfied with the results or processes could choose to write appeal letters
to their tutors. Tutors would then re-evaluate the writing assignments and send their feedback
to the students, but it had been clarified that the reassessing procedure would not guarantee
higher scores.

Data collection and analysis

This study drew on three datasets generated in the two courses (see Table 2). The first was
appealing emails sent by 36 students to their tutors. Since the action of writing an appeal letter
was triggered by peer feedback and followed by teacher re-assessment, what students expressed
in their appeal letters were the in situ feelings, attitudes and emotions activated by features of
specific situations. In order to obtain a comprehensive understanding of students’ affect, this
study also involved students’ posts (in English) on the online discussion forum in Course A and
responses (in Chinese) to a small-scale survey in Course B as the second and third parts of
the data.
6 J. ZHOU ET AL.

Table 3. Category totals for themes identified in students’ appeal letters.

Themes Subthemes Course A Course B Totals
Scores Scores lower than expectation 5 11 16
Scores higher than expectation 0 1 1
Content Disagreement with feedback content 7 18 25
Lack of concrete and/or individualized feedback 5 19 24
Lack of feedback/scores in specific dimensions 6 1 7
Process Engagement of peer evaluator 2 5 7
Logistic problem 2 2 4
Total items of complains 27 57 84

In the online forum of Course A, students were encouraged to respond to two questions
about their feelings of and emotional reactions to peer assessment. In the light of the relatively
lower English proficiency of students in Course B, we designed a survey with closed and open-
ended questions to scaffold students to share their affect. The survey in Course B elicited stu-
dents’ attitudes towards the activity of evaluating peers’ works and being evaluated by peers
(with 5-level Likert items from ‘strongly like’ to ‘strongly dislike’), and the degrees of satisfaction
with their own and peers’ evaluation performance (with 5-level Likert items from ‘very satisfied’
to ‘very dissatisfied’). Open-ended questions followed and requested students to share the rea-
sons. Each letter, post or response was de-identified to comply with ethics requirements.
Data analysis was conducted in two steps. Firstly, we conducted a thematic analysis (Braun
and Clarke 2006) of students’ appeal letters to explore which aspects of peer assessment trig-
gered students’ dissatisfaction. Appeal letters were read several times and then coded by the
first author using the software Nvivo 11. The initial round of analysis generated 11 codes related
to students’ dissatisfaction (satisfaction in several rare occasions) with regard to peer assessment.
In the second round, the authors worked in collaboration and drew a thematic map to illustrate
the relationship among codes and themes. Initial codes formed three main themes (i.e. scores,
feedback content and assessment process).
Secondly, students’ online discussion forum posts and responses to a survey were analysed
following similar inductive analysis processes. Students described their perceptions of peer
assessment and shared their emotions activated in the assessment process. Analysis of these two
datasets added additional data to understand the reasons for students’ grudges and gratitude.
Thirdly, a coding framework was developed through an iterative process of considering
themes, codes and their relationships. The keyword ‘respect’ was repeatedly mentioned by the
students across themes, datasets and courses. This salient theme will be given specific attention
in the discussion section.
Students’ appeal letters and their responses to the open-ended questions in the Course B sur-
vey were written in Chinese. The most pertinent excerpts were translated into English by the first
author. One-third of the excerpts were translated back into Chinese to detect any possible mis-
alignment with students’ original responses. Students’ online forum posts (Course A) were writ-
ten in English, so no translation was involved in this dataset.

Findings
Findings are structured around the two research questions: students’ dissatisfaction revealed in
the appeal letters and their general perceptions and emotions related to peer assessment.

Foregrounded (dis)satisfaction in appeal letters

Thematic analysis of students’ appeal letters indicated that the source of their dissatisfaction
mainly came from three categories: scores, content and process of peer assessment. Table 3
 ASSESSMENT & EVALUATION IN HIGHER EDUCATION 7

shows the totals of each individual occurrence of the theme across appeal letters from Course A
(16 email) and Course B (20 email).

Scores
Five students from Course A and 11 students from Course B complained about scores different
from their expectations, or lower than the scores given by the other two reviewers or tutor in
prior writing tasks. One student said she felt that the reviewer gave her a higher score than her
expectation because she ‘didn’t think the wording reached the level of 10 points’ (Letter 6,
Course B). The majority of appeals against scores were related to the results, in other words, the
appraisal respect.

Content
Most appeals focused on the content of peer feedback, which accounted for nearly three-quar-
ters of the total occurrence of themes in the entire appealing letter data. First, the most fre-
quently mentioned theme was the disagreement with the content of peer feedback, as
illustrated in the following excerpts:
One reviewer pointed out that my essay lacked appropriate transition, but I did have a transitional sentence
at the beginning of every paragraph. These sentences summarized some ideas from the previous para and
introduced what I was going to write. (Letter 1, Course A)

The two reviewers said that the evidence I selected could not fully support my argument. I don’t agree.
(Letter 4, Course B)

The reviewer highlighted many sentences and claimed there were grammatical errors. I disagree with some
highlighted content. For example, I wrote ‘accept education’ in the concluding part. How can this
expression be wrong? (Letter 15, Course B)

In addition, students expressed dissatisfaction toward feedback without concrete examples or

clarification.
The review wrote comments like ‘lack of logic coherence’ and ‘no sign of effective use of cohesive devices’.
The feedback was no more than direct copying from expressions in the rubric. (Letter 5, Course A)

One reviewer just copied some expressions from the rubric without giving concrete examples to justify his
comments. I was confused and dissatisfied. He did not respect my work. (Letter 2, Course B)

Students also complained about receiving only general feedback or scores without getting
specific information from each dimension in the rubric.
Without getting scores from each dimension, I’ve got no idea whether the general score was reasonable
(Letter 15, Course A).

In summary, students’ dissatisfaction with feedback content derived from the lack of certain
necessary attributes, such as accuracy, specificity and concreteness. Some grudges were trig-
gered because the writings were not appreciated by the reviewers (lack of appraisal respect).
Students also began to interpret unspecified and vague feedback as indicating a lack of recogni-
tion respect.

Process
Logistic problems complained about by students included the distorted layout of essays due to
different versions of Microsoft Word software and the inconvenience of submitting their essays
to an unfamiliar platform. Such complaints were more related to specific implementation issues
related to peer assessment as an innovative activity in the course.
8 J. ZHOU ET AL.

Seven students expressed their dissatisfaction or even ‘anger’ due to the disengagement of
their peer reviewers. They sensed reviewers’ relatively low degree of engagement from the self-
contradiction of feedback content (reported by students from Course A) and language selection
and usage (reported by students from Course B):
There were some self-contradictory contents in the feedback, such as comments like ‘well organized but
somewhat messy’. I feel that the reviewer did not read my work carefully. (Letter 7, Course A)

There were seven typos in the feedback. I’m very angry about this. Obviously, the reviewer did not treat
this assessment seriously. (Letter 2, Course B)

The reviewer used the language of Chinese to convey his/her feedback. I was very dissatisfied with this way.
(Letter 13, Course B)

The reviewer used many questions in his comment on the dimension of ‘vocabulary and grammatical
control’. I feel he was uncertain about what he had said. I’m dubious about his judgement, too. (Letter 13,
Course B)

In contrast to these grudges against peer reviewers’ ‘disengagement’, two students continued
their appeal letters with an expression of gratitude to other reviewers for ‘showing respect to
my writing and thoughts’ (Letter 13, Course A).
In brief, students’ complaints against the feedback process were related to two aspects: feed-
back design and reviewers’ attitudes. Compared with feedback content reported in the previous
section, students’ complaints about reviewers’ attitudes were directly linked to the lack of recog-
nition respect, which triggered more intensive and extreme emotions such as anger.

General perceptions and emotions related to peer assessment

It needs to be pointed out that the findings revealed in the appeal letters mainly focussed on stu-
dents’ dissatisfaction. In order to obtain an overall understanding of students’ perceptions and
emotions related to peer assessment, including their positive reactions, this study also analysed
students’ posts in the online forum (Course A) and responses to a small-scale survey (Course B).

Attitudes
Among the 30 posts sharing their general feelings about peer assessment in Course A, 15
students articulated they held ‘mixed feelings’, 13 students elaborated their positive feelings,
and two students only mentioned their negative feelings. Such sharing was generally consist-
ent with Course B students’ responses to the survey (see Figure 1).
From these findings, we can safely say that students generally liked peer assessment. For
those who showed positive attitudes toward peer assessment, they welcomed the valuable peer
comments, the opportunities of reading and evaluating peers’ works, and they even began to
appreciate the value of applying the rubric:
Applying the scoring rubrics when reviewing is a way to learn what makes a piece of writing good and
what makes one dissatisfactory. When judging others’ works, I will also reflect upon my own writing. I will
try to recall whether I have made the same mistakes in my own writing. (Post 12, Course A).

I have a deeper understanding of the argumentative essay structure. When I was evaluating my peers’
works, I used different colours to highlight the different functions of the essay. This process greatly
improved my own analysing ability. It carried great benefits for my own writing. (No. 47’s response,
Course B)

As illustrated in Figure 1, the attitudes of ‘dislike’ and ‘strongly dislike’ were in the minority
and mainly related to being evaluated by peers. Consistent with the patterns identified in
 ASSESSMENT & EVALUATION IN HIGHER EDUCATION 9

Figure 1. Students’ attitudes towards evaluating peers’ works and being evaluated by peers (Course B).

students’ appeal letters, the analysis of forum posts (Course A) and open-ended survey responses
(Course B) indicated that the ‘dislike’ attitudes were linked to the quality of peer feedback, the
attitude of peer reviewers and the scoring element:
Many students have different understandings of the rubric, which leads to chaos and anger. (Post 2,
Course A)

I’m sort of a ‘victim’ of it because for the first time I got some very harsh criticism without justifiable
reasons. (Post 3, Course A)

I think some students didn’t pay enough respect to other people’s work and gave comments and grades
very subjectively. (Post 23, Course A)

Some reviewers were sloppy and harsh. You cannot guarantee other’s working attitude. (No. 7’s response,
Course B)

Emotions and reasons

In the 22 posts responding to the tutor’s question about emotions in the process of peer assess-
ment (Course A), students mentioned a variety of emotions, from gratitude and being flattered
to unsureness, nervousness, disappointment and anger.
The emotion of gratitude was not necessarily activated by high scores. As illustrated in the
excerpts below, students were more likely to feel grateful when their work had been treated ser-
iously by peer reviewers:
I found those reviewers gave very detailed feedback; even some corrected my grammar mistakes.
Thank them for spending time correcting in detail. Thus I accepted my marks without resistance. (Post
9, Course A)

I want to thank those blind reviewers. They commented on what I have done well at first, which made me
feel very happy and willing to accept criticism and suggestion.

So I think although it is very important to point out others’ insufficiency, it is also vital to look for the
advantages of others’ writing and show your appreciation. (Post 10, Course A)
10 J. ZHOU ET AL.

Figure 2. Degrees of satisfaction with their own or peers’ evaluation performance.

In a similar vein, the emotional response of anger was more likely to be triggered by
reviewers’ attitudes and comments than the scores:
One reviewer said something like ‘No efforts can be seen in this piece of work’. I found this extremely
offensive when I first saw it. This statement was rather too assertive and in fact, very mean. My work
may not be excellent, but I did put much effort when doing the assignment. As a matter of fact, I was
so angry that I just closed the file immediately. (Post 13, Course A)

When facing an unfair score without any explanation, I felt angry that how could these people treat others’
efforts like this? I could not see any respect from their feedback. (Post 21, Course A)

Figure 2 illustrated that in Course B, while only one student was dissatisfied with his/her own
performance as an evaluator, 11 out of the total of 69 students were dissatisfied with their peers’
evaluating performance. In the followed open-ended question, most of the 11 students men-
tioned they felt disappointed with peer reviewers’ engagement:
Peer feedback was vague and did not link back to the specific content in my essay. I’ve no idea what the
problem was, and I could not accept his judgement. (No. 59’s response, Course B)

Suggestions
Students also gave their suggestions on how to improve peer assessment. For many students,
the process of reviewing others’ writings and being reviewed was quite fruitful and interesting,
so long as no grading element was involved. Most suggestions were therefore around the issue
of peer grading, proposing to give up the scoring part of peer assessment or to increase the
number of reviewers. Since many students had the negative experience with getting feedback
from a disengaged reviewer, they also suggested tutors design some mechanisms to regulate
peer reviewers, such as requiring student reviewers to support their comments with examples
and making peer review voluntary:
The crucial point is that the reviewers are responsible to present their reasons and foundation of the scores
to the reviewers. If we are provided with convincing explanations, it would actually help us in later writing
rather than cause only our dissatisfaction towards the vicious score-givers behind the screen. (Post 2,
Course A)
 ASSESSMENT & EVALUATION IN HIGHER EDUCATION 11

Another suggestion is about the dialogic channels and opportunities in the peer assess-
ment design:
Although I did have reliable feedback from the teacher, I still feel uncomfortable that I didn’t get the
recognition from the peer reviewers because I didn’t have any chances to argue with them. (Post 3,
Course A)

In summary, based on their experience in giving and receiving peer grades and feedback, stu-
dents have touched upon some heated topics in the field of peer assessment, such as whether
to use it for summative purposes, how to guarantee dialogic opportunities in assessment design,
and how to engage students.

Discussion
This study set out to understand students’ grudges and gratitude related to peer assessment by
analysing their appeal letters, forum posts and survey responses. In contrast to previous studies
(Nesbit and Burton 2006; Ryan and Henderson 2018) which reported students’ dissatisfaction
with grades lower than their expectations, complaints about scores in this study only accounted
for one fifth out of the total occurrence of themes. Despite the differences in their English lan-
guage proficiency, the two cohorts of students held similar grudges when they received vague
feedback without concrete examples and specific improvement advice. They also reported
extreme emotions like anger when they sensed the reviewers’ careless attitude in the peer
assessment process. When reflecting on their peer assessment experiences in platform posts and
survey responses, students generally expressed mixed feelings, praising the process of providing
and receiving feedback and showing resistance to using peer assessment for summa-
tive purposes.
Some mediating factors of students’ experiences with peer feedback resonate with those
reported in previous literature. For example, similar to university students in Australia (Dawson
et al. 2019), Chinese students in this study demonstrated a preference for feedback which was
detailed, personalised and considerate of affect. The tone of feedback carried emotional back-
wash on students’ learning (Pitt and Norton 2017). Vague and judgmental feedback on learning
effort led to students’ refusal to take action (Yu et al. 2018). Students disliked using peer assess-
ment for summative purposes, consistent with findings of peer assessment studies in other cul-
tural contexts (e.g. Patton 2012; Wilson, Diao, and Huang 2015; Sridharan, Tai, and Boud 2019).
In addition to these factors in the cognitive and structural dimensions of feedback, the pre-
sent study elaborates on a particular social-affective element: respect in peer assessment.
Students’ feelings about respect significantly interplay with other factors and mediate their
responses to peer feedback. At the surface layer, students expressed (dis)satisfaction with the
score, content and process of peer feedback. Underpinning these areas are students’ expecta-
tions for the attributes of feedback and reviewers’ attitude; any feedback that failed to meet
their expectations would trigger students’ grudges. The feeling of being disrespected is at the
root of students’ dissatisfaction with peer assessment. Lack of dialogic opportunities with
reviewers further aggravated their dissatisfaction. In contrast, the emotional response of grati-
tude was reported when students felt their works had been treated seriously and respected.
It is illuminating to draw upon Darwall’s (1977) notions of appraisal and recognition respect
to understand students’ emotional responses to feedback. Students felt pleased when they got
appraisal respect from their peers (high scores or positive feedback), but it was the feeling of a
lack of recognition respect (peer reviewers did not read or comment on their works seriously)
that led to most of the students’ dissatisfaction. While previous studies cast the spotlight on
how feedback attributes influence students’ perceptions and emotions (e.g. Lipnevich, Berg, and
Smith 2016; Pitt and Norton 2017), findings in this study support that students considered the
attitudes of feedback providers as equally (if not more) important. Furthermore, students even
12 J. ZHOU ET AL.

tended to interpret the lack of helpful attributes within the feedback as illustrating the reviewers’
sloppy working attitudes, and thus showing a lack of respect to their works and themselves. In
Darwall’s (1977, 39) words, students did not feel that they had been given ‘proper weight to the
fact that they are persons’.
Recognition respect is an ideal attitude for both ends of feedback. On the part of the peer
feedback provider, recognition respect can be embodied by reading peers’ work carefully and
providing elaborative feedback linking to concrete examples. The receiver of peer feedback
might also show recognition respect by considering feedback carefully before rejecting it in a
haste. Similar to the way that some students responded to peer feedback in Zhao et al.’s study
(2017), one student in our study wrote to complain because she disagreed with her peer
reviewers on whether some expressions complied with the grammar rules. If she had respected
the feedback received, she could have realized and admitted the problem through considering
the information in conjunction with external sources (e.g. other peers, teacher or a dictionary).
Students were even more likely to disagree with their reviewers on some subjective and tacit cri-
teria, such as whether evidence is able to support the argument. It seems that both the feedback
provider and receiver need a shared commitment to peer assessment. This finding resonates
with the argument of promoting intersubjectivity in peer feedback (Yu and Hu 2017) and further
specifies a means to this end.

Implications and conclusion

This study on the nature of students’ (dis)satisfaction with peer assessment carries implications
for designing and enacting peer feedback activities. While students’ dispositions greatly shape
whether they demonstrate respect in the feedback process (Forsythe and Johnson 2017), we
argue that taking the ‘respect’ issue into account in designing feedback activities and training
teachers and students enhances students’ experience and uptake of peer feedback.
Students in this study were able to identify and articulate necessary components of high-qual-
ity feedback, and any disparity between peer feedback and their expectations resulted in dissat-
isfaction. Students were also dissatisfied or angry with some features of feedback design, such as
having no chance to negotiate and dialogue with peer reviewers. The finding that students are
not totally ignorant of good feedback design or attributes is a message worthy of being under-
lined in teacher training. The issue of respect deserves more attention during feedback design.
Opportunities for dialogue with peer reviewers may be a mechanism that affords students the
ability to show recognition respect to their peers’ work and feedback. Face-to-face dialogue can
contribute to content clarification and emotion moderation (as reported in Zhu and Carless
2018), and a virtual learning environment offers potential for facilitating dialogue and alleviating
the interpersonal pressure (Ajjawi and Boud 2018).
Explicit discussion about the issue of respect in assessment is fruitful and necessary training
for students. Students are learning and exercising two types of respect in peer assessment. They
evaluate others’ works and decide on whether to show appraisal respect (e.g. in the form of giv-
ing high scores and positive comments); they are also learning to treat others’ work and feed-
back seriously, that is, to show recognition respect to their peers, peers’ works and feedback.
Teachers and students may work together to develop a shared understanding of what respect
may look like in feedback processes. This can be an essential component of nurturing students’
evaluative judgement (Tai et al. 2018) and student feedback literacy (Carless and Boud 2018). It
is also an empowering process for students to firstly be aware of the importance of respect in
peer assessment and then to engage their agency in developing it. Being aware of respect and
embodying respect in practices will enhance students’ university learning and further contribute
to their achievements in their future working environments.
 ASSESSMENT & EVALUATION IN HIGHER EDUCATION 13

Disclosure statement
No potential conflict of interest was reported by the authors.

Funding
This work was supported by the National Social Science Fund of China under Grant 17CYY022.

ORCID
Jiming Zhou http://orcid.org/0000-0003-3797-0441
Yongyan Zheng http://orcid.org/0000-0002-0664-3942
Joanna Hong-Meng Tai http://orcid.org/0000-0002-8984-2671

References
Adachi, C., J. H. M. Tai, and P. Dawson. 2018. “Academics’ Perceptions of the Benefits and Challenges of Self and
Peer Assessment in Higher Education.” Assessment & Evaluation in Higher Education 43(2): 294–306. doi:10.1080/
02602938.2017.1339775.
Ajjawi, R., and D. Boud. 2017. “Researching Feedback Dialogue: An Interactional Analysis Approach.” Assessment &
Evaluation in Higher Education 42(2): 252–265. doi:10.1080/02602938.2015.1102863.
Ajjawi, R., and D. Boud. 2018. “Examining the Nature and Effects of Feedback Dialogue.” Assessment & Evaluation in
Higher Education 43(7): 1106–1119. doi:10.1080/02602938.2018.1434128.
Assessment Reform Group. 2002. Assessment for Learning: 10 Principles. Cambridge, England: Assessment Reform
Group.
Bell, A. R., and C. Brooks. 2018. “What Makes Students Satisfied? A Discussion and Analysis of the UK’s National
Student Survey.” Journal of Further and Higher Education 42(8): 1118–1142. doi:10.1080/0309877X.2017.1349886.
Boud, D., R. Cohen, and J. Sampson. 2014. Peer Learning in Higher Education: Learning from and with Each Other.
London: Routledge.
Boud, D., and R. Soler. 2016. “Sustainable Assessment Revisited.” Assessment & Evaluation in Higher Education 41(3):
400–413. doi:10.1080/02602938.2015.1018133.
Braun, V., and V. Clarke. 2006. “Using Thematic Analysis in Psychology.” Qualitative Research in Psychology 3(2):
77–101. doi:10.1191/1478088706qp063oa.
Brookhart, S. M. 2011. “Educational Assessment Knowledge and Skills for Teachers.” Educational Measurement: Issues
and Practice 30(1): 3–12. doi:10.1111/j.1745-3992.2010.00195.x.
Carless, D. 2013. “Trust and Its Role in Facilitating Dialogic Feedback.” In Feedback in Higher and Professional
Education., edited by D. Boud and E. Molloy, 100–113. New York, NY: Routledge.
Carless, D., and D. Boud. 2018. “The Development of Student Feedback Literacy: Enabling Uptake of Feedback.”
Assessment & Evaluation in Higher Education 43(8): 1315–1325. doi:10.1080/02602938.2018.1463354.
Carroll, D. 2014. Graduate Course Experience. A Report on the Course Experience Perceptions of Recent Graduates.
Melbourne: Graduate Careers Australia.
Darwall, S. L. 1977. “Two Kinds of Respect.” Ethics 88(1): 36–49. doi:10.1086/292054.
Dawson, P., M. Henderson, P. Mahoney, M. Phillips, T. Ryan, D. Boud, and E. Molloy. 2019. “What Makes for Effective
Feedback: Staff and Student Perspectives.” Assessment & Evaluation in Higher Education 44(1): 25–36. doi:10.
1080/02602938.2018.1467877.
Forsythe, A., and S. Johnson. 2017. “Thanks, but No-Thanks for the Feedback.” Assessment & Evaluation in Higher
Education 42(6): 850–859. doi:10.1080/02602938.2016.1202190.
Guo, F., and J. Shi. 2016. “The Relationship between Classroom Assessment and Undergraduates’ Learning within
Chinese Higher Education System.” Studies in Higher Education 41(4): 642–663. doi:10.1080/03075079.2014.
942274.
Han, Y., and F. Hyland. 2019. “Academic Emotions in Written Corrective Feedback Situations.” Journal of English for
Academic Purposes 38:1–13. doi:10.1016/j.jeap.2018.12.003.
Higher Education Funding Council for England. 2016. National Student Survey Results 2016. Accessed September 4,
2017. http://www.hefce.ac.uk/lt/nss/results/2016/
Hounsell, D. 2007. “Towards More Sustainable Feedback to Students.” In Rethinking Assessment in Higher Education:
Learning for the Longer Term, edited by D. Boud and N. Falchikov, 101–113. London: Routledge.
14 J. ZHOU ET AL.

Huisman, B., N. Saab, J. van Driel, and P. van den Broek. 2018. “Peer Feedback on Academic Writing:
Undergraduate Students’ Peer Feedback Role, Peer Feedback Perceptions and Essay Performance.” Assessment &
Evaluation in Higher Education 43(6): 955–968. doi:10.1080/02602938.2018.1424318.
Lipnevich, A. A., D. A. Berg, and J. K. Smith. 2016. “Toward a Model of Student Response to Feedback.” In
Handbook of Human and Social Conditions in Assessment, edited by G. T. Brown and L. R. Harris, 169–185. New
York, NY: Routledge.
Nesbit, P. L., and S. Burton. 2006. “Student Justice Perceptions following Assignment Feedback.” Assessment &
Evaluation in Higher Education 31(6): 655–670. doi:10.1080/02602930600760868.
Nicol, D., and D. Macfarlane-Dick. 2006. “Formative Assessment and Self-Regulated Learning: A Model and Seven
Principles of Good Feedback Practice.” Studies in Higher Education 31(2): 199–218. doi:10.1080/
03075070600572090.
Patton, C. 2012. “Some Kind of Weird, Evil Experiment’: Student Perceptions of Peer Assessment.” Assessment &
Evaluation in Higher Education 37(6): 719–731. doi:10.1080/02602938.2011.563281.
Pekrun, R. 2016. “Academic Emotions.” In Handbook of Motivation at School, edited by K. R. Wentzel and D. B.
Miele, 120–144. New York: Routledge.
Pitt, E., and L. Norton. 2017. “Now That’s the Feedback I Want!’ Students’ Reactions to Feedback on Graded Work
and What They Do with It.” Assessment & Evaluation in Higher Education 42(4): 499–516. doi:10.1080/02602938.
2016.1142500.
Quality Indicators for Learning and Teaching. 2017. 2016 Student Experience Survey National Report. Accessed
September 6, 2017. https://www.qilt.edu.au/docs/default-source/gos-reports/2017/2016-ses-national-report-final.
pdf?sfvrsn=14e0e33c_5
Rowe, A. 2011. “The Personal Dimension in Teaching: Why Students Value Feedback.” International Journal of
Educational Management 25(4): 343–360. doi:10.1108/09513541111136630.
Ryan, T., and M. Henderson. 2018. “Feeling Feedback: Students’ Emotional Responses to Educator Feedback.”
Assessment & Evaluation in Higher Education 43(6): 880–892. doi:10.1080/02602938.2017.1416456.
Sridharan, B., J. Tai, and D. Boud. 2019. “Does the Use of Summative Peer Assessment in Collaborative Group Work
Inhibit Good Judgement?” Higher Education 77(5): 853–870. doi:10.1007/s10734-018-0305-7.
Tai, J., R. Ajjawi, D. Boud, P. Dawson, and E. Panadero. 2018. “Developing Evaluative Judgement: Enabling Students
to Make Decisions about the Quality of Work.” Higher Education 76(3): 467–481. doi:10.1007/s10734-017-0220-3.
van Quaquebeke, N., D. Henrich, and T. Eckloff. 2007. “It’s Not Tolerance I’m Asking for, It’s respect!’ a Conceptual
Framework to Differentiate between Tolerance, Acceptance and (Two Types of) Respect.” Gruppe Interaktion
Organisation Zeitschrift F€ ur Angewandte Organisationspsychologie (GIO) 38(2): 185–200. doi:10.1007/s11612-007-
0015-6.
Wang, W. 2014. “Students’ Perceptions of Rubric-Referenced Peer Feedback on EFL Writing: A Longitudinal Inquiry.”
Assessing Writing 19(1): 80–96. doi:10.1016/j.asw.2013.11.008.
Wilson, M. J., M. M. Diao, and L. Huang. 2015. “I’m Not Here to Learn How to Mark Someone Else’s Stuff’: An
Investigation of an Online Peer-to-Peer Review Workshop Tool.” Assessment & Evaluation in Higher Education
40(1): 15–32. doi:10.1080/02602938.2014.881980.
Xu, Y., and D. Carless. 2017. “Only True Friends Could Be Cruelly Honest’: Cognitive Scaffolding and Social-Affective
Support in Teacher Feedback Literacy.” Assessment & Evaluation in Higher Education 42(7): 1082–1094. doi:10.
1080/02602938.2016.1226759.
Yang, M., and D. Carless. 2013. “The Feedback Triangle and the Enhancement of Dialogic Feedback Processes.”
Teaching in Higher Education 18(3): 285–297. doi:10.1080/13562517.2012.719154.
Yu, S., Y. Zhang, Y. Zheng, K. Yuan, and L. Zhang. 2018. “Understanding Student Engagement with Peer Feedback
on Master’s Theses: A Macau Study.” Assessment & Evaluation in Higher Education 44(1): 50–65. doi:10.1080/
02602938.2018.1467879.
Yu, S., and G. Hu. 2017. “Can Higher-Proficiency L2 Learners Benefit from Working with Lower-Proficiency Partners
in Peer Feedback?” Teaching in Higher Education 22(2): 178–192. doi:10.1080/13562517.2016.1221806.
Zhao, K., J. Zhang, and X. Du. 2017. “Chinese Business Students’ Changes in Beliefs and Strategy Use in a
Constructively Aligned PBL Course.” Teaching in Higher Education 22(7): 785–804. doi:10.1080/13562517.2017.
1301908.
Zhou, J., and C. Deneen. 2016. “Chinese Award-Winning Tutors’ Perceptions and Practices of Classroom-Based
Assessment.” Assessment & Evaluation in Higher Education 41(8): 1144–1158. doi:10.1080/02602938.2015.1066306.
Zhu, Q., and D. Carless. 2018. “Dialogue within Peer Feedback Processes: Clarification and Negotiation of Meaning.”
Higher Education Research & Development 37(4): 883–897. doi:10.1080/07294360.2018.1446417.