This article was downloaded by: [University of Guelph]

On: 15 September 2012, At: 01:14

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Studies in Higher Education

Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/cshe20

How to determine the quality of

students’ reflections?
a b a c
Äli Leijen , Kai Valtna , Djuddah A.J. Leijen & Margus
b
Pedaste
a
Viljandi Culture Academy, University of Tartu, Tartu, Estonia
b
Pedagogicum, University of Tartu, Tartu, Estonia
c
Language Center, Faculty of Philosophy, University of Tartu,
Tartu, Estonia

Version of record first published: 25 May 2011.

To cite this article: Äli Leijen, Kai Valtna, Djuddah A.J. Leijen & Margus Pedaste (2012): How to
determine the quality of students’ reflections?, Studies in Higher Education, 37:2, 203-217

To link to this article: http://dx.doi.org/10.1080/03075079.2010.504814

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-

conditions

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.
 Studies in Higher Education
Vol. 37, No. 2, March 2012, 203–217

How to determine the quality of students’ reflections?

Äli Leijena,b*, Kai Valtnaa, Djuddah A.J. Leijenc and Margus Pedasteb
aViljandi Culture Academy, University of Tartu, Tartu, Estonia; bPedagogicum, University of
Tartu, Tartu, Estonia; cLanguage Center, Faculty of Philosophy, University of Tartu, Tartu,
Estonia
AliLeijen
Studies
10.1080/03075079.2010.504814
CSHE_A_504814.sgm
0307-5079
Society
2011
0Article
00
ali.leijen@ut.ee
000002011
Taylor in
forHigher
and (print)/1470-174X
Research
Education
Francis into Higher
(online)
Education

This article discusses some of the ambiguities related to the concept of reflection
in education, and presents an alternative approach for determining the focus and
Downloaded by [University of Guelph] at 01:14 15 September 2012

quality of students’ reflection. Accordingly, the focus of reflection can vary from
a concrete technical aspect of an experience to the broader societal context of that
experience, and the quality of reflection can be described through successive
stages of argumentation: describing, justifying, evaluating and discussion. The
developed coding schema for determining the focus and quality of reflection was
pilot tested on reflection fragments written by a small sample of tertiary dance
students.
Keywords: reflection; quality of reflection; focus of reflection

Introduction
Developing students’ reflection on their learning and behaviour is currently one of the
major learning goals in higher education. Reflection can generally be defined as a
cognitive process carried out in order to learn from experiences (Moon 2004) through
individual inquiry and collaboration with others (Dewey 1933). It has been pointed
out that reflection leads to deeper learning (Moon 2004), to achievement of more
complex and integrated knowledge structures, and to more accessible and usable
knowledge (Billing 2007). Enhancing this higher order cognitive skill is desired
throughout different disciplines; for example, in teacher training, medicine, social
work and arts education.
The high relevance and inclusion of reflection in several pedagogical practices of
higher education would suggest that the term reflection is well defined and organised
in literature. However, several authors (e.g. Griffiths 2000; Gur-Ze’ev, Masschelein,
and Blake 2001; Hatton and Smith 1995; Van Manen 1995) point out that the
concept of reflection is intangible and ambiguous, with little guidance given in the
literature on how to determine, facilitate and assess reflection in practice. It seems
that the term ‘reflection’ contains a wide range of different or even conflicting
concepts and strategies.
This article aims to unfold some of the ambiguities related to the term and
present an alternative view on determining the quality of reflection using a small-
scale pilot study from tertiary dance education as an example. It is argued that
supporting students’ reflection is very important in dance education (Lavender
1996; Lavender, and Predock-Linnell 2001; Leijen, Admiraal et al. 2008).

*Corresponding author. Email: ali.leijen@ut.ee

ISSN 0307-5079 print/ISSN 1470-174X online

© 2012 Society for Research into Higher Education
http://dx.doi.org/10.1080/03075079.2010.504814
http://www.tandfonline.com
 204 Ä. Leijen et al.

Reflection stimulates students’ awareness of their body and movement experiences,

which is necessary for developing high-quality dance skills. Moreover, reflection is
essential for students to learn how the audience may perceive their performance or
choreographic work. Besides enabling students to discern and evaluate their bodily
activities, and providing opportunities for regulating their learning activities, reflec-
tion also provides information on the socio-cultural surroundings of an individual
(Green 1999, 2001; Marques 1998). For instance, students can learn about how
they relate to other people, the life-situations they find themselves in, which
professional role they wish to develop, but also about the political implications
which shape the social practices within which students function (Leijen et al. 2009;
Stinson 1995).
Although some of the ideas discussed are particularly relevant to dance education
and arts education, the overall discussion regarding the quality of reflection is appli-
cable to higher education in general.
Downloaded by [University of Guelph] at 01:14 15 September 2012

Reflection in education
Reflective dialogue as a method of teaching and learning was already practised during
the time of Socrates; however, today’s concept of reflection in education is often asso-
ciated with the writings of Dewey. Dewey (1933) distinguished between routine
action and reflective action and established the pragmatists’ notion of reflection.
Reflection serves, in his view, the purpose of becoming conscious and thoughtful
about one’s action, the opposite to acting according to a trial and error scheme while
dealing with confusing and problematic situations. The process of reflection, as
described by Dewey, consists of a linear model of successive phases (Rodgers 2002,
851): ‘an experience; spontaneous interpretation of the experience; naming the prob-
lems or the questions that arise out of the experience; generating possible explanations
for the problems or questions posed; ramifying the explanations into full-blown
hypotheses; experimenting or testing the selected hypothesis’. Similar cyclical reflec-
tion models for educational settings have been suggested by Kolb (1984) and
Korthagen (1985), who, in the footsteps of Dewey, promote an active role of students
in finding solutions to complex problems.
Nearly 40 years after Dewey, Habermas (1972) proposed three different kinds of
knowledge-constitutive interests: technical, practical and emancipatory. Each knowl-
edge interest initiates a certain ‘way of knowing’. Each is constructed, guided, and
shaped differently. Following Habermas, Van Manen (1977) found it necessary to
distinguish between technical, practical and critical reflection. Following the emanci-
patory knowledge constitutive interest, critical reflection emphasises the critical posi-
tion of individuals and groups in relation to the actual situation. Reflection involves
questioning existing assumptions, values and perspectives that underlie people’s
actions, decisions and judgements. The purpose of questioning is to liberate people
from their habitual ways of thinking and acting (Procee 2006). This tradition of reflec-
tion is also referred to as the critical social theory perspective on reflection. Inspired
by Habermas, Mezirow (1991) introduced the term perspective transformation, which
signifies the process of becoming critically aware of how and why our assumptions
about the world have come to constrain the way we see ourselves and relate to others.
According to him, transformation in perspectives is only possible through the critical
reconsideration of one’s own perspectives and orientations to perceiving, knowing,
feeling and acting.
 Studies in Higher Education 205

The followers of the pragmatic and social critical theory traditions interpret reflec-
tion differently. The followers of the latter criticise the pragmatists’ perspective for its
cyclical procedure and practical focus. Moreover, Gur-Ze’ev, Masschelein, and Blake
(2001) state that reflection in the pragmatists’ tradition represents the reproduction of
social and cultural order in society and does not provide opportunities for people to
liberate themselves from existing social and cultural patterns. However, scholars of
different or even contradicting traditions are frequently used in a closely intertwined
manner. One particularly flexible term is critical reflection. As described earlier, the
concept of critical reflection implies the acceptance of a particular ideology, assump-
tions and epistemology. However, as Hatton and Smith (1995, 35) point out, ‘some
take it to mean no more than constructive self-criticism of one’s actions with a view
of improvement’. The loose usage of terms creates tangled meanings and confusion
around the concept of reflection in education. Therefore, instead of mixing the two
perspectives, it is useful to consider them separately; both epistemologies in their own
Downloaded by [University of Guelph] at 01:14 15 September 2012

right.

The focus and quality of reflection

Since Van Manen (1977) introduced the three types of reflection, the social critical
theory approach to determining the levels of reflection has dominated in the literature.
The three types form a hierarchical structure. The lowest level of reflection, technical
reflection, is concerned with the efficiency of means for reaching certain goals. The
second, practical reflection, involves an open examination, not only of means but also
of goals, the assumptions goals are based on and the actual outcomes. Mansvelder-
Longayroux, Beijaard, and Verloop (2007) suggest that this stage of reflection is
concerned with constructing personal knowledge and developing awareness of one’s
own identity, beliefs and development. The highest level, critical reflection, is carried
out when personal actions are analysed within the wider socio-historical and politico-
cultural context. For example, Mansvelder-Longayroux, Beijaard, and Verloop (2007)
indicate that this reflection enables critical examination of the social and political
implications of education. Many authors suggest similar hierarchical categorisation of
reflection levels. Some authors prefer to refer to the highest level as dialectical reflec-
tion (Lasley 1992; Taggart 1996) or narrative reflection (Sparks-Langer and Colton
1991). Others (Hatton and Smith 1995; MacLellan 2004; Ward and McCotter 2004)
include dialectical reflection into the categorisation below the highest level, critical
reflection.
Van Manen’s (1977) categorisation of reflection has been criticised for its hier-
archy (Hatton and Smith 1995). Another weakness of the categorisation is its bias
toward social critical theory. This categorisation can be seen as a tool to promote
the social critical theory perspective on reflection. However, since the pragma-
tists’ views on reflection have been placed in the lower level of the categorisa-
tion, this categorisation can also be considered as one that denigrates practical
knowledge and skills. While considering the domain of the current study, tertiary
dance education, or arts education more generally, the categorisation of Van
Manen discredits the core components of the discipline. Harland (2007) pointed
out that the essential ingredients of good arts education have been developing
students’ creativity on the one hand, and techniques and skills related to a specific
arts domain on the other. More recently, a third area has become very important in
arts education, which addresses the meaning in expression and exploration. Leijen,
 206 Ä. Leijen et al.

Lam et al. (2008) have pointed out that students’ reflection in practical dance
classes is concerned both with the subjectivity of the student and with the domain-
specific concepts and principles. In addition, choreographic intentions and execu-
tions, as well as the presentation of dancers’ bodies, could be a topic for critical
reflection in dance theory classes. This means that the practical, or sometimes
even technical, focus of reflection is necessary for developing knowledge and
skills related to the field of art.
Rather than referring to three levels of reflection, Hatton and Smith (1995) suggest
that we consider these as different types of reflection. As they explain, different
contexts in education may lend themselves more towards one kind of reflection;
however, ‘it is important that the types are not viewed as an increasingly desirable
hierarchy’ (35). Alternatively, Tsangaridou and O’Sullivan (1994) and McCollum
(1997) talk about the foci of reflection, while distinguishing technical, situational and
sensitising reflection. Following this division: ‘1) Instructional or managerial aspects
Downloaded by [University of Guelph] at 01:14 15 September 2012

of teaching are considered technical; 2) Contextual issues of teaching characterise

situational reflection; and 3) Sensitising reflection is concerned with social, moral,
ethical, or political aspects of teaching’ (McCollum 1997, 52).
The focus of reflection as a term is useful, as it illuminates the scope of reflec-
tion and is sufficiently neutral to allow us to consider the perspectives of pragmatists
and critical social theorists on reflection on equal grounds. While deciding on the
categorisation of the focus for the current study, we followed two principles; first,
choosing terms which are the most commonly used in literature and, second, select-
ing terms which are as neutral as possible. The most commonly used division is
technical, practical and critical reflection. Given that critical reflection is mostly
associated with the social critical perspective, we chose the term ‘sensitising’
instead, as suggested by Tsangaridou and O’Sullivan (1994) and McCollum (1997),
for our categorisation.
This approach would also make it possible to distinguish different levels of argu-
mentation in technical, practical and sensitising reflections. According to Moon
(2004), a helpful dimension for determining the quality of reflection is superficial-
deep. Moon indicates that most authors view superficial reflection as descriptive.
Deep reflection, on the other hand, involves a ‘functional understanding of the
constructed nature of knowledge and a metacognitive stance’ (96) towards knowl-
edge. Deep reflection is characterised as the ability to revise the ‘meaning structures’
of knowledge. Tsangaridou and O’Sullivan (1994) and McCollum (1997) use the
following aspects to determine different levels of reflection: description (descriptive
information), justification (logic or rationale) and critique (explanation and evalua-
tion) of an aspect related to the area of practice. Inspired by Moon’s definition of deep
reflection, a fourth level could be added to this categorisation. At this level of argu-
mentation, one would go beyond self-evaluation and discuss alternative solutions for
changing one’s practice.
These ideas could be summarised with the following two metaphors: broadening
and deepening. Broadening entails that the focus of reflection can be moved from a
concrete technical aspect to its broader societal context. Deepening means that the
process of reflection entails deepening thinking through successive stages: describing,
justifying, evaluating and discussing.
Finally, it is important to point out the important role of others in the processes
of reflection. Out-of-the-box thinking about one’s experiences, and seeing them
from a more objective perspective, as though they were the actions of another, is a
 Studies in Higher Education 207

challenging task. Several authors agree that reflection needs to happen in a commu-
nity: in interaction with others (e.g. Boud 1999; Dewey 1933; Hammersley-Fletcher
and Orsmond 2005; Moon 1999; Procee 2006; Thorpe 2000). Feedback enables
individuals to share and learn from the perspectives of others on experiences and
ideas, and (re)interpret and develop their own perspectives further. Moreover, peer
feedback can help students move beyond the evaluation and explanation of an expe-
rience and consider alternatives, as indicated by Moon (1999):

Working with others can facilitate learners to reflect and can deepen and broaden the
quality of reflection so long as all the learners are engaged in the process. Another person
can provide the free attention that facilitates reflection, ask challenging questions, notice
and challenge blocks and emotional barriers in reflection. (172)

A pilot study among a small group of dance students was carried out to collect
illustrative data about the focus and quality of reflection, outlined in Figure 1, and the
Downloaded by [University of Guelph] at 01:14 15 September 2012

role of feedback in reflection. The research question of the study was the following:
What is the focus and level of dance students’ reflections in a ballet class and in a
choreography class? Feedback from others was considered as a mutual part of
students’ reflection in the current study.
Figure 1. The foci and levels of reflection.

Methods
Participants
In this study, we analysed written self-reflections and peer feedback accounts
collected in the context of a previous study (Leijen et al. 2009), where dance students
carried out reflection assignments in a ballet course and a choreography course. Self-
reflections and peer feedback accounts were collected from 16 students (3 men, 13
women) who study in a dancer/choreographer bachelor’s programme at a dance acad-
emy in the Netherlands.

Figure 1. The foci and levels of reflection.

 208 Ä. Leijen et al.

Reflection assignments
In addition to regular curricular activities, students were asked to carry out reflection
assignments in either a choreography or a ballet course. Students could choose in
which course they preferred to carry out the assignments. The task of reflection was
different for the two courses. In the choreography course, students reflected on their
choreographic work, which was performed by their peers. In the ballet course,
students reflected on their own practice of a ballet technique. In both classes,
students recorded their dance practices on video, edited these, and uploaded short
videos to a streaming video server. The procedure of the reflection assignment was
as follows:
1. Video viewing and editing were aimed at facilitating students in describing their
experiences.
2. For evaluating the experiences, students were asked to answer questions posted
by the teacher. In the choreography class students evaluated their experiences based
Downloaded by [University of Guelph] at 01:14 15 September 2012

on subjective criteria; for example:

Please evaluate your own spatial decision making in relation to the outcome of your
choreographic intentions. What worked, what did not work? Why? More importantly, as
you look at the video, what new questions and impulses arise for you?

In the ballet class, the teacher provided the criteria for evaluation, for example:

Please evaluate and elaborate on the quality of taking off and landing. What worked,
what did not work? Why? Please make sure that you also point out aspects which you
did very well. Write down your own evaluation using the following criteria, which you
are familiar with from previous classes: 1. Correct placement of heels and use of feet
articulation. 2. Correct body direction in coordination with body balance. 3. Correct
timing and direction of focus.

3. Given that it can be difficult for students to move beyond the evaluation and
explanation of an experience and consider alternatives (e.g. Procee and Visscher-
Voerman 2004), students could formulate relevant topics or issues for which they
wished to receive feedback from peers. The answers from peers were expected to
bring the level of reflection to a higher level, as suggested by Moon (1999). Each
student received feedback from at least two peers on the questions he/she had asked.
Students’ self-evaluations (the answers to teachers’ questions) and feedback to
peers (the answers to peers’ questions), stored in the web-based learning environment,
were collected for further analysis. Besides analysing the quality and focus of
students’ reflection as reported in the current article, students’ and teachers’ interpre-
tation of the value of reflection in relation to students’ learning was also investigated:
these findings are presented in Leijen et al. (2009).

Data analysis
Both qualitative and quantitative data analysis methods were applied in the current
study. The aim of such a triangulation mixed methods approach (Creswell 2005) in
data analysis is to strengthen the analysis quality, and provide a more complex under-
standing of the research problem.
Initially, the qualitative data analysis was carried out. The focus and level of
reflection in the written self-evaluations and peer feedback accounts were analysed by
 Studies in Higher Education 209

two researchers who are familiar with tertiary dance education. First, a coding schema
developed by Tsangaridou and O’Sullivan (1994) and McCollum (1997) was adopted
and further developed in collaboration with researchers (see Table 1). As described
earlier, the focus of reflection can be categorised as technical, practical, or sensitising.
Aspects related to dance technique details or composition elements were considered
technical. Discussions of the execution of choreographic intentions, audience interpre-
tations of the compositional work, as well as interpretations of one’s dance style and
artistic qualities, were characterised as practical reflection. Sensitising reflection was,
firstly, concerned with social, moral, ethical or political aspects addressed within the
dance works. Secondly, we interpreted reflection fragments as sensitising when
students questioned their assumptions regarding choreographic tools and evaluation
criteria of dance works.
Four levels of reflection were distinguished based on the theoretical framework
outlined in the introduction. First, mere descriptions of actions and thoughts were
Downloaded by [University of Guelph] at 01:14 15 September 2012

assigned to be on the level of description. Second, if students provided a rationale or

logic for an action or viewpoint, the fragments were assigned to be on the level of
justification. For example, in Table 1 in the section on technical reflection, the
student provides a logical explanation as to why the hip turnout was not finished.
Third, if students provided an evaluation for an aspect and explained why this expla-
nation was given, then the fragment was interpreted as being on the level of critique.
For example, in Table 1 in the section on technical reflection, the student, after
observation, evaluates that she needs to be much clearer. She explains this by point-
ing out what aspect of her dance is wrong (‘the placement of hips and shoulders’). In
comparison to the justification, the student in this example points to more than just a
rationale of a situation by explaining why she thinks she should be clearer. Finally, if
students moved beyond the evaluation and explanation of what is, and why they
think that is, and pointed out what could be done to initiate changes, and why
changes are needed in the first place, then the fragments were interpreted as being on
the level of discussion. The discussion category was added to the initial coding
scheme of reflection levels used by Tsangaridou and O’Sullivan (1994) and McCol-
lum (1997).
All fragments from self-evaluations and peer feedback were discussed and
assigned with codes after both researchers, who are familiar with dance education, had
agreed. Each fragment was assigned with one code only. Given that the levels of
reflection are cumulative, fragments with a higher reflection level could also contain
information of lower levels (for example, descriptive notes). The codes were assigned
according to the highest level presented in the fragment. Third, all categories were
scrutinised by a language consultant who examined whether fragments corresponded
to the level of reflection. Where the language consultant did not agree with an initial
interpretation of the level, the initially assigned level was reconsidered by a researcher
and the language consultant, and if agreed, changed. Consensus regarding the level of
reflection was reached for all 180 coded fragments. The analyses by three specialists
can be understood as investigator triangulation (Denzin 1970), carried out for quality
assurance of the analysis procedure.
Lastly, the quantitative data analysis was carried out. Descriptive statistics
related to the focus and quality of students’ reflections in both courses were
calculated. Chi-square tests were used to find out whether there were differences
regarding the division of reflection fragments on particular levels and foci in differ-
ent phases of the reflection assignment.
 210 Ä. Leijen et al.

Table 1. Coding scheme with examples.

Technical reflection (examples from the ballet course)
Discussion I think overall I can turn out more. It shows in the passé and in the pique
attitude. I think it will also help my balance if I turn out more. And work
more in pliée. Maybe I’m thinking too much ‘up’ instead of using the ground
well.
Critique Need to be clearer with this. It’s weird to see myself moving, because while
I’m dancing I think that my hips and shoulders are in the right placement,
but they’re not! Always a bit to an other way. Specially with the tombe pas
de burrée.
Justification In the pliée preparation for the pirouette I didn’t relax the foot on the floor,
therefore I did not finish the hip turnout.
Description In the tombé, pas de bourré I’m facing in the right direction with shoulders and
hips.
Downloaded by [University of Guelph] at 01:14 15 September 2012

Practical reflection (examples from the composition class)

Discussion I think in general how you dealt with this theme and which way you choose to
show it, wasn’t bad at all. As we just had so little time I think to deal with
this theme in a more theatrical way was quite ok and well understandable for
the audience. Would this piece be more developed and worked out with more
time investment then it would be a nice choice to then also find another body
language (for instance closer to a dance language). A suggestion to deal with
this complex theme is to not show everything so obvious but to find a way
to bring this topic to the audience in a more encoded way to make the
audience even more think about this and find their own way and their own
thoughts to connect to the ambiguous sexuality.
Critique For me it was clear that the piece was a development of one process. However,
I think that the first and the second part were not well connected. Because
the dancers had a connection with the audience in the first part, but not with
each other, and then the switch to the second part where the dancers have a
contact with each other was too fast as they did not have a connection before.
Justification This piece is not really related to my manifesto, because when I created it I
wasn’t thinking on my manifesto. In this piece my dancers are more kind of
acting. They got a role/character to play with and not a feeling or steps that
are related to feelings or emotions. So it is hard to relate to my manifesto.
Description It seems to be like a story between the two dancers. Ann is waiting for Regina
to ‘grow’ and protecting her in that process and then the ‘story’ begins …
like what kind of problems are coming up in any kind of relationship.
Sensitising reflection (examples from the composition class)
Discussion I think the manifesto can be a very useful tool to evaluate my compositions. At
least when it’s really honest and it really formulates what dance is about for
me. I have to admit at that point that unfortunately I’m not totally convinced
of my manifesto yet. I should rethink again what I wrote in these days. But
for now it was interesting to measure my work on it, because even though
it’s not 100% how I see dance it still reflects factors that are important for
me in dance. And I could see that I somehow did not really go deeply and
consciously into the things that make dance interesting for me. So next time
I create something I will be more aware of what I want to work with and
where I want to go deeper into with my researches.
Critique Somehow it shows how people are in life and in society. How they play with
lives not only with animals but also with human lives. And so for me the
connection between the joke and the piece is good. You are also reminded
of the joke during the whole piece, because of the carrots.
 Studies in Higher Education 211

Table 1. (Continued).
Sensitising reflection (examples from the composition class)
Justification Some it is funny, but in an absurd way. For me it is a kind of ‘war comic’, seem
to be funny but if you look between the lines it is not. So you will also be a
little ashamed when you realise where you are laughing about.
Description For me it is a kind of ‘war comic’, seem to be funny but if you look between
the lines it is not.*
Note: The emphases presented in italic were added by authors.
*This fragment was part of a justification. There were no separate descriptive comments among sensitising
reflection fragments in the current study.

Results
Ballet class
Downloaded by [University of Guelph] at 01:14 15 September 2012

The majority of ballet students’ reflections (79%) focused on technical aspects.

Students were mainly concerned with the execution of technique, focusing often on
specific details of the technique. For example, in the self-evaluation a student noted
the following: ‘Especially with the tombé pas de bure travelling in diagonal, I need to
focus that my hips and shoulders are really travelling to the diagonal and not halfway
to the front’.
The remaining 21% of the ballet students’ reflections focused on practical aspects.
They were more focused on their overall dancing style or more general comments
regarding their dancing, as in the following example: ‘I’m aware of this, but it doesn’t
always show. I also need to be clear with my eyes, what am I looking at? Open eyes,
not within myself busy with the exercise but showing myself and have fun with it’.
Evidence for sensitising reflection was not detected among ballet students’ reflections.
Regarding the level of reflection, students presented descriptions, justifications
and critique fairly equally (see Table 2).
In self-evaluations students most often presented descriptive information (see
Table 3). Moreover, the descriptions among self-evaluations were usually focused on
the specifics of their ballet technique. When students focused on more general aspects
of their dancing (practical reflection), they tended to justify and evaluate their view-
points more often rather than merely describing them.
In feedback, the most often used level of reflection was critique. However, while
considering the focus of reflection, it is evident that the fragments with technical focus
were equally often on the level of justification and critique, whereas the dominant
level of reflection among the fragments with a practical focus was critique.

Table 2. Overall quality of ballet students’ (n = 8) reflection.

Level of reflection Number of fragments
Discussion 8 (7%)
Critique 35 (32%)
Justification 33 (30%)
Description 35 (32%)
Total 111 (100%)
 212 Ä. Leijen et al.

Table 3. The focus and quality of ballet students’ (n = 8) reflection in self-evaluations and
feedback.
Self-evaluation Feedback
Level of Technical Practical Technical Practical
reflection reflection reflection Total reflection reflection Total
Discussion 3 (5%) 1 (8%) 4 (6%) 4 (13%) – 4 (10%)
Critique 13 (22%) 4 (33%) 17 (24%) 10 (34 %) 8 (73%) 18 (44%)
Justification 15 (26%) 5 (42%) 20 (29%) 10 (34 %) 3 (27%) 13 (32%)
Description 27 (47%) 2 (17%) 29 (41%) 6 (20%) – 6 (14%)
Total 58 12 70 30 11 41

We were interested to find out whether peer feedback brings students’ reflection
Downloaded by [University of Guelph] at 01:14 15 September 2012

to a higher level. A chi-square analysis was used to test whether the quality of reflec-
tion in feedback was on a higher level than the quality of reflection in self-evaluations.
Results showed (see Table 4) that, in the case of feedback, the standardised residuals
were positive when the level of reflection was higher, whereas, in the case of self-
evaluations, the only positive standardised residual appeared on the lowest level of
reflection. This shows that the quality of reflection in feedback was on a higher level
than the quality of reflection in self-evaluations. The difference was statistically
significant (χ2 = 9.77, df = 3, p = .021). The biggest difference was found for the third
level.
We also tested whether the focus of reflection differs between feedback and self-
evaluations. Contrary to the findings regarding the quality of reflection, the chi-square
analysis regarding the focus of reflection did not show statistically significant differ-
ences (p > .05) in the ballet class.

Choreography course
The majority of choreography students’ reflections (65%) focused on practical aspects
of their choreography; more specifically, students were mainly concerned with how
their choreographic intentions were communicated with the composition, as demon-
strated by the following example:

I think the main statement I wanted to communicate to the audience through the dancers
attained [successfully reached] most of the people. But I’m not totally convinced if I

Table 4. Chi-square analysis of the distribution of the quality of reflection in different phases
of assignment (self-evaluation, feedback) in ballet class.
Level of reflection
Total number
Reflection phase 1 2 3 4 of fragments
Self-evaluation Count 29 19 17 4 69
Std Residual 1.5 −.2 −1.1 −.5
Feedback Count 6 13 18 4 41
Std Residual −2.0 .3 1.4 .6
Total number of fragments 35 32 35 8 110
 Studies in Higher Education 213

really chose the best way, but anyway it was a first tryout of an idea which could be
developed much further. In the three different steps of the search of details I think espe-
cially the first one, when the performers were looking into the bags of the audience and
finding little papers in it, worked quite well.

Some 22% of the choreography students’ reflections focused on the technical

aspects of their compositions. Here students elaborated on the compositional compo-
nents of their work. For example, a student pointed out the following:

My decision in space was very simple, I just wanted to go from one side of the room to
the other. The characters are developing within the space, that’s why I wanted to make
this easy. I am also questioning myself if the development of the characters is clear
enough. I think if I had more time, I could work more on it.

The remaining 13% of the choreography students’ reflections were characterised

Downloaded by [University of Guelph] at 01:14 15 September 2012

as sensitising reflection. These were concerned with ethical, moral or political aspects
of the choreography process and choreographic work. The following is an example of
sensitising reflection. A student is reflecting on a moral idea communicated with a
dance work. The student was a dancer in the choreography:

I think it is very clear. You put on the stage two women, one is dressed like a lady and
the other one more like someone that doesn’t care how she looks like. You put those two
women together and first I am looking at Ann, like if she is not on my level, that I am
better, therefore I move away from her. But slowly I start to show things that are not
normal to do in public, and Ann is looking very strange at me. But actually somehow this
is the place where everyone is equal, all those things, what I am doing, are the things that
everyone is doing but when nobody sees it.

Regarding the level of choreography students’ reflections, students presented

more often justifications and critique than discussion and descriptive information
(see Table 5).
Looking at the quality of reflection in self-evaluation (see Table 6), it can be seen
that students’ reflection most often contained justifications and critique, regardless of
whether these focused on technical, practical or sensitising aspects of their work.
A chi-square analysis was used to test whether the quality of reflection in feedback
was on a higher level than the quality of reflection in self-evaluations (see Table 7).
No statistically significant difference was found comparing the reflection levels of
self-evaluation and feedback (χ2 = 5.77, df = 3, p = .123). A more detailed analysis
revealed that there were no remarkable differences between feedback and self-evalu-
ation regarding the first three levels of reflection; however, the difference was visible
in the case of the highest level of reflection where the standardised residuals were −
1.2 and 1.1 respectively. This suggests that the overall quality of the feedback was

Table 5. Overall quality of choreography students’ (n = 8) reflection.

Level of reflection Number of fragments
Discussion 13 (19%)
Critique 28 (41%)
Justification 23 (33%)
Description 5 (7%)
Total 69 (100%)
 214 Ä. Leijen et al.

Table 6. The focus and quality of choreography students’ (n = 8) reflection in self-evaluations

and feedback.
Self-evaluation Feedback
Level of
reflection Tech Prac Sens Total Tech Prac Sens Total

Discussion 1 (17%) 1 (5%) 1 (20%) 3 (9%) 3 (33%) 7 (29%) – 10 (27%)

Critique 2 (33%) 11 (52%) 2 (40%) 15 (47%) 5 (56%) 7 (29%) 1 (25%) 13 (35%)
Justification 3 (50%) 8 (38%) 2 (40%) 13 (41%) 1 (11%) 6 (25%) 3 (75%) 10 (27%)
Description – 1 (5%) – 1 (3%) – 4 (17%) – 4 (11%)
Total 6 21 5 32 9 24 4 37

somewhat higher than that presented in the self-evaluations, because relatively more
feedback fragments contained discussions.
Downloaded by [University of Guelph] at 01:14 15 September 2012

In addition, we tested whether the focus of reflection differs between feedback and
self-evaluations. The chi-square analysis did not show statistically significant differ-
ences (p > .05) regarding the focus of reflection in the choreography class.

Summarising the results in two courses

Ballet students’ reflections focused mainly on technical aspects of the ballet technique
execution. Choreography students’ reflections were mainly concerned with the prac-
tical aspects of their choreographic work. Students were mostly concerned with how
their choreographic intentions came out within the dance work. Some choreography
students’ reflections were also characterised as sensitising reflection, whereas this
focus of reflection was not detected among ballet students’ writings.
While considering the focus and quality of reflection among different stages of the
assignment, it can be concluded that the focus of reflection did not differ across differ-
ent stages of the assignment (self-evaluation and feedback) in both ballet and chore-
ography courses. In the ballet class the quality of reflection was higher in feedback
than in self-evaluations. This difference was not found in the choreography class.
However, the results showed that the highest level of reflection – discussion – was
more often presented in feedback than in self-evaluations. Therefore, there is some
evidence to suggest that peer feedback activities facilitated students to reach the
higher level of argumentation while reflecting on their dancing and choreographic
work.

Table 7. Chi-square analysis of the distribution of the quality of reflection in different phases
of assignment (self-evaluation, feedback) in choreography class.
Level of reflection
Total number
Reflection phase 1 2 3 4 of fragments
Self evaluation Count 1 13 15 3 32
Std Residual −.9 .7 .6 −1.2
Feedback Count 4 10 13 10 37
Std Residual .8 −.7 −.5 1.1
Total number of fragments 5 23 28 13 69
 Studies in Higher Education 215

Discussion
In this article we have questioned the relevance of a commonly used approach to the
quality of reflection developed by Van Manen (1977). According to this approach,
three levels of reflection can be determined following a hierarchical order: technical,
practical and critical reflection. The weakness of this approach is that the focus of
reflection (technical, practical and critical reflection) has been used to determine the
value of reflection. Given that reflections on the technical and practical aspects of a
domain are considered lower than conceptual questions, this approach to reflection
has less relevance for educational practices such as arts education, where mastering
the domain-based technical and practical skills is central. In accordance with Tsan-
garidou and O’Sullivan (1994) and McCollum (1997), it was recommended to distin-
guish between the focus and quality of reflection. Three foci were distinguished:
technical, practical or sensitising reflections. In addition, four levels of argumentation
were distinguished in students’ reflection determining the quality of reflection:
Downloaded by [University of Guelph] at 01:14 15 September 2012

description, justification, critique and discussion.

We found that ballet students reflected mostly on the technical issues of the dance
technique. Students presented detailed knowledge about different aspects of ballet
technique. Using the developed coding scheme allowed us to investigate the depth
and the line of reasoning in reflections, rather than considering the writings as low
level, merely because of their technical focus. As Allard and Starkes (1991) showed,
experts of skilled motor performance have greater cognitive skills and domain-
specific knowledge in their particular areas than do other performers. A large
component of learning the skill elements in these domains is learning the vocabulary.
Therefore, from the subject matter point of view, the analysis of technical details is
necessary for students of certain dance styles and techniques. We also found that
choreography students reflected mainly on whether and how their choreographic
intentions transferred to their choreographic work. The translation of choreographic
intentions into physical movements seems to be the most important question regard-
ing composing dances for students. Besides this, students could also reflect on the
composition details presenting a more technical focus, or on the frames of references
in arts and in wider contexts displaying sensitising reflection. In other words, as also
Hatton and Smith (1995) pointed out, different contexts in education may lend them-
selves more towards certain foci of reflection. For example, while reviewing the
examples of the questions teachers set for the reflection assignment, it was noticed
that these questions were aiming at the technical focus. It is the purpose of educators
to guide students into the relevant foci.
Regarding the quality of reflection, the role of others was highlighted as crucial.
Reflecting on one’s experiences and seeing them from a more objective perspective is
a challenging task. Peer feedback is believed to enable individuals to share and learn
from the perspectives of others, and (re)interpret and develop their own perspectives
further (e.g. Hammersley-Fletcher and Orsmond 2005; Moon 1999). The results from
the current study showed that students had the ability to present higher order reasoning
in their writing; however, it seemed to be more difficult for them to deepen the
reasoning in reflection concerning their own actions as opposed to looking at others.
The results provided some evidence that peer feedback can raise the overall quality of
students’ reflection. This finding is remarkable, since it presents an instructional prin-
ciple of how to improve the quality of reflection. A completely different question is
whether the highest quality reflection is always desired. We believe that, similarly to
 216 Ä. Leijen et al.

the focus of reflection, it is the task of educators to guide students towards the neces-
sary quality standards of reflection. Some topics might need mere descriptions or
justifications; however, each discipline also has its core concepts and principles,
which need to be dealt with on the highest level of reflection.
Finally, it must be noted that the current study was conducted with a small sample
of students from a specific educational context. Further research is required to test and
verify the proposed approach for determining the focus and quality of reflection in
different educational contexts.

References
Allard, F., and J.L. Starkes. 1991. Motor-skill experts in sports, dance and other domains.
In Toward a general theory of expertise: Prospects and limits, ed. K.A. Ericsson and J.
Smith, 126–52. Cambridge: Cambridge University Press.
Billing, D. 2007. Teaching for transfer of core/key skills in higher education: Cognitive skills.
Downloaded by [University of Guelph] at 01:14 15 September 2012

Higher Education 53: 483–516.

Boud, D. 1999. Situating academic development in professional work: Using peer learning.
International Journal for Academic Development 4, no. 1: 3–10.
Creswell, J.W. 2005. Educational research: Planning, conducting, and evaluating quantita-
tive and qualitative research (2nd ed.). Upper Saddle River, NJ: Pearson.
Denzin, N. 1970. The research act. Chicago: Aldine.
Dewey, J. 1933. How we think. Buffalo, NY: Prometheus Books.
Green, J. 1999. Somatic authority and the myth of the ideal body in dance education. Dance
Research Journal 32, no. 2: 80–100.
Green, J. 2001. Socially constructed bodies in American dance classrooms. Research in
Dance Education 2, no. 2: 155–73.
Griffiths, V. 2000. The reflective dimension in teacher education. International Journal of
Educational Research 33, no. 5: 539–55.
Gur-Ze’ev, I., J. Masschelein, and N. Blake. 2001. Reflectivity, reflection, and counter-educa-
tion. Studies in Philosophy and Education 20, no. 2: 93–106.
Habermas, J. 1972. Knowledge and human interests. London: Heinemann.
Hammersley-Fletcher, L., and P. Orsmond. 2005. Reflecting on reflective practices within
peer observation. Studies in Higher Education 30, no. 2: 213–24.
Harland, J. 2007, June. Towards a research programme in arts education. Paper presented at
the conference Onderzoek in Cultuureducatie, June 22, Utrecht, the Netherlands.
Hatton, N., and D. Smith. 1995. Reflection in teacher education: Towards definition and
implementation. Teaching and Teacher Education 11, no. 1: 33–49.
Kolb, D.A. 1984. The process of experiential learning. Upper Saddle River, NJ: Prentice Hall.
Korthagen, F.A.J. 1985. Reflective teaching and preservice teacher education in the Nether-
lands. Journal of Teacher Education 36, no. 5: 11–15.
Lasley, T. 1992. Promoting teacher reflection. Journal of Staff Development 13, no. 1:
24–9.
Lavender, L. 1996. Dancers talking dance: Critical evaluation in the choreography class.
Champaign, IL: Human Kinetics.
Lavender, L., and J. Predock-Linnell. 2001. From improvisation to choreography: The critical
bridge. Research in Dance Education 2, no. 2: 195–209.
Leijen, Ä., W. Admiraal, L. Wildschut, and P.R.J. Simons. 2008. Pedagogy before technol-
ogy: What should an ICT intervention facilitate in practical dance classes? Teaching in
Higher Education 13, no. 2: 219–31.
Leijen, Ä., I. Lam, L. Wildschut, and P.R.J. Simons. 2008. Pedagogical practices of reflection
in tertiary dance education. European Physical Education Review 14, no. 2: 223–41.
Leijen, Ä., I. Lam, L. Wildschut, P.R.J. Simons, and W. Admiraal. 2009. Streaming video
to enhance students’ reflection in dance education. Computers & Education 52, no. 1:
169–76.
Maclellan, E. 2004. How reflective is the academic essay? Studies in Higher Education 29,
no. 1: 75–89.
 Studies in Higher Education 217

Mansvelder-Longayroux, D.D., D. Beijaard, and N. Verloop. 2007. The portfolio as a tool for
stimulating reflection by student teachers. Teaching and Teacher Education 23, no. 1: 47–62.
Marques, I.A. 1998. Dance education in/and the postmodern. In Dance, power and difference,
ed. S.B. Shapiro, 171–85. Champaign, IL: Human Kinetics.
McCollum, S. 1997. Insights into the process of guiding reflection during an early field expe-
rience of preservice teachers. Doctoral dissertation, Virginia Polytechnic Institute and
State University.
Mezirow, J. 1991. Transformative dimensions of adult learning. San Francisco: Jossey-Bass.
Moon, J.A. 1999. Reflection in learning and professional development: Theory and practice.
London: Kogan Page.
Moon, J.A. 2004. Reflection in learning and professional development. New York: Routledge-
Falmer.
Procee, H. 2006. Reflection in education: A Kantian epistemology. Educational Theory 56,
no. 3: 237–53.
Procee, H., and I. Visscher-Voerman. 2004. Reflecteren in het onderwijs: Een kleine system-
atiek [Reflection in education: A small system]. VELON Tijdschrift voor Leraren-
opleiders 25, no. 3: 37–44.
Downloaded by [University of Guelph] at 01:14 15 September 2012

Rodgers, C. 2002. Defining reflection: Another look at John Dewey and reflective thinking.
Teachers College Record 4, no. 4: 842–66.
Sparks-Langer, G.M., and A.B. Colton. 1991. Synthesis of research on teachers’ reflective
thinking. Educational Leadership 48: 37–44.
Stinson, S.W. 1995. Body of knowledge. Educational Theory 45, no. 1: 43–54.
Taggart, G.L. 1996. Reflective thinking: A guide for training preservice and in-service practi-
tioners. Doctoral dissertation, Kansas State University.
Thorpe, M. 2000. Encouraging students to reflect as part of the assignment process: Student
responses and tutor feedback. Active Learning in Higher Education 1, no. 1: 79–92.
Tsangaridou, N., and M. O’Sullivan. 1994. Using pedagogical reflective strategies to enhance
reflection among preservice physical education teachers. Journal of Teaching in Physical
Education 14, no. 1: 13–33.
Van Manen, M. 1977. Linking ways of knowing with ways of being practical. Curriculum
Inquiry 6: 205–28.
Van Manen, M. 1995. On the epistemology of reflective practice. Teachers and Teaching 1,
no. 1: 33–50.
Ward, J.R., and S. McCotter. 2004. Reflection as a visible outcome for preservice teachers.
Teaching and Teacher Education 20: 243–57.