“Reframing Assessment: Using social software to collect and organise

learning”

Anne Bartlett-Bragg
Faculty of Education
University of Technology Sydney
Sydney, Australia
Anne.Bartlett-Bragg@uts.edu.au

Abstract: Assessment is a fundamental ingredient in the teaching and learning

process, yet there is an increasing body of literature expressing dissatisfaction with
current assessment policies and practices in higher education that challenges the
appropriateness of assessment to address the future needs of students in a rapidly
changing information rich environment. This paper will provide a framework for
designing assessment with social software and examples of options for alternative,
authentic assessment methods that collect and organise evidence of learning,
suggesting that this process can foster the development of self-directed learners,
resulting in discretely embedded learning activities that promote problem-solving
and knowledge development beyond the boundaries of the classroom and
institution.

Conventional assessment practices tend to focus on students producing work that demonstrates
knowledge of subject content, or a culture of evidence that is reinforced when learning is restricted to
tight, cost efficient methods that relate only to subject matter taught (Shavelson, 2007). The process is
supported by the rationale that to provide motivation, the learners’ attention is directed to content that
is considered to be important within the subject being studied. This approach however, limits how and
what is learned, and the process of learning itself (Boud & Falchikov, 2007).

Concurrently, early implementations of e-Learning systems in educational institutions tended to focus

upon the use of technology as an administrative tool to enhance efficiencies and processes with learner
management systems testing and recording results of students. These strategies in themselves are
irrelevant to the learning process and merely reinforce the controlled, transmission – acquisition
approach to learning and assessment practices. In addition, there is little evidence to suggest that
incorporating these strategies and technologies into existing learning environments has resulted in any
significant change in the learning outcomes (Zemsky & Massy, 2004).

The need to review assessment processes is becoming an expanding body of literature with two
dominant perspectives: firstly – the literature that views assessment as the major influence on learning
and how this effects student approaches to their work and studies. From this perspective arises the call
to rethink the role of assessment and the learning processes, calling for self-assessment, peer feedback,
portfolios, and group work that encourage learning for the future and as an act of informing judgement
(Boud & Falchikov, 2007). And secondly – the integration of technology and associated impact on
assessment processes. Extending now into calls for the use of social software to provide learners with
digital spaces where they can interact, explore, and construct an individualised approach and manage
their own learning. (Attwell, 2005; Bartlett-Bragg, 2007; Owen, Grant, Sayers & Facer, 2006;)

Both perspectives align to the findings from my PhD research and teaching practice as a University
lecturer. The process of students working with social software has evolved into more than a learning
strategy, but also a multi-faceted assessment strategy that incorporated self-assessment, peer feedback,
and the development of self-managed artefacts to be presented as a portfolio at the completion of a
semester.

In the University context, students tend to restrict their study activities to only what is required to be
addressed by the assessment task, resulting in a short term point of reference with limited engagement
beyond the semester timeframe (Boud, 2007). While research conducted into the use of self-publishing
technologies (weblogs and wikis) has demonstrated the potential that social software integrated into the
students’ learning activities presented an opportunity to extend the learning process and incorporate
assessment as a discrete activity, rather than the primary focus of the subject being studied (Bartlett-
Bragg, 2007).

How can social software be used to reframe assessment processes?

Self-publishing software, such as weblogs and wikis, enable the student to gather a collection of
artefacts, develop critical reflection skills, receive peer feedback and potentially engage with a network
beyond the classroom and educational institution. Additional social sharing software applications such
as social bookmarking further encourages the students to collect, distribute and organise resources
considered valuable not only to themselves, but also their network. Aggregation software provides a
central collection platform where the student can subscribe and collect items of interest and be notified
of updates that can be reviewed at a time convenient to them.

Through the process of developing a personalised learning environment, the student is taking
responsibility for the collection and presentation of items in a variety of modes for final assessment,
while feedback and comments from peers and a network that may include participants beyond the
boundaries of the classroom facilitates the potential for self-directed learning to continue beyond the
timeframes of formal studies.

How can social software foster self-directed, lifelong learning strategies?

Social software can provide opportunities to engage with the content, not just the assessment task.
Additionally, discretely embedded in learning activities are digital literacies, personal information
management and self-directed strategies that are transferable to contexts other than the subject.

The underpinning frameworks that have informed the development of pedagogical practices towards
action-oriented, self-directed learning activities using social software applications are outlined below:

1. Pedagogical framework for self-publishing with social software – Bartlett-Bragg (2007)

A pedagogical framework was developed and tested in my PhD research that provides learning and
assessment strategies for the integration of social software into learning environments (Bartlett-Bragg,
2007). The objective of the pedagogical framework is to facilitate the development of independent
learners allowing them to become proficient in the development of collective learning networks using
weblogs and associated social software technologies. The framework can be viewed as an enabler
where multi-linear pathways draw the focus not on the software or technology selected by the educator
to create the learning environment, but on the social aspects of the learning process and strategies to
support the learning experience. Refer to Figure 1 for a graphical representation.

5:
DISTRIBUTED
KNOWLEDGE
•Collaborating
• Distributing
• Networking

2: 3: REFLECTIVE 4: REFLECTIVE
INTERPRETATION MONOLOGUES DIALOGUES

• Structure • Identity • Writing

•Personalisation • Writing • Socialising
• Publishing • Networking

• Concepts
1:ESTABLISHMENT
• Context-examples
• Set-up
 Figure 1: A pedagogical framework for self-publishing with social software

2. Modes of Teaching – Baumgartner (2004)

The educators’ existing pedagogical practices developed through formal studies or influenced by
organisational structures can inhibit the development of learners towards participation within social
software environments and informal learning activities. Baumgartner’s (2004) three prototypical
models for teaching provide a valuable framework to review and reframe pedagogical strategies that
enable the educator to understand their own practice and review their approach to designing learning
and assessment. Refer to Table 1.

Mode 1: Transfer Mode 2: Tutor Mode 3: Coach

(Directed Teaching) (Facilitated Learning) (Informal Guide)
Programmed instruction Problem solving Complex simulations

To teach, to explain To observe, to help, to To co-operate, to support

demonstrate

Production of correct answers Selection of methods and Realisation of adequate

its use action strategies

To know, to remember To do, to practice To cope, to master

Transfer of knowledge Presentation of pre- Action in real situations

determined problems (complex and social)

Table 1: Modes of Teaching

Educators implementing learning and assessment activities in a Mode 1 mindset will be reinforcing a
model that transmits content to an individual student in an asynchronous, self-paced style where the
design is focused on outcome, results driven, directed learning. While educators designing learning and
assessment activities in a Mode 3 mindset will be enabling the students to explore the potential
presented by the integration of social software.

3. Scheme for developing informed judgement - Boud and Falchikov’s (2007)

Boud and Falchikov’s (2007) scheme to develop informed judgement provides a framework for
reviewing assessment tasks to emphasise evaluative expertise. The framework overlays the previous
models and assists the educator to review their designs. The key elements are outlined as five
overlapping components: 1) Identifying self as an active learner; 2) From known to need - identifying
own level of knowledge and the gaps; 3) Practising testing and judging; 4) Developing judgement
skills over time; 5) Embodying reflexivity and commitment.

The key findings from the PhD study focused on the experiences of the adult learner using social
software that was integrated into their learning and assessment processes and provides potential for
educators to re-frame their pedagogical strategies to facilitate students’ development towards self-
directed learners.

An important finding indicated that the inhibitors to effective implementation of social software into
practice required pedagogical strategies to enable the educator to neutralise the potential negative
effects on the learners. These inhibitors are categorised into three broad areas: organisational inhibitors,
including cultural and technical infrastructure; educators’ inhibitors; and learners’ inhibitors.

The frameworks mentioned previously support the educator with the design and implementation of
alternative assessment strategies, and strategies to manage the learner’s inhibitors, but do not address
the organisational cultural inhibitors. Table 2 outlines how the three frameworks can be overlayed to
create processes for learning activities that directly inform assessment processes.
Bartlett-Bragg (2007) Pedagogical
Framework
1. Establishment
Learners are actively creating their
personalised learning environments
with social software eg. weblogs,
wikis, social bookmarking and
aggregation.
2. Interpretation
Learners are developing a structure
and adapting to their perceived
needs.
3. Reflective Monologues
Learners are publishing to their
software platform and establishing
their identity, or finding their voice.
4. Reflective Dialogues
Learners are extending their
learning environment by developing
social networks.
5. Distributed knowledge
artefacts
Learners are collaborating with
others, distributing their work, and
gathering artefacts for review and
reflection.
Baumgartner (2004) Modes of
teaching
Mode 2: Educator is providing
assistance but encouraging learners
to build their own environments.
Some strategies may require
demonstrations of abstract concepts.
Mode 2: Educator is observing and
supporting through demonstration
as learners adapt to their perceived
needs.
Mode 3: Educator is providing
support and activities to assist
learners to progress in a self-
directed manner.
Mode 3: The educator introduces
activities to stimulate action
towards further establishing
connections with other learners.
Mode 3: At this point the educator
could be viewed as the conduit and
a co-participant in the learner’s
network – providing feedback
towards future development.
Boud & Falchikov (2007)
Scheme for developing informed
justdgement
1. Identify self as an active
learner
The process of setting up the
various software applications
actively involves the learner in the
selection and building of their
environments – establishing
engagement in the processes.
2. From known to need
Learners are developing a structure
within their software environment
based on their perceived needs.
3. Testing and Judging
As the learners perform writing and
publishing tasks, feedback from the
educator and peers informs further
development and self-efficacy.
4. Developing skills over time
A period confidence grows with the
self-publishing techniques being
practised, the learners establish
more concrete practices.
5. Embodying reflexivity and
commitment
As both peer and educator feedback
is received the learners continue to
adjust and improve their work.
While gathering artefacts for final
assessment.

Table 2: Overlay of the 3 frameworks with examples

Reframing assessment is not solely about integrating software into the process of assessment, it is also
about reviewing current assessment philosophies and determining how assessment can foster learning.
Social software affords educators the opportunity to personalise the overall learning experience,
including assessment, to enable the learner to collect and organise information and artefacts from
different contexts and situations, and demonstrate and reflect upon their skills and achievements
(Attwell, 2007).

It is timely, to reflect on these issues as our educational institutions espouse the values of lifelong
learning and prioritise practices of collaboration, reflection, personalisation, knowledge sharing and
networks into vision statements and strategic planning documents. Notwithstanding the importance of
social software as the enabler in these processes, without re-framing our practice educators are not
likely to realise the opportunity to incorporate social software and integrated communication networks
that extends the learning beyond the physical boundaries and opinions of the classroom context.

References:

Attwell, G. (2007). The Personal Learning Environments – the future of eLearning?, eLearning Papers,
2(1), 5. Retrieved February 10, 2007, from
http://www.elearningpapers.eu/index.php?page=home&vol=2

Bartlett-Bragg, A. (2007). Reflections on pedagogy: Understanding adult learners' experiences of

weblogs. In T. Burg, & J. Schmidt (Eds.), BlogTalks reloaded (pp. 119-241). Austria: Herstellung.

Baumgartner, P. (2004). The zen art of teaching – Communication and interactions in e-Education.
Workshop presented at ICL2004, Kassel University, Villach, Austria. Retrieved November, 2004, from
http://www.elearningeuropa.info/extras/pdf/zenartofteaching.pdf
Boud, D. (2007). Reframing assessment as if learning were important. In D. Boud, & N. Falchikov
(Eds.), Rethinking Assessment in Higher Education, (pp.14-25). London:Routledge.

Boud, D. & Falchikov, N. (2007). Assessment for the longer term. In D. Boud, & N. Falchikov (Eds.),
Rethinking Assessment in Higher Education, (pp. 4-13). London: Routledge.

Owen, M., Grant, L., Sayers, S., & Facer, K. (2006). Social software and learning. Bristol, UK:
Futurelab. Retrieved July 18, 2006, from
http://www.futurelab.org.uk/resources/publications_reports_articles/opening_education_reports/Openi
ng_Education_Report199

Shavelson, R. J. (2007). Assessing student learning responsibly: from history to an audacious proposal,
Change, 39(1) (pp.26-33).

Zemsky, R., & Massy, W.F., (2004). Thwarted innovation: What happened to e-learning and why. The
Learning Alliance. Retrieved July 20, 2006 from
http://www.thelearningalliance.info/Docs/Jun2004/ThwartedInnovation.pdf