Designing Online Information

Systems for
Portfolio-Based Assessment:
Design Criteria
and Heuristics

Presebted
By:
06503864  Sudeep Mandal
09609124  Divya Manglik
09609148  Avneesh Bhardwaj
09609062  Vini Goel
 Overview
Definitions of portfolio and portfolio-
building process
Benefits of portfolio assessment
Functions of online portfolio assessment
systems
Implications for designing online
information systems for portfolio
assessment.

Designing Online Information Systems for

2 portfolio based assessment September 28, 2010
 • Stakeholders issues and design factors
that will affect the value distribution

• qualitative design issues must be

addressed before focusing on technical
decisions about the hardware, software,
data structures and the database
management systems needed to
implement online portfolio assessment.

Designing Online Information Systems for

3 portfolio based assessment September 28, 2010
 Portfolio
“A collection of evidence
that demonstrates skills,
achievements, learning
or competencies.”

Designing Online Information Systems for

4 portfolio based assessment September 28, 2010
 Portfolio Building Process
 Identify : Areas of Skill
 Development of specific learning
outcomes
 Identification of appropriate learning
strategies
 Identification of performance
indicators
 Collection of evidence
 Organisation and presentation

PORTFOLIO BASED ASSESMENT

TRANSPARENTLY AND DIRECTLY APPLIED IN A PURPOSIVE
WAY
Designing Online Information Systems for
5 portfolio based assessment September 28, 2010
 Benefits of Portfolios for
Assessment and Learning
Portfolios can contain many different types of
evidence and from different sources
They resolve many assessment problems, especially
in equity and moderation
Portfolios provide a “richer picture” of students, their
learning and their competencies
Students are actively involved in their processes
Portfolios are well suited to authentic learning
environments

Designing Online Information Systems for

6 portfolio based assessment September 28, 2010
 Benefits of Portfolios for
Assessment and
Learning(Contd.)
Portfolios can be used in a wide variety of
contexts for the collation of evidence of
fundamental skills
Portfolios provide a means for students to
learn to manage their own professional
development
Portfolios are well suited to assessment in
lifelong learning contexts
There is an educational alignment between
online portfolio assessment and IT related
disciplines
Designing Online Information Systems for
7 portfolio based assessment September 28, 2010
 Stakeholder Groups
Are Students only the Broader Perspective
Stakeholder ?  Student
 Teaching Staff
 Course Co ordinator and
designer
 Enthusiastic
 Academic Line Manager
 Can they  University Authority
demonstrate  Goverment Agencies
their skills  Potential Employers
 Any other  External Assessors
EducationalBenefits  Field Supervisors

Designing Online Information Systems for

8 portfolio based assessment September 28, 2010
 Trends Shaping University
Education
The design of future courses in universities is
influenced by a variety of factors
Evidence-based assessment : Australia , US
and UK
Employability
Quality assurance :Australia
Equity
Plagiarism

Commercialization, modularization and globalization

Designing Online Information Systems for
9 portfolio based assessment September 28, 2010
 Information System
Design:Direct Factors
• Improving the “quality of student
learning and assessment”
• Increasing the reliability of certification
for employers
• Maximising the efficiency of
assessment processes

Designing Online Information Systems for

10 portfolio based assessment September 28, 2010
 Automation of
Administrative Functions
Automatic procedures for identifying elements of work
that a student has not completed.
Collating and comparing marks from each student’s
individual assessment submissions for diagnostic
purposes.
Automatically collating and comparing details of writing
style within and across a student’s assessment
submissions as a means of formative assessment of
student learning.
Assessing graduate attributes in the areas of language
use.
Checking for plagiarism.
Presenting students’ reports or certifications within their
online portfolios.
Designing Online Information Systems for
11 portfolio based assessment September 28, 2010
 Discipline Related Factors
and Technology Choice

A good alignment
between users and
online
technologies is
potentially one of
the most significant
Which One to issue.
use ?

Designing Online Information Systems for

12 portfolio based assessment September 28, 2010
 Information Storage
information storage is the
area that has received
most attention, often at
the expense of education,
assessment and
informatics issues.
Compression of
information in from of
Word, pdf, MPEG and
various other formats.

Designing Online Information Systems for

13 portfolio based assessment September 28, 2010
 Quality Assurance
demonstrating to external assessors the
“goodness” of the product or process.
Quality Assurance primarily concerned with
“quality improvement.”
When Quality Assurance is defined in terms
of Quality Improvement, the two key
quality improvement issues for any
education or assessment program are
transparent moderation and sound
processes of evaluation and feedback.

Designing Online Information Systems for

14 portfolio based assessment September 28, 2010
 Equity Issues
Equity is strongly dependent on students’
technology skills and technology access.
Standardising the hardware and software, and
training students in that hardware and software.
Use of online templates and restrictive modalities of
input.
Intention is to limit the scope for students to gain
assessment advantage through access to more
sophisticated software and hardware, or through
more advanced technical skills.

Designing Online Information Systems for

15 portfolio based assessment September 28, 2010
 Plagiarism
Computed-aided plagiarism is a problem in paper-
based and online formats.
It undermines confidence in assessment processes
and can significantly damage the reputation of an
educational institution.
Traditional approaches not much effective.
New technical forms of addressing plagiarism are
needed.

Designing Online Information Systems for

16 portfolio based assessment September 28, 2010
 Security, Fraud and
Tampering with Records
Flexibility of access raises information security
problems.
Designers of quality courses, however, should
expect to make provision for an appropriate
response to successful or unsuccessful attempts at
IT enabled fraud or security ingress.
A secure interface would also offer the means for
external and internal assessors to certify they have
sighted originals of certificates from elsewhere.

Designing Online Information Systems for

17 portfolio based assessment September 28, 2010
 INTERFACE ISSUES

Online access to the

Provide online access means to build a
to a complete suite of coherent portfolio out of
software individual elements
created elsewhere.

COMMENTARY

OUTCOME:
UNIFIED PORTFOLIO
Designing Online Information Systems for
18 portfolio based assessment September 28, 2010
 Cognitive and Information
Artifacts

Designing Online Information Systems for

19 portfolio based assessment September 28, 2010
 Issues in designing of information
systems

Designing Online Information Systems for

20 portfolio based assessment September 28, 2010
 CONCLUSION
 This paper draws attention to the significant
benefits that online portfolio information
systems
 The paper also draws attention to reasons
why current approaches to designing online
portfolios rarely realise these benefits.
 The paper reports an exploratory
investigation into designing information
systems for online portfolio-based
assessment

Designing Online Information Systems for

21 portfolio based assessment September 28, 2010
 DRAWBACKS
 Focuses only on identifying technical means
 The “online portfolio” consists only of a single
essay, project report or term paper
 Designs for online portfolio assessment
systems are based on an over-narrow view of
value distribution that does not take all
stakeholders into account.
 Designing of online portfolio assessment
systems is not well integrated with overall
course design processes.

Designing Online Information Systems for

22 portfolio based assessment September 28, 2010
 ALTERNATIVE APPROACH
 The identification of the nature and
characteristics of the educational and
institutional contexts
 The identification of potential benefits
and increase in value for all stakeholders.
 The development of heuristics
 The development of an online system
through:
 The use of best practices in course
design,
 The fulfillment of the requirements
 The integration of the designing of the
23
online
Designing Online portfolio system
Information Systems for
September 28, 2010
portfolio based assessment
 A focus on process automation
 Thank
You

Designing Online Information Systems for

24 portfolio based assessment September 28, 2010