LTSN Generic Centre

Assessment
Series No 14

Computer-assisted
Assessment (CAA)
Joanna Bull and
Myles Danson

ltsn

generic centre
Learning and Teaching Support Network
Joanna Bull is Director of Eduology. Eduology provides research, consultancy, advice
and creative solutions for businesses, training organisations, educational institutions and the
public sector. It advises clients on ways to improve the effectiveness of learning and training
within their organisation. Previously, Joanna was Head of Research in Teaching and Learning
at the University of Luton and Project Manager for the Computer-assisted Assessment (CAA)
Centre, a Teaching and Learning Technology Programme (TLTP) project. The CAA Centre
(www.caacentre.ac.uk) was a national centre for expertise, advice and guidance on CAA in
higher education. Joanna has extensive experience of project management and evaluation,
has published widely and has designed and delivered numerous workshops and seminars
nationally and internationally. She has presented at conferences nationally and internationally
and her most recent publications include articles and books on the assessment of student
learning, learning technology and strategies for implementing computer-assisted
assessment.

Myles Danson is the CAA Manager at Loughborough University and has overall
responsibility for the centrally supported CAA systems. He is also the organiser of the
International CAA Conference (www.caaconference.com/), now in its seventh year. Over the
last five years Myles has developed and managed various CAA systems including optical
data capture, networked PC delivery, and most recently, the development and roll out of a
front line web-based CAA system. The latter was funded by the Joint Information Systems
Committee (JISC) and contributed to their Managed Learning Environment projects. The
CAA systems at Loughborough have delivered and marked over 200,000 assessments to
date. Myles engages in project management and income generation including the CAA
Centre project, FDTL and LTSN work and most recently, the JISC funded Technologies for
Interoperable Assessment project (www.toia.ac.uk). He also has substantial experience of
national and international publications, workshops, guest speaking, policy formation and
strategic planning.
Contents 14
Introduction 2

Overview of CAA 3
What is computer-assisted assessment? 3
What is possible with CAA? 3

Advantages and challenges 5

Getting started 8
Planning 8
Piloting 9
Institutional support 9
Further development 9

Effective practice 10
Case studies 10

Using ‘virtual learning environments’ for assessment 13

Standards 15

Question banks 17
What are ‘question banks’? 17
Why are they useful? 17
Question statistics 17
Examples in practice 18

Evaluating practice 19
Why is evaluation important? 19
What aspects of CAA should be evaluated? 19
What approached to evaluation should be taken? 20

Conclusion 23

References 24
 Introduction
The purpose of this briefing is to explore the
role which computers can play in the
assessment process. The briefing
introduces the range of activities which
comprise computer-assisted assessment
(CAA) and discusses the advantages and
challenges of introducing computers into
the assessment process. Ideas and
suggestions on how to get started are
provided and good practice highlighted
through mini case studies of effective
practice. Key issues are also raised such as
standards, the role of question banks and
the importance of evaluation. The briefing
does not discuss the general pedagogical
issues associated with assessment - these
are covered in other briefings and guides.
Neither does it discuss the technical
solutions to computer-assisted assessment
nor provide a detailed methodology for
implementation. However, colleagues
wishing to follow up on the pedagogical and
2
implementation issues of CAA may find the
Blueprint for Computer-assisted
Assessment of use (Bull and McKenna,
2001).
This briefing is aimed at staff and
educational developers and other staff who
teach and support student learning. It may
also be useful to anyone who wishes to gain
an overview of the potential of computers in
the assessment process and the associated
key issues. The briefing provides a useful
starting point for those considering CAA and
could be used as a focus for discussion with
colleagues or as the basis of an introductory
staff development session. The briefing
complements the existing assessment
series, in particular the Guide for Lecturers,
which provides an overall context within
which CAA is one method of assessment
that may be used.
LTSN Generic Centre – Assessment Series 2004
Overview of computer-assisted
assessment
What is computer-assisted
assessment?
Computer-assisted assessment (CAA) is a
broad term which describes the application
of computer technologies to the assessment
process. This may include a variety of
activities which assess knowledge,
understanding and skills using one or more
technologies such as the Internet, intranets,
CD-ROM and optical data capture systems.
CAA may be used for both formative and
summative assessment in the following
ways:
• to deliver, mark and analyse
assignments or examinations (computer
or web-based),
• to collate and analyse data gathered
from optical data capture systems (for
example optical mark readers),
• to record, analyse and report on
achievement, for example through the
construction of online portfolios,
• to collate, analyse and transfer
assessment information through
networks.
There is – in some ways unfortunately – a
plethora of terminology which surrounds the
use of information and communications
technology (ICT) in higher education. The
terminology changes and adapts at the rate
of technological development and can cause
confusion and distrust among academics
and students. Other terms used in the
literature and practice to describe types of
LTSN Generic Centre – Assessment Series 2004
CAA include: web-based assessment,
computer-based assessment, online
assessment, and computer-aided
assessment. While the terminology – and
individual interpretations of it – may vary, the
14
underlying strategies and practices for
making effective use of computers for
student assessment remain constant.
What is possible with CAA?
CAA is most commonly associated with
multiple-choice questions. There is a certain
amount of scepticism about the ability of
multiple-choice questions to test anything
more than basic knowledge. Biggs (1999)
states that ‘MCs assess declarative
knowledge, usually in terms of the least
demanding process, recognition’. The
common misconception goes like this: ‘CAA
is only suitable for delivering objective
assessments, that is those which require the
recognition of a single correct answer.
Objective assessments are comprised of
multiple choice questions and these are
inadequate for testing more than basic
knowledge.’
It is true that there are examples of CAA
consisting solely of multiple-choice
questions and that some of these are not
well designed and do not test more than
basic knowledge. This is often a result of a
lack of understanding about the pedagogy
of designing questions and tests – a
complex and skilled process. However,
there are also examples of CAA which draw
3
 Assessment: A Briefing on Computer-assisted Assessment
on an extensive range of sophisticated
question types by utilising computers to
create questions which would not be
possible using the medium of paper. They
incorporate sound, multi-media and images,
and require students to know, comprehend
and apply their knowledge. The question
types go way beyond the objective format of
‘select one from four’ and require intelligent
consideration rather than guessing. Such
examples of CAA allow skills and abilities to
be tested which could not be assessed
using more traditional methods.
In addition to the ‘objective formats’, there
is a range of related activities which seek to
utilise electronic environments to support
innovative assessment methods. Examples
include gathering assessment information
from online discussion forums and using the
Internet and virtual learning environments to
facilitate peer assessment. Others are
4
developing blended approaches – part CAA,
part paper-based assessments to make the
most of both of these approaches.
CAA offers the opportunity to creatively
extend the range and type of assessment
methods used. It is not a panacea for rising
student numbers and marking overload
though if used appropriately, it can clearly
relieve some of these pressures. Used
formatively to provide detailed and timely
feedback it can support and enhance
student learning in ways which are not
possible with paper-based assessments.
Diagnostically, it can provide an instant
indication of skill and knowledge gaps.
Summatively, it can dramatically reduce
marking loads for large classes and offer
speedy processing of results.
LTSN Generic Centre – Assessment Series 2004
Advantages and challenges
Assessment is a high-stakes activity for both
the learner and the educating institution.
Computerising this process to any extent
highlights not only those issues associated
with traditional assessment, but also poses
new issues. Promotion of a new form of
assessment usually invokes criticisms rarely
considered in the traditional assessment
process. When considering CAA, it is useful
to think about how the traditional process
deals with issues such as candidate
authentication, security and plagiarism, and
to attempt to draw parallel conclusions
where appropriate. There is a major issue of
acceptance of new technologies within the
curriculum and it is constructive to
recognise that standard assessment
techniques, frequently developed with little
strategic planning, are often far from perfect.
Mapping the assessment process can be a
relatively complex task. However, a full
evaluation of this often reveals inadequacies
and opportunities for improvement which
CAA can address. (See Danson et al., 2001,
for an example of an assessment mapping
process.)
At first glance, the key advantage of CAA is
usually perceived as a saving in resources
and time. Automated marking is highly
desirable from both the point of view of the
educator, and if students are properly
informed about the capabilities and benefits
of the CAA – the learner. Quick and often
instant marking and feedback are clear
benefits, leaving educators free from often
LTSN Generic Centre – Assessment Series 2004
laborious, repetitive manual scoring of
assessments, while the learner no longer
14
has to wait for this process to be
undertaken. If designed effectively, CAA can
provide highly effective instant feedback.
Students may take tests at a time and
frequency to suit themselves and, in the
case of open access web-based
assessment, the technology often allows
tests to be taken at a location to suit the
student.
In the case of CAA potential benefits are
available to all parties. The lecturer is free
from manual marking: the student gets an
instant and objective score with specific and
timely feedback. In addition, the detailed
scoring data are already digitised and the
possibility of automated score upload to
central repositories such as a student
records system offers administrative
benefits. This ‘win-win’ scenario provides
great potential for formative, summative and
diagnostic assessment. However, it must be
recognised that CAA ‘front loads’ the
assessment process – the majority of time
and effort is invested in the design of
questions and tests prior to the assessment
taking place. With traditional assessments,
greater time and effort is invested in marking
once the assessment is completed. Creating
pedagogically sound CAA is a highly skilled
activity and should not be dismissed lightly.
The testing medium can be far richer than
paper-based assessment.
5
 Assessment: A Briefing on Computer-assisted Assessment
Examples of screen-delivered CAA include
colour, animations, sound and even video
content, and it is likely that the learner will
interact in different ways (McGuire et al.,
2002). Adaptive testing is also possible
whereby the CAA system serves question
content based on the responses and
performance of an individual student. An
assessment may be repeated several times,
each time presenting different variations of
the same question content. This is an
excellent way of encouraging learning
through the application of skills and is
particularly effective in areas where this is
necessary, such as mathematics. Recent
CAA systems intelligently comprehend
question and answer content, which has
been shown to be useful in algebraic testing
(Ashton and Beevers, 2002) and in the
assessment of free text (Kukich, 2000,
Leacock and Chodorow, 2000).
Reporting features frequently include
analysis of the question and test
performance and offer the potential for
quality control often impossible using
traditional methods. Detailed analysis of
questions and student performance – as
individuals and as groups – can help to
identify gaps in skills and knowledge early in
a course or module. Students at risk can
quickly be identified and offered additional
support. Over a period of time this type of
data can help to inform changes in the
curriculum and – possibly –methods of
learning and teaching.
6
Perhaps one of the most underestimated
challenges for those using CAA is the
gaining of cultural acceptance. We
mentioned in the introduction some of the
perceptions held about CAA as a limited
form of assessment. These, plus some
reluctance to accept the role of technology
in education, can be potent issues to be
faced by those implementing CAA. Despite
the potential time savings, the time and cost
investment may initially seem high. Time
must be spent developing pedagogically
sound questions, and hardware and
software must usually be purchased and
installed. If summative assessment is to be
undertaken, a very secure and robust
system with back-up contingencies is
needed to ensure success.
Student technical skills are less of a
problem in the new millennium. However,
effective support must be put in place to
ensure that no student is disadvantaged by
the testing medium. This may mean
including reference to CAA in appropriate
documentation, handbooks and module or
course descriptors. Students involved in
summative CAA will need the opportunity to
practice with the assessment type and the
software. The logistical and operational
issues of delivering CAA must be planned
for carefully, particularly where summative
assessment is concerned. Traditional
models of examination delivery may have to
be adjusted — for example, running two
sessions of the same examination back-to-
LTSN Generic Centre – Assessment Series 2004
 Assessment: A Briefing on Computer-assisted Assessment
back to allow for student numbers which
exceed the numbers of computers available.
In addition, policies and procedures may
have to be developed to ensure smooth
running of CAA within the context of existing
assessment regulations.
In addition, accessibility needs to be
considered in the design of questions and
construction of online assessments.
Appropriate measures should be put in
place to ensure that assessments are
accessible to all students. Existing policies
and support mechanisms should be
reviewed in light of the introduction of CAA
for summative assessment.
A key challenge is providing appropriate
pedagogical and technical support to
academic and other staff involved in
developing CAA. Both academic and
LTSN Generic Centre – Assessment Series 2004
support staff are likely to need staff
development and training on both the
14
pedagogical and operational aspects of
CAA. There is also an ongoing need for
support to ensure that academic staff are
free to concentrate on developing
pedagogically sound assessments rather
than attempting to install and configure
software on their institutional networks. To
meet this challenge, clear roles and
responsibilities should be discussed and
decided early in the process of considering
CAA. It is important to acknowledge that
CAA requires skills and experience from
across an institution, often involving a range
of staff from academic, IT, library and
administrative departments.
7
 Getting Started
Planning
A good starting point is to conduct a needs
analysis. Consider how assessment is
undertaken currently and where a CAA
system might be most beneficial. It is
important to clearly identify the purpose of
the assessment. Is it to rapidly evaluate
student performance during a course, to
provide end of course/module assessment,
or to give timely formative feedback to
students on their learning? It may be that
there are several purposes for introducing
CAA. It is wise, however, to start small,
usually with small-scale formative or self-
assessment tests. This provides the
opportunity to learn from experience prior to
moving to high-stakes summative
assessment. It is important to remember the
strength of CAA in providing formative
assessment and feedback - instantly,
consistently and specifically — in order to
support student learning.
Once the purpose is defined, the next step is
to identify stakeholders and set up
appropriate consultative processes. This may
be a relatively small group for formative or
self-assessment which is supplementing
existing assessment methods, or a larger
faculty or institution-wide group for
summative assessment. It is particularly
important to involve appropriate IT staff in the
consultative process given that the use of
CAA may well impact on other systems and,
depending on the model of support, can
represent a shift in workload from academic
to support staff. It is useful to recruit other
enthusiasts at this stage to help drive the
process and provide support and motivation.
Review and evaluate the various solutions
available and consider the relative
8
advantages of commercial software or an in-
house solution. The choice and range of CAA
software is increasing so it is important to
determine the desirable features to meet your
needs, but be aware that pleasing all the
stakeholders all of the time is a holy grail and
unlikely to be achieved! For example, the
TRIADS (TRIpartite Assessment Delivery
System) CAA system offers complex
pedagogical features but requires a high level
of skill in writing questions and software use.
The CASTLE (Computer ASsisted Teaching &
LEarning) system, by comparison, is far
simpler to use but has a restricted number of
question types. It may be beneficial to adopt
more than one system to meet several needs.
This, however, raises support and
implementation issues which must be
considered in terms of practicality and
achievability.
In particular, the following are important to
consider when evaluating CAA software:
• Interoperability (IMS standards – see
section on Standards on page 15 below)
• Existing systems and integration
• Scaleability
• Performance levels
• Upgrading procedures and limitations
• Support and maintenance
• Security — particularly where summative
assessment is intended
• Accessibility.
It may also be useful to visit other institutions
with the aim of discovering both successes
and failures. Be aware that there is a general
reluctance to admit failures! The type and
level of support required by staff and
students should also be considered early in
the process. Staff development should be
LTSN Generic Centre – Assessment Series 2004
 Assessment: A Briefing on Computer-assisted Assessment
taken seriously and can help to avoid pitfalls
later in implementation. In particular,
pedagogical staff development should be
undertaken to ensure that assessments
delivered in CAA do not fall into the category
of ‘only good for testing low-level recall’.
It is useful to review existing questions – from
textbooks, the Internet and other sources – as
this can help with the formulation of ideas
about how CAA can be used effectively. It
may also be possible to find existing
questions and tests which can be purchased,
used freely, or shared and adapted for re-use.
Care should, however, be taken to ensure
questions are of a high standard and are
pedagogically and culturally appropriate.
Piloting
It is important to test CAA systems and
assessments, particularly where summative
assessment is to be implemented. Formative
open access assessments can prove a useful
way of gathering information about software
and question performance and student
experience. The level of piloting will depend
on how many students are likely to use the
CAA and on what basis. A series of pilots may
be valuable for high-stakes summative
assessments. Piloting also provides an
opportunity to test operational procedures
and to gain feedback from students. The
original purpose of the CAA should be tested
by piloting to ensure that the system meets
the needs of both students and staff.
Institutional support
It is useful (though not always essential) to
gain institutional or faculty support for CAA.
Because CAA often involves a range of staff,
both academic and support, the aim should
LTSN Generic Centre – Assessment Series 2004
be to foster a culture of ownership and
responsibility. Since traditional roles and job
descriptions may not easily fit within the
context of CAA development and support,
appropriate training and support should be
14
established. It may be beneficial to recruit a
senior manager as a ‘champion’ to help drive
the process forward and to secure resources
where necessary. There may also be existing
support departments (such as staff
development, teaching and learning, and
learning technology groups) that can help to
support and guide the implementation in
conjunction with academic staff. In some
institutions, dedicated CAA support staff are
appointed with this specific remit.
Further development
As CAA develops and more staff and students
become involved, there may be a move from
formative assessment towards summative
assessment. The involvement of larger
numbers of students in CAA and the high-
stakes nature of summative assessment leads
to further considerations, such as:
• Existing examination regulations and
assessment policies and their suitability
for CAA
• Special educational needs
• Back-up procedures and contingency
planning
• Usability and ergonomics
• Bandwidth and stability of connections
for off campus access.
The way in which CAA evolves will vary, and
can be subject-specific. It is important to
acknowledge the time and effort involved in
creating well-designed, valid and reliable CAA
and to be clear about the benefits and
challenges of implementing CAA.
9
 Effective practice
Assessment is a high-profile activity and
critical to student learning. It drives and
motivates students to learn, and changing
assessment will result in a shift in student
learning. For that reason it is important to
embed CAA within existing methods and
strategies of assessment. It should be clear
to students that the CAA they are
undertaking has a distinct and valuable
purpose. CAA which is ‘bolted-on’ to a
course or module, giving the impression that
the lecturer is just trying out some new and
half-formed idea, will not be effective in
motivating student learning. CAA should
offer benefits and complement existing
assessments, rather than duplicating other
assessments or only dealing with trivial skills
and knowledge.
Three mini case studies are presented
below to provide examples of how CAA
might be embedded into a course or
module.
Case Study 1
‘Fundamentals of Human Physiology’ is a
first year core module for undergraduate
Biology. The module prepares students for a
range of modules in their second year,
providing them with essential skills and
knowledge which they will need throughout
their degree. Currently assessments include
10
three coursework assignments and an end-
of-module examination. The course attracts
large numbers of students and lecturers find
that some students, although passing the
coursework, fail the examination. The
coursework assignments represent a
significant marking load and sometimes
students do not receive any feedback on
their learning until halfway through the
module. Following a review of the module
assessments it was decided to replace one
of the early coursework assignments with a
series of five short CAA objective tests.
Each test related to teaching material
covering a two-week period and students
received feedback after the test on each of
their questions. Students had to book
themselves into a computer lab at specific
times to take tests which were invigilated by
support staff. Lecturers received the results
of the tests the following day and
sometimes noticed that significant numbers
of students had failed questions on certain
topics, which they then addressed in the
following week’s seminar. Through the
feedback students gained a clear idea of
how they were progressing with the course
and were motivated to follow up some of the
feedback suggestions regarding further
reading and research.
LTSN Generic Centre – Assessment Series 2004
 Assessment: A Briefing on Computer-assisted Assessment
Case Study 2
‘Understanding Poetry’ is a twelve week first
year undergraduate elective module which
teaches students about critical terms and
theories, the process of developing a poem,
the historical development of poetry, and
some aspects of critical comparison. The
module attracts quite large numbers. For
some, it is a pre-requisite for further study
while for others it is an elective and they
may be unlikely to study literature any
further. The size and mixed literary
experience of the cohort mean that the
number of free-text assessments (essays,
critiques) is limited by what can reasonably
be marked. The lecturer also cannot assume
that all the students have a good knowledge
of those critical concepts which are
important as a basis for developing
understanding and critical analysis skills. In
order to effectively assess the module, a
range of objective and subjective methods
of assessment are used including the
following:
• CAA objective test – to test knowledge
of poetic terms, application of terms
and concepts and understanding of
historical development of the genre.
• Critical comparison of two poems – to
assess ability to critically analyse,
application of terms and concepts and
knowledge of terms.
LTSN Generic Centre – Assessment Series 2004
• Essay – to assess all aspects of the
module, but primarily understanding of
14
historical development of the genre.
• Computer Aided Learning (CAL)
manuscript assignment – to assess
students’ understanding of ‘poetic
process’ from manuscript to
publication.
In this case, the CAA component of the
assessment strategy is both formative, with
self-assessment questions available to
students on demand, and summative, with
an end-of-module exam. It ensures that
students are tested on a wide range of
poetic terms and their application. Students
who are weaker can practice with self-
assessment. This motivates them and has a
positive effect on their other assessments.
The lecturer also finds that she receives
fewer basic and repetitive questions about
poetic terms and concepts during teaching
sessions. The summative assessment
identifies that students have knowledge and
understanding of a core set of critical terms
and concepts.
(This case study is adapted from Bull and
McKenna, 2001).
11
 Assessment: A Briefing on Computer-assisted Assessment
Case Study 3
‘Primary Teacher Education’ is a single
honours degree course which attracts
students from a variety of backgrounds – in
particular students who have worked for a
number of years and those who have few
formal qualifications. In the first year the
course includes modules on
• ICT knowledge and understanding
• Introduction to Mathematics in the
Primary Curriculum
• Introduction to Science in the Primary
Curriculum.
These are key modules and in the past, it
has been found that student skills and
abilities in these subject areas vary
dramatically. Although additional support
sessions are available in Mathematics and
Science, students often do not take these
up or do so late in the course. The course is
intensive and early on includes work in local
primary schools. The lecturers feel that
students need to reach a certain level of
12
competency in these core areas prior to
entering a teaching environment. CAA has
been introduced to provide a diagnostic test
which all students take during their first
week. The test assesses their current
knowledge and understanding of ICT,
Mathematics and Science. The questions
are directly related to the learning outcomes
of these modules and the results give
lecturers a clear indication of where there
are gaps in students’ skills and knowledge.
Students who do not achieve a specified
grade in any one of the three sections of the
test are referred to an academic counsellor
who advises them about supplementary
support sessions in the relevant subjects.
Supplementary support sessions also use
self-assessment CAA which is rich in
feedback and directs student learning
through hints, tips and encouragement.
Students who join the supplementary
sessions can retake the diagnostic test
whenever they wish to check their progress.
LTSN Generic Centre – Assessment Series 2004
Using ‘virtual learning environments’
for assessment
Britain and Liber (1999) state that virtual
learning environments (VLEs) are, ‘Systems
which synthesise the functionality of
computer-mediated communications
software (email, bulletin boards etc) and
online methods of delivering course
materials’. Examples of VLEs in use in the
higher education sector include
‘Blackboard’, ‘WebCT’, ‘Learnwise’, ‘Lotus
Learning Space’ and ‘Top Class’, among
others. Surveys show that in the last two
years many institutions have adopted a VLE
to help support and enhance student
learning.
VLEs can help support and deliver
assessment. Details of assignment and
other assessment requirements,
examination timetables, regulations, and
sources of help and advice can all be made
available for students to access through a
VLE via the Internet or intranet. Email and
bulletin boards can be used to notify or
remind students of deadlines.
VLEs also offer the opportunity to distribute
global and individual feedback to a group of
students about a particular assessment.
This could be achieved by uploading a file
containing global feedback which might
usefully be linked to a discussion session in
which students query and discuss the
feedback and assignment. Feedback on
assessed work can be emailed to individual
students, inviting them to raise questions
(by email, phone, in person) as appropriate.
The online environment may well encourage
some, otherwise reticent, students to voice
their thoughts and concerns.
LTSN Generic Centre – Assessment Series 2004
Some VLEs include the facility to develop
basic objective test questions. These usually
comprise three or four question types
(multiple choice, multiple response, text
match). The functionality of the question
14
features is somewhat limited. However, they
do offer a valuable opportunity to rapidly
provide self-assessment questions which
can be embedded within learning materials,
allowing students to check their progress as
they are working in the VLE. They are a good
way of introducing CAA into the curriculum
and usually provide a simple interface for
question construction.
Beyond the use of VLE to provide
information, feedback and basic objective
test questions there are interesting
developments that indicate a movement
towards the wider use of computers in the
assessment process.
Peer assessment is one area where VLE can
be particularly powerful in allowing students
to engage in critical review, evaluation and
analysis. For example, a cohort of
Economics students is asked to research
and write a report exploring the effects of
expanding the European Union on the
Common Agricultural Policy. Once
completed, students upload their reports to
a secure folder on the VLE. The tutor
anonymises each report and assigns it a
number. Each student is then emailed with
the number of the report they are to assess
according to defined marking criteria and a
series of questions designed to give
feedback to the author. Once students have
completed the peer assessment, they post
13
 Assessment: A Briefing on Computer-assisted Assessment

their marking schedules back to the tutor
who may wish to moderate them prior to
giving them to the students. This activity
could be followed by an online discussion
session, where students comment on the
marks and feedback that they received,
their experience of the process of marking
peers’ work, and how they would approach
the report differently in future. Students
could also be asked to complete a short
objective test on how they approached both
writing the report and marking it. Marks
could be awarded to students for the report,
their peer assessment, and engagement in
the discussion or objective questions.
Robinson (1999) describes a system for
enabling online anonymous peer review
systems and the benefits these offer.
There is also interest in assessing the
contributions which students make to online
discussions. Approaches vary from
awarding a small amount of credit for
‘engaging’ at any level, to attempts to
measure the worth of individual’s
contributions. Indeed, discussions are often
richer learning experiences if they are
assessed, as students feel obliged to
contribute and to interact. Beware, however,
of over-assessing.
To date, no clear models for how this is best
approached have emerged, although the
design of tools which graphically display
levels of participation against defined
criteria should help to resolve some of the
issues associated with computer-mediated
communication and assessment (Kuminck
and Pilkington, 2001).
VLE offer many opportunities to engage
students in a range of formal and informal
assessment activities, and it is perhaps their
ability to integrate learning and assessment
which is particularly powerful. Although
bespoke assessment software offers
greater flexibility for objective style
assessments, VLE have much potential for
enhancing the assessment process,
particular for self- and peer assessment.

14
LTSN Generic Centre – Assessment Series 2004
Standards
Standards encompass content, procedural,
and in the case of CAA, technical areas. All
are equally important in any assessment
process. Question and test content should
be academically effective and processes to
manage this could include peer review of
assessment questions and tests. This is
most often done ‘in-house’ whereby
colleagues are asked to proof questions and
tests. Increasingly, as technology advances
and uptake is more widespread, content is
being made available through third parties in
the form of peer-reviewed question banks.
(Learning and Teaching Support Network
subject centres are beginning to produce
and offer these, as are some commercial
content providers.) As an institutional
question bank expands there is potential to
share material across traditionally disparate
departments where teaching content is
shared. If successful, this can be a huge
benefit to both new teaching staff such as
academic probationers and also established
academics who are keen to adapt and ‘re-
purpose’ questions.
New questions and tests should be
introduced gradually and performance
monitored. The majority of CAA systems
offer statistical item analyses and the use of
these can reveal weaknesses in questions
and errors which may have been neglected
in past paper-based versions. Consideration
of the item analysis will help to improve the
reliability and validity of tests, ensuring they
are an effective measure of student
LTSN Generic Centre – Assessment Series 2004
performance. Further information on item
analysis is provided in the section below on
14
‘Question banks’.
Standard procedures become increasingly
important as larger numbers of students and
staff become involved in CAA. It is possible
to bend the rules to accommodate
individual needs when dealing with a
handful of academics wishing to use CAA
but this can lead to problems where
significant numbers of staff and students
are involved. Adopting standard approaches
and procedures allows resources to be
monitored and adjusted to cope with
demand. In a traditional educational setting,
it should be recognised that peak times
occur throughout the academic year and
these should be borne in mind. Diagnostic
testing at the start of the academic year is
often the first peak. The following weeks see
an ebb in demand as lecture courses are
delivered and content is disseminated to
learners. Once enough content has been
delivered, the demand for testing becomes
greater and peaks again at examination
times.
The technical standards are unique to CAA
as a form of assessment. The language and
wealth of acronyms surrounding standards
initiatives can be confusing. However, the
key issue is that standards will help different
systems to exchange data. This means that,
if deployed successfully, it should be
possible to link student record systems,
15
 Assessment: A Briefing on Computer-assisted Assessment
virtual learning environments and CAA
systems, resulting in single point of entry to
such systems: one user name and
password is all that is ever needed. For
CAA, this could mean that module
registrations can be used to schedule
assessments, that the system could
automatically inform students of which tests
they have to take, and that once completed,
the score is uploaded and ratified.
The IMS Global Learning Consortium is the
most advanced of the standards bodies,
and was originally formed in 1997 as an
Educom project. Its membership is
extensive and a wide range of standards are
currently being worked on. One of these
standards is the Question and Test
Interoperability (QTI) specification. UK
higher education is represented by the
Centre for Educational Technology
Interoperability Standards on international
learning technology initiatives such as IMS.
The QTI defines question types but
separates content from presentation. This
allows questions which are meet this
standard to be imported and exported
between IMS compliant systems. This helps
to avoid situations where the design of the
software means that it is difficult to extract
data held in one system, for example a bank
16
of questions, and input it into another, for
example a better assessment software
package. Another advantage is that such a
standard will allow questions to be shared
across departments and institutions using
compliant software.
Web-based systems rely on web browser
applications which means that security
issues, such as invoking a secure
connection, should be addressed. Web
browsers do pose some additional
difficulties. We must guarantee that the
assessment looks and behaves as expected
across the different browsers and platforms
in use for although the web is ‘cross
platform’, uniformity is not always simple. It
may be necessary to define a specific
configuration to ensure standard
presentation. However, this can be
problematic as settings at the client
machine are hard to control. There are also
differences in Macintosh and PC versions
of the same product such as Internet
TM
Explorer . Reliance on Java TM and ‘plug-ins’
is common in web-delivered CAA and this
must also be enabled on the student
machine.
LTSN Generic Centre – Assessment Series 2004
Question banks
What are ‘question banks’?
Question banks are collections of questions,
each uniquely identified and stored to allow
the automated creation of tests to meet
predefined criteria. Each question has
associated descriptors which may define a
number of features of the question such as
academic level, topic, difficulty, and skill or
knowledge addressed by the question.
Questions can be contributed to (and
withdrawn from) the bank by authors and
users. Question banks are most commonly,
but not exclusively, used with objective
questions. They can be used to store
questions delivered by computer and paper
methods.
Why are they useful?
Question banks contribute to the validity and
reliability of the assessment process by
establishing a common language for
discussing curriculum goals and learning
outcomes. The questions relate directly to
individual tasks (skills and knowledge-
specific) which students are capable or
incapable of demonstrating. Questions are
graded according to difficulty on a scale
within the bank and it is possible to identify
the relative difficulty of particular tasks. This
provides a way to discuss possible learning
hierarchies and ways to better structure the
curriculum. Statistics collected each time a
question is used help to assure the quality of
the questions and assessments.
Question banks can also offer substantial
savings of time and energy over conventional
paper or standard computerised objective test
development. Traditionally assessments are
developed independently from previous
LTSN Generic Centre – Assessment Series 2004
assessment. With question banks, questions –
identified by specific descriptors – are drawn
14
from a bank according to desired criteria and
may be re-used where appropriate.
Question statistics
Statistical measures are used to determine the
characteristics of question and tests and
indicate the worth of each question for
inclusion in a bank. The question is the unit for
analysis rather than the whole assessment,
and each question is evaluated independently
to generate item statistics. For a detailed
description and discussion of the statistics of
question banking see Hambleton,
Swaminathan and Rodgers (1991) and
McAlpine (2002).
The statistics generated offer detailed
information about:
• the questions themselves,
• individual student performance,
• group performance,
• specific strengths and weaknesses, and
• curriculum design.
For example, in terms of the quality of a
question, statistics can identify incorrect
options within an individual question which
are rarely chosen by students. These options
are poor at discriminating between weaker
and stronger students and should be replaced
to strengthen the question. If a question is
answered incorrectly by 90% of a student
group, it would probably be determined to be
too difficult. However, it may also be found
that all the students select the same incorrect
response providing an indication of a common
misconception among the group.
Both students and lecturers can be informed
about the detailed performance of an
17
 Assessment: A Briefing on Computer-assisted Assessment
individual student. Specific strengths and
weaknesses can be identified at a micro level
and progress can be evaluated. At the level of
a course or programme, question statistics
can be usefully employed to help determine
future curriculum design and development.
Questions are usually piloted before being
included in a bank and question statistics
evaluated to weed out or improve any weak
questions prior to inclusion in the bank.
Initially, this can be time-consuming, but by
piloting a few new questions each time tests
are delivered, the bank grows and the quality
of questions is assured. Formative and
summative tests may result in different
statistics for the same questions because the
conditions for taking the test are likely to be
different. For example, multiple attempts may
be allowed in formative tests resulting in a
higher correct response rate.
Although such statistics could be generated
from the grades of traditional paper-based
assessments, the time taken to analyse marks
and overall grades deters most academics.
CAA software automates this process,
making a wealth of data instantly available.
Examples in practice
Computerised question banks are used
widely in the United States (Wainer, 2000) and
the literature discussing the construction and
application of computerised question banks
dates from the early 1980s. The LTSN
Economics Subject Centre is developing a
question bank for academics in the UK
involved in the teaching of Economics at
higher education level. (Further information
can be found at
http://www.economics.ltsn.ac.uk/qnbank.htm
18
– last accessed: 16/12/03). Assessment
materials are being collected, including
paper-based assignments, objective tests,
examination questions, problems and data
response. Questions are described by their
type, subject and academic level. The aim is
to save time and effort for academic staff
across the sector. However, issues of
institutional intellectual property rights have
hindered growth of the bank (Poulter, 2002).
The Electronic and Electrical Engineers
Assessment Network (E3AN), a ‘Fund for the
Development of Teaching and Learning’
project, is creating a bank of questions in a
range of undergraduate engineering topics.
Four UK institutions are collaborating to
develop a peer-reviewed question bank for
electronic and electrical engineering. The
project ‘sees the peer review of questions as
being essential in terms of achieving
ownership of any test banks across the
community. It also believes that the process is
essential in terms of assisting the
establishment of clear academic standards
associated with the test banks’ (White and
Davies, 2000).
Question banks can help to overcome some
of the upfront time investment required by
offering shared resources which can be re-
used and adapted. This may be particularly
useful where common subject matter is to be
assessed for large numbers. The
development and adherence to standards can
help with the technical construction of
question banks, That said, shared question
banks also raise a cluster of thorny issues
around the problems of ownership, copyright
and intellectual property (Bull and Daziel, in
press).
LTSN Generic Centre – Assessment Series 2004
Evaluating practice
Why is evaluation important?
It is important to evaluate educational
innovations, new and adapted methods of
assessment – and indeed established
methods of assessment – in order to judge
their impact on learning, efficiency and
effectiveness. The combination of
computers and assessment often
encourages questions about the
educational and operational effectiveness of
CAA, questions which are rarely asked of
existing assessment methods. This is
largely positive and can prompt a review
and amendment of assessment strategies
and methods overall.
What aspects of CAA should
be evaluated?
Evaluation of CAA can take many forms. For
example, it may be at an individual module
level or at a departmental or institutional
level. In other cases, it may focus on the
student or staff experience or on the effect
of CAA on learning and grades.
Evaluation studies need to be realistic and
to have clearly defined aims and objectives:
often these will relate to the motivations for
introducing CAA. It is highly unlikely that
every aspect of the implementation of CAA
can be evaluated as this would present a
time-consuming and exhaustive task. It is
therefore essential to clearly define the
aim(s) of introducing CAA in order to identify
whether these aims have been met. It is also
important to identify who the evaluation is
LTSN Generic Centre – Assessment Series 2004
for. For example, is it to inform lecturers
about whether students have benefited from
14
additional feedback on their learning? Or is
it to provide evidence for the head of
department that the use of CAA released
time for lecturers to undertake more small-
group teaching?
Those interested in the outcomes of CAA
evaluation might include:
• academics
• members of teaching and learning
committees
• educational technologists
• computer services staff
• examination officers
• senior managers
• quality assurance staff.
IIn order to be achievable, it may be
necessary to focus the evaluation very
specifically, for example, by gathering
qualitative data from a small group of
students about their experiences of CAA, or
by monitoring the volume and pattern of
student access to CAA. Where more time
and resources are available, broader
approaches maybe adopted, perhaps
concerned with gathering experiences,
perceptions and data concerning the use of
several types of CAA from staff and
students in order to provide a generic
measure of the success of its introduction in
a particular course. At an institutional level,
evaluations may be concerned with time
efficiencies, cost effectiveness and
operational practices.
19
 Assessment: A Briefing on Computer-assisted Assessment
Example of evaluations which may be
useful, depending on the type and purpose
of CAA, include:
• comparison of scores between CAA
and paper-based tests
• correlation between CAA tests and
other assessments methods within a
module
• student attitudes towards CAA (ease of
use, anxiety, relevance of content,
accessibility, perceived equity of
system)
• quality and speed of feedback to
students
• quality of questions
• effects of CAA on student study
behaviour
• staff attitudes towards CAA
(educational efficacy, ease of use,
anxiety, use in different educational
levels, perceived).
From McKenna and Bull (2000)
The literature provides examples of
evaluations of several types. Early
comparisons between paper and computer-
based assessments were followed by more
sophisticated triangulations of CAA results
with essay and examination scores
(Kniveton, 1996; Farthing and McPhee,
1999; Perkin, 1999). Students’ experience of
CAA are also reported in the literature
20
(Dalziel and Gazzard, 1999; O’Hare, 2001;
Ricketts and Wilks, (2002). Mulligan (1999)
reports on the impact of regular CAA on
students’ study behaviour. McGuire et al.,
(2002) evaluate the role of partial credit
within CAA and paper-based examinations
and the mode of delivery is also explored by
Fiddes et al., (2002). In the States, the
Educational Testing Service, responsible for
large-scale national testing, has conducted
in-depth and large-scale evaluative studies
of a range of issues relating to CAA.
What approaches to evaluation
should be taken?
Approaches to evaluating learning
technology generally have been developed
over the last twenty years and Oliver (1998)
summarises the difficulties associated with
using learning technology:
• ‘the effect of IT is not consistent across
subject or age groups (Hammond,
1994);
• high and low ability learners benefit
from different types of software (Atkins,
1993);
• lack of expertise amongst students or
teachers can create difficulties
(Hammond, 1994); and
• it can be extremely difficult even to
specify or measure “educational value”
(Mason, 1992)’.
LTSN Generic Centre – Assessment Series 2004
 Assessment: A Briefing on Computer-assisted Assessment
These challenges apply equally to the
evaluation of CAA, with the additional
problem that assessment can be both
culturally and politically sensitive – with
many issues of power and control from both
an academic and student perspective
(McKenna, 2001).
Evaluation can serve a number of purposes
and several roles are described in the
literature (Oliver 1998, Draper, et al. 1994).
Although the terminology may vary, the
principles appear to be commonly
represented.
Formative evaluation
Such evaluations seek to identify and
evaluate the use and impact of CAA with the
purpose of feeding evaluative data back into
the design of the system. Formative
evaluations commonly identify problems
with resources and propose solutions
(Cronbach, 1982).
Summative evaluation
Summative evaluations provide information
at a particular point in the implementation
process when a specified stage has been
reached. They are often focused on a
particular, well-defined question, seeking to
measure achievement of learning outcomes.
LTSN Generic Centre – Assessment Series 2004
Illuminative evaluation
These evaluations attempt to discover
factors and issues which are important to
certain groups of stakeholders rather than
14
measuring the success using standard
measures. It is common practice to seek to
identify and explain problems in adoption
using illuminative evaluation.
Integrative evaluation
Integrative evaluation takes a broad
approach to the adoption of learning
technology. The aim is to find ways of
improving teaching, learning and
assessment generally through the
introduction of technology. Illuminative and
integrative evaluations are often conducted
hand-in-hand, as the former frequently
provides the basis for the latter.
The importance of defining what you wish to
evaluate cannot be underestimated! Small
variations in wording or process can have
considerable impact. Asking, ‘What factors
influence how students perform at CAA?’
suggests an exploratory study, which seeks
to identify influences on performance.
However, asking ‘Which of the following
factors influences how students perform at
CAA ?’ suggests a comparative study,
possibly involving a controlled experiment.
21
 Assessment: A Briefing on Computer-assisted Assessment
The extent to which the evaluation is
exploratory is likely to determine the
methods used. Asking ‘what’ questions
indicates that few pre-conceptions of the
answers are held, and that the study is
largely exploratory in nature – in the
example above, the factors that will
influence learning need to be discovered in
the course of the study. When asking
‘which’ questions, the factors which
influence learning have already been
identified and the evaluation is conducted to
measure their influence on students. In this
situation the evaluation is less exploratory.
Qualitative methodologies, which ask open
questions – such as interviews,
observations, concept maps and focus
groups – are suited to explorative studies.
22
Checklists, experiments and quantitative
data collection techniques require a
framework for questions to be fixed in
advance.
There is a range of methodologies which
can be used to conduct evaluations and the
selection of different methods will depend
on the purpose of the evaluation and the
time and resources available. Cook (2002)
provides a good overview of the range of
methodologies which can be used to
evaluate CAA, while the Evaluating Learning
Technology toolkit (Oliver and Conole, 1998)
provides a practical guided methodology.
LTSN Generic Centre – Assessment Series 2004
Conclusion
The opportunities afforded by computer-
assisted assessment, both formatively and
summatively, are many. Students benefit
from timely and specific feedback on their
learning and a chance to practise skills and
monitor their own progress. Lecturers can
use CAA to complement and enhance
existing assessment methods, to broaden
the range of knowledge assessed, and even
extend the limitations of paper-based
methods. Rapid and detailed feedback to
staff can also help to guide the development
of the curriculum, while automated marking
releases time which can be spent on richer
forms of face-to-face interaction with
students. Assessment quality can be easily
assured and there is a level of consistency
and objectivity in marking which is not
possible for traditional forms of assessment.
However, time and effort is required in
designing pedagogically effective questions
and tests. A rethinking of assessment – and
often teaching and learning – is sometimes
prompted by the introduction of CAA, and
this can prove challenging as new ways of
working need to be found. The operational
and technical issues need careful planning
and resourcing and should not be under-
estimated, especially where summative
assessment is undertaken.
LTSN Generic Centre – Assessment Series 2004
Technology develops at a furious pace and
keeping up can seem daunting. However,
14
with appropriate support, both
pedagogically and technically it is possible
to use CAA to make assessment more
effective and more efficient. A variety of
opportunities are emerging which offer
exciting possibilities for CAA. Sharing
questions through banks, and standards
compliance, will ease the necessity for
reinventing questions and will open up
opportunities for re-use and creative
adaptation. Virtual learning environments
provide opportunities to support and
enhance assessment and together with
software developments and integration, are
beginning to encourage the assessment of
skills and abilities which would be not
attempted on paper.
CAA is still in the relatively early stages of
development. It requires creativity and often
challenges well-established traditions of
assessment but has powerful potential for
both educators and students.
23
 References
Ashton, H.S. and Beevers, C. (2002) Extending
the Flexibility in an Existing On-line
Assessment System, Proceedings of the 6th
Annual CAA Conference. Loughborough
University, Loughborough, UK, 9-10 July,
2002, 3-16.
Atkins, M. (1993) Evaluating interactive
technologies for learning. Journal of
Curriculum Studies, 25 (4), 333-342.
Biggs, J. (1999) Teaching for Quality Learning
at University, Society for Research into Higher
Education and Open University Press,
Buckingham, 175.
Britain, S. and Liber, O. (2000) A Framework for
Pedagogical Evaluation Of Virtual Learning
Evironments, JISC Technology Applications
Programme. Bristol: Joint Information Systems
Committee report.
Available from:
http://www.jtap.ac.uk/reports/htm/jtap-
041.html (Accessed:
January, 2004)
Bull, J. and Daziel, J. (2003) Assessing
Question Banks. In Littlejohn, A. (ed.) Reusing
Online Resources, London: Kogan Page.
Bull, J. and McKenna, C. (2001) Blueprint for
Computer-assisted Assessment. CAA Centre,
ISBN: 1-904020-00-3,
http://www.caacentre.ac.uk.
Cronbach, L. (1982) Issues in planning
evaluations. In Cronbach, L (ed.) Designing
evaluations of educational and social
programs. Jossey-Bass, San Francisco.
Cook, J. (2002) Evaluating Learning
Technology Resources, LTSN Generic Centre
and the Association for Learning Technology,
http://www.ltsn.ac.uk/genericcentre/index.asp
?id=17149.
Dalziel J. and Gazzard, S. (1999) Next
generation computer-assisted assessment
software: the design and implementation of
WebMCQ. In Proceedings of the 3rd Annual
CAA Conference,. Loughborough University,
Loughborough, UK, 16-17 June, 1999, 61-74.
Danson, M. et al. (2001) Large Scale
Implementation of Question Mark Perception
V2.5. In 5th International CAA Conference
Proceedings. Loughborough University. 2001.
24
Draper, S. et al. (1994) Observing and
Measuring the Performance of Educational
Technology. TILT report no.1, University of
Glasgow.
Farthing, D. and McPhee, D. (1999) Multiple
choice for honours-level students? A statistical
evaluation. In Danson, M. and Sherratt, R.
(eds), Proceedings of the 3rd Annual CAA
Conference, Loughborough.
Fiddes, D. J. et al. (2002) Are Mathematics
Exam Results affected by the Mode of
Delivery? Alt-J 10 61-69.
Hambelton, R. K., Swaminathan, H. and
Rogers, H. J. (1991) Fundamentals of Item
Response Theory, Sage Publications,
California.
Hammond, M. (1994) Measuring the impact of
IT on learning. Journal of Computer Assisted
Learning, 10, 251-260
Kniveton, G. H. (1996) A correlation analysis of
multiple-choice and essay assessment
measures’ Research in Education, 56.
Kukich, K (2000) Beyond Automated Essay
Scoring. IEEE Intelligent Systems, 22-27.
Kuminnek, P.A. and Pilkington, R.M.(2001)
Helping the Tutor Facilitate Debate to Improve
Literacy using CMC. In Proceedings of IEEE
International Conference on Advanced
Learning Technologies: Issues, Achievements
and Challenges, Madison, Wisconsin, 6-8
August, 261-266.
Leacock, C. and Chodorow, M. (2000)
Automated Scoring of Short Answer
Responses, Research Report, ETS
Technologies.
Mason, R. (1992) Methodologies for Evaluating
Applications of Computer Conferencing. PLUM
report no. 31. Open University, Milton Keynes.
McAlpine, M. (2002) Design Requirements of a
Data Bank. Bluepaper Number 3, CAACentre,
University of Luton,
http://www.caacentre.ac.uk.
McGuire, G.R., Youngson, M.A., Korabinski,
A.A. (2002) Partial Credit in Mathematics
Exams - a Comparison of Traditional and CAA
Exams. In Proceedings of the 6th International
Conference on Computer-assisted
Assessment, Loughborough University,
Loughborough, UK, 9-10 July, 2002. 223 -230.
LTSN Generic Centre – Assessment Series 2004
 Assessment: A Briefing on Computer-assisted Assessment
McKenna, C. and Bull, J. (2000) Quality
assurance of computer-assisted assessment:
practical and strategic issues, in Quality
Assurance in Education, 8(1).
McKenna, C. (2001) Who’s in control?
Considering issues of power and control
associated with the use of CAA: a discussion
session. In Proceedings of the 5th International
Conference on Computer-assisted
Assessment, Loughborough University,
Loughborough, UK, 2- 3 July, 2001, 305-308.
Mulligan, B. (1999) Pilot study on the impact of
frequent computerized assessment on student
work rates, In Danson, M. and Sherratt, R.
(eds), Proceedings of the 3rd Annual CAA
Conference, Loughborough.
O’Hare, D. (2001) Student Views of Formative
and Summative CAA. In Proceedings of the 5th
International Conference on Computer-
assisted Assessment, Loughborough
University, Loughborough, UK 2-3 July, 2001,
371-386.
Oliver, M., (1999) A framework for evaluating
the use of educational technology. Evaluation
of Learning Technologies project, University of
North London
Oliver, M. (1998) Evaluating Learning
Technologies: a toolkit for practioners. Active
Learning, 8, 3-8. See also:
http://www.unl.ac.uk/tltc/elt/toolkit.pdf
Perkin, M. (1999) Validating formative and
summative assessment. In Brown, S., Bull, J.
and Race, P. (eds.), Computer-Assisted
Assessment in Higher Education, Kogan Page,
London.
Poulter, M. (2002) Personal Communication
with Dr Joanna Bull, April 2002.
Ricketts, C. and Wilks, S. (2002) What Factors
Affect Student Opinions of Computer-assisted
Assessment? In Proceedings of the 6th
International Conference on Computer-
assisted Assessment, Loughborough
University, Loughborough, UK, 9-10 July,
2002, 307-316.
Robinson, J. (1999) Computer-assisted Peer
Review. In Brown, S., Bull, J. and Race, P
(eds.), Computer-Assisted Assessment in
Higher Education, London: Kogan Page.
Wainer, H. (2000) Computerised Adaptive
Testing: A primer, Lawrence Erblaum
Associates, New Jersey.
White, S. and Davies, H. (2000) Creating Large-
scale test banks: a briefing for participative
discussion of issues and agendas. In 4th
International Computer-assisted Assessment
conference proceedings, University of
Loughborough, UK June 2000
Available from:
http://www.caaconference.com
(Accessed: January 2004)
Further Reading
and Resources
Computer-assisted Assessment Centre
website - http://www.caacentre.ac.uk
(Accessed: January 2004)
CASTLE Assessment System -
http://www.le.ac.uk/castle/
(Accessed: January 2004)
TRIADS Assessment System -
http://www.derby.ac.uk/assess/talk/
quicdemo.html
(Accessed: January 2004)
Centre for Educational Technology
Interoperability Standards -
http://www.cetis.ac.uk
(Accessed: January 2004)
Educational Testing Service -
http://www.ets.org/research/newpubs.html
(Accessed: January 2004)
Economics Learning and Teaching Support
Network Question Banks
http://www.economics.ltsn.ac.uk/qnbank.htm
(Accessed: January 2004)
Engineering Assessment Network (2002)
http://www.ecs.soton.ac.uk/E3AN/
(Accessed: January 2004)
Mostert, E., Knoetze, J. G. (2001)
Implementing an Electronic Portfolio
Assessment Strategy: Multiple Pathways for
Diverse Learners. In Proceedings of the 5th
International Conference on Computer-
assisted Assessment, Loughborough
University, Loughborough, UK 2- 3 July, 2001,
349-357.
The Learning and
Teaching Support Network
Generic Centre
The Learning and Teaching Support Network (LTSN) The LTSN Generic Centre
is a network of 24 Subject Centres, based in higher Assessment Series Guides for:
education institutions throughout the UK, and a Senior Managers

Generic Centre, based in York, offering generic Heads of Department

information and expertise on learning and teaching Lecturers
issues that cross subject boundaries. It aims to
Students
promote high quality learning and teaching through
the development and transfer of good practice in all
Briefings:
subject disciplines, and to provide a ‘one-stop shop’
Assessment issues arising from key skills
of learning and teaching resources for the HE
Assessment of portfolios
community.
Key concepts: formative and summative,
criterion and norm-referenced assessment
The Generic Centre, in partnership with other
organisations, will broker information and knowledge Assessing disabled students
to facilitate a more co-ordinated approach to Self, peer and group assessment
enhancing learning and teaching. It will:
Plagiarism
• work with the Subject Centres to maximise the
Work-based learning
potential of the network;
• work in partnership to identify and respond to key Assessment of large groups

priorities within the HE community; Problem-based Learning

• facilitate access to the development of Computer-assisted Assessment
information, expertise and resources to develop
Implementing an Institutional Assessment
new understandings about learning and teaching.
Strategy

Published by
Learning and Teaching Support Network (LTSN) LTSN Generic Centre
The Network Centre, Innovation Close,
Assessment
York Science Park, Heslington York YO10 5ZF
Series No 14
For more information, contact the
Generic Centre at the above address or A Briefing on Computer-assisted Assessment
ISBN 1-904190-53-7
Tel: 01904 754555 Fax: 01904 754599
£75.00 (for full set)
Email: gcenquiries@ltsn.ac.uk
www.ltsn.ac.uk/genericcentre

© LTSN, January 2004

All rights reserved. Apart from any fair dealing for the purposes of research or private study, criticism or review, no part of this
publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, graphic, electronic,
mechanical, photocopying, recording, taping or otherwise, without the prior permission in writing of the publishers.