Lifelong Learning Network Peer Evaluation report

Higher York, Kent & Medway LLN, Sussex Learning Network

Report compiled January May 2008

Contents

Executive summary

Context

A comment about structure and process

Background and observations

Overview of approaches adopted

12

Meeting of LLN Chairs

17

Impact

20

Evaluating the Evaluation

20

Appendices
Appendix A Meeting preparation (staff)

21

Appendix B Meeting preparation (questions)

22

For supplementary information please see the following:

Individual business cases for Lifelong Learning Network funding
Individual LLN annual reports to HEFCE
LLN Key facts
Interim Evaluation of Lifelong Learning Networks (CHERI), January 2008

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

Peer Evaluation: Executive Summary

Executive Summary
Organisation & Governance
Indicator of success 1: All 3 LLNs have demonstrated the flexibility to
accommodate key personnel (core team and Board/Exec level) changes.
Governance structures have not necessarily followed business plan.
Comment:
Structures have demonstrated capability to manage projects (and also
flexibility of agenda) but it is too soon to tell if these structures will be
appropriate for sustainability.
Team autonomy was agreed to be important factor. Whilst teams are in
place to manage the delivery, they must feel able to drive some aspects (in
addition to Boards). It is difficult to balance the role of the core team and
LLN partners as driver.
The fluidity of the partnership approach has meant interest from new
partners to join has been accommodated in most cases.
Indicator of success 2: FE and HE partners using LLN to carry out other
activity
Comment:
The balance of power between partners is also significant and this can
determine the effectiveness of the LLN as a platform for other agendas.
Access to key people has been refreshing although its recognised that
decision making at Board level doesnt necessarily filter down. Some
processes are better if instigated top-down, some bottom-up.
Stakeholders/Partnership & Buy-in
Indicator of success 3: Additional Student Numbers (ASNs) have been useful
for achieving initial buy-in for all 3 LLNs.
Comment
ASNs have been helpful in opening up planning discussions and their
flexibility of use has been useful especially for FEC partners. However,
ASNs cannot be a basis for sustainability and must be managed in a
pragmatic way in the future.
Indicator of success 4: whilst the 3 LLNs have taken different approaches to
using a distinct identity, the LLN brand has helped achieve buy-in both
internally and externally.

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

Comment
The brand has been important to LLNs in different ways (eg. learner
facing, staff development, engagement with local authority).
Progression
Indicator of success 5: All 3 LLNs have a large number of progression
agreements, the concept of which has been a useful tool for change.
Comment:
Progression agreements are not ends in themselves but a very useful tool
to challenge thinking and introduce process (actual number of progression
agreements does not necessarily equate to level of changes within
infrastructure as referred to above).
Progression accords and agreements should be seen as one part of
progression, as with curriculum.
Looking forward, the further development of the PA as a vehicle will need
managing, as will the maintenance of the relationships built.
LLNs have little experience of working across borders with different models
so this may be a challenge for the future.
Indicator of success 6: The focus of activity has been on the development of a
progression infrastructure as opposed to documentation.
Comment:
Credit and progression services are central to the culture change driven by
LLNs.
Curriculum
Indicator of success 7: New opportunities have been brought to LLN
partners.
Comment:

Demonstrating cause and effect with regards to new provision has been
difficult for LLNs, given the timescales involved in developing new
provision.

LLNs have been involved in gap analysis and opportunity spotting

(particularly in the area of work based learning and the development of
existing provision).

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

LLNs were responsible for setting their own priorities including sectors.
Sectors have been a useful starting focus for LLNs but can also be limiting

Indicator of success 8: 2 LLNs have agreement on use of credit which is

being seen as central to engaging work based learners.
Comment
The issue of credit has generally taken second place behind the
development of progression agreements. LLNs are now pushing this
agenda in line with national developments.

An explicit mechanism for monitoring credit recognition could be built in as

LLNs go forward.

Information, Advice & Guidance (IAG)

Indicator of success 9: Staff development for existing practitioners is high on
the agenda and needs to remain so. Capacity building is central to the
development of IAG services.
Comment:
LLNs are not only about clarifying the existing landscape, but also they are
creating new things (such as progression agreements) which need IAG to
sit alongside them.
There are different views across the 3 LLNs about offered a 1:1 service.
Relationships with AimHigher tend to work well although 2 out of the 3
LLNs felt they were increasingly overlapping when considering
progression from Diplomas.
Student Engagement
Indicator of success 10: Students are engaged at varying levels across all 3
LLNs. The work with students is very labour intensive but is generally very
valuable to partners.
Comment:
Capturing the student voice maybe problematic as LLNs move forward to a
stronger focus on employer lead provision. Engagement has been with a
variety of types of student so work based learners have not been
neglected as such (just not the focus).
Tracking
Indicator of success 11: Tracking is regarded as successful where the data
has been useful. The main areas are therefore monitoring cause and effect

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

(LLN teams/feed into development process), measurement of impact of LLN

(partners), and measurement of achievements (HEFCE).
Comment:
There have been difficulties in demonstrating linkages within the short
timescale (especially effect on retention and achievement).
Tracking ASN learners is not enough. Further in-depth work is most
valuable (such as samples studies conducted by Sussex Learning
Network).

Recommendations:
Continue to demonstrate flexibility in response to new policy initiatives; promote to
partners and external stakeholders.
Encourage role of staff development, specifically within the area of IAG, within LLN
and increase activity if capacity is available.
Explore ways in which to make use of existing data for tracking purposes. This
may include linking with LSC (L3 learners). Consider alternate ways to measure
effectiveness of progression agreements, possibly in collaboration with other LLNs
(regionally of nationally).
Share resources and approaches with other LLNs, to avoid duplication, develop
consistency where appropriate, and to maximise the impact of the resource
invested in LLNs nationally.
The further development of the progression agreement as a vehicle will need
managing, as will the maintenance of the relationships built. An explicit
mechanism for monitoring credit recognition could be built in to any plans for
sustainability.
Learn from the various student engagement projects but look to include those not
already in the system, particularly people who are in the workplace.

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

Peer Evaluation: Full Report

Context
As part of the HEFCE evaluation strategy for Lifelong Learning Networks (LLNs),
individual LLNs have been asked to take part in an evaluation exercise with
colleagues from other LLNs. It was the intention of the LLNs involved to adopt a
critical friend approach in order to evaluate each individual network, and highlight
areas of strength and weakness, areas of good practice, issues that have arisen and
lessons learnt.
Other LLNs were undertaking two-way evaluations and the tripartite exercise in this
case was unusual. It brought together three LLNs of varying sizes, locations and
stages of development - Higher York LLN, Kent and Medway LLN and Sussex
Learning Network. The unique opportunity provided by involving three networks has
provided an interesting platform for review and in some part has enabled a more
objective outlook for the whole exercise.
Structure and process
It was felt that an approach which did not attempt to encompass all aspects of LLN
activity was favourable in order to focus effort and maximise the likelihood of results
which would be both meaningful and useful to all participants. The LLNs involved
agreed to spend two days on the exercise and put together a list of areas that would
be covered and the individuals that would be involved (App 1). It was agreed that
interviews/meetings with the key individuals would take place and in order to focus
discussion, a series of questions were drawn up under each heading (App 2). This
would then be followed by a meeting of the three Chairs of the LLN
Board/Management groups. Subsequent discussion highlighted a need for a further
meeting where peers met to agree on the main indicators of success highlighted in
the Executive Summary.
During preliminary discussion, the three teams decided on which key areas they
would focus on, these were made up of the 7 areas recommended through the
National Forum (Organisation and Governance; Partnership; Information, Advice and
Guidance (IAG); Student Voice; Progression; Curriculum; Tracking/re-engaging with
the addition of two other areas: Sustainability and Employer Engagement. However
discussions did not allow for the latter two areas to be commented on separately and
reference to these is made throughout.
This paper attempts to tease out some of the core themes across the 3 networks,
comparing some of the approaches followed, the impacts of activity, and highlighting
areas of joint concern. Recommendations set out below are primarily for the three
networks involved but could also apply to other LLNs depending upon their stage of
development.
As this document is primarily for use within the three networks involved, there is an
assumption of some background knowledge. Further information on the three LLNs
can be found in the documents listed on page 2.

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

Main findings and observations

1. Organisation and Governance
The initial focus of the three networks namely the city (York); Kent and Medway
and the county of Sussex are important factors in shaping their respective
approaches their early activity and should be one measure of their impact.
All three LLNs had high-level ownership form the outset, with VCs and equivalents
being involved in the development of the business case.
In each of the three LLNs the core teams have been identified as the main change
agents but each has adopted a different approach, influenced by the pre-existing
partnership context for their work.
It is difficult to identify one particular organisational model as the most effective as
the three LLNs have adopted different models which are working in different ways
each one having advantages and disadvantages.
o Centralised model no staff at partner institutions so FHEI involvement is
therefore essential and requires a lot of ground work by core team (time
consuming). Aims to embed LLN activity from beginning.
o Distributed model/s possibly more difficult to manage but goes some
way in ensuring partner buy-in. Some problems with balance between
lead institution as employer as teams can then be thought of as
belonging to single institution rather than LLN. Can result in differing
levels of delivery between groups as there is reduced input from core
team.
Governance is about balancing accountability and control with the need to ensure
that ways are found to engage all stakeholders in decision making. The core teams
are the main change agents and have been allowed to get on and deliver.
Engagement means finding ways of working to meet the different needs and
ambitions of partners. The motivation and ambition of stakeholders inevitably leads
to them valuing different aspects of what the LLN can deliver.
The role of the lead institution is differently interpreted by the various LLNs. It is a
HEFCE used device for ensuring accountability and not one that need determine
the way LLNs are governed or managed.
Managing the transition from the approval of a business case to the stage of
becoming fully operational has led to under spends that in turn lead to delivery
being a two year rather than three year exercise.
Two LLNs have moved to a position of commissioning work rather than inviting
partners to go through a bidding process as this was seen as a barrier for partners.
Collaboration between Widening Participation practitioners has been very positive
for one LLN, although within others it is recognised that LLN activity overlaps
institutional and/or AimHigher territory and can sometimes lead to complications.
Funding - the distribution of LLN funds is approached differently in each LLN.
Each LLN directs a substantial part of their funding to cover staff costs with the
remaining funds being available to partner institutions (these development funds
vary as a proportion of the total budget). All three LLNs have operated a bidding
process for the allocation of development funding, however at least one LLN is
moving away from this to more of a commissioning approach.
2. Partnership and Buy-in
The defining characteristics of the LLNs are shaped to a great extent by the nature
and history of the partnerships on which they are based.

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

 Most LLNs are led by HEIs, however this is not the case for Higher York which
contains two HE and two FE partners and so the balance of power is shared
equally which produces an interesting dynamic.
We cannot underestimate the impact of key individuals or stakeholders in shaping
and to some extent determining the impact of the LLN
One measurement of buy-in appears to be the extent to which the LLNs featured in
institutional strategic plans. For one LLN, this is variable across partners
documentation, for another LLN this is very positive and the LLN features
significantly in partners plans.
LLNs have demonstrated flexibility in being able to engage and incorporate new
stakeholders and sectors, respond to new policy initiatives and present the
collective voice of HE in different fora. The external environment is not fixed even
in the very short term and many stakeholders are in a constant state of flux.
Collaborative work with Aimhigher was common and all LLNs recognised the need
for a dialogue; existing partnerships created by AH helpful. Joint projects were
being pursued and in one case, a jointly funded post was being discussed.
All had initial challenges linking with Train to Gain although running training events
for brokers was seen as one positive contribution.
3. Progression Agreements
Overall, networks were of the same mindset that the PA was a tool and that a long
term culture change was the desired outcome. It was also felt that the existence of
agreements would go some way in encouraging FE/HE dialogue post the current
funding period and some structures had already been put in place to ensure this.
Considerable progress has been made in all three networks and all have
agreements in place.
A range of marketing strategies are being considered but there is also a need to be
cautious as the shelf-life of a PA is limited; PA web resources are being developed
and there is evidence of particular good practice from one network .
LLNs have found this a time-consuming process and the debate around offering
learners a guarantee has yet to be settled within some LLNs. Getting the
curriculum staff on board has been difficult but now there is a greater
understanding through improved dialogue and colleagues are positive about what
PAs can provide. ASNs have not been fully utilised here due to the short
timescales involved.
Staff development in this area is essential to bring about culture change and keep
progression in the mindset of partner staff.
The development of PAs for employees/employers is at varying levels however,
one LLN is currently working on agreements between a number of local employers
and colleges / universities, relating to progression onto work-based foundation
degrees. Particular successes are in areas of Sport & Leisure, and Travel, Tourism
& Retail.
Generally, the need to monitor learners using PAs is viewed as significant but
thinking is at varying stages regarding how information will be captured. In one
case, it will be the responsibility of the LLN, in another it will be the responsibility of
the institution involved to identify learners on a PA route.
4. Curriculum

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

 Each LLN had clearly defined sectors which were named in their business cases.
This has allowed LLNs to focus their activity and has been beneficial in that it has
been manageable as it has enabled prioritisation, but has also meant that their
approach to some areas has been limited. The need to clearly define boundaries
has been essential.
Curriculum teams have been involved in different areas of activity ranging form
the development of PAs, to curriculum development and contributing to IAG
materials.
Engagement with employers has been at the sector level and this appears to be
the preferred method (as opposed to a blanket approach from the LLN). This has
also enabled links with SSCs.
Curriculum development has been encouraged by LLNs which have provided a
platform for joint planning, background information and statistics and in some
cases additional student numbers.
Some LLNs have developed new courses/modules specifically for types of
learners rather than being linked to specific curriculum areas.
The three LLNs had used different models to distribute ASNs to partners and there
was evidence that model two had more potential to influence curriculum
development as it was a more flexible approach.
Evidence of a direct link between LLN sponsored curriculum planning/
development; ASNs and recruitment of learners from target audiences attributable
to IAG activity will be difficult to find in the three year period.
Networks have experienced some problems with marketing of courses seen to be
LLN programmes and there is debate about the institution or the LLN should be
responsible for this.
Work around Credit Frameworks varies across LLNs. In one network, good progress
has been made in agreeing a joint credit framework across all network institutions.
5. IAG
Approach taken varied according to what was already available to learners. The
three LLNs are targeting different categories of learners via their IAG e.g. existing
students, those on particular pathways or predominantly those in work or returning
to work with no assumed knowledge of HE .
There is a lack of evidence about the most effective role the LLNs can play in IAG
e.g. from providing a direct service to enquirers to using the web to support
potential learners and intermediaries. There appeared to be a consensus
regarding the LLN constituency for the purposes of IAG there was a focus on
adults in or entering work as these were the group for whom there was little IAG
for HE level study.
There is a heavy emphasis on staff development in this area both for the LLN
staff themselves and for staff within partner institutions.
Moving towards models which incorporate IAG for employers providing a onestop shop.
6. Student Engagement

All LLNs saw engagement with students as valuable, and a range of approaches
have been used from focus groups, to working with Students Unions and specific
student led projects (involvement in areas of development such as web,
informational materials etc.).

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

10

 Value attached to supporting student achievement either via specific initiatives or

awards.
There is a desire to capture voice of non-traditional students and work has been
done with vocational learners, mature learners and those embarking on CPD
programmes; also looking at those not currently in the system in order to identify
and highlight possible barriers to institutions.

7. Tracking/re-engaging
All three LLNs have received a large number of ASNs but view this as only one
measurement of success. The issue of tracking LLN students linked to ASNs is
that working within the timescales, and with existing data collection points that
HEIs and FECs adhere to, as well as the differences in reporting requirements for
HEIs and directly funded FECs, there are limits to the effectiveness of tracking
processes.
There is also a need to be aware of the limits of tracking 'flagged' learners as a
means of measuring and evidencing impact: these methods do not pick up
learners who have been positively affected by any of the range of LLN activities
(e.g. broader IAG provision) that cannot be tracked in this way. The extent to which
learners flagged as LLN learners have come to HE as a result of LLN activity will
also vary - in some cases these will be learners on a course supported via LLN
ASNs - there may be no other connection between the course and LLN activity.
The learners may not be local, and it is impossible to say whether they would have
come to HE without the LLN's activity.
Additional student numbers have had a very different impact on the formative
development of LLNs. There is no clear evidence about which model works best.
The unanticipated consequences of monitoring and tracking LLN students do place
additional burdens on institutions.
One LLN has approached the LSC for help with data and by matching ILRs with
enrolment data from HEIs its possible to get a valuable insight into local
progression patterns of Level 3 learners.
8. General Observations
There was a lack of a sharp or well defined focus on widening participation. LLNs
were not linked to institutional targets or strategies. WP goals have become implicit
rather than explicit. Saw the relationship with Aimhigher as one way of contributing
to this agenda.
All resolved work with employers more but the field is likely to become even more
crowded with the new emphasis on employer engagement for HE. We can
anticipate more SDF projects will be funded that could lead to confusion with the
role of LLN.
The idea of creating the one-stop shop for employers finding out about HE is a
favoured option but this is complicated by the existence of Business Managers in
many HEIs who see it as their role.
Staff development is viewed as evidence of engagement of practitioners in the LLN
and a way of building networks; also viewed as supporting the new staff working
for the LLN. There is debate about whether all this provision should be recognised
and built into a CPD model for institutions to build in to their current offer for
employees.

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

11

Approaches
A summary of the main approaches within each area is shown in the tables below.

CN/E:/HY/Peer Eval/Peer Evaluation Report DRAFT 8

12

Table 1. LLN Peer Evaluation: Overview of approaches adopted

Approach

Benefits of approach?/ Have any

changes been necessary?

Examples of Good
Practice

Comment

Element reviewed: Organisation

Higher York

Structure: Board (FE, HE & city

council), Executive group,
curriculum + working groups.

Having middle level group (Executive)

has been key both in terms of driving
agenda and delivering objectives.

Shared post funded by

neighbouring LLN recently
appointed

Retention of staff within

core team has been poor.

Kent &
Medway

Structure: Exec Board,

Management group, Operational
Managers Group, Co-ordinators,
Curric groups

Co-ordinators role to Champion the

LLN within their institution has provided
good attendance at events and positive
feedback from meetings.

Co-ordinators benefited
from staff development

Lengthy recruitment
process.

Structure: Board (FE, HE &

economy) curriculum teams and
interest groups. Core team is policy
driver.
Two staged approach HE then
engagement of FECs
Element reviewed: Partnership & buy-in

Variety of membership on Board meant

a lot of initial work to ensure full
engagement.

HE in FE group

6/7 months to get team in

place.

Higher York

Two stage process - have

concentrated initially on embedding
within partner institutions.
Pre-existing partnerships; spent first
year ensuring ownership; building
on current relationships held by
member institutions

Sussex
Learning
Network

Range of stakeholders involved;

brand of SLN viewed as important
both internally and externally;
decision not to engage directly with
employers but work with employer
reps.

Curriculum groups have

provided platform for
engagement with local
employers (see below).
Good pre-existing
partnership working has
needed to be built on in
developing the LLN agenda
and ensuring continuing
buy-in.
links with FDF now regional
(with other SE LLNs)

Buy-in aided by fact that

HY was existing
partnership.

Kent &
Medway

Has been deliberately inward looking to

date this could have impacted upon
some relationships (eg. with local
council).
Good Commitment of institutions that
have found different ways to engage
with the LLN; strong links with many
external stakeholders

Sussex
Learning
Network

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

SLN see as holding brokering role;

hope that brand with provide incentive
for collaborative relationships to
continue

Levels of engagement vary

- some stakeholders appear
less engaged currently (as
they have only recently
become involved);
important to view roles of

13

Table 1. LLN Peer Evaluation: Overview of approaches adopted

Approach

Benefits of approach?/ Have any

changes been necessary?

Examples of Good
Practice

Comment

external stakeholders
individually; limited
engagement with SSCs.

Element reviewed: Progression

Higher York
Looking at guaranteed
consideration culture change
more important than PA docs.
Kent &
Medway
Sussex
Learning
Network

Programme-programme
agreements, sector-wide and
institutional agreement
Viewed as tool for culture change
which have forced staff to think
differently and course content and
entry requirements. Limited
development of PAs to those which
would be meaningful and have
impact

Element reviewed: Curriculum

Approach
Higher York

HY provided platform for

collaborative development (inc.
dialogue with employers where
appropriate); ASNs useful as
incentive for development. Each
group has different priority.

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

Beginning to change culture within

institutions

Has been lengthy process

not helped by challenges of
offering guarantees (ongoing).

FECs see as aid to boost internal

progression; key benefit has been staff
interaction culture change
Focus on meaningful progression
routes.

Curriculum Groups being

established with
progression overview.
Progression Management
Group will continue to meet
post-SLN

Benefits of approach?/ Have any

changes been necessary?
ASNs have enabled some curriculum
development but timescales very tight
to be innovative. Flexibility of focus has
been helpful.

Examples of Good
Practice

ASNs not incentive for

development of PAs

Comment
Ongoing issues over whose
responsibility it is to market
of specific LLN
programmes

14

Table 1. LLN Peer Evaluation: Overview of approaches adopted

Approach

Kent &
Medway

A wide range of curriculum

development mostly based on credit
framework including CPD, short
courses, bridging and Work Based
Learning

Sussex
Learning
Network

Cross curricular activity and shared

learning is enabled via the
Curriculum Team; awareness
raising of FDs has been focus.

Element reviewed: IAG

Higher York
Currently offer 1:1 service

Kent &
Medway

Sussex
Learning
Network

Key focus on adult learners in

workplace; Co-ordinators received
formal IAG training to supplement
current institutional services (not
replace); working closely with other
providers/agencies (AH, Next Step)
Focus was on developing IAG
resources and promotion; staff
development/ awareness raising
activity. Now moving to 1:1 service
(This comment relates to addition of
facility on website to email the IAG
team if web info not sufficient.)

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

Benefits of approach?/ Have any

changes been necessary?

Examples of Good
Practice

Comment

Good employer links within

areas of Built Environment,
Manufacturing &
Engineering and social and
health care

Most employer engagement

has been through sectors

Difficult to keep track of activities due to

structure questions over value for
money in some groups

Common approach to Level

0 unsuccessful.

1:1 not sustainable planning to move

to model based on staff training/
upskilling. Need to recognise that more
than one approach is necessary issues need to be identified within each
institution and addressed individually as
practices differ.
Regional model being considered

Also attempting to coordinate approaches to

schools.

Has sustainability in mind

Range of joint
AimHigher/LLN projects.

Some materials developed

in partnership with
Aimhigher; development of
on-line PDP facility
underway.

Mystery Shopper activity

re FD enquiries has had
varying impact.

15

Table 1. LLN Peer Evaluation: Overview of approaches adopted

Approach

Benefits of approach?/ Have any

changes been necessary?

Examples of Good
Practice

Comment

Element reviewed: Student Engagement

Higher York

Initial stages focused on

collaborative SU activity now
moving towards an approach which
will encompass non-trad students in
LLN
Kent &
The LLN has challenging learner
Medway
engagement targets to meet. Also
focus groups (existing students,
mature, CPD), student
competitions, early leavers project
Sussex
Programmed learning experiences
Learning
involving students in LLN; website
Network
which includes vocational learners
forum (blog)
Element reviewed: Data tracking

Enabled joint working within SUs but

hasnt yet provided student voice.

Range of opportunities to enable

engagement

Blog currently under-utilised

Higher York

Will primarily be through ASNs, but

also collect IAG data and partner
institutions progression stats.

Achievement of ASNs not necessarily

linked with LLN activity so more needs
to be done in the area to capture work
of LLN.

Kent &
Medway

LSC agreement to being pursued to

provide ILR data of all L3 learners
then match with HEI enrolment
data; tracking IAG interventions and
obtaining HESA reports
Tracking just under 1000 ASNs;
commissioning study to review data
from HESA; qualitative report on
SLN learners

Less onerous on LLN staff - reliant on

third parties such as LSC and HESA

The real challenge is

capturing evidence of the
breadth of impact that the
LLN achieves, rather than
just those elements which
are easily measured.
Baseline and impact data
gathered through LSC and
HESA.

Sussex
Learning
Network

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

Valuable feedback for institutions

Working with LSC for L3

learner details

Some difficulties moving

from Model 2 Model 1
ASNs which has
complicated data collection

16

Meeting of Chairs 6 February 2008

Present: Julian Crampton, Colin Mellors, Margaret Noble
Preamble
1. The Chairs welcomed the exercise and saw considerable value in shared
critical learning. They noted that the three-way dialogue had already been
useful and would encourage further co-operation where this would be helpful.
Self-help links at practitioner level were seen as especially useful.
2. Other LLNs were undertaking two-way evaluations and the tripartite exercise
in this case was an unusual. It brought together LLNs with differences in
several respects size, funding level, history, mission, age etc and this
made for a rich mix of experience. Despite these differences there were
some strong common themes both in terms of factors that influenced
progress to date and likely plans going forward.
3. One strong common theme was the importance of a high-quality core team
(with appropriate translational/brokering skills) and the three Chairs felt lucky
in this respect. They agreed to pass this view to the teams as part of the
evaluation process.
4. Also, all three Chairs were firmly committed to the initiative and saw
significant opportunities for using the partnership platforms to take forward
other relevant institutional/government agendas. (see below)
Key Issues for Chairs
5. Several themes emerged from the perspective of the Chairs.
Partnerships
There was real value in effective partnerships and recognition that these
take time to form if they are to be more than token groupings in response
to a particular (and, perhaps, temporary) HEFCE initiative. All three felt
that their LLNs provided the basis for durable partnerships that,
appropriately shaped, could add future value to joint working on relevant
issues.
Timing
The three-year time horizon was difficult. Apart from the HR issues (time
taken to recruit staff; likely resignations in latter stages), it takes time to
establish the partnership and even longer to achieve appropriate culture
shift as well as to demonstrate impact/outcomes. Five years would have
been much more appropriate.
Value to Partners
It was felt that showing value to partner institutions was critical
contribution to mission and/or bottom line was important if the
partnerships were to survive. Although all partners need to benefit in
some way of other, the nature of such benefits will vary widely from one

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

17

institution to another. This should not be viewed as a problem but does

call for a fairly sophisticated approach to how institutional value is
identified and the various internal discussions are brokered.
There was also an interesting discussion about what happens when one
partner goes it alone on a particular issue. Again this was not viewed as
particularly problematic so long as all partners have the chance to
participate it is inevitable that some activities will appeal/benefit more than
others. LLN partnerships are a means of enabling but not straightjacketing.
Communicating
For reasons outlined above, it is important that LLN teams capture and
describe (in a form that will be easily accessible) key
values/outcomes/benefits of their work. Partner institutions need to know
how LLNs are adding value if they are to continue their support for them
and, possibly, contribute to future funding. LLNs could, for example,
create more links with Knowledge Transfer work of Universities. Such key
messages will also be important to positioning LLNs nationally.
There was a short discussion about how national LLN messages are put
across and the importance of demonstrating value rather than seeming to
take the grant dependency approach that sometimes colours aspects of
WP activity.
Impact/outcomes
HEFCE will be looking for hard evidence of outcomes. Some will be
about process new curricula, provision, credit, progression
arrangements etc but they will also be looking for outcomes in terms of
numbers progressing. This will be difficult both in terms of time (three
years is a short period to put in place the processes and to recruit and
progress learners) and in terms of methodology (tracking systems are at
an early stage of development and there is the difficulty of identifying
causal factors that affect recruitment and progression patterns).
Nevertheless these issues will need to be addressed. The establishment
of baseline data, even if at quite a generic level, would be important in
estimating distance travelled.
Also, the three Chairs voiced the strong view that a key outcome was
culture shift and styles of working. This was difficult to measure but might
prove one of the most important legacies of LLNs.
Sustainability
What are we seeking to sustain a staff team, a set of processes, a new
approach/style of working? We need to avoid the simple of approach of
sustainability for its own sake. More important was the durability of LLN
objectives.
Indeed, should LLNs be trying to put themselves out of business within a
reasonable period of time through embedding (a view implicit in HEFCEs
approach to funding LLNs)? The consensus was that there would be
continuing need for a (very) small team operating between institutions to

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

18

undertake core relationship management and to act as

broker/translator/lubricant. All three chairs saw LLNs as essentially
facilitating mechanisms and this would represent the mainstream core
such as IAG, staff development, etc. The longer term challenge is to
focus on who is being taught what, where, by whom, and what are they
learning and how has this changed their lives.
Moving Forward
Key to the future will be to use the LLN platform for other purposes. In
essence, LLNs have been created to ease supply arrangements in
respect of vocational learners and such an approach is equally valid for
other purposes (eg CPD, employer engagement). If the platforms created
by LLNs can demonstrate their value and be innovative in addressing
new agendas then they have a useful and continuing role in the HE
partnership landscape.
Importantly, HEFCE/DIUS need to be much more aware of the
considerable potential of LLNs in taking forward new agendas. Awareness
seems low at the moment. The recent 2008/09 HEFCE grant letter from
DIUS identified the opportunity for some, not necessarily all, LLNs as
building on the promise of LLNs. In the case of our three LLNs, the focus
could be much more on delivering on the promise. Other areas identified
in the grant letter as key priorities for Government and therefore HEFCE
are around FE/HE collaboration, employer engagement and the skills
agenda. HEFCEs Board paper on employer engagement clearly sets out
their thinking around a much more entrepreneurial market driven
approach and specification of the methodology for recruiting customers
and meeting their needs. The LLNs are ideally placed to respond to
these priorities. Key themes for the future could therefore be around:
a. Curriculum review and development
b. Progression
c. IAG
d. LLNs as current platforms for excellent FE/HE partnerships. Clearly
partnerships of LLNs within larger regions would be particularly well
placed to respond to regional issues, challenges and opportunities (as
well as sharing good practice).
e. LLNs as excellent, existing platforms for innovative ways to develop
the employer engagement strategies for HE and FE.
Key Messages
6. We considered some key messages. These were:
For LLN Teams

demonstrating value to partner institutions

evidencing outcomes/impact (inc tracking data)
using the partnership platform to take forward future opportunities
looking for innovative ideas to take such initiatives forward

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

19

For LLN Boards

As above plus

using their networks and radar to spot developing opportunities for

using the LLN platforms

For Institutions
helping to deliver mission and corporate objectives
the opportunity to trial new approaches to engaging learners
the value of the LLN in enabling institutions to develop, and
mainstream, responses to the developing agendas
For HEFCE/DIUS

the enormous value of LLN platforms for taking forward other

Government agendas. This needs a real push and we agreed to think
more about how we might communicate this message

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

20

Evaluating the Evaluation

The organisational model adopted can and will inevitably lead to different outcomes.
The ways in which each LLN is currently set-up will significantly influence future LLN
activity (post the initial funding period) and the impact of each model will be tested as
LLN funding reduces.
It was considered difficult to demonstrate the effectiveness of many approaches
LLNs have taken given the timescales involved. This is also hindered by the fact that
many LLNs are themselves struggling to measure their impact especially when
reporting the more softer outcomes specified in original business cases.
The peer evaluation exercise has been both an interesting and useful one. The
process itself has been allowed to grow organically, much in the same way that LLNs
have. Whist participants set aside an amount of time to interview other LLN staff,
there was little attention paid to what might be deemed as successful for this
exercise. For this reason, and those explained above, it was difficult to move away
from a descriptive report to one which pulled out those success criteria which LLNs
may not have focused on instantly and may not appear in business plans.
In order to draw something from the peer evaluation, colleagues felt that outcomes
should pull together and build upon previous monitoring and evaluations (such as the
Interim Evaluation conducted in the latter have of 2007), this called for a further
meeting to explore some of the indicators which the LLNs identified for success. This
has been useful for the LLNs involved which have draw up, or are in the process of
drawing up impact and process measures in order to demonstrate their impact.

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

21

Appendix 1
Peer Evaluation: Meeting Preparation (Staff)

Preparation for 17th/18th January, 2008

We agreed that we would ask those members of the three teams participating
in the evaluation to agree a set of bullet points around the main areas for
consideration. These would provide the baseline for each area so that we
covered common ground. As a start we agreed to complete a grid that
identifies who is responsible in each LLN.
Area
Organisation and
governance,
including finance
Partnership

SLN
Chris Baker

Kent & Medway

Hugh Joslin

Higher York
Colin Mellors

Sarah Hardman

Hugh Joslin

Colin Mellors

IAG

Remona Puri

Nicola Bedford

Remona Puri

Nicola Bedford

Progression

Maureen
Haywood
Maureen
Haywood
Sarah Hardman

Viki Faulkner

Jessica Grant

Curriculum

Chris Baker

Viki Faulkner

Jessica Grant

Tracking/reengaging

Sarah Hardman

Hugh Joslin

Nicola Bedford

Sustainability

Chris Baker

Hugh Joslin

Jessica
Grant/Colin
Mellors

Employer
engagement

Chris Baker

Viki Faulkner

Jessica Grant

Student voice

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

22

Appendix 2
Peer Evaluation: Meeting Preparation (questions)

Organisation and governance including finance

1. Are the objectives of the LLN clear, understood and accepted by the key partners? With
hindsight would they be different in anyway?
2. What are the key governance features? What works well; what might be better?
3. What are the key features of the management/staffing structure? What works well; what might
be better?
Partnership
1. How far had internal buy-in been achieved? Do partner institutions recognise the benefits of
the LLN?
2. Who are the other key stakeholders? How engaged are they?
3. What role has staff development played in building the partnership? How effective do you feel it
has been ?
4. Are partnership working arrangements sufficiently embedded to continue post HEFCE
funding?
IAG
1. What model is the LLN using to deliver IAG?
2. How have you worked with other IAG providers?
3. What information materials (including web resources) has the LLN developed and how do you
know they are effective?
4. What role does staff development play in relation to IAG delivery?
Student Voice
1. How is the LLN trying to engage with students?
2. What methods have been used to encourage participation and how are you assessing their
effectiveness?
3. How has the LLN benefited from engaging with students?
Progression
1. What real difference will the progression agreements make?
2. How will you know learners, providers, advisers understand and use agreements?
3. What are the threats to the effectiveness of the agreements, and how are these being
addressed?
4. How will the agreements be maintained and new ones added after HEFCE funding ends?
Curriculum
1. How has curriculum development been informed/changed due to the LLN activities?
2. How strategic has curriculum development been; how has it fitted with learner demand and
local and regional economic and social priorities?
3. What model has been used for developing curriculum? What works well; what might be better?
4. Has the new or revised curriculum attracted students and can you prove that it is making a
difference?
Tracking/re-engaging
1. What is the LLN tracking and why?
2. What use will the tracking data be put to?
3. How much useful tracking data will have been produced by the end of the LLNs current funding
period?
4. How will the LLN ensure institutions continued use of student numbers within appropriate
(vocational) areas of the curriculum (especially if operating model 1)?

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

23

Appendix 2

Sustainability
1. What plans exist for longer term sustainability of a) the objectives of the LLN b) other
opportunities that have emerged from this partnership?
2. Has the initial funding model worked?
3. What do you see as the main threats to sustainability and how are you planning to address
them?
Employer Engagement
1. What model(s) of employer engagement has the LLN used/is using? What works well; what
might be better?
2. How effective are the relationships you have with other agencies supporting employer
engagement and progression? How could they be improved?

/var/www/apps/conversion/tmp/scratch_5/262703337.doc

24