Research and Evaluation Toolkit

Research & Evaluation
Toolkit
This work is licensed by Middlesex University Higher Education Corporation under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0
International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/ or send a letter to Creative Commons, PO Box
1866, Mountain View, CA 94042, USA.
Acknowledgements and contact
This toolkit is the work of Neil Kaye and Julie Haddock-Millar, working as part of the Middlesex University
Business School within the Centre for Enterprise and Economic Development Research (CEEDR). It is licensed
by Middlesex University Higher Education Corporation under the Creative Commons Attribution: Non-
Commercial. Any use of the toolkit must acknowledge Middlesex University as creators and licensors and
attribute it to its authors.
Middlesex University is a well-established and thriving British university based in London and also has major
campuses in Dubai, Malta and Mauritius. With institutions on three continents – and students hailing from
more than 140 countries – Middlesex offers outstanding teaching, learning and research that knows no
geographical or academic borders. The University in London consists of 3 faculties (Arts and Creative
Industries; Science and Technology, and Professional and Social Sciences).
The Centre for Enterprise and Economic Development Research (CEEDR) is a leading academic research
centre specialising in entrepreneurship and small and medium-sized enterprises (SMEs), and focusing on SMEs
finance and growth, social and sustainable enterprise, labour market analysis (employment, skills and quality
of work) and local and regional economic development in the UK and internationally. The Professional Practice
group in the Department of Management, Leadership and Organisations (MLO) specialises in mentoring,
coaching and leadership development and has a long-established track record in mentoring research and
practice both nationally and internationally.
For more information about the research, or on the Research & Evaluation Toolkit, please contact:
Dr Julie Haddock-Millar, Neil Kaye,

Associate Professor of Human Resource Management Research Consultant
E: j.haddock-millar@mdx.ac.uk E: neil.kaye1@gmail.com
Page 2 of 100
Table of Contents
1. How to… carry out survey research .............................................................. 5
2. How to… carry out interview research ........................................................ 13
3. How to… carry out focus group research .................................................... 21
4. How to… carry out visual methods research .............................................. 29
5. How to… analyse numerical data ................................................................ 37
6. How to… analyse textual and visual data .................................................... 57
7. How to… write an evaluation report ........................................................... 63
Appendix 1: Questionnaire templates .............................................................. 71
Appendix 2: Interview schedule templates ...................................................... 85
Appendix 3: Focus group topic guide templates .............................................. 93
Page 3 of 100
Page 4 of 100
Toolkit
1. How to…
carry out survey
research
Page 5 of 100
Contents
1. Introduction to survey research
2. Designing your questionnaire
3. Carrying out your survey
Source: Rose, Sutton & Jarvis (2014) Family Arts Festival Evaluation and Audience Research Toolkit,
http://www.familyarts.co.uk/wp-content/uploads/2015/05/Family-Arts-Festival-Evaluation-Toolkit.pdf
Page 6 of 100
This How to…. guide provides an overview of survey research and is intended to assist researchers with
limited experience to carry out programme evaluation through the use of surveys. It outlines the
principal steps involved in designing, administering and collecting questionnaires and should be
consulted in combination with the questionnaire templates: 1a (long), 1b (medium) and/or 1c (short).
These templates, attached at the back of the main guide, are intended to provide an example and a
starting point for conducting your own survey research. To this end, users are encouraged to think
about their own specific research needs and to adapt the template accordingly before collecting data.
1. Introduction to survey research

Surveys can be used to collect information about people’s attitudes, values, beliefs and behaviours in a
standardized and consistent way. However, it should be kept in mind that there is a limit to the amount
of information you can get from a survey and more in-depth information about people’s motivations,
ideas and experiences might be better explored using interviews (see: How to… guide: 2. Carrying out
interview research), focus groups (see: How to… guide: 3. Carrying out focus group research) or visual
research methods (see: How to… guide: 4. Carrying out visual methods research).
Surveys allow you to gather information from lots of participants in an efficient and cost-effective way.
They are a useful and widespread method of carrying out research for a wide range of purposes,
including programme evaluation.
2. Designing your questionnaire

Before designing your questionnaire, you need to be clear about:
 What is the objective of your research or evaluation?
What are the specific questions you are interested in answering, e.g. how satisfied are
participants with the programme overall?
Template: the templates created for this toolkit provide an example of questionnaires that can
collect information from participants evaluating a mentoring programme.
 How much time and resources do you have to carry out your research or evaluation?
Whilst you might want to collect as much information as possible from your participants, you
should be realistic in terms of what you can achieve with the time, labour and other resources at
your disposal.
Page 7 of 100
This should include considerations both around questionnaire design (e.g. how long should the
survey be?) and scale of data collection (e.g. how many participants are you able to survey?)
Template: short, medium and long examples are provided and can be adapted to the specific
requirements and constraints of your research or evaluation.
 Who are you seeking to obtain information from?
Are you planning to survey mentees, mentors, other staff members? Do you need to use separate
questionnaires for different participants? Do you need to ask them different questions? Do you
need to ask them questions using different wording?
Template: provides examples that can be used for both mentors and mentees. It can, however,
be adapted to fit one or other audience – or, indeed, a different type of participant altogether.
 How are you going to analyse the information once you have collected it?
Having an idea about how you are going to analyse and present your data allows you to focus on
which are the most important questions you want to ask (removing those that are unnecessary).
It also lets you know the best way of asking a question (e.g. if you want to be able to summarise
the information as percentages or provide short quotes from respondents)
Template: uses a number of different types of questions (see below), some of which elicit
numerical data and some of which produce textual data (See How to.. guides: 5. Analysing
Numerical Data and 6. Analysing Textual or Visual Data).
Once you are clear about what your research is for, who you are surveying and how you want to use
the information you get, you can begin to construct your questionnaire. This brings together all of the
required components to enable you to get the information you need from your participants.
All surveys contain the following elements:
1) Title
2) Introduction – this provides a brief explanation of the research or evaluation
3) Questions – this is the bulk of the survey, where you are obtaining information from your
participants
4) Concluding statement – thanks participants and gives them contact information if they want to
get in touch
Page 8 of 100
1) Title
Should be concise and relate to the content of the survey, e.g. ‘Mentee Evaluation of Mentoring
Programme’
2) Introduction
Usually provides a summary of the research so that participants are clear on why they are being asked
to complete the survey. This part should also include information about confidentiality and any other
ethical issues relating to use of the information (particularly personal or sensitive information) provided
by the participant (see the example provided by the templates). An indication or how long the
questionnaire will take to complete is also useful to provide.
3) Questions
The questions you include in the questionnaire depend entirely on the purposes of the research or
evaluation being done. In general, there are two types of questionnaire questions: ‘open’ and ‘closed’,
commonly divided by theme.
Open Questions Closed Questions

Respondents can answer in their own Help to clarify the meaning of a question to
terms/words respondents
Can be too vague or difficult to answer Force answers
properly (and it’s difficult to make options exhaustive
and mutually exclusive)
Are good to explore topics Reduce variability of answers
Unusual/unexpected answers are allowed Allow comparability of answers
Time consuming to answer, code and analyse Easy to answer and process
Examples:
Open question: ‘What did you find to be the most beneficial part of the programme?’
(write in your answer) ………………………………………………………………………………………………..
Closed question: ‘Which of the following did you find to be the most beneficial part of the programme?’
(Tick one box only)
a) Improving my business management skills
b) Developing my business plan
c) Increasing my sales performance
d) Accessing new markets
e) Other
Page 9 of 100
In addition, questions can be:
 categorical – respondents are placed in one of several categories, e.g. a) male; b) female
 multiple choice – when respondents can choose more than one response, e.g. ‘Why did
you decide to take part in the programme? Select all responses that apply to you.’
 ranking – asking respondents to rank there experience on an ordinal scale (typically from 1
to 5), e.g. ‘To what extent did the programme meet your initial expectations? Answer from
1=not at all to 5=completely’
 numerical – when respondents are required to provide an answer that is an exact number,
e.g. ‘How old are you?’ (NB this does not include questions which have pre-defined ranges,
for example: 18 to 25, 26 to 35, etc.)
CAUTION! hen
designing your survey questions be sure to:
 Avoid very general questions – be specific; each question should have a purpose
 Remove unnecessary questions
 Avoid technical terms – remove or change words that participants are unlikely to
understand
 Be as clear and unambiguous as possible – avoid ‘double-barrelled’ questions (e.g. ‘Are you
satisfied with your mentor’s level of knowledge and number of industry contacts?’)
 Be careful not to use ‘leading questions’ – make sure your questions do not unfairly
prompt or encourage participants answer in a particular way (e.g. ‘Would you agree that
this mentoring programme met all of your initial expectations?’)
4) Concluding statement
Your concluding statement should thank the participant for taking the time to complete the survey. In
addition, you should provide contact information so that your participant(s) can follow up with any
queries they might have. You should also explain how you intend to use the responses and, if
appropriate, where participants can find the published results.
3. Carrying out your survey

Before carrying out your survey it is always advisable to pilot the questionnaire – if only with a handful
of your colleagues. This process allows you to identify:
a) How long it takes to complete the survey

b) Whether any questions are difficult to understand
c) If questions are understood in the same way by all participants
d) Whether there are any questions participants feel are missing from the survey
Page 10 of 100
Following your pilot, you may want to shorten the questionnaire or amend the wording of some of the
questions.
Once your questionnaire is designed and ready to be completed by your participants, there are several
ways in which you can collect responses:
Online – this method is one of the simplest and most cost-effective. The questionnaire is constructed
in (or input into) an online interface (e.g. Survey Monkey). Participants are provided a weblink and
can complete the survey from any computer with internet access.
WhatsApp – a questionnaire is sent via WhatApp to participants, who complete the survey on their
mobile phone and send it back to the given number (again via WhatsApp)
Postal – paper questionnaires are sent out to participants, who are required to complete the survey
and return it to a given address.
Telephone – the questionnaire is administered over the telephone by a researcher, who completes
the survey by inputting the responses provide to him/her by the participant
Face-to-face – a researcher administers the survey with the participant in the same place, providing
responses to the questions
Page 11 of 100
Page 12 of 100
Toolkit
2. How to…
carry out
interview research
Page 13 of 100
Contents
1. Introduction to interview research
2. Designing your interview schedule
3. Arranging your interviews
4. Carrying out your interviews
Page 14 of 100
This How to…. guide provides an overview for researchers with limited experience and is intended to
assist them to carry out programme evaluation through the use of interviews. It outlines the principal
steps involved in designing, administering and collecting qualitative data through one-to-one interviews
and should be consulted in combination with the interview schedule templates: 2a (long), 2b (medium)
and/or 2c (short).
starting point for conducting your own interview research. To this end, users are encouraged to think
about their own specific research needs and to adapt the template accordingly before beginning their
data collection.
1. Introduction to interview research

Interviews, like surveys, can be used to collect information about people’s attitudes, values, beliefs and
behaviours, alongside information about participants’ profile (e.g. gender, age, ethnicity, location,
position in the organisation, etc.). However, they are more suited to collecting richer information about
a topic, focussing on the story behind a participant’s experience and understanding in more depth why
and how individuals feel or act the way they do. Because interviews collect a greater amount of textual
information, they are often carried out with a smaller number of participants.
Interviews can be carried out in person (face-to-face), over the telephone, or via the internet (e.g. with
an application like Skype). Whilst one-to-one interviews can elicit a lot of in-depth information, they
have clear resource implications. It should be kept in mind, therefore, whether participants'
experiences and attitudes might be better explored using surveys (see: How to… guide: 1. Carrying out
survey research), focus groups (see: How to… guide: 3. Carrying out focus group research) or visual
research methods (see: How to… guide: 4. Carrying out visual methods research).
In terms of programme evaluation, interviews can allow you to gather information from participants
that cover very specific aspects of the programme, that highlight particular challenges facing the
programme within a specific context and that allow participants to fully explain their experiences.
However, it must be kept in mind that one individual’s experience is not necessarily indicative of issues
that are widespread or representative of participants’ attitudes more generally.
2. Designing your interview schedule
Carrying out an interview requires designing an interview schedule to guide the conversation so that
you can lead your participant to talk in depth about the topics in which you are interested. Interview
schedules can vary in length and format and can be ‘unstructured’, ‘semi-structured’ or ‘structured’.
Most commonly, the semi-structured interview is used, which allows participants to speak freely about
their experiences without the rigidity of a very structured, survey questionnaire-style format, but also
keeping them within a pre-selected number of answers to specific themes or topics.
Before designing your interview schedule, you need to be clear about:

Page 15 of 100
What are the specific topics or themes you are interested in finding out about, e.g. how satisfied
are participants with the programme?
Template: the templates created for this toolkit provide an example of interview schedules that
can collect information from participants evaluating a mentoring programme.
your disposal.
This should include considerations both around interview schedule design (e.g. how many topics
or themes you can cover? How long do you have to interview your participants?) and scale of
data collection (e.g. how many participants are you able to interview? How much data are you
able to analyse?)
Are you planning to carry out interviews with mentees, mentors, other staff members? Do you
need to use separate interview schedules for different participants? Are you more interested in
one group than another? Do you want to find out different things from different participants?
which are the most important topics or themes you want to investigate (prioritising those that
are most important in the interview schedule).
Template: arranges questions thematically to facilitate analysis of textual data at a later date
(See How to.. guide: 6. Analysing Textual and Visual Data).
Page 16 of 100
Once you are clear about what your research is for, who you are intending to interview and how you
want to use the information you get, you can begin to design your interview schedule.
A semi-structured interview schedule will usually contain the following elements:
1) Introduction and ethical consent – this provides a brief explanation of the research or evaluation
and obtains explicit consent from the participant to be part of the research
2) Questions – structured using a small number of themes to allow participants to talk freely
around the topics
3) Prompts – using follow-up questions or probes to ask a participant to answer in more depth
4) Concluding remarks – thanks participants and gives them contact information if they want to
get in touch
1) Introduction and ethical consent
The interviewer should provide a summary of the research so that participants are clear on why they
have been asked to participate. Before asking any of your main interview questions, you should obtain
informed consent from the participant, providing information about confidentiality, how responses will
be stored securely and used and the respondents’ ability to withdraw from the research or refuse to
answer a question should they choose (see the example provided by the templates).
2) Questions
The questions you include in the interview schedule depend entirely on the purposes of the research
or evaluation being done. You should use open-ended questions to allow participants to respond in
their own words, providing in-depth information and explaining the reasons behind their responses.
The schedule should set out the broad topics or themes to be covered. It is not necessary to follow the
same order for each participant, although it is usual to prioritise those themes you are most interested
about earlier on in the interview to ensure you have enough time to explore them fully.
Example:
Question: ‘How well matched do you feel that you are with your mentor/mentee?’
Page 17 of 100
3) Prompts
Broad topics can be followed up using one or more probing questions to narrow down the scope of the
response if you are not eliciting the data you are looking for. You can also use follow-up questions to
home in on specific examples or explore a participant’s response in more depth.
Example:
Question: ‘How well matched do you feel that you are with your mentor/mentee?’’
(prompt): Why do you feel that was the case?
(prompt): Did they meet your initial expectations? In what ways did they meet them or not meet them?
Your concluding statement should thank the participant for taking part in the research. In addition, you
should provide contact information so that your participant(s) can follow up with any queries they
might have. If you intend to re-contact the participant, you should let them know when and how you
will do this. You should also explain how you intend to use their responses and, if appropriate, where
and when participants can find the published results.
3. Arranging your interviews

Once you have designed your interview schedule, you can begin to collect your data. Whether you
decide to conduct the interviews in person, over the telephone or via the internet, you should ensure
that you allot enough time for each participant to talk about each of the themes mentioned in your
interview schedule.
You should record the interviews either by taking notes or by using audio or video recording equipment.
However, you must ensure that participants consent to being recorded in this way. Having a record of
the interview will help you to analyse the data you collect, including lifting out illustrative quotes and
identifying consistent themes across interviews (see: How to… guide: 6. Analysing textual and visual
data).
Page 18 of 100
4. Carrying out your interview
When you are carrying out your interview, there are a few principles you should seek to follow – the
‘five Es’:
Establish a rapport with your participant by introducing yourself and explaining the aim of
the interview. Put the participant at ease, ensure them that their answers are confidential
and start by asking very general, easy-to-answer questions.
Encourage expression of their own attitudes, experiences and beliefs, in their own words
and without prejudice or value judgement of responses.
Elicit contextual factors and try to establish the circumstances around which something
happens or the organisational culture that exists. E.g. “How do you meet or have contact
with your mentor [/mentee]?”
Elucidate inconsistencies and tensions by highlighting and exploring where a participant

may have contradicted themselves or if they acted against their initial instincts E.g. “To
what extent did your mentor [/mentee] meet your initial expectations?”
Explore justifications and rationales for actions or beliefs. Probe participants as to the
motivations for their action or the reason why they feel a particular way E.g. “Why did you
decide to join the mentoring programme in the first place?”
CAUTION!
When conducting your interviews be sure to:
 Use clear open-ended questions – avoid closed questions or those with a ‘yes/no’ answer
 Avoid double-barrelled questions
 Leave enough time for your participant to think about their response
prompt or encourage participants answer in a particular way
 Avoid commenting on answers – try not to make any value-judgements on what your
participants says
 Observe non-verbal communication – take note of your participants’ body language and
not just the words they are saying
Page 19 of 100
Page 20 of 100
Toolkit
3. How to…
carry out
focus group research
Page 21 of 100
Contents
1. Introduction to focus group research
2. Designing your focus group topic guide
3. Arranging your focus groups
4. Carrying out your focus groups
Page 22 of 100
assist them to carry out programme evaluation through the use of focus groups. It outlines the principal
steps involved in designing, administering and collecting qualitative data through focus group
discussions and should be consulted in combination with the topic guide templates: 3a (long), 3b
(medium) and/or 3c (short).
starting point for conducting your own focus group research. To this end, users are encouraged to think
about their own specific research needs and to adapt the template accordingly before beginning your
data collection.
1. Introduction to focus group research

Focus groups, like interviews, collect rich, qualitative information from participants about their
attitudes, values, beliefs and behaviours. However, the distinctive feature of focus groups is that
interaction between group members is encouraged such that responses flow as a result of group
dynamics and from participants ‘bouncing off’ one another when discussing a topic.
Focus group research is typically carried out with small groups of around 6 to 10 participants, who share
similar experiences or interests. Typically, people are less inhibited in a group and more willing to
provide elaborate and detailed accounts – especially when they feel backed up by other group
members.
Focus groups are conducted as a group discussion led by a moderator, who should encourage open
disclosure from all participants and explore the different opinions, ideas and experiences expressed in
relation to a pre-defined set of topics. Whilst many people’s views can be collected during one focus
group, it is advisable to carry out several focus groups for any one piece of research or evaluation, as
each discussion will have its own dynamic and themes will arise in one focus group that were
overlooked in another.
Conducting focus groups elicits a lot of in-depth information. This can take time to analyse fully, which
has clear resource implications. It should be kept in mind, therefore, whether participants' experiences
and attitudes might be better explored using surveys (see: How to… guide: 1. Carrying out survey
research), interviews (see: How to… guide: 2. Carrying out interview research) or visual research
methods (see: How to… guide: 4. Carrying out visual methods research).
In terms of programme evaluation, focus groups of colleagues can provide insights that might not
emerge from one-to-one interviews as one person’s experience may echo another’s (or contrast
sharply), allowing the group as a whole to explore and clarify the circumstances and motivations behind
this. It also allows access to participants’ work-based knowledge, their ‘jargon’ and shared norms,
within a safe environment.
However, focus groups can be difficult to convene, getting everyone in one physical place to meet can
be hard to arrange. For this reason, it might be useful to consider using ‘natural’ groups of participants,
who meet regularly anyway, although this will clearly have implications about whose voice(s) you are
excluding by doing this. Group dynamics are also important to bear in mind and it is important to avoid
any clear conflict of interests (e.g. business competitors or mentors and mentee pairings together) that
might prevent participants from being willing to disclose certain information.
Page 23 of 100
As with interview data, it must be kept in mind that one individual’s experience is not necessarily
indicative of issues that are widespread or representative of participants’ attitudes more generally.
Furthermore, some people do not express their opinions well in a group situation (e.g. lack of
confidence) and it may be useful to follow up with individual participants after the focus group to
produce feedback stimulated by the discussion.
2. Designing your focus group topic guide

Carrying out a focus group requires designing a topic guide to steer the discussion, although the
emphasis is on a light-touch approach that encourages spontaneous and free-flowing participation. For
this reason, the topic guide should contain a small number of open-ended questions to provoke
thoughtful and reflective debate. Additionally, focus groups may make use of scenarios or vignettes or
other visual materials (see: How to… guide: 4. Carrying out visual methods research) as a starting point
for discussion.
Before selecting your focus group topics for discussion, you need to be clear about:
What are the most important topics or themes you are interested in finding out about, e.g. how
satisfied are participants with the programme?
Template: the templates created for this toolkit provide an example of topic guides that can
collect information from participants evaluating a mentoring programme.
your disposal.
This should include considerations both around topic guide design (e.g. how many topics or
themes you can cover? How long do you have to carry out the focus group?) and scale of data
collection (e.g. how many focus groups are you able to conduct? How much data are you able to
analyse?)
Are you planning to carry out focus groups with mentees, mentors, other staff members? Do you
need to use separate topic guides for different participants? Are you more interested in one
group than another? Do you want to find out different things from different participants?
Page 24 of 100
which are the most important topics or themes you want to investigate (prioritising those that
are most important in the topic guide).
Template: arranges questions thematically to facilitate analysis of textual data at a later date
(See How to.. guide: 6. Analysing Textual and Visual Data).
Once you are clear about what your research is for, who you are intending to conduct focus group with
and how you want to use the information you get, you can begin to design your topic guide.
A focus group topic guide will usually contain the following elements:
1) Introduction and ethical consent – this provides a brief explanation of the research or evaluation
and obtains explicit consent from the participant to be part of the research
2) Questions – structured using a small number of broad themes to allow participants to talk freely
around the topics
3) Prompts – using follow-up questions or probes to ask participants to answer in more depth on
a topic
4) Concluding remarks – thanks participants and gives them contact information if they want to
get in touch
1) Introduction and ethical consent
The focus group moderator should provide a summary of the research so that participants are clear on
why they have been asked to participate. Before asking any of your main questions, you should obtain
informed consent from each participant, providing information about confidentiality, how responses
will be stored securely and used and the respondents’ ability to withdraw from the research or refuse
to answer a question should they choose (see the example provided by the templates).
2) Questions
The questions you include in the topic guide depend entirely on the purposes of the research or
evaluation being done. You should use a small number of open-ended questions to allow participants
to respond in their own words, using these as a starting point to provoke further discussion within the
Page 25 of 100
group. Ideally, the focus group should follow a format that engages participants, explores the topics and
finally summarises participants’ views.
Examples:
Engaging questions:
‘How do you reflect on the outcomes of your mentoring relationship?’
‘How have you benefitted professionally (and personally) from the mentoring relationship?’
Exploring questions:
‘To what extent has your experience with your mentor influenced your career aspirations and
why?’
‘Can you describe a specific occasion where you had a positive experience with your mentor?’
‘If you were a mentor, what, if anything would you change about your relationship?’
Summarising questions:
‘Of all the support you have received which would you say was the most useful?’
‘Do you have any further comments you’d like to add about the support you have received?’
3) Prompts
Whilst, in general, you should maintain impartiality and avoid commenting on participants’ responses,
you should use prompts and follow-up questions to get focus group members to speak in more depth
on a topic. These could include asking participants to elaborate on an anecdote or to explain or justify
their motivations or actions. Encouraging the group participants to speak at greater length also provides
greater opportunity to stimulate discussion amongst other members of the focus group.
Your concluding statement should thank the participants for taking part in the focus group. In addition,
you should provide contact information so that your participants can follow up with any queries they
might have. If you intend to re-contact the participants, you should let them know when and how you
will do this. You should also explain how you intend to use their responses and, if appropriate, where
participants can find the published results.
Page 26 of 100
3. Arranging your focus groups
Once you have designed your topic guide, you can begin to collect your data. Ensure that your focus
group can be conducted in a comfortable environment where you won’t be disturbed. You should
ensure that you allot enough time for each group to allow a thorough discussion of the themes
mentioned in your topic guide.
It is advisable that two people are present during the focus group: the moderator should lead the
discussion, whilst it is the responsibility of the assistant moderator to take notes and operate the
recording equipment.
You should record the focus groups using audio or video recording equipment and/or by taking notes.
However, you must ensure that participants consent to being recorded in this way. Having a record of
the focus group will help you to analyse the data you collect, including lifting out illustrative quotes and
identifying consistent themes across focus groups (see: How to… guide: 6. Analysing textual and visual
data).
4. Carrying out your focus group

When you are carrying out your focus groups, there are a few principles you should seek to follow:
The moderator should establish ground rules for the focus group. These could include:
 asking participants to turn off mobile phones;

 referring to each other by first name;
 emphasising that there are no right or wrong answers, only differing points of view
 allowing everyone to be heard, listening to each other’s view before giving their own
It is the role of the moderator to allow debate to flow, whilst also keeping the discussion on track,
balancing the relevance of the discussion with the flow of conversation.
He/she should try to involve everyone and to note body language and acknowledge group dynamics.
The moderator should encourage explanation or clarification of stories or anecdotes by asking

participants to think about their motivations behind their decisions, or inviting other group members
to comment on these experiences.
Finally, they should close the focus group by summarising what has been discussed and allowing
participants a chance to add anything they feel has been overlooked.
Page 27 of 100
CAUTION!
When conducting your focus groups be sure to:
 Use clear open-ended questions – avoid closed questions or those with a ‘yes/no’ answer
 Avoid double-barrelled questions
 Leave enough time for your participants to think about their response
prompt or encourage participants answer in a particular way
 Avoid commenting on answers – try not to make any value-judgements on what your
participants says
 Observe non-verbal communication – take note of your participants’ body language and
not just the words they are saying
Page 28 of 100
Toolkit
4. How to…
carry out
visual methods research
Page 29 of 100
Contents
1. Introduction to visual methods research
2. Including visual methods in your research design
3. Researcher-produced visual data
4. Using pre-existing visual data
5. Participant-produced visual data
Page 30 of 100
assist them to carry out programme evaluation through the use of visual research methods. It outlines
the key principles involved in designing visual research methods to collect qualitative data and should
be consulted in combination with the How to… guides on: 2. Carrying out interview research, 3. Carrying
out focus group research and 6. Analysing textual and visual data.
1. Introduction to visual methods research

Visual research methods are a useful way of eliciting information from participants beyond asking them
to reflect verbally on their attitudes, values, beliefs and behaviours. Typically, it requires participants to
think more carefully about their responses, encouraging self-reflection. It also entails a more inclusive
approach, allowing for difference learning styles and participant expressive preferences. Visual
methods facilitates creativity of expression and can reveal insights that might not easily be expressed
using words alone.
Visual data relates to a whole range of non-verbal information, including (but not limited to)
photographs, drawings, videos, maps, animations, advertisements, social media images and diagrams.
There are three types of visual research methods that can be employed in your research:
 researcher-produced visual data (e.g. photographs of on-going practice);
 using pre-existing images to examine a topic or theme (e.g. showing a short film or picture as a
starting point to discuss an issue); and
 participant-produced visual data as part of the fieldwork (e.g. where a participant is asked to
select or create a picture or image in response to a topic or theme).
The use of visual research methods may complement other research methods and is often used in
conjunction with, or as part of a wider interview or focus group discussion.
Visual research methods can reveal insights that may not be easily expressed using words alone. For
this reason, whilst the data they produce can clearly add value to any piece of evaluation or research,
it can also be time-consuming to analyse fully, which can have resource implications. It should be kept
in mind, therefore, whether participants' experiences and attitudes might be better explored using
surveys (see: How to… guide: 1. Carrying out survey research), interviews (see: How to… guide: 2.
Carrying out interview research) or focus groups (see: How to… guide: 4. Carrying out focus group
research) alone.
In terms of programme evaluation, using visual research methods can engender a more collaborative
and participatory approach and can allow participants to feel more involved in the research process.
Beyond the textual data revealed through interviews and focus groups, the production of visual data
can help colleagues to express their thoughts, motivations or attitudes more accurately, with a wider
range of tools with which to do so.
Page 31 of 100
2. Including visual methods in your research design
Including some form of visual methods in your research design requires some forethought as to how
and why you are intending to use the visual data. What type of visual data are you planning to use or
produce? Does it relate to photographs, videos or drawings, for example? How will this data add value
to your research?
As with other research designs, before beginning to collect your data, you need to be clear about:
What are the most important topics or themes you are interested in finding out about, and how
can visual data add value to this.
your disposal.
This should include considerations both around research design (e.g. how many topics or themes
do you want to explore using visual research methods?) and scale of data collection (e.g. how
many participants are you able to collect data from? How much data are you able to analyse?)
Are you planning to carry out research with mentees, mentors, other staff members? Do you
want to use or produce different types of visual data from different participants? Are you more
interested in one group than another? Do you want to find out different things from different
participants?
Having an idea about how you are going to analyse and present your visual data allows you to
focus on which are the most important topics or themes you want to investigate in this way. How
are you planning to analyse the data and incorporate the findings with those arising via other
research methods?
Page 32 of 100
3. Researcher-produced visual data
The researcher uses visual methods to record an interaction or exchange. This could be by taking
photographs, drawing a sketch or filming a process.
The idea behind using images or film recording is to try to capture ‘the complex’, producing a visual
representation of what is happening, without it being mediated through someone’s verbal description
of the experience. Recording people in a ‘naturalistic’ setting allows you to let participants carry out
their lives as normally as possible, before you revisit and analyse the data at a later time.
In terms of programme evaluation, examples of carrying out visual research in this way could include:
 Photographing participants whilst they are undergoing their training
 Filming meetings between mentees and mentors
 Recording participants within the context of their business practice
 Constructing a relational diagram of how participants interact with business partners,

suppliers or clients
4. Using pre-existing visual data

The research shares a photograph, short piece of film or other visual material to introduce a topic or
theme for discussion. This can occur with one participant, as part of a one-to-one interview, or with
several participants within the context of a focus group. This is a more collaborative process, which
involves a dialogue between researcher and participant as they explore the meaning behind the visual
stimulus or using it to trigger recollections of experiences or situations.
Photo-elicitation (as this is called), has several benefits including establishing a rapport between
researcher and participant(s), providing structure to the interview or focus group and breaking down
power dynamics by introducing a collaborative element to the research.
Page 33 of 100
Using the visual material as a starting point, you should encourage the participant(s) to think about:
 How to interpret what is going on in the photo/video/etc.
 What thought processes are occurring
 How this relates to their own practices
 Recalling their own practices
 Examine the motivation and meaning behind their own practices
5. Participant-produced visual data

Researchers invite participants to select or create a picture or image (or other visual material) in
response to a topic or theme, or which they feel best represents a notion or experience. Whilst
participants will have different levels of artistic skill or confidence, you can ask them either to draw a
picture or select an image from a newspaper, magazine or the internet, etc.
The idea is for the participant themselves to produce a visual representation to convey how they feel
about an issue or to illustrate their attitude towards a situation. This collaborative approach helps
participants to feel more involved in the research process and the use of images means they can provide
more creative and insightful data than an oral interview would typically allow.
You can ask participants to carry out this exercise during the interview or focus group but, it may be
more effective to allow them time and space to think about their composition. Informing them of this
aspect of the research some days in advance allow participants the opportunity to reflect more deeply
on the issue or topic, to be more creative in their response, and to use a range of different methods to
produce their material (e.g. taking a selection of photographs, or consulting a number of magazines).
Alternatively, this could be given as ‘homework’, to be done after the main interview or focus group
discussion to allow the thoughts and ideas raised to crystallise before the participant selects or creates
a visual representation in response to a given theme or topic.
In terms of programme evaluation, this exercise could allow participants to reflect on the overall
organisational culture in their business, or the dynamic between the mentor and mentee, or how their
business has changed throughout the mentoring programme.
Page 34 of 100
CAUTION!
Using visual research methods entails a number of ethical considerations:
 Always obtain informed consent – especially if photographing or recording participants or

other people who may feature in the visual materials
 Be careful to ensure anonymity - do photographs or other visual materials contain
identifiable images of people? Can contextual information in the images be used to
identify people? Can participants be identified by their handwriting or drawing style?
 Be clear about how you intend to use the images – ask participants for permission to reuse
their drawings, photographs, etc., in any report or analysis being produced. Think about
whether including images of people or places is necessary or desirable for published work.
 Think about issues of copyright – are there legal constraints to being able to use images or
symbols, selected by participants, in your report or published work?
Page 35 of 100
Toolkit
5. How to…
analyse
numerical data
Page 36 of 100
Contents
1. Introduction to survey data analysis
2. Collating and arranging your data in Excel
3. Using pivot tables to summarise your data
4. Using charts and graphs to visualise your data
5. Producing tables and charts using online survey tools
Page 37 of 100
assist them to carry out programme evaluation by analysing numerical data. It outlines the principal
steps involved in collating and analysing statistical data collected through surveys and other
quantitative methods. This guide should, therefore, be consulted in combination with the previous How
to… guide: 1. Carrying out surveys.
This guide is intended to give a broad overview of producing descriptive statistics within the context of
conducting a mentoring programme evaluation. It assumes the user to have a basic understanding and
working knowledge of Microsoft Excel or similar spreadsheet applications. It will show users how to
produce tables, charts and graphs to visualise quantitative data in a report-friendly format.
1. Introduction to survey data analysis

Once you have collected your survey data through the use of paper or electronic questionnaires, you
can begin to explore what the data are saying and how this relates to the research or evaluation as a
whole. Data analysis is a process which involves several steps and requires you to make decisions about:
how you want to present the data, what story you want it to tell, what you want to include and what
you think is unnecessary, and how your findings link into the wider research/evaluation aims.
It is helpful to think about the following 5 issues when you are undertaking your survey data analysis:
What are the implications of the data? What is the data showing, what does it mean about your
respondents’ views and experiences?
What relationships and trends are evident? Is there any clear relationship between how respondents
answer? Are there trends over time, or are there any patterns amongst respondents with similar
backgrounds, or in the same location?
How do the findings relate to other information or literature? Are the findings of your analysis what you
expected to find? Are they surprising in any way? Think about why this might/might not be.
What actions might be considered as a result of the findings? How will the information you have revealed
be used to inform your practice or organisational strategy?
Is there additional information or research that should be conducted? Is there data that you were not
able to collect that you feel is important to obtain? Has anything been revealed that you would like to
know more about?
With these questions in mind, it is important to conduct your data analysis in a systematic way so that
you are clear about what story the data is telling and what patterns and trends are revealed. The next
section explains how to use Excel to organise your data, to summarise it in tables, and how to visualise
the data in preparation for including it in your final report. Additionally, the benefits of using online
survey tools for data analysis are also outlined.
Page 38 of 100
2. Collating and arranging your data in Excel
Excel is often used just as an electronic worksheet, to organise lists or tabular data.
However, this software also provides some very powerful tools to manage data - particularly
quantitative datasets - perform statistical analysis and produce charts and other outputs.
Typically there are two options for getting your data into Excel:
1. Enter the data manually (see below).
2. Import the data in a suitable format from, for example, an online survey tool (see Section 5).
If you are going to enter your data manually, use a single worksheet to hold all the data in your dataset
and set up the worksheet with variables (questions) as the columns and the cases (respondents) as the
rows. An individual cell, therefore, contains an individual respondent’s answer to a specific question.
It is useful to give each of your respondents a unique ID number so that their responses can be double-
checked with the original copies of the questionnaire. You should also give headings to your variables
that explain what the responses represent – using question numbers (e.g. Q1, Q2, etc.) can get
confusing with large numbers of questions.
Data that is ‘nominal’ – i.e. responses fit into discrete categories that cannot be ranked or ordered in
any way – can be entered in words. For example: gender can be entered as ‘Female’, ‘Male’ or ‘Other’.
Be careful to enter responses with the same value in exactly the same way (spelling, formatting, etc.)
Data that can be ordered or that are values on a scale – e.g. responses to ‘ranking’ questions (‘Overall,
how satisfied are you with your mentoring relationship?’) – can be entered as the number
corresponding to the respondent’s score.
Figure 1
Page 39 of 100
Once the data is entered into Excel, you can begin to analyse the data, explore relationships and
trends, and prepare it for presentation. Excel provides a wide range of statistical tools. Some of the
most commonly used include:
SUM – returns the sum total of a range of cells
COUNT – counts the number of cells in a range that contains numbers
COUNTA – counts the number of cells in a range that contains any data
AVERAGE – returns the average (mean) of selected numbers
To use these (or other) functions, you can access the appropriate dialog box by selecting the ‘Insert
function’ button on the ‘Formulas’ header.
Figure 2
3. Using pivot tables to summarise your data

A useful shortcut for producing tables that can easily be exported into a report is to use ‘pivot tables’.
Pivot tables are an interactive worksheet table that can quickly and easily analyse and summarise large
amounts of tabular data. They are highly flexible and can be quickly adjusted depending on how you
need to display your results.
To create a pivot table, select all of your data and click on the ‘PivotTable’ button from the ‘Insert’
header (see figure 3). Clicking ‘OK’ on the pop-up dialog box provides you with a blank pivot table on a
new sheet in the same Excel workbook (figure 4).
Page 40 of 100
Figure 3
Figure 4
You can use the tick-boxes on the right-hand side to include variables in your table. For example,
selecting age group and gender will automatically assign these to the ‘rows’ of the table. You can drag
and drop these fields to rearrange the format of your table.
Page 41 of 100
Figure 5
Figure 6
You must also include a variable that can be used to assign values. The easiest way to do this is to drag
and drop the ‘ID’ variable into the ‘values’ box.
By default, this gives the sum of these values – this needs to be changed to counts for this data to be
meaningful (i.e. the number of respondents).
Click on the ‘Sum of ID’ variable in the values box and select ‘Value field settings’. In this dialog box you
can select ‘Count’ as the way to summarise the data (see Figure 7).
Page 42 of 100
Figure 7
The resulting table provides a cross-tabulation of respondents’ age group by gender:
Figure 8
The pivot table format can also be used to summarise the data percentages of the total, rather than
counts. To do this, return to the ‘Value field settings’ for the ‘Count of ID’ variables and click on the
‘Show values as’ tab. The drop-down menu here provides a number of options – selecting the ‘% of
grand total’ option will give percentages for each cell in the table:
Page 43 of 100
Figure 9
You can even display both counts and percentages in the same table. This involves dragging and
dropping the ‘ID’ variable down into the values field again. This will duplicate the variable in this field
(again, the default provides the sum of the values).
Figure 10
Change the format of this variable, as above, using the ‘Value field settings’ option – selecting ‘Count’
from the ‘Summarize values by’ tab. You can also change the name of the field for the table here or
adjust the number formatting using the ‘Number format’ button.
Page 44 of 100
Figure 11
The final table can be copied and pasted into a Word document or into another sheet in your Excel
workbook for any further formatting changes. For more information about how to organise your
findings within a report, see How to … guide: 7. Writing an evaluation report.
Female Male Other Total

No. % total No. % total No. % total No. % total
18 to 35 years old 1 6.7% 0 0.0% 0 0.0% 1 6.7%
36 to 45 years old 0 0.0% 3 20.0% 0 0.0% 3 20.0%
46 to 55 years old 4 26.7% 0 0.0% 0 0.0% 4 26.7%
56 to 65 year old 0 0.0% 3 20.0% 1 6.7% 4 26.7%
66 + 0 0.0% 3 20.0% 0 0.0% 3 20.0%
Grand Total 5 33.3% 9 60.0% 1 6.7% 15 100.0%
Figure 12
Page 45 of 100
4. Using charts and graphs to visualise your data
Charts and graphs are useful ways of exploring numerical data and can be used to summarise datasets
in a visual manner and to show patterns and relationships between different variables. They can be
used to interrogate the information you have collected as well as to present your findings in a visually-
appealing, user-friendly way.
There are many different types of graphs – amongst the most commonly used are:
 Bar charts
 Column charts
 Line graphs
 Pie charts
 Scatterplots
The same information can often be represented by different kinds of charts, or by organising the same
graphs in different ways. It is always important, therefore, to consider which is the most appropriate
type of chart for the data you want summarise and for the information you want to highlight.
Bar charts and column charts
Bar charts are useful to compare values of a variable in two or more cases. Bars are displayed
horizontally with length proportional to the values they represent. Column charts are the same as bar
charts, except that the bars are displayed vertically.
Clustered bar/column charts are useful to compare values across categories. Similarly, stacked
bar/column charts divide a single bar/column into segments to display values for different categories.
In each of these cases, the procedure in Excel is similar and involves highlighting the data you wish to
visualise as a chart before clicking the ‘Insert column or bar chart’ button under the Insert’ header:
Figure 13
Page 46 of 100
Selecting one column of data will produce a standard bar/column chart:
Figure 14
Selecting more than one column of data will create a clustered bar/column chart (figure 15), or a
stacked bar/column chart (figure 16):
Figure 15
Figure 16
Page 47 of 100
You can enter and edit chart elements, such as a Chart Title, variable and value labels, legend, axis
labels, etc., via the ‘Add chart element’ button under the ‘Design’ header:
Figure 17
Line graphs
Line graphs are most often used to display trends over time. They show related information by
drawing a continuous line between points on the chart. For example, if you have collected
respondents’ scores at a number of different time points, you can plot the underlying trend for the
average score over time using a line chart:
Figure 18
Page 48 of 100
Pie charts
A pie chart is a circular chart divided into ‘portions’ representing the size of each category relative to
the overall total. Whilst they are a useful tool to summarise data about one variable (e.g. what
proportion of your respondents answered in a particular way), they are not appropriate to explore
relationships between two or more variables.
They are accessed in Excel in the same way as for other charts –under the ‘Insert’ header, via the ‘Insert
pie chart’ button:
Figure 19
Scatterplots
Used to visualise the relationship between two numerical variables (i.e. not for data that represents
categories), scatterplots assign a point on a graph that corresponds to the value of a response to one
question (x-axis value) and the value of a response to a second question (y-axis value). This enables you
to examine whether and how responses to these two questions are correlated. For example, you may
be interested to see whether there is a relationship between respondents’ satisfaction with the
mentee/mentor matching process and their overall level of satisfaction with the mentoring
programme. A scatterplot chart assigns a point for each respondent on the basis of their responses to
both questions:
Page 49 of 100
Figure 20
Adding axis titles and a best-fit trendline (via the ‘Add chart elements’ button) can help to clarify what
relationship exists between responses to these questions.
Once you are happy with the formatting of your chart in Excel, it can be simply copy-and-pasted into a
Word document and used to visualise the findings of your evaluation, and as a means of summarising
and describing the responses to your survey.
5. Producing tables and charts using online survey tools

As discussed in How to… guide: 1. Carry out survey research, there are several online survey tools
available to help you administer your questionnaire. These include: SurveyMonkey, Qualtrics,
SurveyGizmo, PollDaddy, LimeSurvey, and many others. (NB you may have to subscribe to access certain
features).
Each of these services will have a different user interface, although they will all share similar basic
functionality. This includes being able to export the survey data directly into Excel, which enables you
to analyse the data and produce pivot tables, charts and graphs, as described above.
However, one of the benefits of using an online survey tool is that they will also be able to produce
tables and charts through the user interface, before exporting these directly into your evaluation
report.
Most online survey tools provide a thorough User Support service and helpful FAQs section to assist in
the data analysis using their own interface.
Page 50 of 100
For comparison, some of the above examples can be produced using Qualtrics and SurveyMonkey:
Qualtrics
In Qualtrics, the ‘Data & Analysis’ header has an option to create ‘Cross tabs’, through which you can
summarise respondents’ answers to two (or more) questions on your questionnaire (see below):
NB: Qualtrics does not provide ‘Total percentages’, so ‘Column percentages’ are given in blue
Figure 21
Page 51 of 100
Using the ‘Reports’ header, a number of options for visualising your data are available, including a range
of different graphs (‘visualisations’) that can be created and exported as image files, e.g. a column chart
(figure 22) or a stacked column chart (figure 23):
Figure 22
Figure 23
Page 52 of 100
SurveyMonkey
In SurveyMonkey, you can use the ‘+Compare’ function in the ‘Analyze Results’ page to create cross-
tabulations. Selecting the categories of one question you would like to compare (i.e. cross-tabulate)
with the others in your survey.
(Note, however, that more detailed analysis in SurveyMonkey often requires to upgrade to a paid
subscription to the service.)
Figure 24
NB: SurveyMonkey does not provide ‘Total percentages’, so ‘Column percentages’ are given in above numbers
Figure 25
Page 53 of 100
Graphs can be created on the same screen, using the ‘Customize’ tab to explore different ways to
present the data, amending the default charts that appear in the ‘Analyze Results’ page:
Figure 26
Figure 27
Page 54 of 100
Toolkit
6. How to…
analyse
textual and visual data
Page 55 of 100
Contents
1. Introduction to analysis of textual and visual data
2. Familiarisation with your data
3. Coding, indexing and organising your data
4. Exploring themes and relationship in your data
5. Drawing conclusions and presenting your data
6. Analysing visual data
Page 56 of 100
assist them to carry out programme evaluation through the analysis of textual and visual data. It
outlines the principal steps involved in collating, organising and analysing data collected as part of one-
to-one interviews and focus groups, including the use of visual research methods. It should therefore
be consulted in combination with the previous How to… guides: 2. Carrying out interviews, 3. Carrying
out focus groups and 4. Carrying out visual research methods.
This guide is intended to give a broad overview of how to organise and analyse responses to interviews
and focus groups in a thematic way, within the context of conducting a mentoring programme
evaluation. Through the application of thematic analytical techniques, it provides the user with a
framework within which they can present the key findings arising from the collected textual and visual
data.
1. Introduction to analysis of textual and visual data

Once you have collected your data through the use of one-to-one interviews or focus groups, you can
begin to explore what the data – in the form of transcripts, field notes, or still or moving images – are
saying and how this relates to the research or evaluation as a whole.
Data analysis is a process which involves several steps and requires you to make decisions about: how
you want to present the data, what story you want it to tell, what you want to include and what you
think is unnecessary, and how your findings link into the wider research/evaluation aims.
It is helpful to think about the following 5 issues when you are undertaking your data analysis:
What are the implications of the data? What is the data showing, what does it mean about your
respondents’ views and experiences?
What relationships and trends are evident? Is there any clear relationship between how respondents
answer? Are there trends over time, or are there any patterns amongst respondents with similar
backgrounds, or in the same location?
How do the findings relate to other information or literature? Are the findings of your analysis what you
expected to find? Are they surprising in any way? Think about why this might/might not be.
What actions might be considered as a result of the findings? How will the information you have revealed
be used to inform your practice or organisational strategy?
Is there additional information or research that should be conducted? Is there data that you were not
able to collect that you feel is important to obtain? Has anything been revealed that you would like to
know more about?
With these questions in mind, it is important to conduct your data analysis in a systematic way so that
you are clear about what story the data is telling and what patterns and trends are revealed. The next
section explains how to undertake a thematic analysis of your textual data, organising it using the
process of coding and by using themes as a framework for summarising your findings. Additionally, the
analysis of data elicited through visual methods is also outlined.
Page 57 of 100
2. Familiarisation with your data
The form that the data takes can vary according to how it was collected. Typically voice recorded
interviews or focus groups will be transcribed, to a greater or lesser extent. At one end of the spectrum,
you might have noted down only very short extracts or indicative quotes to summarise what was said;
at the other end, there is a complete verbatim transcription, undertaken from a recording at a later
date. Whilst a more complete transcription allows for a fuller analysis, there are clearly time and
resource implications involved in laboriously transcribing a large amount of data word-for-word.
Once you have your textual data, the first stage towards analysing it is to familiarise yourself with the
content. This involves reading and re-reading the transcripts literally – looking at the content and
structure of the dialogue – interpretively – gauging what the data mean and what the respondents are
describing – and reflexively – asking what is the subtext, what assumptions are being made and what
other factors are affecting the respondents’ answers.
3. Coding, indexing and organising your data

Once familiar with the data, the next step is to start thinking about the themes, patterns and
relationships that are arising from the data. Organising your initial thoughts can be done through the
process of coding.
Coding aims to reduce the amount of data by summarising it into smaller categories and to analyse the
data by extracting the meaning of what is being said. Common words, phrases or recurrent themes are
assigned a ‘code’ which expresses an underlying concept. The same code can be used to pick out
themes that occur within the same participant’s response, as well as across responses from different
participants.
An easy way of organising your initial codes is by using a coding frame. This can be derived from the
interview schedule or focus group topic guide, which will typically have been designed to explore
themes in a logical sequence.
For example, a mentee interview might have begun by exploring why someone joined the programme,
before asking what specific areas of advice they were seeking, how useful the training they received
was, etc. Organising the responses within a coding frame allows you to condense and summarise the
main themes in one response and highlight recurrent themes that occur across different interviews.
Code
Reasons for joining Specific areas of advice Training

Sub-code programme
Finance: Regulation Very useful
“I really didn’t have a clue “quote” “quote”
about how to get a
Quote business loan, etc.”
(Interviewee 1)
Networks Business management Needed more
“quote” “quote” “quote”
Mandatory Exporting
“quote” “quote”
Page 58 of 100
4. Exploring themes and relationships within the data
Once you have organised your initial codes, the next stage is to collate these codes and look at those
that cut across all transcripts and identify the main themes that emerge. Codes can be clustered under
an overarching theme, or removed from further analysis if it appears to be redundant. This is the point
at which you can begin to look at the linkages between the codes and themes, exploring relationships
across the whole dataset (i.e. that appear in all – or most – transcripts). You can also look for causal
relationships, trying to pinpoint what experiences lead to which responses, and whether common
responses exist amongst the respondents.
Organising your codes within a ‘classification tree’, as shown, provides you with a visual guide as to how
you can summarise the findings arising from your data:
In drawing conclusions within each theme, think not just about the evidence that supports your point,
but also whether any opposing views are expressed and why this might be the case. Exploring both the
consistencies and contradictions that exist in the data in this way will allow you to present ‘generic’
findings whilst also allow for the examination of case-specific issues.
5. Drawing conclusions and presenting your data

The better organised your codes and themes are, the clearer you will be able to present your findings.
Using these themes as a framework, you should be able to discuss what conclusions can be drawn from
your data, backing these assertions up with evidence in the form of quotes from your respondents.
As outlined in the introduction to the guide, you should return to the key issues when choosing how
best to summarise and present your data:
 What are the implications of the data?

 What relationships and trends are evident?
 How do the findings relate to other information or literature?
 What actions might be considered as a result of the findings?
 Is there additional information or research that should be conducted?
More detailed information on how to write and present your evaluation report can be found in the How
to.. guide: 7. Writing an evaluation report.
Page 59 of 100
6. Analysing visual data
As discussed in more detail in How to… guide: 4. Carrying out visual methods research, there are several
ways of using visual methods in research and evaluation. The use of visual research methods typically
complements other research methods and is often used in conjunction with, or as part of, a wider
interview or focus group discussion.
Depending on the type of visual research methods used, how the data produced is analysed also varies.
As outlined in the previous guide, visual data can be researcher-produced (i.e. using images or
recordings to capture an interaction or exchange); it can be used as a way of stimulating a dialogue
between researcher and participant (e.g. photo-elicitation); or it can be participant-produced (i.e. with
participants producing or selecting a visual representation to convey how they feel about an issue or to
illustrate their attitude towards a situation).
Clearly, the latter two methods elicit textual data through participants’ responses in relation to a
visually-inspired stimulus. Researcher-produced video recordings of participants can also provide
textual data in the form of transcribed verbal interactions. This data can be included in standard textual
data analytical techniques, as outlined above. However, importantly, you have visual information that
you can include to evidence your findings and illustrate (literally) the conclusions you present.
When there is no textual data produced by the participants, it remains up to the researcher producing
and collating the visual data to interpret the images in a thematic way. For example, if photographs are
being taken of a number of mentor/mentee interactions, are there commonalities or differences
between body language, facial expressions, how the participants are positioned in relation to each
other, etc.? Here the evidence is purely visual and the researcher’s interpretation of what is going on
provides the overarching narrative. Undertaken by an expert in the field, however, this can also be an
effective method of evaluation.
Page 60 of 100
Toolkit
7. How to…
write an
evaluation report
Page 61 of 100
Contents
1. Executive summary and front matter
2. Description of the programme
3. Description of the evaluation
4. Writing up the findings
5. Conclusions and recommendations
Page 62 of 100
assist them to write a comprehensive evaluation report following data collection. It outlines the
principal steps involved in presenting the overall evaluation project, its purpose, methodology and
findings. To this end, this guide should be consulted (where appropriate) along with the other How to…
guides in this toolkit:
 1: How to… carry out survey research;

 2: How to… carry out interview research;
 3: How to… carry out focus group research;
 4: How to… carry out visual methods research;
 5: How to… analyse numerical data; and
 6: How to… analyse textual and visual data.
This guide takes you through step-by-step each of the sections that your final programme evaluation
report should include and provides broad advice and guidance as to how you should approach writing
each of them. Of course, the exact layout and content of the report will depend on the evaluation’s
purpose, methods, and findings and this guide should be seen as a starting point from which to adapt
your report to the specific context of your own research.
1. Executive summary and front matter

Your final report should be given a clear title that lets readers know what the evaluation or research is
about. You should provide a table of contents in your report to allow readers to locate a specific section
of interest. If your report contains lots of tables, charts or images, consider also including a ‘list of
figures’ with your table of contents.
It is good practice to include a short Acknowledgements section out the outset of the report, outlining
the contribution made to your research by participants, colleagues and/or collaborators. This may also
contain contact information for the authors of the report to allow readers to follow up or query any
point in relation to the research.
It is a useful idea to include an Executive Summary as a brief overview of what is contained in the full
report. The idea of the executive summary is to provide a short summary for people who may not read
the whole document. For this reason, it should cover the main points, highlighting key findings and
recommendations from the evaluation. The summary should be around 1 to 2 pages in length and may
be disseminated separately from the report, providing an efficient means of sharing key findings with
a larger audience or a potential funder.
It is advisable to write the executive summary after the full report is finished. You should read the
original document fully and highlight the key ideas and important points. You can then re-write the
highlighted sentences briefly, skipping the unimportant details. The executive summary should contain
the following details in brief form:
 Purpose of the programme

 Programme activities, setting and target population
 Purpose and scope of the evaluation
 How the evaluation was conducted
Page 63 of 100
 Overview of findings or outcomes
 What the main recommendations are
Think about who the potential audience will be for the executive summary – it is likely to have a broader,
less specialist readership than the full report.
2. Description of the programme

The first main section of the full report should provide an introduction to the programme that is the
target of the evaluation. Here, you should give an overview of the programme, outlining its purpose
and objectives, who it is designed to help, what the planned outputs, outcomes and impacts are, and
how the activities are intended to meet its objectives.
How extensive the description should be will depend on whether the intended audience has any
previous knowledge of the programme or not. You should not assume that everybody will know; at
the same time, you should not burden your readers with a lot of unnecessary or technical details.
Things you should include in this section of the report include:
 Origin of the programme: how the programme began, why it was set up and what its
original mission was.
 Aims of the programme: what are the programme’s purpose and objectives? What are
the planned outputs, outcomes and impacts?
 Programme activities: how the programme is put into operation or delivered.
 Participants in the programme: who is the target population for the programme? Who
are the participants? Who delivers the programme?
 Revisions to the programme: how the programme has changed since it began; whether
there have been any major changes in objectives or activities – and why these were
made.
3. Description of the evaluation

Following the description of the programme, the next section of the report should describe why and
how your evaluation was undertaken. In this section, you should explain to the reader what the purpose
and scope of the research was, and describe how and why you selected the research methods you used.
In terms of the purpose of the evaluation, think about whether this was in response to a particular
situation (e.g. following an expansion of the programme) or at the request of a specific group of
stakeholders (e.g. in response to participants’ concerns over one aspect of the programme). Is the
evaluation being carried out to establish a benchmark for future evaluations? Is this part of a regular
series of evaluations?
You should also be clear as to the scope of the evaluation undertaken, outlining what it does and does
not intend to achieve. Is it focused on answering a specific set of research questions? What are the
objectives of the evaluation? Objectives of the evaluation might include: assessing the relevance,
Page 64 of 100
effectiveness, efficiency, impacts and sustainability of the project and its activities. However, these
should be realistic, in line with the time and financial resources available.
This section should also include a description of the research methods you used to carry out your
evaluation:
 Did you use surveys, interview or focus groups? Did you use more than one of these? Did you
combine them with visual methods? What type(s) of data did you collect?
You should provide an explanation of how the evaluation was designed and why the specific methods
were selected. For example, whether you were trying to obtain a broad representation of how satisfied
participants were with the programme, or whether you wanted to look at how individual experiences
shape participants’ practices.
This section also outlines who took part in the evaluation, why you selected (or excluded) specific
groups of people. Did different groups of people take part in different stages of the evaluation? Did the
same people participate in more than one stage? How many people took part and how were they
approached? You should also outline how you addressed any ethical issues related to the research (e.g.
maintaining confidentiality and anonymity) and how you sought to minimise risk to participants (e.g.
potential conflicts of interest, attention to cultural sensitivities, etc.).
You should also provide a description of your research tools (e.g. questionnaire or interview schedule)
or include them in full as an appendix to the report. This explains to the reader how you intended to
answer the research questions or fulfil the objectives of the evaluation.
4. Writing up your findings

Once you have set out the programme and detailed why and how your evaluation was undertaken,
your report should present the findings that emerge from the research. Your findings should be
organised in a coherent way and focus on answering your research questions or addressing the
objectives, as outlined in the previous section.
This is the main part of the report as it presents the information that comes out of the data you have
collected. Writing up your findings is a process of presenting your data analysis, whether this is using
tables and charts to summarise survey data (see How to… guide: 5. Analysing numerical data) or
interpreting key themes elicited from participants in interviews and focus groups (see How to… guide:
6. Analysing textual and visual data), or a combination of both.
The first step in presenting your findings involves organising the data you have collected in a way that
will allow you to address your objectives. Think about the types of information you will need to
summarise to determine whether or not you have met your objectives. Read through your analysis and
ensure that it is understandable to your readers.
Use your research questions or stated objectives as a framework to present your findings. Under each
of the objectives, present the evidence from the data analysis and describe how it relates to meeting
the objective. Use data tables, charts or illustrative quotes to provide examples of your interpretation
of the findings. Explore alternative explanations, if appropriate, and offer what you deem to be the best
explanation. Discuss what your findings highlight and how they relate to each other. Are they what was
expected, or did something new and unanticipated emerge?
Page 65 of 100
Example:
One of your objectives was to evaluate to what extent participants were satisfied with the
programme and why this was the case.
Presenting your findings in relation to overall satisfaction could involve:
1) Presenting the proportion of participants who were satisfied or not satisfied with the
programme in a data table:
% of
No.
total
Very dissatisfied 4 11.4%
Dissatisfied 7 20.0%
Neutral/Don’t know 2 5.7%
Satisfied 8 22.9%
Very satisfied 14 40.0%
Total 35 100%
The table shows that most respondents (62.9%) were satisfied with the programme, with
two-fifths stating that they were “very satisfied”. However, a significant minority (31.4%) of
respondents reported being “dissatisfied” or “very dissatisfied” with the programme.
2) Presenting the main themes that emerged from interviews with (satisfied) participants as to
why they were satisfied:
When asked why they were satisfied with the programme, a key theme that emerged from
interviewees was that they felt that it had helped to improve their business. For example,
one mentee, Robert, stated: “I couldn’t have done it without the programme; it has
increased my turnover in the short time I have been involved”. Another participant, Sandra,
spoke about how the programme had improved her business’s reach: “I have been able to
access a much wider range of clients since working with my mentor”.
3) Presenting the main themes that emerged from interviews with (dissatisfied) participants as
to why they were not satisfied:
Of those participants who reported not being satisfied, the main reason appeared to be a
lack of visible results. Harry, who has been on the programme for more than 12 months said
that he had “not seen much difference in turnover or profits”. Maria agreed that her
business had “made only small progress since I began in the programme; I am still waiting
to see a real improvement”.
Page 66 of 100
5. Conclusions and recommendations
The final section of the report seeks to bring together the findings of the evaluation and draw
conclusions from what has been presented in the previous section. It should also provide
recommendations for what might be changed or implemented in the future.
Conclusions and recommendations should be stated clearly and precisely; they might be best presented
as a list of bullet points highlighting the main findings and suggested actions to achieve maximum visual
impact.
Your conclusions section should focus on interpreting your findings. In particular, think about:
 What do your results say about your programme? Is this what was expected?
 Were the programme’s objectives met or not – why was this the case?
 Were there any external circumstances that impacted upon the programme’s effectiveness?
 Did different research methods or stages elicit different results? What are the implications of
this?
 Pay attention to and discuss ‘deviant’ or extreme cases – why might these have occurred?
 How have your results changed or stayed the same since the previous evaluation? Do your
results differ from evaluations of similar programmes?
 What other information would you like to have to assist in making further recommendations?
Think about how the conclusions of the evaluation can underpin recommendations for the
improvement or modification of the programme. As you list your recommendations, it is useful to
explain the basis for each recommendation and why you believe a particular recommendation will
improve the programme. You should aim to include a reasonable number of recommendations that
you feel are realistic to implement and which will have the biggest impact. Whilst your suggestions
might pertain to the specific programme under evaluation, think about including recommendations
that might have wider implications for other stakeholders, organisations, funders or communities.
Example:
 It is recommended that induction training for mentors is delivered in a more formal and
systematic way, and is available in a range of formats.
At the moment, mentors are required to undertake self-directed training online before they
embark on the programme. The findings of the evaluation show that a significant number
of mentors feel that the induction training is not adequate. Participants in the evaluation
also emphasised that they would like to received training on a one-to-one or group basis,
in addition to the current self-directed format.
Page 67 of 100
Page 68 of 100
Toolkit
Appendix 1:
Questionnaire templates
Page 69 of 100
Questionnaire template – long (20 questions)
[Organisation name]
[Street address]
[City, State, Postal code]
[Phone no.]
Mentoring Programme Evaluation

Please take a few minutes to fill out this survey. We are currently evaluating our mentoring
programme and, as a mentor/mentee who has participated in the programme, we welcome your
feedback. Your experiences will help to shape the programme’s future development. The survey
should take around 30 minutes and all your answers will be kept confidential. Thank you for your
participation.
Profile
Q1. What age are you?

 18 to 24 years  25 to 34 years  35 to 44 years
 45 to 54 years  55 to 64 years  65 years or over  Prefer not to say
Q2. Are you…?

 Male  Female
 Other  Prefer not to say
Q3. Do you consider yourself to be part of any ethnic, national, cultural or social minority
group?
 Yes (please state) ___________________________  No  Prefer not to say
Q4. What is the highest form of educational or professional vocational qualification that you
have attained?
 Postgraduate degree (e.g. Masters)  Undergraduate degree (e.g. Bachelors)
 Higher education diploma/certificates  Secondary education
 Primary education  Other professional/vocational qualifications
 No formal qualifications  Other (please explain) ________________
Programme Participation
Q5. How long have you been in your current mentoring relationship?
 Less than 3 months  3 to 6 months  6 to 12 months
 12 to 18 months  18 to 24 months  Over 24 months
Page 70 of 100
Q6a. At what stage of development was your/your mentee’s business when you started the
mentoring programme?
 Just an idea (e.g. without a business plan in place)
 Planned but not yet started trading (e.g. with a business plan in place)
 Just started trading
 Established trading and ready to grow
 Established trading and growing
 Other (please explain) _______________
Q6b. What stage of development is your/your mentee’s business currently at?

 Just an idea (e.g. without a business plan in place)
 Planned but not yet started trading (e.g. with a business plan in place)
 Just started trading
 Established trading and ready to grow
 Established trading and growing
 Other (please explain) _______________
MENTEES ONLY: Q7. Why did you decide to participate in the mentoring programme?
Select all that apply.
 It was compulsory for me to participate  To improve my business management skills
 To improve technology skills  To improve professional skills
 To improve my personal skills  To access advice to set up my business
 To improve access to new business contacts  To find ways to access new markets
 To access advice to grow and develop my business skills
 To access external finance and improve
financial skills
 To develop my business plan  To understand my strengths & developmental
needs
 Other (please explain) _______________________________________
Training
Q8. Did you receive any induction training before starting your mentoring relationship?
 Yes |  No
Q8b. If yes, how useful was the training to you?

No use at all      Very useful
Q9. Since starting on the mentoring programme, have you received any other training?
 Yes |  No
Q9b. If yes, how useful was the training to you?

No use at all      Very useful
Page 71 of 100
Mentor/Mentee Matching
Q10. Is your mentor/mentee the same age/gender/ethnic or social background to you?

a)  Same age  Younger than me  Older than me  Not sure/Don’t know
b)  Same gender  Different gender  Not sure/Don’t know
c)  Same ethnic or social background  Different ethnic or social background  Not sure/Don’t know
Q11. Overall, how effective has the mentee and mentor matching process been for you?
Not effective at all      Very effective
Q11b. Please explain your answer in more detail:
Mentoring Evaluation
Q12. What evaluation tools have been used to assess your mentoring relationship?
Select all that apply.
 Online survey
 Email feedback
 Handwritten feedback
 Verbal feedback
 Formal face-to-face meeting
 Other (please state) _____________________
Q13. How often is your mentoring relationship evaluated by your organisation?

 Weekly
 Twice monthly
 Monthly
 Quarterly
 Less frequently than quarterly
 Not sure/Don’t know
Programme Effectiveness
Q14. How effective has the mentoring programme been in meeting your initial expectations?
 Exceeded initial expectations  Met all initial expectations  Met some but not all expectations
 Met only very few expectations  Did not meet expectations at all
Page 72 of 100
Q15. To what extent has the mentoring programme been able to assist you/your mentee in
the following areas?
Not at all A little Somewhat Significantly Completely
Facilitated access to finance     

Assisted with starting-up a business     
Assisted with developing an existing
    
business
Access to new markets     
Lead to increased sales performance     
Helped save a business and stabilise it     
Helped create new jobs in the
    
business
Become more innovative     
Improved business management skills     
Improved business financial
    
management skills
Improved business presentation and
    
promotional skills
Improved business networks     
Improved confidence in running a
    
business
Increased personal communications
    
skills
Increased personal confidence and
    
self esteem
Improved decision making     
Developed new markets     
Improved technology skills     
Developed my business plan     
Learned about my personal strengths
    
and development areas
Mentoring Relationship
Q16. How easy has it been to establish a relationship of trust with your mentor/mentee?
Very difficult      Very easy
Q17. How easy was it to agree clear objectives for your mentoring relationship?
Very difficult      Very easy
Page 73 of 100
Q18. Which of the following roles did your mentor/you fulfil?

Select all that apply
 Adviser  Coach  Confidant  Consultant  Counsellor
 Friend  Guardian  Listener  Role model  Sponsor
 Teacher  Other (please state) ______________________________
Q19. Overall, how satisfied are you with your mentoring relationship?
Very dissatisfied      Completely satisfied
Q19b. Please explain why in more detail:
Overall Impact
Q20. What has been the overall impact of the programme on your/your mentee’s business?
Very negative      Very positive

E.g. Have you seen any improvements due to the programme; has turnover increased or decreased; has there
been a change in the size or scale of the business
Additional Feedback
Please list any areas in which the programme could be improved:
Page 74 of 100
Please share any additional comments:
Thank you for taking the time to fill out our survey. We rely on your feedback to help us improve our
programme and your input is greatly appreciated.
If you have any queries about this survey or evaluation research, please contact:
[Insert name and contact details here]
Page 75 of 100
Questionnaire template – medium (10 questions)
[Organisation name]
[Street address]
[Phone no.]

should take around 20 minutes and all your answers will be kept confidential. Thank you for your
participation.
Profile

Q2. Are you…?

 Male  Female
Q4a. At what stage of development was your/your mentee’s business when you started the
mentoring programme?
 Just an idea (e.g. no business plan in place)  Planned but not yet started trading (e.g.
business plan in place)
 Just started trading  Established trading and ready to grow
 Established trading and growing  Other (please explain)
__________________________
Q4b. What stage of development is your/your mentee’s business currently at?

 Just an idea (e.g. no business plan in place)  Planned but not yet started trading (e.g.
business plan in place)
 Just started trading  Established trading and ready to grow
 Established trading and growing  Other (please explain)
__________________________
Page 76 of 100
Q5. Is your mentor/mentee the same age/gender to you?

a)  Same age  Younger than me  Older than me  Not sure/Don’t know
b)  Same gender  Different gender  Not sure/Don’t know
Q7. How effective has the mentoring programme been in meeting your initial expectations?
 Exceeded initial expectations  Met all initial expectations  Met some but not all expectations
 Met only very few expectations  Did not meet expectations at all

    
business
    
business
    
management skills
Page 77 of 100

    
promotional skills
    
business
    
skills
    
self esteem
    
Q9b. Please explain why in more detail:
Overall Impact

E.g. Have you seen any improvements due to the programme; has turnover increased or decreased; has there
been a change in the size or scale of the business
Page 78 of 100
Additional Feedback
Page 79 of 100
Questionnaire template – short (7 questions)
[Organisation name]
[Street address]
[Phone no.]

should take around 5-10 minutes and all your answers will be kept confidential. Thank you for your
participation.
Profile

Q2. Are you…?

 Male  Female
Page 80 of 100

    
business
    
business
    
management skills
    
promotional skills
    
business
    
skills
    
self esteem
    
Page 81 of 100
Overall Impact
Additional Feedback
Page 82 of 100
Toolkit
Appendix 2:
Interview schedule
templates
Page 83 of 100
Interview schedule template – long
INTERVIEW SCHEDULE

Thank you for agreeing to take part in this interview. It should take approximately 45 minutes to an
hour.
We are currently evaluating our mentoring programme and, as a [e.g. mentor/mentee], we are keen
to hear about your experiences and ideas to help shape the programme’s future development.
There are no right or wrong answers; your thoughts and experiences are what matters. Anything you
say will be treated in the strictest confidence and your identity and personal information will be
protected and anonymised in any published material, although we may use direct quotes from your
interview for the purposes of illustrating the findings of the research. To help us analyse your
answers later, would it be okay to record the interview using a voice recorder?
The questions will focus primarily on your professional experiences in relation to the programme,
however, you are free to choose not to answer any specific question, or to withdraw from the
research at any time, without giving a reason.
[We would like to interview you again in the future and, with your permission, will be in contact
again in approximately 6 months’ time].
Thanks again for your participation.
1. Participant profile, background and experience
1.1 Tell me a bit about yourself

(Prompts: background, business history, current role and experience)
2. Entry onto programme and current engagement
2.1 How long have you been working with your mentor [/mentee] on the programme?
E.g. New to the programme, mid-point, end point
2.2 What, if any, is your previous experience of mentoring?
2.3 Why and how did you join the programme?
(Prompts: mandatory, access finance, business planning, specific knowledge & skills, HR, Strategy,
Marketing)
2.4 What specific areas did you needed support with [/could you provide expertise in]?
(Prompts: issues around regulation; exporting to international markets, work-life balance, access to
finance etc.)
Page 84 of 100
3. Training and support
3.1 What training and/or support did you receive prior to starting your mentoring relationship; what
was most useful and why?
3.2 What training and/or support did you receive during your mentoring relationship; what was most
useful and why?
3.3 What additional training and/or support would help you during your mentoring relationship?
4. Exploring the mentoring relationship
4.1 What were you looking for in a mentor [/mentee] (qualities, experience, industry, personality,
male/female etc.)?
4.2 How well matched do you feel that you are with your mentor and why (did the mentor [/mentee]
meet your expectations)?
(Prompts: What matching processes/approaches were used and what worked well or less well?)
4.3 How do you meet or have contact with your mentor and how effective was this?
4.4 Who steered the conversation and took responsibility for the managing the mentoring
relationship?
4.5 What were the main topics you discussed in your mentoring relationship?
(Prompts: business planning, laws & regulation; exporting to international markets, work-life
balance, access to finance etc.)
4.6 How did the topics change at different points in your relationship (beginning, middle, end)?
4.7 What key skills did your mentor [/mentee] demonstrate at different points in your relationship
(beginning, middle, end)?
5. Exploring mentoring relationship outcomes
5.1 How do you reflect on the outcomes of your mentoring relationship?
5.2 How have you benefitted professionally (and personally) from the mentoring relationship?
(Prompts: Improved confidence, communication skills, networking, access to new markets,
overcoming barriers, etc.)
5.3 How has your [/mentee’s] business benefitted from the mentoring relationship?
(Prompts: increase in revenue, profit, new employees, new markets, innovation)
5.4 Can you describe a specific occasion where you had a positive experience with your mentor
[/mentee]?
5.5 Can you describe a specific occasion where you and your mentor [/mentee] overcame a
challenge/obstacle together?
5.6 If you were a mentor [/mentee], what, if anything would you change about your relationship?
Page 85 of 100
6. Evaluation of the mentoring programme
6.1 What improvements, if any, would you suggest could be made to the programme to support you
[/your mentee] and your [/your mentee’s] business? How should these be adopted?
6.2 Have you received any other forms of support and how useful have they been?
(Prompts: funding, consultant advice etc.)
6.3 To what extent has mentoring helped you and your business in comparison to other forms of
support?
6.4 What other support would you like to receive from the organisation and why?
Thank you for taking part in the interview. Your input is greatly appreciated and will help to improve
our programme in the future. We will let you know when the findings of this research are published
and where you will be able to access them.
In the meantime, if you have any queries about this interview or evaluation research, please contact:
[Provide name and contact details here]
[With your permission, we would like to interview you again in 6 months’ time. Would that be
okay?]
Page 86 of 100
Interview schedule template – medium
INTERVIEW SCHEDULE

Thank you for agreeing to take part in this interview. It should take approximately 20-30 minutes.

(Prompts: background, business history, current role & experience)
Marketing)
2.2 What specific areas did you needed support with [/were you able to provide expertise in]?
(Prompts: issues around regulation; exporting to international markets, work-life balance, access to
finance etc.)
3.1 What training and/or support did you receive prior to starting your mentoring relationship?; what
about during your relationship?; what was most useful and why?
Page 87 of 100
Interview schedule template – medium
3.2 What additional training and/or support would help you during your mentoring relationship?
4.1 What were you looking for in a mentor [/mentee] (qualities, experience, industry, personality,
male/female etc.)?
4.2 How well matched do you feel that you are with your mentor [/mentee] and why (did the mentor
[/mentee] meet your expectations)?
(Prompts: business planning, laws/regulation; export to international markets, work-life balance,

access to finance)
5.2 How have you benefitted professionally (and personally) from the mentoring relationship?
(Prompts: Improved confidence, communication skills, networking, access to new markets,

overcoming barriers, etc.)
5.3 How has your [/your mentee’s] business benefitted from the mentoring relationship?
okay?]
Page 88 of 100
Interview schedule template – short
INTERVIEW SCHEDULE

Thank you for agreeing to take part in this interview. It should take approximately 10-15 minutes.
(Prompts: background, business history, current role & experience)

Marketing)
3.1 What training and/or support have you received in relationship to your mentoring relationship?;
what was most useful and why?
Page 89 of 100
Interview schedule template – short
4.1 How well matched do you feel that you are with your mentor [/mentee] and why (did the mentor
(Prompts: business planning, laws and regulation; exporting to international markets, work-life
(Prompts: How have you benefitted? – Improved confidence, communication skills, networking,
access to new markets, overcoming barriers, etc.)
okay?]
Page 90 of 100
Toolkit
Appendix 3:
Focus group topic guide
templates
Page 91 of 100
FG topic guide template – long
FOCUS GROUP TOPIC GUIDE

Thank you all for agreeing to take part in this focus group. The discussion should take approximately
1 hour to 90 minutes.
We are currently evaluating our mentoring programme and, as [e.g. mentors/mentees], we are keen
protected and anonymised in any published material, although we may use direct quotes for the
purposes of illustrating the findings of the research.
To help us analyse your answers later, my colleague will be taking notes and we would also like to
record the focus group using a tape recorder. This is a group discussion, so feel free to respond to
me and to other members of the group. However, it is important that everyone can hear what
others are saying, so if only one person could speak at a time, it will also allow the tape recorder to
pick up clearly what is said. Can I also ask that all mobile phones are put on ‘silent’ or turned off.
[We would like to speak to you again in the future and, with your permission, will be in contact again
in approximately 6 months’ time].
1.1 Let’s start by everyone introducing themselves and how long they have been involved in the
programme
2.1 What do you feel are some of the main challenges facing young entrepreneurs at the moment?
(Prompts: access to business networks, lack of experience, access to finance, negative

stereotypes)
2.2 How does the mentoring programme seek to overcome these challenges?
Page 92 of 100
2.3 Why did you decide to join the programme?
(NB: allow everyone to give their own motivations)
Marketing)
3.1 Tell me about the kind of training and/or support you received prior to starting the mentoring
relationship
(Prompts: what was most useful and why?)
3.2 What about training and/or support you received during your mentoring relationship?
3.3 What additional training and/or support do you think would be useful?
4.1 How do you reflect on the mentor/mentee matching process?
 What were you looking for in a mentor [/mentee] (qualities, experience, industry, personality,
male/female etc.)?
 How well matched do you feel that you are with your mentor and why (did the mentor
 What matching processes/approaches were used and what worked well or less well?
4.2 Can anyone speak about how their mentoring relationship works in practice?
 How often do you meet or have contact with your mentor and how effective was this?
 Who steered the conversation and took responsibility for the managing the mentoring
relationship?
 What were the main topics you discussed in your mentoring relationship?
4.3 Does anyone have an example of how their relationship has developed over time
 did the topics change at different points in the relationship (beginning, middle, end)?
 What key skills did your mentor [/mentee] demonstrate at different points in your
relationship (beginning, middle, end)?
Page 93 of 100
 How have you benefitted professionally (and personally)?
(Prompts: Improved confidence, communication skills, networking, access to new markets, etc.)
 How has your [/mentee’s] business benefitted?
5.2 Can anyone describe a specific occasion where they had a positive experience with their mentor
[/mentee]?
5.3 Can anyone describe a specific occasion where they and their mentor [/mentee] overcame a
challenge/obstacle together?
6.1 What improvements, if any, would you suggest could be made to the programme
(Prompts: different types of support; How could these be adopted?)
6.2 Have you received any other forms of support and how useful have they been?
(Prompts: funding, consultant advice etc.)
6.3 Overall, to what extent has mentoring helped you and your business in comparison to other
forms of support?
7. Other comments
7.1 Do you have any further comments you’d like to add about the support you have received?
Thank you for taking part in the focus group. Your input is greatly appreciated and will help to
improve our programme in the future. We will let you know when the findings of this research are
published and where you will be able to access them.
In the meantime, if you have any queries about this focus group or evaluation research, please
contact:
[With your permission, we would like to speak with you again in 6 months’ time. Would that be
okay?]
Page 94 of 100
FG topic guide template – medium

45 minutes to an hour.
programme
Marketing)
Page 95 of 100
3.1 Tell me about the kind of training and/or support you received in relation to the mentoring
relationship
3.2 What additional training and/or support do you think would be useful?
 What were you looking for in a mentor [/mentee] (qualities, experience, industry, personality,
male/female etc.)?
 How often do you meet or have contact with your mentor and how effective was this?
 Who steered the conversation and took responsibility for the managing the mentoring
relationship?

Page 96 of 100
6.2 Overall, to what extent has mentoring helped you and your business in comparison to other
forms of support?
7. Other comments
contact:
okay?]
Page 97 of 100
FG topic guide template – short

20 - 30 minutes.
programme

Marketing)
Page 98 of 100
3.1 Tell me about the kind of training and/or support you received in relation to the mentoring
relationship.
7. Other comments
Page 99 of 100
contact:
okay?]
Page 100 of 100


Research and Evaluation Toolkit

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Research and Evaluation Toolkit

Uploaded by

Copyright:

Available Formats

Research & Evaluation

Dr Julie Haddock-Millar, Neil Kaye,

1. Introduction to survey research

2. Designing your questionnaire

 What is the objective of your research or evaluation?

 Who are you seeking to obtain information from?

All surveys contain the following elements:

2) Introduction – this provides a brief explanation of the research or evaluation

Open Questions Closed Questions

(write in your answer) ………………………………………………………………………………………………..

3. Carrying out your survey

a) How long it takes to complete the survey

1. Introduction to interview research

2. Designing your interview schedule

Before designing your interview schedule, you need to be clear about:

 Who are you seeking to obtain information from?

A semi-structured interview schedule will usually contain the following elements:

1) Introduction and ethical consent

3. Arranging your interviews

Elucidate inconsistencies and tensions by highlighting and exploring where a participant

1. Introduction to focus group research

2. Designing your focus group topic guide

 What is the objective of your research or evaluation?

 Who are you seeking to obtain information from?

1) Introduction and ethical consent

‘How do you reflect on the outcomes of your mentoring relationship?’

4. Carrying out your focus group

 asking participants to turn off mobile phones;

The moderator should encourage explanation or clarification of stories or anecdotes by asking

1. Introduction to visual methods research

 researcher-produced visual data (e.g. photographs of on-going practice);

 What is the objective of your research or evaluation?

 Who are you seeking to obtain information from?

 Photographing participants whilst they are undergoing their training

 Filming meetings between mentees and mentors

 Recording participants within the context of their business practice

 Constructing a relational diagram of how participants interact with business partners,

4. Using pre-existing visual data

 How to interpret what is going on in the photo/video/etc.

 What thought processes are occurring

 How this relates to their own practices

 Recalling their own practices

 Examine the motivation and meaning behind their own practices

5. Participant-produced visual data

 Always obtain informed consent – especially if photographing or recording participants or

1. Introduction to survey data analysis

1. Enter the data manually (see below).

SUM – returns the sum total of a range of cells

COUNT – counts the number of cells in a range that contains numbers

AVERAGE – returns the average (mean) of selected numbers

3. Using pivot tables to summarise your data

The resulting table provides a cross-tabulation of respondents’ age group by gender:

Female Male Other Total

Bar charts and column charts

5. Producing tables and charts using online survey tools

1. Introduction to analysis of textual and visual data

3. Coding, indexing and organising your data

Reasons for joining Specific areas of advice Training

5. Drawing conclusions and presenting your data

 What are the implications of the data?

 1: How to… carry out survey research;

1. Executive summary and front matter

 Purpose of the programme

2. Description of the programme