Professional Documents
Culture Documents
Partnership Satisfaction Impact Survey Final
Partnership Satisfaction Impact Survey Final
FITT Project
Page 1 of 21
Interreg IVB
Project no. 020A
FITT Project
TABLE OF CONTENTS
Contents
I
INTRODUCTION ........................................................................................................................ 3
II
SATISFACTION SURVEY......................................................................................................... 4
II.1
What? ........................................................................................................................................ 4
II.1.1 Definition ............................................................................................................................... 4
II.1.2 Satisfaction survey in Practice ............................................................................................... 4
II.1.3 Pros & Cons ........................................................................................................................ 7
II.2
Why?.......................................................................................................................................... 7
II.2.1 Rationale: Why was this established?.................................................................................... 7
II.2.2 Impact: Why is it included in the process as an essential part? ............................................. 8
II.3
Outcome .................................................................................................................................... 8
II.4
II.5
Another approach to satisfaction survey................................................................................ 9
II.5.1 What? ..................................................................................................................................... 9
II.5.2 Why?.................................................................................................................................... 10
II.5.3 Outcome............................................................................................................................... 10
III IMPACT ASSESSMENT........................................................................................................... 11
III.1 What? ...................................................................................................................................... 11
III.1.1
Definition......................................................................................................................... 11
III.1.2
Impact Assessment in Practice ........................................................................................ 11
III.1.3
Pros & Cons .................................................................................................................. 13
III.2 Why?........................................................................................................................................ 13
Rationale.......................................................................................................................... 13
III.2.1
III.2.2
Impact: Why is it included in the process as an essential part? ....................................... 14
III.3
Outcome .................................................................................................................................. 14
III.4
IV ANNEX 1 ..................................................................................................................................... 15
Page 2 of 21
Interreg IVB
Project no. 020A
FITT Project
INTRODUCTION
Partnership management is essential to assure the best relationship and cooperation, business or
research, between TTOs and industrial partners. A good and structured partnership management
facilitates the daily work of Technology transfer officers and increases the chances to successfully
transfer research results.
Based on experiences, knowledge and expertise within the FITT consortium, three practices have been
identified as crucial in partnership management:
The Satisfaction Survey enables to identify weaknesses and strengths of partnerships and to improve
on future collaborations. It can result in establishing mutual trust and long-lasting relationships, as
well as gaining the partners loyalty.
The Impact Assessment enables to measure the impact of transferred research results to industrial
partners by identifying the benefits coming from the transferred technology and the impact of the
transferred results on the partners organizations. On the other hand, Impact Assessment is an effective
tool to justify technology transfer activities to various stakeholders, mainly policy makers.
This document explains these two practices and identifies the value added for each of them from the
Technology Transfer Officers point of view. The examples given come from the Public Research
Center Henri Tudor & INRIA.
Partnership Agreement is treated in a separate practice in the FITT Toolbox.
Page 3 of 21
Interreg IVB
Project no. 020A
FITT Project
II
SATISFACTION SURVEY
II.1 WHAT?
II.1.1 Definition
The satisfaction survey is a powerful tool that shows how the organization is perceived by
partners1. In this manner, the organization has a continuous feedback on its activities and can
assess its role in the economical and social environment. This satisfaction survey involves some
organizational impacts like:
Another version of the satisfaction survey includes a mirror survey: a survey carried out
internally by interviewing the staff that has been involved in a project carried out in collaboration
with a partner. This additional survey enables to identify the gap between partners satisfaction
degree and its perception from inside of the organization.
The evolution of the results over time proofs whether the new policies, actions and measures have
improved the way in which the organization interacts and exchanges with its partners.
II.1.2 Satisfaction survey in Practice
The satisfaction survey evaluation consists of specific parts which will be explained in detail in
the following section of the document:
1. Selection of satisfaction indicators
2. Getting the questionnaire ready
3. Selection of the partners to be interviewed by phone followed by a face to face interview
(based on the answers from the phone interview)
4. Running the survey
5. Results analysis and reporting
6. Results communication
The quality of the provided services (training sessions, project results, etc).
In the context of PROs the word partners represents either clients or partners with a contractual or noncontractual relationship. Throughout the document we will use the word partner for both partners and clients,
without distinction.
CRP Henri Tudor
INRIA
Page 4 of 21
Interreg IVB
Project no. 020A
FITT Project
The quality of the relationships between partners and organizations business people (persons
in contact with partners: responsible of external relationship, responsible of project definition,
etc).
The relationship with the teams in charge of the execution of the project or of those providing
the services (project team staff, trainer, etc.).
The administrative aspects (billing, partnership agreements, contractual matters, etc.).
Additional elements of the services provided (other services, documentation, support, etc).
Logistic aspects (delays, quality of the project management).
Communication aspects (quality of the information shared, access to the information, etc.).
Follow-up (partner support services, exploitation of the results).
Quality/price ratio if relevant.
A specific one for training and educational activities if it is relevant to your organization.
Another one for other services (professional services, participation in projects, etc).
The phone is a suitable channel for larger numbers of interviews. In this case, the questionnaire
(questions) must be adapted to fit the constraints intrinsic to this communication channel:
The order in which questions are asked is very important. To reduce the bias induced to the
interviewee, it is good practice to order the questions as follows:
1.
2.
3.
4.
Global satisfaction.
Evaluation of the different aspects related to the quality of the provided services.
Evaluation of the satisfaction related to the contacts with staff.
Open questions (3) to identify the required improvement, strengths and new services to
be developed.
5. Identification of the interviewee profile: partner, project which he/she was involved, etc.
After the interview session by telephone, a small number of face to face interviews (around 10) is
highly recommended. These interviews will facilitate the identification of improvement
opportunities based on dissatisfactions mentioned during the interview. These questionnaires
must use more qualitative questions to clearly identify the causes behind the expressed
dissatisfaction.
II.1.2.3 Selection of the partners and contacts to be interviewed
The interviewees are selected from the partners base according to following criteria:
They were in contact with the organization during last year AND the collaborations
(professional services, partnership in a project, etc.) are already finished during the
evaluation year.
Page 5 of 21
Interreg IVB
Project no. 020A
FITT Project
The selection of the sample (out of those who filled in a specific satisfaction
questionnaire at the end of a training or educational service) is made up from two
different types of partners:
o Institutional partners: ministries, chambers, professional associations, research
organisms and universities. These organizations must have long lasting
relationships with our organization on a contractual or non-contractual basis.
o Other partners: the organizations that were in contact with us during the year
before the survey and where the collaboration already finished.
Exploitability of results
Information flow between partners and the organization
Respect of deadlines
Collaboration management
Price/quality ratio
Project follow-up
Quality of the provided services
Specificity of the proposed solutions
Clarity of the proposed services
Reactivity
Understanding of the partners needs
As mentioned before, these criteria must remain unchanged from survey to survey to see how the
satisfaction scores evolve over time and to measure the impact of the actions taken. An important
aspect of this analysis is the identification of those criteria with the biggest impact on the
partners global satisfaction. These key criteria are identified through the interviews with the
lowest scores. The results of the statistical analysis will be reported, including the following
subjects:
The overall survey report covers the partners satisfaction survey, the mirror satisfaction survey
and the training/educational activities satisfaction survey.
Page 6 of 21
Interreg IVB
Project no. 020A
FITT Project
Pros
This technique has enabled CRP Henri Tudor to remain aligned with the partners
needs and expectations.
After 4 years we have observed improvements on some of the satisfaction criteria.
This shows that the measures taken after the survey have had a positive impact on the
partners satisfaction.
The consciousness of the gap between internal and external perception helps to align
the staffs point of view with that of the partners (the mirror survey results).
Cons
The satisfaction criteria could be judged as being too general (the whole
organization), without providing feedback to the organizations departments, service
units or the research teams.
The evaluation criteria must evolve over time as the CRP Henri Tudors activities
evolve. However, changing the criteria quite often jeopardizes the impact of the
actions taken over time.
II.2 WHY?
II.2.1 Rationale: Why was this established?
As an ISO (ISO 90012)certified organization, we must respect quality levels and processes in our
activities. This certification is only one of the motivations, others are:
Know the points of view of our partners and position the organization in function of this
insight.
Gather those points of view in a neutral contact or context (not a commercial one).
Use good or outstanding results as relevant and selling arguments for:
o Potential partners
ISO 9001:2000 specifies requirements for a quality management system where an organization:
1. needs to demonstrate its ability to consistently provide products that meet customers needs and are
compliant to applicable regulatory requirements.
2. aims to enhance customer satisfaction through the effective application of the system, including processes
for continuous improvement of the system and the assurance of compliance to customers and applicable
regulatory requirements.
Page 7 of 21
Interreg IVB
Project no. 020A
FITT Project
o
o
Prospects
Employees
We need a pool of active partners ready to be involved in our research activities (on
contractual or non-contractual basis).
The same partners are sometimes involved in our technology transfer activities as a
natural continuation of their participation in our research projects.
Some partners are involved in more strategic activities at CRP Henri Tudor, like helping
to define the research roadmap at corporate and department levels.
Other partners are involved in the advisory boards and working groups in research
projects (formal or informal).
II.3 OUTCOME
The outputs may differ depending on the selected evaluation criteria. Some criteria have evolved
positively, other ones negatively. Nevertheless, the global perception on CRP Henri Tudor has
improved over time. However, it is still too early to state its impact on the organization.
The satisfaction survey has been totally integrated into the CRP Henri Tudor culture and even has
become a landmark in the CRP Henri Tudors annual report. No unexpected or undesirable side
effects have been observed yet.
Next steps and evolutions:
Measure of CRP Henri Tudors innovation impact
Analysis of the brand value
Carry out a specific satisfaction survey focusing on the type of collaboration (service
supply or collaboration in research projects)
Link satisfaction indicators and organizational performance indicators
Page 8 of 21
Interreg IVB
Project no. 020A
FITT Project
II.5.1.1 Definition
INRIA has started to work on the process of evaluation of transfer activities within research
collaborations. This evaluation is meant to analyze the performance of a PRO (Public Research
Organization) from the point of view of the partner, as well as of the PROs staff. This allows not only
the measurement of partners satisfaction, but also the detection of the possible gap between internal
and external perception. The results can provide useful conclusions on different aspects of partnerships
and the possible improvements.
The guidelines describe the objective of each category of data and the main questions to be asked in
order to obtain the right data. The meaning of each group of questions is explained to be able to
interpret the results. The construction of the questionnaire is of high importance for the efficiency of
the process and the reliability of the results. The guidelines propose a structure for each question and
possible options for the answer (for details see annex 1). The emphasis is put on data collection and
therefore we avoid the use of open questions. It is a strong recommendation to avoid all ambiguities
and to privilege clear and simple questions, even if some issues are difficult to approach in this way
(e.g. impact of the results).
Nevertheless, some further work needs to be done to draft a concrete questionnaire on this basis to
evaluate a partnership. The questionnaire should have two versions: one for the PRO and one for the
partner. The time span of evaluation should be ideally from two to three years, starting from the
moment when the first data is available. A diversity of contributors should be involved, if possible.
CRP Henri Tudor
INRIA
Page 9 of 21
Interreg IVB
Project no. 020A
FITT Project
Once collected, the data should be updated after this period on the aspects of usage and exploitation of
the results (12 to 18 months after the end of the collaboration), so that its analysis could bring a truly
valuable outcome.
Pros
o
o
o
o
The evaluation guidelines are oriented at research partnerships, which make them a
valuable tool for the PRO to perform a specific analysis.
The goal is to be able to assess the perception of research collaboration by the actors
that have been involved in the collaboration - a research team and an industrial partner
- and to measure a possible gap in the satisfaction perceived from both sides.
Corrective measures and adaptation of collaboration criteria can then be initiated.
The open format allows creation of different versions of the questionnaire depending
on the survey method (online survey, data collected by TT officers, interns or external
organizations, etc.); it also allows adjustment of the content and creation of a specific
questionnaire for each partnership.
The guidelines can be adapted to evaluate collaboration within other networks as well
as for European projects.
Cons
o
o
There is no ready to use questionnaire, but it has to be drafted using the guidelines.
The evaluation takes quite long in order to guarantee relevant results. Also obtaining
the engagement of various contributors after the end of the project can be difficult, in
particular on the partners side.
These guidelines should be completed in the future with details on methods of
analysis of the data and presentation of the results.
II.5.2 Why?
II.5.3 Outcome
The process of evaluation of research partnerships has not yet been introduced in INRIAs activities.
The reflection phase has come to an end and now the measures should be taken to start collecting and
analyzing data on partnerships, using the guidelines presented above.
Page 10 of
21
Interreg IVB
Project no. 020A
FITT Project
III
IMPACT ASSESSMENT
III.1 WHAT?
III.1.1 Definition
As defined by the EARTO (European Association of Research and Technology Organizations),
Research and Technology Organizations (RTOs3) are specialized knowledge organizations dedicated
to the development and transfer of science and technology to the benefit of the economy and society.
As a RTO, the main mission of CRP Henri Tudor is to work for a continually better society for
Luxemburg. Therefore the Impact Assessment for research activities and transfer is really important,
even if it is not an easy activity to manage.
Impact Assessment is a process that assesses the change in the well-being of the individual or the
performances of organizations that can be a result of a specific project, service provided or transferred
research outcome.
In the short term, the Impact Assessment for RTO is considered as an activity to measure the impact of
research and transfer activities on social, economical and environmental fields.
III.1.2 Impact Assessment in Practice
In past years, CRP Henri Tudor tried to implement this practice without a real success. The main
reason already identified was that the impact assessment was carried out too early (just after the end of
research projects) and with no appropriate impact indicators. In fact, experience has shown that each
decision and action taken in an organization has an impact on the whole of the organizations services
and activities. That made it difficult to isolate and assess how the transferred result increased the
performance of the partners organization.
CRP Henri Tudor is currently reworking and rethinking the process of Impact Assessment. The review
is based on best practices from other European RTOs. As the process itself is at an early development
stage, this document will describe the best practices identified and give guidelines for an Impact
Assessment practice that will be used for the new Impact Assessment Process in CRP Henri Tudor.
III.1.2.1 Impact in the research chain
In order to identify the process of research and transfer activities of a RTO, and to point out the stage
with the biggest impact, here is a picture of a simplified research project chain.
In this document we will consider that a PRO is a special case of an RTO. The difference only refers to the
status of the organization itself (public or private). We will not make any difference between both terms.
CRP Henri Tudor
INRIA
Page 11 of
21
Interreg IVB
Project no. 020A
FITT Project
Category of impacts
Example of indicators
Impact
on
environment
social
and
physical
Competitiveness Improvement
Product/Service/Process/ Creation/Improvement
Cost-savings
R&D Efficiency Improvement
Research Methods Creation/Improvement
Patent Applications
Expertise strengthening
Research Methods Improvement
Scientific
Impacts:
scientific
publications,
conference and seminar presentation
Improvement of networking between research
partners, firms, etc.
Network creation
Domestic networks, Global networks
Organizational and social innovations
Support of decision making through expert
consultancy and governmental advice
Participation in legislative and strategic planning
Norms, Regulation and standards
Material/Resources and/or Energy Consumption
Reduction
Regional development and growth Promotion
Safety Promotion
Infrastructure Development
Page 12 of
21
Interreg IVB
Project no. 020A
FITT Project
Pros
Cons
Impact indicators are difficult to identify and evaluate.
For partners, impacts from research and transfer collaboration are sometimes difficult
to isolate from the other actions engaged by the organization. Every action engaged
in a specific department or service has an impact on the other ones making it difficult
to assess the contribution of the improvement measure in question.
A long timeframe between the end of the research project and the Impact Assessment
is required to get reliable results. That can be an issue for some partners that need to
observe or assess the results quickly.
III.2 WHY?
III.2.1 Rationale
As mentioned in the introduction, as a RTO, CRP Henri Tudor has the mission of to work for a
continually better society. Because of their dedication to this mission, CRP Henri Tudor needs to
demonstrate that its activities have a real impact on society. Impact assessment is the best way to
qualify the effects of its research and transfer activities. As a public funded RTO (PRO), CRP
Henri Tudor, as most of the RTOs, needs to justify how the public money has been invested by
showing results and translate them into Key Performance Indicators (KPI). For the moment,
KPIs are only based on scientific publications, spin-off creation and license distribution. It would
be good if also the impact indicators were integrated in those KPIs. Another reason is that this
practice also allows being closer to partners. It gives the opportunity to be aware of the gap
CRP Henri Tudor
INRIA
Page 13 of
21
Interreg IVB
Project no. 020A
FITT Project
between internal RTOs perception and the real perception that partners have on RTOs research
and transfer activities.
To be more efficient, Impact assessment should be linked to the satisfaction survey.
III.2.2 Impact: Why is it included in the process as an essential part?
In spite of the low success of implementing this practice for the first time, CRP Henri Tudor is
convinced that Impact Assessment is essential to justify and improve its reputation as an
organization in the society. This practice is not easy to implement and really needs to be adapted
case by case for each organization. The most sensitive part is the selection of the right impact
indicators. This selection is closely related to the core activity of the research organization and the
partners.
At CRP Henri Tudor, this practice will be linked to the corporate strategy. By getting assessments
of the impact of our activities in the economic, environmental and social context, CRP Henri
Tudor will be able to justify their role as a valuable RTO in the country. Moreover, this will proof
to the Ministries that CRP Henri Tudor is investing in public research projects with a real impact
on the countrys society and its economy.
The communication activities around Impact Assessment results can follow the same steps as
presented in the Satisfaction survey section (III.1.2.6). From a partnership point of view, the
impact assessment survey can be used as a strategic tool to keep the partners in the loop. At the
same time it is an excellent external communication tool tell to our partners how important it is for
CRP Henri Tudor to have a positive impact on their activities.
III.3 OUTCOME
The outputs differ depending on the selected impact criteria. Without real implementation, it is too
early to claim if the Impact Assessment survey will contribute to or will be responsible for a
performance improvement of CRP Henri Tudor.
Page 14 of
21
Interreg IVB
Project no. 020A
FITT Project
IV
Annex 1
Very
satisfied
2
Not
Applicable
0
Page 15 of
21
Interreg IVB
Project no. 020A
FITT Project
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Page 16 of
21
Interreg IVB
Project no. 020A
FITT Project
Very
satisfied
Not
Applicable
Not
satisfied
1
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Page 17 of
21
Interreg IVB
Project no. 020A
FITT Project
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
0
Very
satisfied
2
Not
Applicable
Page 18 of
21
0
Interreg IVB
Project no. 020A
FITT Project
Very
satisfied
2
Not
Applicable
0
Page 19 of
21
Interreg IVB
Project no. 020A
FITT Project
Page 20 of
21
Interreg IVB
Project no. 020A
FITT Project
Page 21 of
21