Interview

DETAILED PRESENTATION OF THE TOOL

Why and when is the interview used?

What are the reasons for using interviews? When should the interview be used? What are its advantages and its limitations? At what stage of the evaluation is the interview conducted? What are the pre-conditions for its use in evaluation?

Why and how is the interview used in country/region evaluation?

What specific tasks have to be completed during the preparation for the interview? What are the types of interviews? How are respondents chosen? How does the evaluator schedule his appointments?

How is the interview carried out?

How is the interview prepared? How is the interview conducted?

How is the quality of the interview controlled?

At what stage of the evaluation is quality control carried out? What are the key issues for control?

Why and when?

What are the reasons for using interviews? o What is the purpose of an interview? o Where does this tool come from? o Three types of interviews and their contribution to the evaluation When should the interview be used? o What kind of information does the interview collect? o Can the interview be combined with other collection tools? o How should the interview be conducted? o From whom does the interview collect information? At what stage of the evaluation should the interview be conducted? What are the advantages and the limitations of the interview? o The advantages o The limitations What are the pre-conditions for the use of the interview in evaluation? o The time span o Human resources o Financial resources

WHAT ARE THE REASONS FOR USING INTERVIEWS?

What is the purpose of an interview?
The use of interviews in evaluation is inevitable. This tool collects information and points of view, and analyses them at each stage of the evaluation. The interview usually takes the form of a face-to-face discussion between the evaluator and the interviewee.

Where does this tool come from?

Interviews are used in many fields, such as psychology, ethnology, sociology, anthropology, and philosophy. In psychology, this tool focuses on motivation, or the reasons for a person's behaviour. The interview is thus used to study human actions and to help with the understanding of the human psyche. In ethnology, the interview aims mainly at collecting direct observations. In sociology, the interview is used in all activities.

Three types of interviews and their contribution to the evaluation

Unstructured interviews
The interviewee expresses himself/herself freely and can discuss unplanned topics, because there is no predetermined set of questions. The evaluator intervenes only to generate and develop questions relating to the interviewee's comments. This type of interview is particularly interesting at the start of an evaluation, in order to get a global view of the subject, and identify the major topics and issues.

Semi-structured interviews
The evaluator modifies the interview guide's instructions with additional questions, in order to develop useful areas of inquiry during the interview. This type of interview is the most frequently used, particularly when the evaluator knows sufficient about the aims and the main questions to pose during the evaluation.

Structured interviews
The evaluator follows strictly the interview guide's instructions. He asks different interviewees the same set of questions, in the same order, and using the same words. The evaluator avoids generating and developing additional questions, and the interviewee is not given the opportunity to express himself/herself freely. Answers to each question tend to be short. Structured interviews are seldom used in evaluation, where the evaluator needs to adapt to the situation. However, they can be used to classify points of view and information about the impact of a project/programme by categories. Thereafter, the evaluator can use the results of these interviews to design a questionnaire, with a view to analysing the impact of the project/programme. Semi-structured interviews are the most commonly used tool in evaluation and are the subject of further guidance.

WHEN SHOULD THE INTERVIEW BE USED?

What kind of information does the interview collect?
The interview may be used as a quantitative collection tool; however, it is mostly a qualitative device. In evaluation, the use of interviews is simple, quick, and affordable. The interview collects:

Facts and information for the verification of facts Opinions and perspectives Analyses Suggestions Reactions to the evaluator's hypotheses and conclusions

However, a series of interviews is not sufficient to quantify an event, because the answers are not standardised. This is the main difference between the interview and the questionnaire.

The added value of the interview

Among other advantages, this tool is essential for the development of analyses because it collects information taken directly from the context. Thus, it provides a good indication of what motivates stakeholders to act, their various perceptions of the programme's aims, problems encountered and effective outcomes.

Can the interview be combined with other collection tools?

The interview is a useful device for developing hypotheses and analyses. It can highlight the programme's aims and dynamics, the stakeholders' rationale, and the organisation of the various opinions and perceptions of the programme. Depending on the type of observation tool used and on the stage of the evaluation, the interview can be used in combination with other tools:

To test the main questions to address at the beginning of the evaluation with a view to preparing a series of focus groups.

To be the main observation tool, and be supported by a questionnaire (if the country's general context allows it) or, for example, a focus group of beneficiaries.

How should the interview be conducted?

Usually, the interview takes the form of a face-to-face discussion. This arrangement is particularly effective, as the relationship builds on trust and interviews supplement written information. However, beyond verbal information, other elements should be taken into account, such as the context, the general mood of the session, the people likely to influence the interviewee, etc, as well as the interviewee's reactions: hesitation, silence, eye contact, etc.

From whom does the interview collect information?

The interview is a suitable tool for collecting information, analysing and forming conclusions from a limited (but essential) number of respondents, such as:

Partners and people in charge of the evaluated policy or programme Strategic institutional stakeholders Main operators and people in charge of the programme implementation Representatives of beneficiaries

To find out more:

List of the categories of stakeholders and interviewees How are respondents chosen in the particular case of country/region evaluation?

AT WHAT STAGE OF THE EVALUATION SHOULD THE INTERVIEW BE CONDUCTED?

Types of interview appropriate to various stages of the evaluation
Stages of the evaluation Desk phase: implementation of the methodology and preparation of the mission to the country Type of interview Preparatory interview used for the design of questionnaire grids and the selection of respondents. At this stage, interview guides should be flexible and aimed at highlighting the topics on which the interview is based. The interview is designed to collect information and perspectives. Several interview guides should be developed to correspond to the different categories of respondents, and to the major topics and issues. In-depth interviews for presenting and investigating the issues, used to collect reactions to the evaluator's findings and analyses. At this stage, the evaluator can test the relevance and the feasibility of his/her conclusions. The interview's contribution to the evaluation It strengthens the basis for the choice of major topics and issues, and completes the questionnaire grid. Its structure should be flexible.

Collection of information in the country and from the European Commission

It collects information on the programme's objectives and outcome from the people in charge of the programme, operators, stakeholders and beneficiaries.

Analysis and preparation of judgements

It collects feedback from respondents. It gives in-depth information and reformulates old questions.

WHAT ARE THE ADVANTAGES AND THE LIMITATIONS OF THE INTERVIEW?

The advantages

The interview is appropriate for project evaluation, as well as for more complex evaluations, such as sector evaluations and country/region evaluations, where it should be combined with other tools.

The limitations

WHAT ARE THE PRE-CONDITIONS FOR THE USE OF THE INTERVIEW IN EVALUATION?
The time span
The preparation for the interview does not take long.

To find out more:

How to prepare an interview? How to conduct an interview?

One expert will not be able to conduct many interviews per day and, therefore, the number of interviews which can be carried out during the mission is limited. In practice, at the interviewee's request, the expert may conduct an interview with several respondents at the same time. Thus, this particular usage of the interview increases the opportunity for collecting the information required in a relatively short time.

Human resources
Interviews must be conducted by a trained professional. The necessary skills are:

Thorough knowledge of the major topics and issues addressed in the evaluation Excellent interviewing skills:

The ability to quickly understand the respondent's perspective (his/her interest in the interview, whether he/she has expressed himself freely, whether he/she has committed himself) in order to be interactive and, where appropriate, modify the questions

Financial resources
Costs depend on the number of interviews and their physical location in the country. However, apart from professional fees and transportation costs, the interview itself does not lead to substantial costs.

Why and how is the interview used in country/region evaluation?

What are the tasks to be completed during the preparation for the interview?
o o

The hypotheses and questionnaire grid development The evaluator's tasks during this first stage

What are the types of interviews?

o o

During the first stage During the second stage

How are respondents chosen?

o o o

Distinguish between the different roles of officials from institutions Go beyond formal statements Balance the interviews between officials and beneficiaries

How does the evaluator schedule the appointments?

o o

The pre-conditions for appointment making How to schedule the appointments?

WHAT ARE THE TASKS TO BE COMPLETED DURING THE PREPARATION FOR THE INTERVIEW?
The hypotheses and questionnaire grid development
In country/region evaluation, the evaluator needs to define the strategic questions quickly, in order to be efficient during the information collection.

Three elements must be closely linked together:

First hypotheses leading to the development of questionnaire grids

Questionnaire grids leading to the selection of respondents Interviews, testing and modifying the first hypotheses, as well as consequential changes to the questionnaire grids, which in turn may revise the selection of respondents

The evaluator's tasks during this first stage

Select the types of information to collect
The evaluator should:

Organise the information to be collected on strategic questions between key and secondary interest Devise a methodology to limit the collection of documentation. As country/region evaluation covers so many diversified fields, the evaluator risks dissipating his efforts if this stage is neglected

Develop and organise the questions

The evaluator should classify key questions by their aims and categories of interviewees.

Determine the list of the respondents

See: How are respondents chosen in the particular context of country/region evaluation?

Settle the order in which respondents will be met

See: How are the appointments scheduled?

WHAT ARE THE TYPES OF INTERVIEWS?

Different types of interviews can be distinguished during the two main stages of country/region evaluation

During the first stage

Preliminary interviews setting the programme's purpose and boundaries
These consist of meeting people who have a global vision of the policies implemented, their objectives and impact, and who belong to relevant institutions. These interviews can avoid bias in the process of an evaluation arising from reliance on officials in charge of policy definition and monitoring.

Interviews with different categories of stakeholders

During these interviews, the evaluator meets the stakeholders who have been participating in the process of drafting, definition, negotiation and the operational choices for the implementation of policies. Thus, the relevance and impact of geographical strategies rely on what different stakeholders have done in the process of drafting and negotiation.

During the second stage

Interviews with operators and beneficiaries
The use of interviews to analyse the impact of a programme on beneficiaries in country/region evaluation is not usually advisable because of the large number of people involved, representative problems, etc. However, interviews can be conducted prior to the implementation of other data collection tools, such as focus groups and questionnaires.

Thus, interviews should focus on particular categories of beneficiaries and/or precise questions dealing with various hypotheses.

To find out more:

Can the interview be combined with other collection tools? At what stage of the evaluation is the interview conducted?

HOW ARE RESPONDENTS CHOSEN?

In country/region evaluation, this issue is crucial. It is less a question of knowing how to collect information, than of determining what the information's content should be, and from whom it can be obtained, in the context of short missions and wide fields of inquiry. Choosing the categories of respondents is particularly important:

To identify people who have benefited from the implemented policies To identify those who have played a strategic role in the programme To identify those who have played an intermediary role To identify those who could have been behind obstructions or unplanned effects (such as groups with diverging interests, intermediary groups intervening in the middle of the process, target groups of the policies / the programme, etc.) To analyse current strategies and the various rationales considered

To find out more:

How are the different categories of stakeholders defined during the preparation stage?

Distinguish between the different roles of officials from institutions

The list of officials who should be met can be long because of the nature of the country/region evaluation, which can cover numerous programmes and projects. Thus, a selection must be made consistent with the budget allocated to the evaluation, and especially when the mission is very short. The evaluator should make a distinction between key interviews and "etiquette" interviews; he/she should also think of the quickest and cost-effective way to conduct them. Once the roles of officials are identified, the evaluator must locate the people who have agreed to be interviewed, corresponding to this typology. This selection belongs to the evaluation process and becomes clearer as the first hypotheses are being developed.

Go beyond formal statements

In country/region evaluation, the evaluation's political dimension and economic implications tend to complicate some interviews with institutional authorities and respondents concerned with their hierarchy. The evaluator should be able to "decode" the information he/she is given, by taking into account the context in which the interview is conducted. A way to avoid formal statements implied by this situation is to carry out an informal interview (for example, away from the institution to which the respondent belongs).

Balance the interviews between officials and beneficiaries

The evaluator must be careful to avoid devoting all the time in his/her mission to interviews with officials, to the detriment of beneficiaries. He/She must check the interviewee's "representativeness", which is

always relative and sometimes self-proclaimed. Several questions may help find the respondent's interest in the evaluation and his/her "representativeness":

To which group does the respondent belong? What is his/her professional background? What is his/her influence on the programme's strategy? Does he/she have direct or indirect interests in the programme? Are they explicit or implicit? What is his/her opinion about the policies / the programme? How did he/she become a representative (has he/she been elected, designated)? Is he/she representative of the entire group he/she stands for? How many respondents should be interviewed to collect representative information and perspectives?

HOW DOES THE EVALUATOR SCHEDULE THE APPOINTMENTS?

The pre-conditions for appointment making
During the preparation for his/her interviews, the evaluator should be practical and:

Fix an order of importance for the collection of data and, after developing the first set of questions, select who should be interviewed See: What specific tasks have to be completed during the preparation of the interview in country/region evaluation? Identify quickly on-site informants and stakeholders. They often hold a position at the centre of many sources of information. Thus, they can advise the evaluator, and suggest "good" respondents, who will be able to give accurate answers. Identify respondents who would be better interviewed in a group Consider the possibility of interviewing people on the telephone, which could save time Avoid overestimating the number of interviewees that can be met during a short mission

How to schedule the appointments?

The particular context of country/region evaluation implies taking additional precautions, in addition to theusual precautions, in the making of appointments. The evaluator should:

Schedule in advance at least part of the interviews that he/she needs to conduct on-site Ensure that the evaluation's clients (for example, the European Commission Delegation or the national authorities), or the local consultants organise the first set of interviews on his/her arrival Make time available for unscheduled appointments made necessary because of new perspectives identified during his mission. As a consequence, the evaluator's hypotheses can be extended or modified.

With a view to being more efficient, the evaluator could:

Use the appointment making process as an opportunity to communicate to the respondents all the information they need to know: the purpose/ of the evaluation, what questions they should prepare for, and the interview/ duration. This is also the appropriate moment to check the respondent's profile against the planned content of the data collection. Distribute an interview guide prior to the interview. This is appreciated by all parties because it is time-saving and directs the respondent's information gathering to specific documentations.

How is the interview carried out?

How is the interview prepared?

Stage 1: list the categories of stakeholders and interviewees o Examples of stakeholders' groups o What are useful questions to bear in mind for the selection of interviewees? Stage 2: design the questionnaire grid Stage 3: schedule the interviews o Make the appointments o Send interview guides prior to the appointment o Conduct additional interviews

How is the interview conducted?

HOW IS THE INTERVIEW PREPARED?

Stage 1: list the categories of stakeholders and interviewees
The evaluator should define the categories of stakeholders he needs to meet, and then select a sample group for each category.

Examples of stakeholders' groups

Agents in charge of the operational strategy / the implementation Actors specialised in a sector of intervention Operators working at a local level / at a national level Technical operators: representatives / on-site operators Public sector agents (working for the state, the administration, etc.) / private sector agents (working for the civil society, NGOs, etc.), Beneficiaries

The list of the stakeholder categories will evolve as the evaluation progresses. Thus, additional interviews should be anticipated. Some choices are obvious (for example, the agent in charge of the implementation of a nationwide programme), but others are more complex. Moreover, the evaluator must consider how he will meet the interviewee, which depends on:

The respondent's accessibility (will the evaluator meet him in person or only a representative) The respondent's availability

To find out more:

How are respondents chosen in country/region evaluation?

What are useful questions to bear in mind for the selection of interviewees?

Within the relevant institution, who knows the programme/the politics well? Who is in charge?

Who is in charge of implementation, of monitoring? Of which aspects of the programme is the respondent in charge? Who will be able to trace the programme's evolution, in the context of a rapid turnover of the implementation team?

During the interviewees' selection, the evaluator should be careful to distinguish between direct witness' testimonies and second-hand testimonies.

Stage 2: design the questionnaire grid

Questionnaire grids, and interview guides derived from them, vary with the categories of respondent and the latter's links with the evaluated issue. Grids should include all instructions, themes and questions which the evaluator wants to discuss with the respondents. The stages required to organise the grid

The questionnaire grid is an intermediary between the evaluation study's design and its implementation. Design of the interview guide

The evaluator should allow the interview to remain as close to an open discussion as possible and accept discursive answers, inconsistencies, pauses, hesitations, etc. The questions can be written in the questionnaire grid, or be asked spontaneously, in order to complete, to probe, to give new insights and to challenge the respondent's answers.

Stage 3: schedule the interviews

This organisational stage depends on the evaluation stage within which the interview is implemented, and on the respondent's availability. The selection criteria must clearly indicate the priority of the people to be met. Furthermore, the evaluator must be careful about:

The risk of key officials being unavailable His/Her short mission span Travelling time Scheduling too many interviews, leading to the cancellations of appointments due to poor planning or unexpected events Respecting his/her appointment, such as the time, although conventions vary from one country to another Conducting the interview's introductory stage slowly, in order to establish rapport and respect traditions and customs Unplanned and additional interviews which have to be organised because a respondent has suggested an "interesting" informant who may be interviewed, or because a key interviewee is now available Tiredness: too many interviews conducted in a day may affect the capacity of the interviewer to listen

The evaluator must find a balance between:

Scheduling 4 to 6 interviews a day seems to be a good compromise between the two visions.

Make the appointments

This stage constitutes the first contact between the interviewee and the evaluator. Therefore, the choice of the person in charge of this task is very important. It is often better to schedule appointments in advance. However, time should also be allocated for unplanned interviews, particularly in the context of country/region evaluation. The evaluator must know how to resist pressure to meet as many respondents as possible. Organising focus groups can solve this problem.

Send interview guides prior to the appointment

This stage is optional, however, it often eases the interview's course by giving the respondent the opportunity to:

Understand the topics on which he/she has to talk Collect all the necessary documentation Ask the evaluator whether the participation of other respondents during the interview is possible

It will save time for both the respondent and the evaluator, and help them feel at ease. Most of the time, the quality of the interview depends on this.

Conduct additional interviews

They are often necessary after the first elements are collected and during the progress of the evaluation. Thus, they must be programmed into the planned timetable and budget.

Quality control

Check-list for evaluators Check-list for managers

CHECK-LIST FOR EVALUATORS

Questions Preparation and design
Does the list of respondents meet the needs of the evaluation's methodology? Have alternatives been planned by the evaluators in case of cancellations of appointments with the actors? If any, has the issue of "representativeness" been solved? In interviews with representative stakeholders belonging to the evaluation's spotted category, has the respondent's " representativeness " been checked? Do the interview grids cover all the evaluation issues? Does the design of the interview guides vary sufficiently to meet the needs of different categories of stakeholders?

Answer

Implementation
Have the evaluators controlled and checked the information collected? Does the intended format designed for the debriefing highlight the differences between reliable information and opinions? Is the diversity of perspectives, expressed by the various categories of stakeholders, explicitly exposed?

CHECK-LIST FOR MANAGERS

Questions
Does the list of respondents meet the needs of the evaluation's methodology? Does the design of the interview guides vary sufficiently to meet the needs of different categories of stakeholders? Is the homogeneity of the interviews assured? Does the intended format designed for the debriefing highlight the differences between reliable information and opinions? Is the diversity of perspectives, expressed by the various categories of stakeholders, explicitly exposed?

Answer