Impact Monitoring in Value Chain Kenya

Impact Monitoring
in Value Chain Promotion
Concepts and Application
First sequence of thoughts, prepared for the Training Course
„Promoting Promoting Value Chains for Agribusiness

Development in Africa“
held in Nairobi, 4th - 7th April 2005
Heike Höffler, GTZ Kenya
Deutsche Gesellschaft für

Technische Zusammenarbeit (GTZ) GmbH
Heike Höffler Kenya
Impact Monitoring: Concepts
and Application
1 A Concept for Impact Monitoring

1. What to monitor? What are impacts?
2. GTZ Impact Model
3. Theory: Six steps to set up a monitoring system
2 Measuring Impacts in Value Chain Promotion

1. Impact Hypotheses – chain of changes
2. Scope of influence
3. Benchmark Data, Milestones and Indicators
4. Methods of Measuring Impact
5. Using Monitoring Information
3 Practice: PSDA Impact Monitoring System

2
and Application

2. GTZ Impact Model


3
Monitoring is an ongoing activity!
1 • Projects and Programmes need to follow-up their

progress
• Successes and failures need to be looked at
while projects and programmes are running
• Monitoring is an internalised process of team
communication, continuously undertaken while
implementing,
whereas
• Evaluation is an act of stopping implementation
to reflect past activities (but drawing from
information from monitoring).
4
The World of Impact Monitoring

Performance Monitoring Implementation Monitoring
Benefits
Output Monitoring Outcome

Input Monitoring
Activity Monitoring
Impact Model
Impact Monitoring
Outreach
Impact Assessment
Impact Chain
Impact Indicators
Process Monitoring
Outputs
Evaluation Result-based Monitoring 5


Performance Monitoring Implementation Monitoring
Benefits
Output Monitoring Outcome

Input Monitoring
Activity Monitoring
Impact Model
Impact Monitoring
Outreach
Impact Assessment
Impact Chain
Impact Indicators
Process Monitoring
Outputs
Evaluation Result-based Monitoring 6

Performance Monitoring Impact Monitoring Impact Assessment
Process Monitoring Result-based Monitoring Evaluation
Input Monitoring Impact Indicators Outreach
Activity Monitoring Impact Model Outcome
Implementation Monitoring Impact Chain
Output Monitoring
Outputs Benefits Impacts 7

What to monitor in projects –

Performance & Impact Monitoring
1 Performance Impact
• Observing outputs • Focussing on effect
against planned of outputs: impacts!
activities
• Observing the direct
• Providing benefit of outputs
information for
project management • Strategic steering of
implementation
• Day-to-day activity
 To self-evaluate
 To trigger short-term whether activities
adjustments in contribute to
operation objectives 8
Impacts are ...
1 ... changes that have a causal - or at least a

plausible - link to a project/programme
... a change of circumstances as a consequence of
an intervention, it can be intended or unintended,
positive or negative.
... there: from the first moment of intervention and
they continue to occur all the time.
... rather the result of social interaction than a
straight-forward interventions
... the result of complex interactions and thus, a
complex matter to deal with!
9
Why concentrating on Impacts?
• Broad international discussion late 90ies

1
• Criticism about the efficiency of development
cooperation
• Criticism about the Monitoring and Evaluation
system of GTZ
AURA
• BMZ is a contractor, GTZ is an agent

• New structure of the project documents
• Changes in political dialogue
• Changes in Report obligations
10
The GTZ Impact Model (I)
1
• Impact Orientation has become a principle of
GTZ‘s corporate development.
• „quality at entry“  „quality at exit“
• „what has been done“  „what has changed“
11
Model of Interaction
Direct
Use of Benefit
Indirect
Project
Output
Benefit
Engel, P., The Social Organization of Innovation, 1997

The GTZ Impact Model (II)
Indirect
1 Benefit
Direct
Benefit
(Goal)
t i on ?
tr i bu
At n !
o
Use of
Outputs
r v at i
Obs e
Outputs
Poverty
Alleviation
Activities
put
GTZ In Partner
Illustration: which of the 100
Bricks did we donate?
14
The GTZ Impact Model (II)
Indirect
1 Benefit
Direct
Benefit
(Goal)
Use of
Outputs
diff.
Impact
Outputs
Levels Poverty
Alleviation
Activities
put
GTZ In Partner
The GTZ Impact Model (III)
Pro-poor Rural Economomic Growth

1
Indirect Benefits Increase in rural
employment & income
Attribution Gap
Direct Benefit (Goal) Producers can access international markets

Use of Output Market information is used to change production standards
Output Presentation of Research to stakeholders and publishing

Activities Market research for a strategic agricultural product
Inputs Advisory Services in the Ministry of Agriculture

16
Consequences for GTZ Projects
• Impact Models form the essential methodological part in

1 project design!
• Project reports are reduced to the information relevant for
political intervention, i.e.
– Changes in risk
– Changes in assumptions
• and: IMPACTS! i.e.
– Which changes can be observed?
– Which impacts can be plausibly attributed to the project?
– Are there unintended impacts?
Changes in Monitoring! 17
18
Theory: Six Steps (GTZ)
1
Step 1: Identify the System Boundaries
Step 2: Agree on Purpose and Procedures for
Results-based Monitoring
Step 3: Agree on Results Hypotheses
Step 4: Review Indicators and Define Milestones
Step 5: Conduct Data Survey
Step 6: Using Monitoring Results
GTZ 2004: Result-based Monitoring: Guidelines for Technical

Cooperation Projects and Programmes
19
Theory: Six Steps (Herweg/Steiner)
1
Step 1: Involvement of Stakeholders and
Information Management
Step 2: Review of Problem Analysis
Step 3: Formulation of Impact Hypotheses
Step 4: Selection of impact Indicators
Step 5: Development and Application of Impact
Monitoring Methods
Step 6: Impact Assessment & Follow-up
Herweg, K & Steiner, K. 2002: Impact Monitoring and Assessment,

Vol. I & II 20
Theory: Seven Steps („MAPP“, GDI)
1
Step 1: Preparing a Life Line
Step 2: Preparing a Trend Analysis
Step 3: Cross-checking with other sources
Step 4: Compiling an Intervention List
Step 5: Developing the Influence Matrix
(connecting trends and interventions)
Step 6: Developing the Impact Profile
Step 7: Attribution of Impacts to MDGs
DIE 2004: Briefing Paper: Impact Analysis of Development
21
Cooperation is Feasible
and Application

2. GTZ Impact Model


22
Impact Models: A chain of changes
What do we expect to happen in value chain promotion?
Intervention;
i.e. extension
Specific Primary Logistics Final Con-

Input producers centres, Traders sumers
providers Industry
2
Specific Production Trans- Trade Consumption
Inputs formation
Specific
Trans- Trade Consumption
Inputs
formation
C H A N G E !
23
Defining the Impact Hypothesis (I)

Direct Benefit (Goal) Small-scale producers access international markets
Use of Output Farmer group gets EuroGap certified

providers Industry
2 Specific
Inputs
Production Trans-
formation
Trade Consumption
Output Farmer Groups are setting up an Internal Control System
Activities Training on EuropGap Compliance and Certification
Inputs Advisory Services for Extension Providers
24
Defining the Impact Hypothesis (II)

Direct Benefit (Goal)
Use of Output

providers Industry
2 Specific
Inputs
Production Trans-
formation
Trade Consumption
Output
Activities
Inputs
25
Defining the Impact Hypothesis (III)

Use of Output

providers Industry
2 Specific
Inputs
Production Trans-
formation
Trade Consumption
Output
Activities
Inputs
26
Defining the Impact Hypothesis (IV)

Use of Output

providers Industry
2 Specific
Inputs
Production Trans-
formation
Trade Consumption
Output
Activities
Inputs
27
Defining the Impact Hypothesis (V)
Direct Benefit (Goal) • Each intervention undertaken

along a value chain should
Use of Output follow an impact model;
• thus: each value chain to be
promoted needs a concept of
2 impact hypothesis.
• All impact hypotheses should

Output
be derived from the project
goal;
Activities
• thus need to be derived from
Inputs
the impact indicators! 28
Impact Hypotheses – Food for Thought:
• Pre-formulation of impact
chains can lead to mono-causal
conclusions.
• Linear following of impact
r of chains can fade out the
2 a n g e
a l ity!
D aus complex interaction of
n o - c
mo interventions and impacts.
• The higher the impact level, the
more a context oriented
approach is needed, taking into
account development trends.
29
Defining the Scope of Influence (I)

Specific single interventions

providers Industry
2
Inputs formation
30
Defining the Scope of Influence (I)

Specific single interventions

providers Industry
2
Inputs formation
31
Defining the Scope of Influence (II)

Chain promotion at all stages

providers Industry
2
Inputs formation
32
Defining the Scope of Influence (III)

Full chain promotion, all stages, all levels

Inputs formation
micro

providers Industry
2
Associations
meso
Subsector-specific BDS providers
Subsector-specific Technical Agencies

macro
Local Government, Providers of Utilities / Infrastructure
National Government (Line Ministries) & Public Agencies 33

Impact Hypothesis - Conclusion

Inputs formation

providers Industry
2
Associations
Subsector-specific BDS providers
Subsector-specific Technical Agencies
Local Government, Providers of Utilities / Infrastructure
National Government (Line Ministries) & Public Agencies 34

Benchmarks, Milestones & Indicators
Benchmark Data & Information

The state of the art at given point in time (i.e.
programme start)
Milestones
2 Desired output at a given point in time for (programme)
achievements
Indicators
Determinants for checking whether activities led to the
expected output (performance) or expected implications
(impact).
35
Benchmarks, Milestones & Indicators (I)
Programme level
Indicators
Value chain level
2 t
0
Milestones
Benchmark Progress
Data Reports
Evaluation
36
Benchmarks in Value Chains

Inputs formation
Usage of Outreach: Number of Market Quality

Inputs How many processors Concentra-
Food Safety
are tion
Usage of Degree of
involved? Product
2 services vertical Entry
Diversity
Employment concentra- Barriers
tion and
Gross Price
integration
margins Volatility
Regional Seasonality
distribution
Trade
margins
37
From Benchmarks to Milestones

Inputs formation
3 processing In 18 Month:
firms 2 more firms
Output
Low process established
2 quality Activities
At least 2 certified
High costs of under ISO xxx
Inputs
processing Cost-effectiveness
of at least 2 firms
improved by x %
Milestones serve as process indicators.

They are directly derived from programme indicators
38
and the impact hypothesis.
Benchmarks & Milestones in Value Chains

Inputs formation
Try to build these „couples“ per stage in a chain that

2 you are targeting with your project.
Not all Benchmarks need to be quantitative data;
qualitative description also serves the purpose.
Market Analyses need to take the need for benchmarks
into account (into the TORs!)
We might not be the first to undertake this – keep
looking for partners in gathering data! 39
From Milestones to Indicators –

or was it the other way round?
Direct Benefit (Goal) Programme Indicator(s)
Use of Output
2 Value chain Indicator(s)
Production
Several Milestones in a chain
Output
Single Milestone
Activities
Inputs
40
Benchmarks, Milestones & Indicators (II)
The number of
profitably operating Impact Indicator,
private processors Direct Benefit (Goal) Component 3,
doubles (2002: 6) Phase 2
Use of Output

Inputs formation
2
3 processing In 18 Months:
firms 2 more firms
Output
Low process established
quality Activities
At least 2 certified
High costs of under ISO xxx
Inputs
procesing Cost-effectiveness
of at least 2 firms
improved by x % 41
...
42
Example: Employment
The number of
primary producers
doubles.
3 (Research Primary Customers

Institute) Large processing companies 1
Producers 1 Market 1
n = 20 n = 400 n = 450
SM Primary
Producers 2
Retailers A 2
Industrial n = 20
n = 6000 Customers
SMEs Market 2
n = 2400
Importers Retailers B 3
n = 40 n = 200
43
Example: Value Added
Specific Production Trans- Trade Con-

Inputs formation sumption
Sales Prices / unit 25,00 60,00 100,00

The value
3 Cost / unit 23,00 27,00 30,00 added at
Income / unit 2,00 8,00 10,00 producer
level
increases
Distribution in %
from 25%
- of value-added 25 % 35 % 40 % to 35%.
- of income 10 % 40 % 50 %

providers Industry
44
Example: Better Business Relations

Specific Primary Final Con-

Input producers Traders sumers
providers
Interviews show
that small cale Key informant interviews
farmers have show that wholesale
improved traders are more satisfied
relationship to with the
traders and trust quality/quanity/reliability of
them. produce delivered by small
scale farmers.
45
Indicators in Value Chains
„There is increasing interest in, and use of, private

sector tools for performance and impact measurement;
i.e. consumer market research tools for measuring
changes in markets and private sector business tools
for measuring service provider performance.“
2 Some examples:
• Customer satisfaction
• Repeated customers
• Number of enterprises demanding a service
• Satisfaction with last service purchased
• Percentage of women-owned enterprises 46
Methods of Measuring Impacts

• Market Studies
2
• Key Informant Interviews
Where to measure in
• Point of Leverage –
the chain?
comparisons (before – after)
• Participatory Monitoring What data – qualitative
or quantitative?
• Time-series
Which degree of
• etc... participation? 47
Example: Spider Web Diagrammes
Product Quality
10
Use of Services 5 Consumer Satisfaction
2 Beginnging
0
after 3 years
after 10 years
Soil fertility Farm Income
Growth in Exports
48
E-Val
GTZ tool to catch different perceptions of projects and

programmes.
There are only subjective opinions to changes and

2 impacts
Triangulation of opinions:
GTZ,
partners, and
target group
49
Using Impact Monitoring Systems
• Focussing on effect
of outputs: impacts!
• Observing the direct
benefit of outputs
2 The objective of • Strategic steering of
Impact Monitoring implementation
was ....
 To self-evaluate
whether activities
contribute to
objectives
50
Observe direct Benefit Self-evaluation Steering
Participation! Team Steering

2 Meetings Committee
Discussion with
Partner Reports
Stakeholders
Meetings Annual
Finetuning
Other Progress
Communication donors Reports
Private
Sector
51
More ideas – let‘s be creative!
Communication of Go- No-go- Decisions

Results
Publications
2
Replanning
Newsletter
Accountability
Business Fora
52
It‘s better to be approximately

right
than precisely wrong!
( John Maynard Keynes )
Please email your comments & questions to heike.hoeffler@gtz.de
53
and Application

2. GTZ Impact Model


54

Impact Monitoring in Value Chain Kenya

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Impact Monitoring in Value Chain Kenya

Uploaded by

Copyright:

Available Formats

Impact Monitoring

in Value Chain Promotion

Concepts and Application

First sequence of thoughts, prepared for the Training Course

„Promoting Promoting Value Chains for Agribusiness

Heike Höffler, GTZ Kenya

Deutsche Gesellschaft für

1 A Concept for Impact Monitoring

2 Measuring Impacts in Value Chain Promotion

3 Practice: PSDA Impact Monitoring System

1 A Concept for Impact Monitoring

2 Measuring Impacts in Value Chain Promotion

3 Practice: PSDA Impact Monitoring System

Monitoring is an ongoing activity!

1 • Projects and Programmes need to follow-up their

The World of Impact Monitoring

Output Monitoring Outcome

Evaluation Result-based Monitoring 5

The World of Impact Monitoring

Output Monitoring Outcome

Evaluation Result-based Monitoring 6

The World of Impact Monitoring

Performance Monitoring Impact Monitoring Impact Assessment

Process Monitoring Result-based Monitoring Evaluation

Input Monitoring Impact Indicators Outreach

Activity Monitoring Impact Model Outcome

Implementation Monitoring Impact Chain

Outputs Benefits Impacts 7

What to monitor in projects –

Impacts are ...

1 ... changes that have a causal - or at least a

Why concentrating on Impacts?

• Broad international discussion late 90ies

• BMZ is a contractor, GTZ is an agent

The GTZ Impact Model (I)

Engel, P., The Social Organization of Innovation, 1997

The GTZ Impact Model (II)

The GTZ Impact Model (II)

The GTZ Impact Model (III)

Pro-poor Rural Economomic Growth

Direct Benefit (Goal) Producers can access international markets

Output Presentation of Research to stakeholders and publishing

Inputs Advisory Services in the Ministry of Agriculture

Consequences for GTZ Projects

• Impact Models form the essential methodological part in

Theory: Six Steps (GTZ)

GTZ 2004: Result-based Monitoring: Guidelines for Technical

Theory: Six Steps (Herweg/Steiner)

Herweg, K & Steiner, K. 2002: Impact Monitoring and Assessment,

Theory: Seven Steps („MAPP“, GDI)

1 A Concept for Impact Monitoring

2 Measuring Impacts in Value Chain Promotion

3 Practice: PSDA Impact Monitoring System

Impact Models: A chain of changes

What do we expect to happen in value chain promotion?

Specific Primary Logistics Final Con-

Defining the Impact Hypothesis (I)

Use of Output Farmer group gets EuroGap certified

Specific Primary Logistics Final Con-

Output Farmer Groups are setting up an Internal Control System

Activities Training on EuropGap Compliance and Certification

Inputs Advisory Services for Extension Providers

Defining the Impact Hypothesis (II)

Specific Primary Logistics Final Con-