EVALUATION REPORT

Action for Global Health

Evaluation Report
December 2007

prepared by

Jim Coe
Harm-Jan Fricke
María Becerro Torres
Jean Martial Bonis Charancle
Elena Lucchi
Dörte Pommerening
Jeremy Smith

for

staff, managers and organisations involved in the

Action for Global Health network

Contents:

part one: summary report ................................................................. 2

summary ...................................................................................................... 2
recommendations .......................................................................................... 8
part two: findings and conclusions .................................................... 9
1. introduction ........................................................................................... 9
2. project rationale and relevance .............................................................. 10
3. progress to date................................................................................... 11
4. organisational and national contexts ....................................................... 12
5. plans and planning ............................................................................... 15
6. strategy .............................................................................................. 17
7. influence and outcomes ........................................................................ 19
8. health policy agendas ........................................................................... 20
9. policy analysis and positions adopted ...................................................... 21
10. public communications .......................................................................... 23
11. cross-national working .......................................................................... 24
12. network coordination and management................................................... 24
13. governance and accountability ............................................................... 26
14. internal communications ....................................................................... 27
15. working groups .................................................................................... 28
16. decision making ................................................................................... 28
17. identity ............................................................................................... 29
18. network membership ............................................................................ 30
19. general conclusion ................................................................................ 32
part three: survey findings .............................................................. 33
Introduction ................................................................................................ 33
State of the AfGH Network ............................................................................ 34
Internal Communication Flow ........................................................................ 38
General Appreciation of the Network .............................................................. 39
appendix: evaluation purpose & methodology ................................. 40

page 1
 EVALUATION REPORT

part one: summary report

summary

introduction

1. This evaluation report considers the progress made by the Action for Global
Health (AfGH) network during its first year of operation. It concentrates on the
development of the Network against its stated objectives, the processes used to
achieve the aims of the Network, and on opportunities to develop the Network‟s
operation in the future.

2. The primary concern in conducting this evaluation is to provide an opportunity for

the Network‟s members to consider experiences to date and learn from them so
that future working can be more effective. Ideally the report will act as a tool to
encourage and stimulate internal reflection on the work done to date, including
its notable achievements, and focussing on important issues that still require
resolution.

3. Assessments given in this report draw on information and opinions gathered from
AfGH and partner staff and managers, documents supplied by the Network, and
from a limited number of external sources. A draft report was presented to AfGH
internal stakeholders for comment and attempts have been made to address the
feedback received in this final report.

first year achievements and the key challenges now

4. Thanks to a great deal of energy from Network members, and despite sometimes
significant organisational problems, AfGH has been able to achieve virtually all of
the objectives it set itself for its first year of operation. It is commendable that
the basics of the network are in place. The structure is sound, plans have been
delivered, much has been produced, and an identity has been forged. We believe
that in achieving these things, the network has laid a good foundation for the
development of future work.

5. In building on work to date, inevitably much still needs to be done in order to

fulfil the Network‟s goals and objectives. We suggest that certain aspects of
Network operation need explicit attention, particularly those relating to member
relations and mutual understanding, strategy and planning, and management
and decision-making.

rationale for the network

6. The rationale for the existence of the AfGH Network is generally felt to be sound
by both internal and external stakeholders, and the establishment of a European
network was said to be timely.

page 2
 EVALUATION REPORT

7. The AfGH network appears to be fulfilling a need by advocating for global health
and better health services as a „cross-cutting‟ issue. Whilst there remain some
issues about what this means in practice, the fundamental notion seems sound.

8. The extent to which the Network adds value, particularly at national and
organisational levels, is sometimes questioned. This is particularly so where
overlapping work is done by other networks, organisations or institutions.
However in other national contexts the added value of AfGH seems clear. Our
own view is that it is too soon to comment with any force on the benefits and
added value of national and cross-national levels to date. In any case there is a
challenge to be met to establish sound national rationales for how the Network
can best add value in different contexts and strategies to support this, based on
a good understanding of the national political and policy concerns, and within the
context of an over-arching EU-wide approach.

9. Application of a cross-cutting (as opposed to an intervention- or disease-specific)

approach to issues of global health and development is relevant, timely and
needed in order to hold governments to account for their stated commitments to
the delivery of the Health MDGs. What this focus means and how best to
communicate it - for example by members with disease- or intervention-specific
remits - needs further clarification.

organisational and national contexts

10. Network members are diverse, for instance in size, culture, experiences, and
status within national and international NGO communities. Such differences can
be accommodated within a network such as this one. There is, however, a need
to acknowledge and address the implications arising from these differences,
ensuring that members respect each other‟s contexts and have clear expectations
of each other.

11. There are differences too in ideological and practical buy-in to the Network, and
differences in opinion about benefit to member organisations. Mutual awareness
of the existence of diverse opinions about such issues – let alone thinking about
its implications - seems to have emerged gradually and relatively recently. It has
not yet been properly addressed.

12. In addition differences have been created from the outset in the allocation of
resources to the different countries taking part, with the UK having access to
three full-time staff members and others having to rely on two. Given the
differences that exist between members and their national situations, progress is
being made at different speeds, with some members having adopted a generally
reactive stance to network initiatives.

planning

13. There has been significant, and successful, effort to ensure that project activities
and milestones for year one have been achieved. In effect, this has been
achieved in something like eight months. Some feel however that this has been
at a cost in terms of quality or meaningful participation, particularly where
planning on issues other than those outlined in the proposal was concerned.

page 3
 EVALUATION REPORT

14. Concerns around the speed of, and mechanisms for decision-making were
especially highlighted by respondents.

strategy

15. To date, work has focused on achieving identified activities and milestones. This
has left little time to reflect on the Network‟s broader strategic aims and
objectives. From the available information, it is not evident that AfGH strategies
make the best use of the network‟s resources. Nor is it clear that there has been
sufficient exploration of the external operating environment. A clearer strategic
rationale for the work, and greater focus within it, is therefore needed.

16. Such a strategic rationale and focus should include the design of approaches to
engage with and influence identified target decision makers and influential
audiences. To support this, better understanding of the policy and advocacy
environments in which the Network operates needs to be systematically
developed.

17. Sharper strategies are needed to feed into efforts to improve prioritisation of
policy agendas, so that the network can develop achievable objectives and
operate to greatest effect in achieving its goals.

18. The Network is battling huge forces. To make its contribution in the most
effective way, AfGH will need to be very clear how it can bring its limited
resources to bear in ways that makes most effective use of them. This requires
greater attention too to ways of assessing progress against goals and objectives.

influence and outcomes

19. As already anticipated in the evaluation Terms of Reference, little can be said at
this stage about the influence and outcomes of the AfGH project to date.

20. Members generally feel that, while it is early, AfGH is increasingly being
recognised, by both NGOs and targets. The „Health Warning‟ report and
dissemination work done to date is helping to forge good relations according to
respondents. What limited information we were able to gather from external
sources appears largely to support these internal perceptions.

21. Currently, evidence of network influence - let alone outcomes - is not being
recorded, and not systematically analysed by the Network. Occasional external
evaluation cannot be a replacement for ongoing internal analysis, and a system
and approach should be designed to enable it to do that.

health policy agendas

22. Progress on the development of joint positions on health policy has in many ways
been remarkable, given the difficulties faced and the timescale in which this has
been achieved. The challenge now seems to be to make such progress
practically significant by building on individual members‟ strengths without the
network losing sight of its broader health and development mandate and
intention.

page 4
 EVALUATION REPORT

policy analysis

23. The production of the „Health Warning‟ report caused some significant problems –
largely due to pressures of time – and the final reports are acknowledged to have
limitations. However, their publication is seen as useful by both internal and
external respondents. Internal respondents generally suggested that external
views of the report were more positive than the views that they themselves had
of it.

24. The current consensus on the quality of the Network‟s policy analysis provides a
basis on which to develop stronger joint positions. A comparative analysis of
current published policy asks of the Network and its individual members could
help to identify areas of policy consensus, divergence and hiatus.

public communications

25. The network has identified a number of broad audience groups, including
governments in the different national arenas (including the EU institutions),
NGOs, EU citizens, media, and the private (healthcare and research) sector.
Development of public communications needs more work and is closely related to
the development of the network‟s (influencing) strategies, addressing such
questions as: who are the priority audiences that need to be engaged or
addressed? Are they different in different national contexts? Which potential
channels can be used to reach them?

cross national working

26. There has been some disappointment that to date the Network has insufficiently
been able to benefit from what is possibly its greatest asset: its international
character. Improvements in acquiring and sharing intelligence and in joint
strategising can provide AfGH with the key benefit of advocacy networking:
members being able to advocate common positions simultaneously, across
different arenas, in order to exert multiple routes of influence.

coordination and management

27. The work of the Coordinator and Coordination team was generally seen as
supportive of members‟ and the Network‟s needs, providing a steer and support
to the project. Responses from members do indicate however that there is a
constant need to be sensitive to the difficult balance between providing
leadership and facilitating members‟ involvement and participation.

28. In particular, it seems that Network decision-making sometimes suffers because

decisions are taken at short notice, with insufficient time given to thought and
debate. Difficulties are exacerbated by language issues.

29. Structuring members‟ involvement in decision making in ways that make it easy
for all to participate, including through scheduling in more time for translation
and for national consultations before Steering Group meetings, would help to
improve the quality of decision making and, it seems, of the decisions made.

page 5
 EVALUATION REPORT

governance and accountability

30. There appear to be some divergent views (for example between some in the
Project Accountability Group and others outside it) about where accountability
and final decision making lies within the Network.

31. Respective roles of Steering Group, Advocacy Officer Meetings and Working
Groups appear not to be always clear, at least not for many members.

32. In our view the Network‟s engagement with and accountability to Southern (Third
World) partners and groups needs examining and strengthening for reasons of
credibility, consistency and effectiveness.

internal communications

33. Comments from internal respondents sometimes demonstrated a lack of

appreciation of others‟ working contexts, and there was a lack of awareness of
what others were calling for in „their‟ report.

34. To some extent, it seems that the pressure to deliver activities on time has
precluded the development of solid internal communications. This should not be
overstated however since survey findings indicate relatively high levels of
satisfaction with internal information flows. Nevertheless, telephone conferences
are not particularly seen as useful by some (because of language issues) and
face-to-face meetings are felt to lack variety in approach and format.

working groups

35. Working Groups are regarded as a potentially useful tool but they seem not to
have functioned fully to-date.

decision making

36. Moving forward in a strategically coherent fashion requires that members have
full confidence in decision-making mechanisms. The main issue seems to be
about lack of preparation time, with the result that people don‟t feel equipped to
take the right decisions.

37. Given the organisational issues involved in developing the work of the various
decision-making groups, a balance will have to be found between face-to-face
work and meetings and virtual meetings and work through e-mail and internet
exchanges.

identity

38. As expected at this stage of the project, recognition of AfGH as an entity is still at
a low level. In some countries the Network is explicitly „branded‟ and promoted
in its own right, while in other countries member organisations primarily refer to
AfGH as one of their activities.

page 6
 EVALUATION REPORT

39. It would be helpful for the Network to develop some common principles around
how members communicate their membership of AfGH.

network expansion

40. One of the stated intentions of the Network is to expand over the coming years.
This will need thought and debate to consider how this should best be done in
ways that ensure that future expansion contributes strategically to the
achievement of AfGH goals. Key questions to ask in this respect include: what
added value would the network want to gain from a new member; what benefit
would the new member get when joining AfGH network; what possible obstacles
will need to be faced – both by the network and the new member; what added
value can the network offer candidate members?

conclusion

41. The Action for Global Health Network has, during its first year, established a good
basis for itself. It is seen as credible and its concerns are felt to be timely.

42. In developing its work in the coming year priority should be given to improving
internal communications and decision making processes and to developing
coherent strategies focussed on the Network‟s main concern and main policy
targets.

page 7
 EVALUATION REPORT

recommendations

We highlight below the key areas that we recommend the Network focuses attention
on. Additional suggestions for action are made in the main text, in bold.

The Network should:

R1. Identify, review and clarify its primary strategies. This should be done based
on an analysis of (internal) strengths and weaknesses and (external)
opportunities and threats. Sharp, focused strategies should feed into efforts
to prioritise policy agendas better, so that the network can develop
achievable objectives and operate to greatest effect in achieving its goals.

R2. Ensure that systems for gathering information about policy contexts and
targets, and systems for reporting provide meaningful intelligence about
strategies and their effectiveness, as well as record the delivery of activities
and milestones.

R3. Review planning and decision making processes and establish procedures
that include:
laying out consultation guidelines for national convenors;
building in time so that all can contribute equally; and
adopting measures to ensure the network‟s focus is preserved and
applied.

R4. Develop ways of working that promote and encourage cross-national

collaboration, and review who might be best placed to lead on which aspects
of work, and - on this basis - consider allocating lead responsibility for key
work to specific organisations.

R5. Set out a process and timetable for resolving questions of membership
expansion, including:
What added value would the network want to gain from a new member?
What benefit would the new member get when joining AfGH network?
What possible obstacles will need to be faced – both by the network and
the new member?
What added value can the network offer candidate members?

page 8
 EVALUATION REPORT

part two: findings and conclusions

1. introduction

We summarise in the appendix the purpose of this report and the approach we have
taken to gather evidence for it.

This version of the report is an updated version of an earlier draft that was circulated
to members for comment. We have attempted to respond to all comments in this
new (and final) version of the report.

Our primary concern in conducting this evaluation is to provide an opportunity for

the Network‟s members to consider experiences to date and learn from them so that
future working can be more effective. During its first eight to twelve months the
Network has achieved a great deal, in effect delivering what it set out to do in that
period. This report should ideally act as a tool to encourage and stimulate internal
reflection on those achievements, but with a focus on the important issues that have
been given less or no attention to date.

We regret that the draft report has not been effective in generating such internal
reflection. Few Network members were able to give feedback on the draft report,
and many found it difficult to find the time even to read it. Clearly this reflects a
specific failure with this evaluation process as well as specific flaws with the report (it
was felt to be too long and not sufficiently clear for non-English speakers).

But we are concerned too that this relative lack of engagement with the process may
also be symptomatic of a wider concern - highlighted too through interviews carried
out as part of our research - that opportunities for reflection within the network are
too limited or not taken up sufficiently (because of pressures of work and other
factors).

In addition, the nature of the responses received on the draft report seems to
confirm the point being made elsewhere that opportunities to input and shape the
future of the network are currently disproportionately taken up by UK network
members (because of issues of staffing capacity and language, amongst others).
This is not meant as a criticism of the UK partners or of others, but it reveals an
inequality of participation that the Network needs to address.

Although written responses elicited comments such as “a helpful contribution” and

“good, well written and thoughtful”, our efforts to describe the challenges facing the
Network were met by mixed responses. Some felt the criticisms made were “harsh”
in some places, whereas others felt the report was “too diplomatic” and did not get
to the heart of some of the issues. In fact, whilst we respect both of these
perspectives, we have sought to maintain a balance: we are keen that issues of
concern be raised not ignored, but keen also to express them in sensitive and
constructive ways.

We hope that we have raised issues arising sufficiently well to allow the Network and
its members to follow up on them, by identifying and addressing key priorities over
the coming months and years. For this reason, we very much welcome the plans

page 9
 EVALUATION REPORT

already underway to ensure that timetabled processes will be developed in

response to the recommendations we make.1

2. project rationale and relevance

The rationale for the AfGH project is expressed in the July 2006 ActionAid
International et al proposal to the Bill & Melinda Gates Foundation as follows:

“Advocacy for global health has a long tradition in Europe but is almost
entirely focused on specific health topics. Moreover, cooperation between
groups working on different health topics has been limited, with little
advocacy for global health as a whole, traditions of advocacy varying
considerably across European countries and little attempt at coordination
across Europe. This proposal sets out a programme for developing the
capacity of European civil society to advocate collectively for global health,
using the Health MDGs as a focus”.2

Both internal AfGH and external respondents to the evaluation suggest that this
rationale is a valid one – at least at a cross-European level.

Speaking generally, it is clear that decision makers naturally find it easier and more
efficient to deal with groupings rather than individual organisations. Networks offer
the prospect of benefit through increased capacity development, improved quality
and sophistication of analysis and strengthened engagement and influence. The case
for networking to achieve change is also well illustrated in research and literature
relating to joint working in advocacy.

More specifically, we heard the case that, in the past, Health NGOs have not been
well coordinated at a European level and often appear to have been in competition.
The effect on EU policy-makers has been that they “end up getting mixed
messages”.3

And the establishment of a European network was said to be timely. Given that aid
budgets in most cases are comparatively high, and growing, the benefits of
successful joint advocacy are potentially significant. But health, according to some,
is seen as “old-fashioned” - in Brussels, for example, where the Commission seems
more concerned with promoting good governance. And with fears in the EU around
the increasing influence of China, one response seems to be an inclination to put
fewer conditions on the use of development funds. The fact that AfGH is asserting a
specific health agenda is therefore seen as a necessary and valuable counterbalance
to these challenging trends.

At national level, we heard that the current state of the debate – in Germany and the
UK for example – meant that the network‟s formation was timely too: “There is not
much pressure on Governments to fulfil their promises”, and, “we bring a vigilance
function that was not covered before”. The fact that the network is essentially

1
Text in bold highlights particular issues which we suggest the Network should consider through
reflection, discussion and resolution in the development of future work.
2
page 6 of the proposal
3
Unless otherwise indicated, statements in quotation marks have been taken from respondents.

page 10
 EVALUATION REPORT

advocating to governments to implement (international) decisions that have already

been made is seen as favourable to both national and international collaboration.

Both in intention and - according to internal respondents - in practice, the network

allows its members to advocate to and engage with key audiences and targets in a
(more) systematic manner. Also significantly, the AfGH network appears to be
fulfilling a need by advocating for global health and better health services as a
„cross-cutting‟ issue. Whilst there remain some issues about what this means in
practice (as we discuss later), the fundamental notion seems sound. As one external
respondent noted, with particular reference to AfGH being able to bring practical
experience to a so far largely theoretical debate, “AfGH is entering the debate at the
right point”.

However, in some countries overlapping networks exist. This means that, for some
respondents at least, it is not clear what the AfGH network is bringing that is not
already covered by other institutions, networks, or alliances: “We bring a force but
not necessarily something new”. In different contexts, there is a different
perspective. In France, for example, the analysis was that, “Health and development
was covered by large NGOs working independently with a vertical approach. We are
occupying an empty space”.

We are only able to present an overview of AfGH‟s role and contribution in the
different national arenas in which it operates. More work seems to be needed to
identify the added value of AfGH in each of these contexts. This should also take
into account the relative strengths and weaknesses of AfGH members in relation to
other NGOs in the country. The challenge is to establish how the Network can
best add value in different contexts and strategies to support this, based on
a good understanding of the national political and policy concerns, and
within the context of an over-arching EU-wide approach.

3. progress to date

In normal circumstances, networks evolve in response to potential partners

recognising a need. This means that, at least theoretically, the structure and
membership of a network derives from its purpose. In the case of AfGH, “the
network was created because of funds and not the other way around”. Combined
with the situation that members generally did not know each other, and with the fact
that year one plans were designed by people other than those responsible for
implementing them, this could be seen as an unpromising start.

Given this, we think it commendable that the basics of the network are in place. The
structure is sound, plans have been delivered, much has been produced (setting up
the website, fact sheets, reports etc.), and an identity has been forged: all this
despite some inevitable delays, for example in appointing staff.

The fact that a shared set of top-level messages and policy recommendations has
been developed and agreed has been a sign of a good start too. As many internal
respondents mentioned, this has been a difficult process: “We have to find
agreement along the way and not before the beginning of the project; that would
have been the ideal”. Although many network members recognise the limitations of
the current policy positions adopted, the general sense is that they form a good
basis, potentially at least, for the development of future advocacy.

page 11
 EVALUATION REPORT

Difficulties and shortcomings are to be expected, certainly in the first year. And it is
worth stressing that the Network has only been functional for a few months. From
the information which we‟ve gathered and from our analysis of it, we have not
identified major flaws within the intentions or set-up of the project. That should be
seen - and is meant - as high praise.

At this stage, it makes sense to look at the value of the network in its potential
rather than in its achievements to date. So it is encouraging to hear that externals
generally saw the establishment of the network as a positive development both in
terms of health related objectives and in terms of creating synergies at European
level. Indeed, for one external respondent, the value of AfGH's work is already
apparent: “health as an issue has become more clearly identified; the issue is more
prominent, which is in part attributable to AfGH and an indication that working
together has been beneficial”.

Our own view is that it is too soon to comment with any force on the benefits and
added value of national and cross-national levels to date. The important thing at this
stage, therefore, is to look at whether the conditions are in place to ensure future
effective working.

It seems that the starting point for the further development of the network is
relatively good: a good spirit and interest in the work in hand exists. But it is
important too to recognise the challenges facing the network, so that members can
reflect on them and then work out what needs to be different.

4. organisational and national contexts

In developing the project proposal, the partners that were brought together seem to
have been contacted primarily because of existing personal relationships. Given the
pressures on time, this may have been inevitable, but the result has been a network
of very diverse, if not disparate, NGOs. This is not necessarily a bad thing, but it
requires careful management. And for AfGH currently it is not yet obvious how one
member‟s strengths complements those of another, nor how one member‟s
weakness in a particular aspect of work can be accommodated through another
member‟s strength.

Without wishing to overstate the differences amongst the network‟s members, the
fact remains that these include:

culture: as shown in organisational processes such as those around internal

decision making, and Advocacy Officers‟ ability to take decisions on behalf of an
organisation, which varies from NGO to NGO;
size: both in absolute terms across the network and in relative size in national
contexts;
orientation: some having a focus on vertical (i.e. disease or other specific
intervention) issues, others concentrating on broader (horizontal) health themes;
language: with English being the project‟s working language across the Network
and the first language for some members, for others it is their second or third
language;
connections and relations with other NGOs: with some members “at the margins
of the NGO community” and others much more at the centre;

page 12
 EVALUATION REPORT

staff assigned to the project: some are new in their organisation, while others
have been employed in the same organisation for some time;
staff involvement in AfGH work: some are involved on a full-time basis, others
are part-time;
practical support from within the member organisation: for example, support
from media specialists is in some cases available and sometimes not (in which
case those assigned to the AfGH project have to do everything);
organisational support for advocacy: for some members advocacy is a core part
of the function of the NGO, in others it is less so;
organisational expertise: significant differences between members in, for
instance, capacity to carry out policy analysis, or media work, as well as
variations in levels of experience of negotiating with decision makers, public
campaigning, etc.; and
the specific position of the partners based in Brussels: these partners rely on the
input and approval of the other network members in a way that those operating
at „national‟ levels do not.

Such differences can be accommodated within the network over time (as staff and
organisations get used to each other and to the task in hand). In general it helps to
acknowledge and address them specifically: differences, if not well understood, may
lead to misinterpretations and lack of trust. Such understanding is particularly
needed for what may be called cultural differences: mentalities and philosophies
relating to work and its organisation. From early reports, the current peer reviews
(probably better called exchange visits rather than „reviews‟) have gone some way
towards developing the necessary mutual understanding. But further appreciation of
what these differences entail will help avoid situations where there are attempts by
members to impose rigid ways of thinking and acting, with the risk of corresponding
inaction or disengagement by one or other member organisation or network
„country‟.

What may be more difficult to deal with are the different levels of organisational
support to the AfGH project: commitment to the project within the different member
organisations varies. Some internal respondents report that “buy-in is a little shaky
in some places [in the member organisation]” or that “[involvement in the network
is] tolerated but not strongly supported”. Buy-in, or its absence, relates closely to
real or perceived tensions between the network‟s relatively broad health agenda and
a member‟s more specific remit.

In some cases, the level of integration of the project within a member organisation is
low (particularly in organisations with disease specific remits given that AfGH is
seeking to work „horizontally‟). As one respondent said, “[it feels like the
organisation is] hosting a stranger”, since there is very little cross-over with other
work done by the member NGO.

The survey results confirm that there are currently mixed views about the Network
and its value to its members (see Part 3 of this report). It is striking that those
whose responses across all the bi-polar questions tend to be high (indicating
dissatisfaction with the various elements of network functioning and performance)
are much more likely also to score the network low in terms of its „importance‟ and
„benefit‟ as well as its „effectiveness‟. The evidence from the survey, supported by
interview findings, is that the majority of members are broadly positive about the
network and its ways of operating, but that there is a bloc of organisations whose
representatives are significantly less positive about the network than the others.

page 13
 EVALUATION REPORT

Going forward, this could result in problems if the difference of perception and
attitude between groups of members is not reduced.

This may be partly about different expectations: for example, some hope for their
organisation‟s profile to be raised, others have less desire for this. But the opinion
that, “nothing has been done to date that could not have been done anyway in my
own organisation's name” - although perhaps at one extreme - illustrates a
perception that is potentially damaging for the success of the network.

Differences in levels of ideological and practical buy-in, and differences in opinion

about network interests and benefits, are likely to create problems for the AfGH
project if they are not resolved. Evidence from the literature emphasises that
networks are most effective when the goal of a network is consistent with, and
furthers the mission of each organisation entering into it and when network members
consider the priorities of the network their own.

Awareness within the network of the existence of such diverse opinions about buy-in,
interests and benefits – let alone thinking about its implications - seems to have
emerged gradually and relatively recently. It has not yet been properly addressed.
There is a need to promote mutual understanding of constraints and needs, and of
contributions that people in different organisations can make.

As part of this, the Network should look more at the strength of each organisation,
and consider how best tasks might be divided accordingly. This was a view that
many informants agreed with, expressing a sense that specific NGOs‟ expertise has
not yet been exploited enough, for example in the production of fact sheets or in the
thematic development of the project.

Given the differences that exist between members and their national situations, one
consequence for practical work has been that some members have adopted a
generally reactive stance to network initiatives. As a result, national partners‟
opportunity to develop working relationships amongst themselves has been quite
low. In other cases, notably in the UK, network partners have been able set up
intricate joint working systems based on close alignment, with significant
communication between themselves.

Progress is being made at different speeds. As a result, on the one hand, a sense of
frustration was communicated to us by some members that some organisations are
not making an optimum contribution to the network, or at least not visibly. On the
other hand, there are feelings that some members are seeking to lead others in the
work of the project in ways that are inappropriate and not fully sensitive to cultures
and constraints. The very busy first-year workplan, with its tight deadlines, has
exacerbated some of these tensions.

One issue to consider in particular is that the network has tended to be northern
European - in particular UK – dominated, with others - in particular members in
Spain and Italy – having to work hard not to be removed to the margins. Amongst
other reasons for this is that in its design, the project gave some precedence to UK
NGOs, through providing funding for three full-time posts while in other countries
two were funded. In dealing with the implications of this, the Network must balance
the need for efficiency and effectiveness – recognising, facilitating and building on
members‟ good quality work - with the importance of a sense of mutual
accountability and appreciation of others‟ contexts.

page 14
 EVALUATION REPORT

Differences in the pace and quality and focus of work across the network must not be
allowed to affect its future effectiveness. Recognising diversity of contexts should be
the starting point for developing greater sensitivity to aspects such as timing,
working methods, and decision-making needs of network members. In the words of
the proverb: “if you want to go quickly, go alone; if you want to go far, go together”.

Trust is the glue that holds networks together but is difficult to develop where there
are significant cultural differences, language barriers, different capacities and
capabilities and organisational pressures. Likewise, trust cannot be developed where
members do not know or value each other. Although the current exchange visits
(„peer reviews‟) provide an important „getting to know you‟ mechanism, AfGH staff
should give explicit attention to working towards becoming a „virtual team‟, aware
and confident in the skills and values they each bring, and acknowledging others‟
added value.

One model of joint working identifies that successful collaboration develops in

phases:
from attention to „form and focus‟;
to „organisation and action‟;
to „achievement and transformation‟ (Taylor Powell & Rossing).

It seems that the AfGH network, with early pressure to deliver on milestones and
outputs, has to some extent jumped in at phase two, which means that some of the
phase one aspects have not yet been resolved.

To foster improved trust and joint working, guiding principles and values
which members relate to and work with each other should be established,
including around:

the need for collaborative action;

a common understanding of the practicalities of mutual reliance;
mutually agreed parameters regarding expectations
respect for diversity enabling all voices to be heard;
acknowledgement of power differences within the Network, but with
a commitment to equality in decision-making.

5. plans and planning

The fact that the network has delivered on its plans is something that we think
members should be both very pleased with, and also sceptical about.

The work that this involved was hard and occasionally “very stressful”, in particular
in the run-up to the release of the national reports. Although this does not seem to
have seriously affected the quality of work produced, it has led to the results being
more limited than might have been the case had more time been available.

There has been significant effort to ensure that project activities and milestones for
year one have been achieved. Some feel that this has been achieved at a cost:
“Sometimes we have been more worried to carry out the agreement with Gates
Foundation than to do the advocacy work with quality”. Campaigners are often in
the position where there are constraints in place that make it difficult to gather,
share and learn from information and intelligence, and apply the lessons in the

page 15
 EVALUATION REPORT

future. Often this boils down to problems of finding the time. It will be important for
the Network to ensure there is meaningful space where reflection can take place in a
considered manner, and to ensure that opportunities to reflect are valued, and not
seen as an unaffordable luxury, or a diversion from action.

Planning of activities other than those outlined in the proposal seems to have been,
in some settings at least, rather unstructured and reactive. Whilst the survey results
suggest that, overall, members themselves see planning and implementation of
plans as a relative strength, there is a sense too that there could be a clearer focus
on what‟s important (see part 3).

In particular, plans will need to be responsive to concerns around speed of, and
mechanisms, for decision-making. In the first year, there have been sensitivities
about this, highlighted in the inability of Italy and Spain to launch the report at an
apparently jointly-agreed time. (Italian and Spanish partners are aware of the
sensitivities, but report that they were unhappy about the deadline from the start.
Other partners have expressed frustration about the perceived delays and profess to
not understand why they happened.)

More generally, various informants mention that there has been pressure to make
decisions in teleconferences when they feel insufficient time has been available for
preparation and consideration: “we put pressure on ourselves that is not necessarily
justified”. Such pressure to decide „in an instant‟ has been particularly felt when it
comes to taking positions on issues raised by members. This indicates that longer
lead-in times for decision-making are needed. It also perhaps suggests that the
network covers too much ground and that there is a need for agreeing (annual)
criteria or guidelines that establish a focus on specific global health themes and
create a framework for debate and development of those selected themes. What
falls outside those criteria or guidelines would be for members‟ individual decision
and not as part of the Network. This should then enable the network to react “fast
but accurately” to aspects that fall inside the agreed guidelines.

Reasons for this pressure on decision making are not always clear; after all, as one
member pointed out (in a line of thought echoed by others), AfGH is not operating in
an emergency situation. It‟s right that there is a strong impetus behind the work,
and a sense of its urgency and importance, but ways of working need to be both
sustainable and appropriate to ensuring strategic effectiveness. It may be that the
heightened pressure of delivering the first year milestones in what was actually less
than a year has created temporary difficulties for the network, but our experience is
that these things have a habit of becoming established ways of working unless
reconsidered explicitly.

We believe that greater flexibility of timeframes, taking into account the varying
available staff time, would not only help bring improvements in working relationships
but also help to ensure a higher quality of advocacy activity and thus improved
results. Key to this will be to ensure that quality assessments – as different
from largely quantitative milestones - are identified and established as
important for AfGH members and the project.

page 16
 EVALUATION REPORT

6. strategy

Strategies deployed in this first year have largely been in line with those outlined in
the proposal to the Gates Foundation. They have not been significantly developed.
Although the proposal gives a sketch of components of strategy, it does not give
much of an assessment of possible approaches that might be best suited to
achieving the aims of the project. From the available information, it is not evident
that AfGH strategies make the best use of the network‟s resources (“we do not know
yet the strengths of the members”). Nor is it clear that there has been sufficient
exploration of the external operating environment (for instance through external
environmental analysis, contact mapping, identifying influencing routes, etc).

Some internal respondents, however, feel that the outline of strategy as shown in
the proposal is sufficient: “We have the right tools. The key will be to use them at
the right moment”.

Other respondents suggested that the proposal gives too much attention to
milestones and activities (driven it seems by the requirements of the grant funder)
and that a greater focus on achieving success (influence on political and social
change) may be best achieved using approaches other than those already outlined in
the proposal, mentioning that, for instance, political engagement strategies are not
in place and have not been adequately developed.

Whichever of these opinions one endorses, there is a generally recognised need for
greater clarity on strategic vision, especially with many respondents stressing that
strategy coherence is (and will continue to be) under pressure due to the tendency of
the members to try to align the network agenda with their institutional priorities.

We suggest that strategic analysis of both internal and external contexts should be a
priority for the network. This would entail assessing internal strengths and
weaknesses and external opportunities and threats.

Some building blocks for this have been put in place, for instance through the
development of some joint policy positions, and through collaborative work to date.

However, a structured approach to this is needed to replace the current tendency to

focus on activities instead of explicit attention to the creation of strategic results. As
one interviewee put it, the network has so far been “always looking forward, without
reflection”. A clearer strategic framework would give much-needed criteria with
which to assess the value of particular aspects of work, including for instance
attendance at and follow up to external events.

The Network should identify, review and clarify its primary strategies. This
should be done based on an analysis of (internal) strengths and weaknesses
and (external) opportunities and threats. Sharper strategies should feed
into efforts to prioritise policy agendas better, so that the network can
develop achievable objectives and operate to greatest effect in achieving its
goals.

Improved political information could also be valuable in helping the network to

establish sound political positioning - balancing dialogue, support and criticism -
bearing in mind that the policy environment is reported to be particularly sensitive in
some arenas – in Brussels, for example, where relationships between NGOs and the

page 17
 EVALUATION REPORT

Commission have, according to some, become quite conflict-prone. The idea, floated
in the funding proposal, to test positions with focus groups made up of allies within
target institutions could perhaps be revived.

The kind of approach we are outlining here would also help support the creation of a
monitoring and evaluation system with indicators to assess progress towards goals
and objectives. At the moment, such indicators seem to be lacking, or if they exist,
members do not seem to be aware of them, despite the claim in the proposal that
“indicators and baselines have been established or will be in year 1 of the project”.

We have a concern, that some internally also identified, that the network to date has
been too focused on delivering a set of activities, without taking sufficient time to
present a coherent and transparent sense of the social and political change it is
seeking, and the pathways towards achieving it.

It is not yet clear that the Network has a common or clear sense of what would help
demonstrate progress towards achieving goals, beyond delivery against milestones
which are typically measures of activity only.

Some dimensions to this wider question could include the following:

focus of enquiry possible benchmark

activities milestones achieved
network processes money effectively disbursed;
quality of planning, monitoring and evaluation;
members‟ satisfaction with systems and procedures;
lessons learnt are identified and serve as the basis for
reformulating strategies and activities
political outcomes network is seen by key audiences as a reference for global
health;
levels and quality of engagement with decision makers;
global health issues are high on national and international
agendas;
participation of network members in the design of the health
policies strategies;
ODA for health increased
organic outcomes growth in the network;
relationships between members;
motivation and performance of individual members;
enhanced capacity to act jointly;
network sustainability;
network acknowledged as key partner by organisations in
the South;
participation of South organizations in the network
impact 2015 targets on track/met
improved health of poorest people in South

This list is designed to be illustrative rather than definitive. The point is that the
network would benefit from developing a clear and shared sense of what it is trying
to achieve, and how it will know if it is on course to do so. Externally conducted
reviews like this one are not a substitute for the Network itself clarifying its
aspirations and then ensuring that it has good information about the extent and
speed of progress towards them, so that tactics, strategies and ways of working can

page 18
 EVALUATION REPORT

be adjusted as necessary along the way. This would include, for example, tracking
how the network itself is evolving as well as recording the reaction from key
audiences and targets to the network and its issues (and in both cases reviewing
approaches, activities and strategies in this light).

The network is battling huge forces. To make its contribution in the most effective
way, the Network will need to be very clear how it can bring its resources to bear in
ways that makes most effective use of them.

As part of this, the Network will need to set out ways that ensure that
approaches to information gathering and reporting provide meaningful
intelligence about strategies and their effectiveness, as well as recording
the delivery of activities and milestones.

7. influence and outcomes

As already anticipated in the evaluation Terms of Reference, little can be said at this
stage about the influence and outcomes of the AfGH project to date. It is, however,
a concern that evidence of network influence - let alone outcomes - is not being
recorded, suggesting that a more systematic approach to political intelligence
gathering needs to be put in place.

Some, largely anecdotal, examples of AfGH influencing the debate in some countries,
or moving the issue up the agenda, have been mentioned to us or appear in peer
review reports. At least one MEP (Wolfgang Wodarg, SPD) refers to AfGH on his
homepage, endorsing AfGH‟s analysis. And the opinion of internal respondents that
AfGH‟s information supply is appreciated is corroborated by another MEP, who noted
that the network is important for her work as a source of information. She
commented on the well-organised website which enables her to follow the debates
about health and development issues.

Members generally feel that, while it is early, AfGH is increasingly being recognised,
by both NGOs and targets. The report and dissemination work done to date is
helping to forge good relations according to respondents, including with
organisations interested in becoming members (see section 18). Other NGOs are
noticing the issue raised by AfGH (e.g. Coordination Sud in France) and express an
interest in meeting to explore possible shared interests. Increased recognition of
health as a key development issue has led, in Italy, to the issue being brought into a
draft law for international cooperation. And in France, we understand that AfGH
members have been invited by the government to comment on their latest health
strategy.

As discussed in the section on Strategy (section 6), there is a need for the Network
itself to set up means of gathering political intelligence through its day-to-day work,
as well as systems for interpreting findings. This should enable it to be in a position
to make judgements about the effectiveness of strategies and tactics pursued, based
on evidence of achievements (and failures). Any future external evaluation of the
network should have the purpose of looking to corroborate, challenge and possibly
provide alternative interpretations to the internal analysis. Such external evaluation,
however, cannot be a replacement for internal analysis.

page 19
 EVALUATION REPORT

8. health policy agendas

While one partner commented that AfGH members have coalesced around a common
agenda, and that the network “has advanced much quicker” than others with which
they have experience, another felt that a shared agenda was, “still far off ... at the
moment AfGH is all over the place, engaging in any and every discussion on health
systems … over-ambitious”.

These diverse views reflect the fact that, given the short timescale, progress on joint
positions has in many ways been remarkable, but more needs to be done to make
these meaningful and feasible.

The potential value of working across vertical issues is that it can be a bridge
between different interest groups. And certainly, the evidence from the ground in
Southern developing countries, as we understand it, is that integration of vertically
constructed health programmes into a general healthcare focus represents an
important way forward in strengthening health provision.

Thus, the aspiration is a sound one, but to achieve it involves a difficult balancing
act. The challenge is to find ways to build on individual members‟ strengths without
the network losing sight of its broader mandate and positioning.

One external respondent saw it as “impressive that AfGH started off as a network of
disease-specific organizations and is moving towards the overarching health systems
agenda”. However, the respondent continued, “there is still a dual message in the
report. It is asking for both disease-specific funding as well as for funding for health
systems. AfGH needs to be clearer whether they are advocating on horizontal or
vertical issues. At the moment there are mixed messages”.

These external views on the nature of the Network‟s vertical versus horizontal
approach possibly reflect the fact that members themselves don‟t always seem to be
wholly clear about the Network‟s position on this.

There are concerns amongst some members that the current inability to talk about
intervention or disease specific issues means that the network is not sufficiently
making use of their strengths or needs. But at the same time, it is right that these
pressures are counter-balanced: “the partners have to regularly revisit and confirm
their shared agenda and conceptual identity [i.e. the horizontal (cross-cutting)
theme of health]; otherwise there is a danger of becoming another AIDS network”.

It would certainly help to clarify key issues around policy and

communication, for instance:

is it appropriate to advocate on intervention/ disease specific issues as a

means into the broader health and development debate?
if ‘yes’, when would this be appropriate?
how can the network keep focused on the cross-cutting health and
development agenda when this may in some circumstances be more
difficult to communicate than intervention/disease specific issues?

The need for more rigorous identification of when best to intervene is identified, for
example, by this response: “There are so many things happening … that we are

page 20
 EVALUATION REPORT

running behind the agenda. It is a struggle to keep up ... we would like to steer the
agenda more”.

As mentioned elsewhere, this can partly be addressed through better forward

planning. For example, an annual assessment of forthcoming European and national
advocacy opportunities (that relate to the network‟s priorities) would allow the
position to be developed in advance.

Obviously, sometimes work is reactive, and cannot be wholly anticipated. For

example, the recently launched International Health Partnership was initiated by the
governments of Germany, the UK, Norway and Canada, but now includes France,
Italy and others as well as the European Commission and UN agencies. This seems
like an initiative requiring significant attention from the Network. Responding to
such interventions and opportunities requires both clear strategic parameters and
also capacity (time and ways of working).

From the feedback received, however, it is not clear if current ways of working
promote the operation or development of a systematic, integrated approach. For
instance:

the policy Working Groups, by all accounts, “are not up and running yet” and the
fear has been expressed that they have become too general. This may be
because of organisational reasons (for example because their terms of reference
are not sufficiently detailed) or because of planning reasons (for instance
because the overall strategies or external analyses are insufficient to give the
required guidance).
members are finding it difficult to keep pace with the need to take positions on
various issues raised by other members. As mentioned previously this may be
due to a lack of sufficiently detailed set of priorities. It also indicates for some
members that the current policy of „no comment = yes‟ needs adjusting, as it
“gives an advantage to those who make propositions”, risking a more diffuse
approach with a reduced focus on the Network‟s core intention.

To help resolve these issues, there is a need to agree criteria or guidelines

establishing a focus on specific global health themes. These guidelines
would create a framework for debate and development of a common
position on the selected themes. Any such guidelines should incorporate
attention to systematic ways of working that enable speedy, accurate
reaction when required.

9. policy analysis and positions adopted

Comments by network members on the policy reports were somewhat muted. They
were cited as useful internally by a number of members: “it is a good to have the
report as a tool for communication and advocacy”. One Italian partner saw it as “an
opportunity to step back and reflect on specific topics, although the lack of available
data at national level made it difficult to produce a realistic picture”. In Germany,
internal comments noted that the “end product is ... not as relevant for the
[national] policy audience and NGO scene [as it could have been]”. At an EU level,
according to one interviewee, the report was “insufficiently tailored towards the
interests and expertise of the audience”. While, from the UK, an opinion was

page 21
 EVALUATION REPORT

expressed that, “the first two sections contain some broad evidence that was already
in the public domain but didn‟t get down to details”.

Internally too there was mention that the report was based on a “weak consensus
and [with] recommendations that sometimes lacked teeth”. The process adopted for
developing the report also received some criticism as being too top-down: national
sections were derived from the international, and the opinion was expressed that this
should have been the other way round (or at least ideally in both directions
concurrently).

Interestingly, internal respondents generally suggested that external views of the

report were more positive than their own, stating that they had received positive
feedback from other civil society organisations, and that it had been noted in
government circles, in some cases leading to access and debate within decision-
making circles.

The timing of the evaluation makes it difficult for us to corroborate this sense (for
example, the Spanish report was launched on October 15th, after interviews had
taken place). And in any case, we have only heard from a very limited number and
range of external respondents. Those external respondents who could comment
largely share the internal view that the reports have been useful in publicising the
NGOs' thinking, and as a reference text. One MEP indicated that, “The report gave a
good and also very detailed overview of the work of the EU and [national]
government to date … it did point out shortfalls that undisputedly exist”. Another
external respondent said that the report was “of fairly good quality, given that the
information was still moving”; another commented similarly, along the lines that it
was good given the circumstances. However, we did hear critical comments too,
with one official for example identifying “mistakes which undermined its value …
[and] it had a limited vision”.

Notwithstanding this, it seems to be the case that the reports met a need: “a good
product at the right time – what was needed was a set of fairly crude messages”.
The reports have established a foundation; this will need to be built on: “the network
needs to update its messages to show it understands the broader context ... To do
so would boost [its] credibility. To not do so means you fall back into the group of
NGOs just making noise and whose messages are crude”. With this in mind, it is
perhaps encouraging to note that a number of external respondents made positive
mentions of the AfGH newsletter.

Development of the report also raises the question, for the design of future
propositions by AfGH, as to whether the Network sees itself as a producer of new
knowledge, and/or whether it wants to be a collator and disseminator of already
existing knowledge and information produced elsewhere. The answer to this would
indicate the Network‟s ambitions and the extent to which AfGH is restricted or not by
its members‟ existing knowledge and expertise.

One current problem is that many members are not at all sure what others are
calling for (for example in section 3 of the national reports). Although the ability to
frame an issue successfully and jointly in an international network can be particularly
difficult, there is a need to give time to this if the network wants to achieve
meaningful results.

The current consensus, although described as “weak” by some respondents, provides

a basis on which to develop stronger joint positions. Network members must create

page 22
 EVALUATION REPORT

the time and energy to enable this to happen. In order to identify and expand
areas of policy consensus, the Network should conduct a comparative
analysis of current published policy asks of the Network and its individual
members.

10. public communications

The proposal to the Gates Foundation identified in effect five broad audience groups:

governments in the different national arenas (including the EU institutions);

NGOs;
EU citizens;
media; and
the private (healthcare and research) sector.

Such broad audiences are obviously too general to be of much use in practical
advocacy work without further refinement. Some of these groups, for instance, may
not be target audiences, but instead may be channels through which other (target)
audiences are approached. This is an area that needs more work and is closely
related to the development of the network‟s (influencing) strategies, addressing such
questions as: who are the priority audiences that need to be engaged or addressed?
Are they different in different national contexts? Which potential channels can be
used to reach them?

A general public communication tool currently available to AfGH is its website.

Although in the early stages of development and not yet widely used, its potential
usefulness is indicated by some comments from externals (see section 7). We
understand that plans are afoot to improve accessibility to the site (including through
languages other than English), and to improve its potential relevance (including
through expanding the links to articles relevant to AfGH and to activities in individual
countries). Such development should increase what looks like limited use of the site
currently (around 1800 visits in one month covering October-November, and a
majority of visitors originating in the UK, Belgium and the USA). Improvements to
the site are also likely to increase reference to AfGH on other sites. A search on
Google for internet sites which mention AfGH gives 604 references for “Action for
Global Health” (which also covers French and German language references), 372 for
“Acción por la Salud Global” and 851 for “Azione per la Salute Globale”.

Specifically considering public audiences, it seems that different opinions exist,

perhaps reflecting different national contexts. In the UK for example, partners have
only limited capacity to make health and development a „public issue‟ (and in any
case there is an existing public campaign on this issue run by a major NGO), while
Italian members are talking about the need for wider public (as well as NGO)
support, in order to put pressure on decision makers. One respondent suggested the
desirability of a joint popular event to create public „noise‟, and some members
would like to allocate a communication specialist for the production of
communication materials. However, it would seem that a more developed set of
public communication strategies is needed first.

In developing these strategies, the question has, firstly, to be: what support is really
needed and where from (in order to achieve the core aims of the network)? And,
secondly, if it is needed, is it possible? Public campaigns are very resource intensive

page 23
 EVALUATION REPORT

especially when the media environment is reported as a difficult one in different

locations. Positive media coverage of the launch was mentioned by Italian members,
for example, but not highlighted elsewhere.

Approaches towards public communications should be considered as part of

the wider strategy development process that we advocate in section 6.

Part of the answer to these questions will depend on the broader, and global,
advocacy campaigning environment on health and development issues. Ensuring
that members are being informed of - and where advantageous relate to - other
relevant campaigns is likely to be a central task for the Coordination Team.

11. cross-national working

Some members are disappointed that to date the Network has insufficiently been
able to benefit from what is possibly its greatest asset: its international character.
German staff, although aided at short notice by UK members, did not always feel
well supported by other network members at the G8, for example.

In an international network, operating at different levels of influence can bring vital

benefit. At a relatively simple level, for example, German AfGH staff mention that
they are often asked by politicians and others where Germany stands in relation to
other countries in terms of MDG performance and policies. Evidence on this could
provide a powerful advocacy tool, but it cannot be answered without information
from other countries. Through better shared intelligence and joint strategising, AfGH
can fulfil a key benefit of networking. Members will be able to advocate common
positions simultaneously, across different arenas, in order to exert multiple routes of
influence.

As part of its development of ways of working that promote and encourage

cross-national working, AfGH should review who might be best placed to
lead on which aspects of work, and - on this basis - consider allocating lead
responsibility for key work to specific organisations.

For example, there may be particular lead roles which the Brussels groups could
usefully and productively take on, with an EU-wide perspective; and Working Group
leads may be in a position to take on a more proactive and strategic role on the
network‟s behalf.

To be most effective, these kinds of approaches may depend on some reallocation of

resources between members from that currently agreed.

12. network coordination and management

The network coordinator is generally respected, was widely - although certainly not
universally - praised for his approachability and for the support he has given, and
additionally commended for displaying energy, finesse and balance.

Inevitably, there were criticisms too. These were mostly along the lines that the
coordination function has been seen as being too controlling to date. Certainly, the

page 24
 EVALUATION REPORT

literature stresses that the coordinator‟s emphasis should be on quality rather than
control and that the coordinator‟s role is to encourage innovation and focus.

But in this case there are complexities to be negotiated flowing from the
Coordinator‟s multiple roles: firstly, as lead in a network aspiring to be democratic
and participatory; secondly, as project manager, answerable to managers in
ActionAid and to the Gates Foundation for the network delivering against plans; and
thirdly as conduit for representing ActionAid‟s positions, as partner, within Network
discussions. This is clearly a very difficult balancing act, especially given some of the
issues we raise below about governance.

Other more specific complaints – e.g. that in one country the Coordinator has been
too operationally involved in national affairs – would seem to be more easily
resolved.

The point is that there is a balance to be struck between strong leadership and
participatory decision-making, and between trust and formalisation. Basically, the
more top-down and centralised the network, the more you trade off participation for
efficiency, speed and leadership.

In relation to whether the coordination function delivers a good balance, opinions are
mixed. This only serves to reinforce the fact that one person‟s momentum is
another‟s feeling that there is unwelcome direction from above. The sense from the
survey results was that the coordination function is currently too operational. There
were some suggestions of “laborious [over-]consultation”, with the accompanying
recommendation that the Coordinator should take more of a leading role in resolving
things, “rather than always setting up a group to look into things when issues are
raised”. Others seem to value the approach taken, however, welcoming
opportunities for involvement. The key challenge seems to be how to structure
members‟ involvement in decision making in ways that make it easy for all to
participate whilst ensuring there is good quality involvement and that good quality
decisions result.

There was a widely felt concern that the need for immediate answers - for positions
on health policy issues for example - has affected the quality of decision-making.
Clearly, more time needs to be scheduled in for translation and for national
consultations before Steering Group meetings. We heard from several sources that
national convenors often go into Steering Group meetings/conferences unprepared
(without having consulted on or considered issues for discussion), with the result
that decisions have been taken, or implicitly accepted, that turn out later not to have
been wise. One input suggested that the network is working at “a pace and style
that … does not suit non-English speaking countries with only two staff”.

The challenge is to provide a greater focus, in order to promote network

cohesiveness, enabling a more measured decision making process. As part of this, a
clear eye should be kept on decision-making arenas. At least in the early days, the
Coordination Team has involved itself in processes of agreeing policy, which, for one
partner, are more properly the role of the policy officers collectively. It was felt now
though that there is becoming a greater and more appropriate separation between
the functions of coordination and policy setting. In addition, the Steering Group
should gradually withdraw from any focus on operational issues, on the
basis that, as Advocacy Officers develop enhanced working relationships,
they can be trusted to get on with it. The increasing risk otherwise would be

page 25
 EVALUATION REPORT

that the Steering Group makes decisions on points that have already made their way
into daily practice.

As it is identified who can do what, where expertise resides, there can be more
bilateral communication, with less having to go through the coordination hub.

13. governance and accountability

Although the Steering Group is established to “govern the running of the Network”4,
one member of the Project Accountability Group described it as “a forum for
consultation”.

And there was a view expressed too from within Project Accountability Group, as the
accountable body (to the Gates Foundation), it is for ActionAid to take a strongly
controlling view of the network: “it is important that partners are part of the process
and that there is a certain amount of consensus, but in the end we need to get the
message out”. These comments should not be taken as representing the collective
view of PAG members. As emphasised by another PAG member, a successful
network will only be created if members‟ authority within the network is recognised
and respected. But they perhaps highlight a discrepancy worth exploring further.

In any case, these comments represent only a narrow interpretation of the extent of
the network‟s accountability to others. Whilst lines of accountability seem to be
clearly manifesting themselves through ActionAid International, and thus to the
donor, they are less obvious in the other direction. Accountability to members in
networks is generally maintained through a funding relationship: network
Secretariats tend to be part-funded at least through members‟ contributions; in this
case, as funding flows the other way, members‟ recourse is somewhat more limited,
to withdrawal (with its accompanying costs).

Another line of accountability runs towards partners and people in the South, the
network‟s perceived beneficiaries. Current relationships with groups and people in
the Third World appear very tentative and indirect: stronger in some organisations,
weaker in others, but in all cases not explicitly addressed through AfGH processes.
The extent to which the Network speaks or can speak on behalf of partners in the
South is therefore left vague. The survey results indicate that members recognise
this concern (see part 3) even though the internal sense seems to be that this is not
a key priority to address.

Our own view is that one key source of the Network legitimacy is its links with
Southern, developing world, partners. Drawing on other (i.e. Southern) perspectives
and experiences is, for a network such AfGH, important:
in terms of moral consistency: given that members make the case to decision
makers that they should be listening to grassroots voices;
for effectiveness: it is easy for officials to ask, who do you represent?
to help ensure message consistency: if Southern campaigners have different
messages to those disseminated through Northern networks, then this can be
extraordinarily damaging; and
for credibility: to help ensure that policy proposals offer credible solutions to
practical, on the ground, health and development issues.

4
Steering Group ToR, January 2007

page 26
 EVALUATION REPORT

Engaging with civil society groups in developing countries is about coordinating

advocacy activities as well as informing AfGH's own positions. The network should
be connected to stakeholders in the South to enable information sharing, including
sharing of plans and political information – in other words providing access to
information that Southern groups often find very difficult to obtain. As one external
put it, “the network needs antennae, if not legs, in the developing world and also a
part of its brain should be there”.

The need for enhanced links also reflects the fact that, “the problem is not just with
the donors, but with the recipient [governments], which need [influencing] by CSOs
directly”.

We believe, therefore, that AfGH should look at ways of forging of links with
Southern regional counterparts and/or a caucus of civil society informants
by building on members’ existing contacts and collaborations. As part of
this, AfGH, could consider what kind of access partners in developing
countries can have to the network (e.g. as a second circle of members? as
observers?)

14. internal communications

Information exchange is at the core of any network. As already noted, comments

from interviewees to this review sometimes demonstrated a lack of appreciation of
others‟ working contexts. There was even a lack of awareness of what others were
calling for in reports: as one put it, “I‟m not sure if common positions are being
advocated across Europe”. In some countries, joint working structures (between
national partners) are very well developed; others have found it more difficult to set
these up. To some extent, it seems that the pressure to deliver activities on time
has precluded the development of solid internal communications.

This should not be overstated. Survey findings indicate relatively high levels of
satisfaction with internal information flows. But some of the mechanisms do not
seem to be working as well as they ideally would. For example the teleconferences
are not a useful tool for some (because of language issues), especially given the
point already made regarding insufficient preparation time. Long face-to-face
meetings have a tendency to sap energy: suggesting the need to find different ways
to interact. Face-to-face meetings are important but perhaps different formats
should be adopted, involving a variety of appropriate methods of work. In particular,
there is a need to expand the network‟s multi-lingual facility: language barriers
exacerbate the likelihood of UK partners (and of others with English as a first
language) dominating the discussions and proceedings.

It is important that barriers to participation are addressed; and some reorientation is

required to ensure this. Successful networks ensure that those with least resources
and power have the opportunity to participate in a meaningful way and also that
there is willing input of resources by members, particularly those most advanced, for
benefit of all.

page 27
 EVALUATION REPORT

15. working groups

The Working Groups seem not to have functioned fully to date. There is a need to
clarify their purpose and content, as well as their nature and relation to the overall
themes (horizontal or transversal). They are regarded as potentially a good tool,
even though they were not very active in year 1. It will be important to give more
strength to them in year 2.

For one partner, there is an opportunity for the Working Groups to “play an
increasingly strategic role in how the network functions”. Working Groups could be a
primary means to demonstrate that AfGH “is a supportive structure to bring people
together”, for example, acting as a policy interface with other (non-member) groups.

16. decision making

Network decision-making processes work best when there are low levels of formal
controls with high levels of coordination and facilitation. You cannot consult
everyone on everything all the time so it is vital there is confidence in others. Some
formal aspects between members are no doubt needed in this case, given the
contractual relationship that ActionAid has with the Gates Foundation, but ideally
these would not have to be invoked during normal interactions between members.

As noted elsewhere, there are frustrations about the perceived slowness of decision-
making but the main issue seems to be more about lack of preparation time, with
the result that people don‟t feel equipped to take the right decisions. One
consequence of this is that decisions are made but not always followed through.

Given the organisational issues involved in developing the work of the various
decision-making groups, a balance will have to be found between face-to-face work
and meetings and virtual meetings and work through e-mail and internet exchanges.
This will require the development, at the outset, of clear guidance on which aspects
of reflection, creativity, strategy, and implementation will need to be discussed
through which format.

Moving forward in a strategically coherent fashion requires that members have full
confidence in decision-making mechanisms. To ensure this, the Network should
review planning and decision making processes and establish procedures
that include:

laying out consultation guidelines for national convenors,

building in time so that all can contribute equally, and
adopting measures to ensure the network’s focus is preserved and
applied.

As part of this, we suggest that the Network should conduct an assessment of

the contribution that member organisations and staff feel they can make to
the Network, as well as their perceived needs from others. This information
should form the basis for developing ways that Advocacy Officers and
others can deliver more focussed and targeted contributions.

On policy content, for example, there was a feeling that the Advocacy Officers could
be better supported by specialists within the Network, rather than left to deal on

page 28
 EVALUATION REPORT

their own with content-specific issues. Overall, “it is [still] not clear who should do
what... we could have a more structured division of tasks”. Again, to achieve a
better balance, members will need to know others‟ strengths and weaknesses.

One concern raised in relation to this is that there could be “over-reliance on those
members of the network who are performing well … [which] may result in some
national partners being unable to conduct the policy and advocacy work that is
required of them”. Clearly this concern, amongst others, would need to be taken
into account in identifying who can best support whom, when.

Whatever the systems and processes employed, it is ultimately a question of having

people who are committed, empathetic and flexible.

17. identity

Anecdotal evidence suggests that the network is becoming better known

progressively, but that recognition is still at a low level. This is to be expected at
this stage in its lifetime. Internally, as indicated in survey results, there is an overall
sense that that a strong identity/brand has been created. Certainly, the fact that the
network has been so proactive in working internationally to create a distinctive
identity has helped create a potentially valuable platform for future joint working,
especially in tandem with the fact that effort has been invested in establishing joint
messages and joint positions. Network branding seems to be positively regarded
internally, notwithstanding one or two complaints about the cost.

However, the identity of the Network and the use of the Network as an entity in its
own right vary from country to country. In France for example, AfGH e-mail
signatures and links to the AfGH website show that members promote the Network in
their day-to-day work. In other cases, for example in Italy, much more reliance is
placed on individual members taking forward AfGH ideas, with separate AfGH
branding not (yet) or less in vogue.

Some feel difficulties in separating out when they are speaking for their own
organisation, when for AfGH and when for AfGH in its national configuration.
However, the confusions to date are within the bounds of what would be expected in
a new network, and it appears that the parameters are becoming clearer. The best
way to resolve these kinds of questions is through developing members‟ sense of
association with the network, in the ways touched on earlier: through clarifying and
expanding the space for overlap between members‟ and network aims.

Alongside the issue of whether and when members can speak for the Network rather
than their own organisation, the issue of whether and when members should speak
for the Network was touched on, but without an obvious sense that this was a major
concern. The case was made that for some of the bigger organisations involved, it
does not necessarily make sense always to highlight that they are speaking on behalf
of the network, as they are well known themselves within the arenas in which they
are operating.

It may be right in some circumstances that a pragmatic approach is adopted, with

groups with national profile sometimes at the forefront of messages. Over time,
however, it will be important to see a growing use of the Network‟s brand by

page 29
 EVALUATION REPORT

members. Partner organisations should be able and willing to talk for the whole
AfGH network and not only on their own behalf.

The Network should seek to agree a set of common principles around how
members promote their membership of the Network, allowing for flexibility
of implementation according to national and organisational contexts.

18. network membership

Whilst we respect opinions (expressed in Berlin for example) that network growth is
not a priority issue for consideration now, we did gain the impression from (internal)
interviews there is a divergence of opinion on this issue as well as an unhelpful lack
of clarity about how the network might grow.

In general, it seems to us that at this stage the Network is only partially ready to
meet the intention expressed in the proposal to the Gates Foundation, namely:
“While the core of the network will consist of the funded partners, their early remit
will be to act as a resource for other organisations in the same country/location”.

Various dimensions of membership expansion need further consideration:

timing

We heard the view that the network has yet to focus on a genuine shared agenda
and that this is a precondition for further expansion. Hence the opinion of one
partner that “expansion of the network should not take place until the third year and
only when [we] have a clear agenda”.

But although there is clearly some merit to the argument that internal processes
must be sorted out first, it doesn‟t seem to be as simple as that. The case was also -
rightly - made that the network now needs to “build on this first year by pushing to
see [its] agenda taken up in other fora and by other networks”. To achieve this,
there is need for “a clear strategy about how [AfGH] relate[s] to the wider NGO
community”. But this cannot really be made clear unless there is clarity around what
kind of engagement is wanted from potential allies: are they joining something?
signing up to something? announcing themselves as supporters in some way?

In any case, we understand that there is already some dialogue with potential
members. It sounds like time will be needed to establish a clear common position on
expanding membership. The sooner this debate starts, the better.

is there a need for growth?

organisationally?
In the view of externals, the network would be stronger if there was involvement of
some of the leading European NGOs; and at national levels too, some NGOs were
cited as obviously “missing”. Internal opinion on this seemed to vary both between
and within national contexts, although respondents to the survey tend to support the
statement that “the logic of existing membership is not clear”. And given the
organisations were fairly randomly assembled within the network in the first place, it
seems likely that there would be strategic benefit in bringing others in.

page 30
 EVALUATION REPORT

geographically?
As one internal interviewee put it, “it is a bit silly to be called a European network
with presence only in five member states and Brussels”. So, for some, growth is
imagined in terms of number of countries first, rather than within the countries with
existing partners. This is not just a question of increasing representation across the
EU, but also of developing relations, through newly acquired partners, with allies in
certain „progressive‟ states (such as the Netherlands, Sweden). Ruling out future
Network membership from such countries, because their governments are not
strategic targets for AfGH, would not make the best use of experiences that are
available from within these states. The current Network could (potentially) learn
much from and be strengthened by advocacy experiences of allies in these countries.

the offer

It will be important that members understand the distinction between “working with
others” and “bringing others on board” – the latter implies that others sign up to
pre-determined agendas, the former that they have some involvement in shaping
the agenda.

More importantly, there are obvious sensitivities around expanding the network
without the availability of funding. The network has been established on a somewhat
artificial basis. Networks are generally sustained by a sense of ownership: members
work within a network, not for it. In the case of AfGH, that dynamic will be slightly
different given that the attraction of participation is at least partly financial. To
introduce into this context a different tier of (non-funded) members is potentially
problematic. This requires very delicate handling, in the „recruitment‟ stage but also
in terms of what kind of functional arrangements would be required within a two-
tier-funded network of this type.

It is also worth noting that there is some scepticism around Gates-funded networks,
including amongst decision makers who are asking what Gates‟ “agenda” is, and
even amongst some potential members, one of whom expressed themselves
“distrustful” of the Gates Foundation (and by implication of Gates Foundation
supported work).

AfGH should set out a process and timetable for resolving questions of
membership expansion, including:

what added value would the network want to gain from a new member?
What benefit would the new member get when joining AfGH network?
What possible obstacles will need to be faced – both by the network and
the new member?
what added value can the network offer candidate members

As part of this process, it would help if the Network could look at membership issues
from a broad perspective. This would ideally include a review at some point of the
appropriateness of existing funding arrangements and assessment of whether
additional sources of funding might be available to the Network.

page 31
 EVALUATION REPORT

All this should be considered in the context of some early outlines of how the
Network sees itself developing over the next five years, and beyond the project
funding period too.

19. general conclusion

The Action for Global Health Network has, during its first year, established a good
basis for itself. It is seen as credible and its concerns are felt to be timely.

In developing its work in the coming year priority should be given to improving
internal communications and decision making processes and to developing coherent
strategies focussed on the Network‟s main concern and main policy targets.

page 32
 EVALUATION REPORT

part three: survey findings

Introduction

The survey was sent back by 22 respondents, a response rate of 63%. Among those,
11 are Advocacy Officers (AO), 9 are managers and 2 are members of the network
coordination team.
6 network partners have sent two contributions (i.e. one each from the Advocacy
Officer and manager) : GHA, MDM, International HIV/AIDS Alliance, Terre des
Hommes Germany, CESTAS and AIDOS.
5 have sent one contribution (AO or Manager): FPFE, Deutsche Welthungerhilfe,
Medicus Del Mundo, EPHA, Interact Worldwide.
Five Action Aid respondents send back the survey, including 2 network
coordination staff, and 1 PAG member.

A significant finding of the survey is the high variability of responses, demonstrated

by increased “standard deviation”. The following table is an example of how
responses are widely spread for some questions. [The scale identifies a range from
strong agreement with the first statement of the bipolar question (-3) to strong
agreement with the second statement (+3)]

-3 -2 -1 0 1 2 3
members' participation
high levels of
in the network is
participation are
22 1 5 3 2 7 3 1 limited by lack of
occurring at all levels
commitment and/or
in the network
other problems

An important variability factor is difference of opinion between Advocacy

Officers and the coordination team in one group, and managers in a second
group. The analysis shows that opinions vary on many points between the
two groups. Even more striking is that these two groups are frequently (6
times out of 22 questions) on opposite sides in their response to bipolar
questions (such as the one above).

In the following section, we indicate the average response for each bipolar
question and specify when the difference is important between the two
groups of respondents. Difference of opinions may well be only a reflection of
one group being involved on a daily basis in the network life, while the other
is less involved. Nevertheless we consider it important for the network‟s
future that these differences are acknowledged and discussed.

Overall, the results of the survey confirm and strengthen the information collected
during the interviews. We also suggest that a „barometer‟, such as the one designed
on this occasion, could play a constructive role in future evaluation work carried out
by the AfGH network.

page 33
 EVALUATION REPORT

State of the AfGH Network

The network does not yet have strongly established strengths. Only
three questions (out of 22) got an average response of less than –1.

the network’s structure is the network‟s structure is

appropriate given its unsuitable for delivering its
purpose -3 -1.32 0 purpose
+3

internal information is
quality of internal
inadequate or is badly
information flows is high
-3 -1.24 0 packaged
+3

the network adds value to

the network is no more than
the work of individual
its parts
members -3 -1.09
+3

In other words, in the opinion of respondents, the three main strengths of

the network are its structure, the internal information flow, and adding value
to the work of individual members. According to the literature on networks,
“adding value” is a key attribute of the “network advantage”. It is therefore a
good sign that this element is positively highlighted by the AfGH network
barometer at this early stage in its development.

Managers and Advocacy Officers have responded quite differently to one

question out of three in this group. Managers consider more strongly that the
network‟s structure is appropriate given its purpose.

AO +
TOTAL Managers
Coordination
The network's structure is appropriate given its
purpose / the network's structure is unsuitable for -1.32 -1 -1.78
delivering its purpose

page 34
 EVALUATION REPORT

Six bipolar questions got an average response between –0.33 and –0.91.
Given that the network is relatively new, the average obtained for any of
these six questions is not particularly worrying: there is time and opportunity
to improve on these aspects.

the network depends on the

the network has a strong
identity/brand of individual
identity/brand -3 -0.91
+3
members

Planning is focused and

Plans are poor and/or remain
agreed actions are
-3 -0.76 unimplemented
implemented +3

network relationships are network relationships are

cooperative -3 -0.57 competitive
+3

clear & appropriate

agreements do not exist or
working agreements are in
are unclear or inappropriate
place -3 -0.41
+3

the network demonstrates

lessons learnt are not
a strong commitment to
identified and applied
learning -3 -0.38
+3

the network acts

members‟ advocacy takes
simultaneously in different
place in isolation
arenas -3 -0.33
+3

Managers and advocacy officers have responded quite differently for one
question out of the six in this group. Managers and AOs do not have the
same perception on the quality of the working agreements in place. This
needs clarification.

AO +
TOTAL Managers
Coordination
Clear & appropriate working agreements are in
place / agreements do not exist or are unclear or -0.41 0.08 -1.11
inappropriate

page 35
 EVALUATION REPORT

Six bipolar questions got an average response between –0.05 and –0.23. On the
issues covered by these questions, respondents overall see the network in a sort of
middle ground, with plenty of room for improvement.

the network facilitates the network has not improved

creation of new knowledge knowledge on the issues
-3 -0.23
+3

members consider network there is little overlap between

goals their own network and members‟ goals
-3 -0.18
+3
the balance between strong decision making is either too top-
leadership and participative down or unclear
decision making is good -3 -0.14
+3

levels of trust between levels of trust between members

members are high are low
-3 -0.14
+3

members share with and limited focus on joint capacity

learn from others building
-3 -0.10
+3

connections with important

the network is a key player
stakeholders are not established
-3 -0.05
+3

According to the literature on networks, “trust” and “building capacity” are

key attributes in taking full advantage of this organisational model. Given the
current scores, it seems that there is a priority for the network to build trust
between its members and to find ways to increase sharing and learning.

Managers and AOs have responded quite differently to two questions out of
the six in this group. AOs are more convinced than managers that the
network is already a key player and are more inclined to find the leadership
balance good:

AO +
TOTAL Managers
Coordination
the network is a key player / connections with
-0.05 -0.38 0.5
important stakeholders are not established
the balance between strong leadership and participative
decision making is good / decision making is either too -0.14 -0.23 0.67
top-down or unclear

page 36
 EVALUATION REPORT

Finally, 7 bipolar questions got nil or positive average responses. Those elements are
not necessarily the main current weaknesses (or risks) of the network but the issues
raised need to be addressed.

high levels of participation are members’ participation in the

occurring at all levels in the network is limited by lack of
network -3 0 commitment and/or other
+3
problems
the network’s and individual
the network’s and individual
members’ resources (staff
members’ resources (staff
and finance) are adequate -3 0
and finance) are insufficient
given the task of AfGH +3
given the task of AfGH

The network focuses on the the network lacks focus on

essential what is important
-3 0.05
+3

coordination is focused on coordination is too

leadership, quality and innovation -3 0.14 operational
+3

the work programme is

the right strategies are in place
-3 0.2 mostly reactive
+3

there is a clear rationale for the logic of existing

network membership membership is not clear
-3
0.55 +3

the network has strong links with,

network accountability to
and accountability to,
-3 0.68 beneficiaries is low
beneficiaries
+3

Managers and AOs have responded quite differently to three questions out of
seven in this group. Particularly important is that participation is perceived so
differently between the two groups.
AO +
TOTAL Managers
Coordination
high levels of participation are occurring at all levels in the
network / members' participation in the network is limited by 0 0.38 -0.56
lack of commitment and/or other problems
the network's and individual members' resources (staff and finance) are
adequate given the task of AfGH / the network's and individual members' 0 0.38 -0.63
resources (staff and finance) are insufficient given the task of AfGH
the right strategies are in place / the work programme is
0.2 0.38 -0.14
mostly reactive

page 37
 EVALUATION REPORT

Internal Communication Flow

The validity of the information matrix shown below is somewhat diminished

by the fact that not everyone involved in the network responded to the
survey. The matrix shows that the coordination team is clearly the
centre of the network and that no other node exists. Some organisations
appear to be at the edge of the network in terms of communication flows,
while others (e.g. Stop Aids Alliance) have regular interactions with a
significant number of network partners.

You have had You have had

You have had You have had
no or regular and
only a few some
virtually no significant Respondents
interactions interaction No
BASE interaction interaction Average
with this with this response
with this with this response
partner partner
partner partner
(3) (2)
(4) (1)
22 0 2 4 13 3 19
the Coordination Team
3.58
22 6 5 6 3 2 20
Marie Stopes International
2.3
The European Public Health 22 10 4 3 1 4 18
Alliance 1.72
22 5 4 3 6 4 18
Stop Aids Alliance
2.56
22 6 5 4 3 4 18
Deutche Welthungerhilfe
2.22
22 6 6 4 3 3 19
Terre des Hommes
2.21
Federación de Planificación 22 9 3 6 2 2 20
Familiar Estatal 2.05
22 9 3 5 3 2 20
Medicos del Mundo
2.1
22 8 5 1 3 5 17
Global Health Advocates
1.94
22 8 5 1 3 5 17
Medecins du Monde
1.94
22 6 5 5 3 3 19
CESTAS
2.26
Associazione Italiana Donne 22 8 4 4 3 3 19
per lo Sviluppo 2.11
22 9 3 6 3 1 21
TB Alert
2.14
The International HIV/AIDS 22 7 5 5 3 2 20
Alliance 2.2
22 6 5 6 2 3 19
Interact Worldwide
2.21
ActionAid International 22 7 2 5 4 4 18
national office 2.33

page 38
 EVALUATION REPORT

General Appreciation of the Network

Three questions were asked to respondents with regard to their general

appreciation of the network. Some respondents stressed that their current
responses had nothing to do with their expectations for the future.

For these questions, in addition to separating the respondents between

Advocacy Officers + Coordination Team, and managers, we also separated
the respondents between Action Aid (including network coordination) and
others. Given the size of the Action Aid group (5 respondents), the analysis is
statistically of little value. However, results are shown here because they
may indicate differences worth exploring further.

Scale from 1 to 9, with 1 Other

AO +
very low, 5 average and 9 TOTAL Action Aid network Managers
Coord
very high partners
What is your overall 22 5 17 13 9
appreciation of the network
5.32 6.4 5 5.46 5.11
effectiveness?
How important is it for 21 4 17 13 8
your organization to be
6.43 6.75 6.35 6.15 6.88
part of this network?
Does your organisation 21 4 17 13 8
benefit from being part of
5.67 7.75 5.18 5.85 5.38
the network?

Respondents overall are not overly enthusiastic about the current

effectiveness, which does not come as a surprise given the responses to the
bipolar questions. The level of importance (6.43) is a good sign given the
network age. Importance is currently higher for smaller organizations. It will
be interesting to see if importance grows in larger organizations in the
coming years.

One possibly significant feature to note is that, whilst overall the picture that
emerges is generally positive, around a third of respondents tend to have a
much more negative view about the Network and its key aspects:

mode (response given average number of

more frequently than respondents scoring
any other) 4 or under
effectiveness 6 5.32 4
importance 9 6.43 6
benefit 9 5.67 8

Those who score these questions low (compared to others) tend also to score
the bi-polar questions relatively high (compared to others).

page 39
 EVALUATION REPORT

appendix: evaluation purpose & methodology

In April 2007, Action for Global Health (AfGH) asked for ideas and suggestions for an
external evaluation of the first year of this five year networking and advocacy
project. The request was that the evaluation should consider:
the development of the Network in Year 1 against the stated objectives;
the functioning of the Network;
plans and delivery in each country;
the impact on decision-makers (which, it was recognised, would be limited in
Year 1); and
recommendations for ways of improving the Network‟s operation. 5

The authors of this report submitted ideas for such an external evaluation in May
2007. More detailed implementation plans were designed and agreed with the AfGH
Evaluation Working Group in July 2007. We identified that we would give particular
attention to analyses of findings, and make recommendations, relating to:
functioning, planning and delivery at national levels;
functioning, planning and delivery at cross-Network levels; and
the functioning of the Coordination team.

To fulfil our task, we have gathered information through the following methods:

a review of documentation, including plans and reports, made available to us by

AfGH through its extranet;
telephone interviews with 29 internal staff and others involved in the operation
and management of the Network, (Advocacy Officers, national convenors, project
manager and coordination staff, and Project Accountability Group members);
telephone interviews with 15 external informants from across the various
countries involved in the network, generally those sympathetic to the network‟s
aims, including parliamentarians, officials and staff from other (non-AfGH)
organisations;
a review of best practice within advocacy networks, through a literature review
(which disproportionately draws on UK and US sources) and discussion with a
small number of informants involved in other (advocacy) networks operating at a
European level; and
a questionnaire survey circulated to 36 internal staff and others involved in the
Network (see part 3 of this report).

Information gained from available peer review reports, developed by AfGH members,
has been drawn on as an additional source of information. We also benefited from
the opportunity to present some very preliminary findings at the Advocacy Officers‟
meeting in Berlin in October.

The significant amount of evidence gathered was initially collated in a series of „mini-
reports‟: these reflected views on the network from different perspectives (from the
various „national arenas‟ (including Brussels), from the Coordination Team, from the
Project Accountability Group, etc.) Further collation of all available information and
further analysis of findings then took place leading to the production of this report.

5
‘Action for Global Health: monitoring and evaluation strategy/tendering’, April 2007

page 40
 EVALUATION REPORT

bibliography

Church et al 2003. Participation, Relationships and Dynamic Change: New Thinking

on Evaluating the Work of International Networks. Development Planning Unit,
University College London
Creech, H. Ramji, A. 2004. Knowledge Networks: Guidelines for Assessments. IISD
Creech, H. Willard, T. 2001. Strategic Intentions. Managing knowledge networks for
sustainable development. IISD
Eade, D. 1997. Capacity-Building: An approach to people-centred development.
Oxford
Keck, M. & Sekkink, K. 1998. Activists Beyond Borders: Advocacy Networks in
International Politics. Cornell University Press
Kelly, L. 2002. International Advocacy: Measuring performance and effectiveness
Perkin, E. & Court, J. 2005. Networks and Policy Processes in International
Development: a literature review. Overseas Development Institute, Working Paper #
252
Shimmin, S. & Coles, G. 2007. Campaigning in Collaboration. NCVO
Taylor-Powell, E. Rossing, B. Evaluating Collaborations: Challenges and Methods
van Zee and Engel 2004. Networking for Learning. European Centre for Development
Policy and Management
van Tuijl, P. & Jordan L. 1999. Political Responsibility in Transnational NGO
Advocacy. Bank Information Center, Washington DC
Wilson-Grau, R. Nunez, M. 2006. Evaluating International Social Change Networks: A
Conceptual Framework for a Participatory Approach

page 41