Impact Assessment and Project Appraisal

ISSN: 1461-5517 (Print) 1471-5465 (Online) Journal homepage: www.tandfonline.com/journals/tiap20

The adoption of the logical framework in

international development projects: a survey of
non-governmental organizations

Ruggero Golini, Paolo Landoni & Matteo Kalchschmidt

To cite this article: Ruggero Golini, Paolo Landoni & Matteo Kalchschmidt (2018) The
adoption of the logical framework in international development projects: a survey of non-
governmental organizations, Impact Assessment and Project Appraisal, 36:2, 145-154, DOI:
10.1080/14615517.2017.1354643

To link to this article: https://doi.org/10.1080/14615517.2017.1354643

Published online: 28 Jul 2017.

Submit your article to this journal

Article views: 4207

View related articles

View Crossmark data

Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=tiap20
Impact Assessment and Project Appraisal, 2018
VOL. 36, NO. 2, 145–154
https://doi.org/10.1080/14615517.2017.1354643

The adoption of the logical framework in international development projects:

a survey of non-governmental organizations
Ruggero Golinia , Paolo Landonib and Matteo Kalchschmidta
a
Department of Management, Information and Production Engineering, Università degli Studi di Bergamo, Dalmine, Italy; bDepartment of
Management and Industrial Engineering – Politecnico di Torino – Corso Duca degli Abruzzi, Torino, Italy

ABSTRACT ARTICLE HISTORY

Non-governmental organizations (NGOs) play a fundamental role in International Development Received 30 January 2017
but their projects often lack efficiency and effectiveness. A key aspect is the proper use of project Accepted 31 May 2017
management tools and methodologies. The Logical Framework (LF) has been suggested as a KEYWORDS
key tool in this sector. However, the extent and impact of its use by NGOs is still not known. Logical Framework;
Based on almost 500 questionnaires collected from project managers working on International international development;
Development (ID) projects in NGOs around the world, we show that the LF is broadly adopted, NGO; impact assessment
especially for large projects and ‘soft’ projects. Moreover, we demonstrate that when used in the
various phases of a project’s life-cycle, the LF contributes significantly to the achievement of
superior project impact.

1. Introduction namely the Logical Framework (LF). The LF is a matrix

Compared to emergency projects, which provide
immediate assistance to populations hit by wars or nat-
ural disasters (Durning 2014; Gore and Fischer 2014),
International Development (ID) projects aim to long-term
improvements for the beneficiaries in terms of health,
education or living conditions in general. ID projects
attract relevant flows of public funds: in 2013 about $
135 billion have been poured into international develop-
ment by DAC member countries.1 However, also private
funding, even if difficult to be estimated has a relevant
share. For instance, in 2010, private funding for human-
itarian assistance was estimated to be about one-third
of the total ($ 5.8 billion on a total 18.22). Nevertheless,
empirical and anecdotal evidence (e.g. TED Talks3) and
recent researches (Lovegrove et al. 2011; Zimmermann
and Smith 2011; Ika and Saint-Macary 2012) show that
the success rate and long-term impact of such projects is
often below the expectations. This glitch is often associ-
ated to a lack of a professional managerial approach and
understanding of the context (Ahsan and Gunawan 2010;
Ika 2012), especially in NGOs (Newcomer et al. 2013). As a
consequence, several authors have called for additional
research (Youker 1999; Golini and Landoni 2014), in par-
ticular regarding project appraisal tools and approaches
able to adapt the best practices in project management
with the complex context these projects have to deal
with (Ebrahem 2003; Khang and Moe 2008).
In response to this call, in this paper we focus on one
of the most specific tools developed for ID projects,
(normally a 4 × 4 matrix) that summarizes the project’s
goals, activities, assumptions, indicators, and sources of
verification in order to measure and report the achieve-
ment of objectives (Coleman 1987; Gasper 1997; Akroyd
1999). An example of the LF is reported in Figure 1.
Despite its importance and widespread use by gov-
ernmental organizations (Landoni and Corti 2011), its
level of adoption by NGOs is still unknown. Moreover,
as we will explain in the literature review, this tool has
been subject to several criticisms, so that its actual effec-
tiveness should be further investigated. Accordingly, on
the basis of a large-scale survey, the goal of this paper
is to shed light on the following aspects: the extent to
which the Logical Framework is adopted by NGOs; the
factors that induce its adoption; and its impact on project
performance.
The rest of the paper is structured as follows. First, we
provide a review of the literature on the LF, highlighting
the gaps and thus formulating our research questions.
Next, we illustrate the methodology used to answer
them, and then set out and discuss the results. Finally,
we draw the main conclusions and highlight the princi-
pal limitations and future developments of this research.
2. Literature review
ID projects have specific characteristics such as the pres-
ence of a large number of different stakeholders in terms
of interests, power and culture (Youker 1999; Saad et al.
2002; Diallo and Thuillier 2005; Zhai et al. 2009; Steinfort

CONTACT Ruggero Golini ruggero.golini@unibg.it

© 2017 IAIA
146  R. GOLINI ET AL.
Project Description Indicators Sources of Verifications
Overall Objectives
Purpose
Results
Activities
Figure 1. Example of logical framework.
2010) that cope generally with instable environments
(Youker 1999; Diallo and Thuillier 2004; McCarthy and
Zakrajšek 2007; Khang and Moe 2008; Zhai et al. 2009; Ika
et al. 2012) and with the presence of intangible project
outputs (e.g. improvement in living conditions or educa-
tion) (Khang and Moe 2008; Ahsan and Gunawan 2010;
Steinfort 2010; Ika et al. 2012) making these projects
particularly challenging to be managed and evaluated,
especially in their social impact (George 2001; Esteves et
al. 2012; Golini et al. 2017).
In 1970, Baum introduced a specific approach for ID
projects based on the ‘project cycle’, and specific tools
(e.g. the Logical Framework) were developed in the
following years, see e.g. Coleman (1987). In recent dec-
ades, the project cycle approach has been adopted by
development agencies because it offers a structured
framework of techniques that enables people to work
together while bringing the project’s objectives into
focus (Ahonen 1999; Landoni and Corti 2011). However,
while the project cycle concept is shared also by busi-
ness-oriented project management guidelines (e.g. the
PMBOK Guide® by the Project Management Institute),
some of the tools, like the Logical Framework, have
remained confined to the international development
field. This has led to the scant integration of standard
project management tools with the Logical Framework
that causes issues in the management of ID projects
(Golini and Landoni 2014). To gain better understand-
ing of these issues, in this paper we focus on the Logical
Framework, its diffusion, and its actual contribution to
project performance.
The origins of the Logical Framework in the context
of ID projects can be traced back to 1969, when two
US-based consulting firms (Fry Associates and Practical
Concepts, Inc.) developed it for the United States Agency
for International Development (USAID) (Solem 1987).
Thereafter, the LF was adopted by many other gov-
ernmental agencies around the world (Landoni and Corti
2011). Today, there are many versions and variations in
the terminology used, but the Logical Framework main-
tains its typical structure of a matrix that breaks a pro-
ject down into its component parts in order to facilitate
management (Cracknell 2000). One of the best-known
versions of the tool is probably the one adopted by the
European Commission.
Preparation of the LF is an iterative process. As new
parts of the matrix are completed, information previously
Assumptions gathered must be reviewed and, if necessary, revised.
After completion of the first column (Hierarchy of
Objectives), the second step consists in identification
of the fourth column of the matrix (Assumptions).
Assumptions are usually identified during the project
planning phases through analysis of stakeholders,
objectives and tasks. When the project description and
assumptions have been drafted, the next step is to iden-
tify indicators that may be used to measure and report on
the achievement of objectives (column 2) and the sources
of information (column 3). The indicators describe the
project’s objectives in operationally measurable terms
(quantity, quality, time).
Use of the LF can provide several advantages in the
various phases of a project by remedying many of the
limitations of other tools (Gasper 1997; Bell 2000). In the
first phase (initiation), it helps to achieve a better problem
analysis and definition of objectives through systematic
analysis of problems and by placing the project within
a broader context. Next, it can be used in the planning
phase to set objectives, establish measurement systems,
and identify and manage risks. Furthermore, it can be
used in the execution and control phase to monitor pro-
gress and manage scope changes and risks. Finally, it can
be used in the concluding phase to check, measure, and
report the project’s achievements. The LF can also pro-
vide several organizational advantages. First, it helps to
define different levels of management responsibility and
gives greater accountability to the project manager for
the risks and the results. Second, it allows better stake-
holder involvement and communication by its simple
and clear format. Given all these advantages, the LF is
extensively adopted not only in governmental agencies,
but also in NGOs (Golini et al. 2015). However, despite its
many advantages, the LF has been also significantly crit-
icized (Gasper 1997; Crawford and Bryce 2003; Couillard
et al. 2009; Ika and Lytvynov 2011). The main limitations
reported by the literature are: oversimplification, deter-
ministic and time-invariant logic, lack of flexibility, lack
of accountability, unclear terminology, and lack of inte-
gration with other project management tools. In particu-
lar, the LF is considered to be sometimes too simple to
represent the complexity of a project, and it may omit
some important elements (such as cultural aspects or
the objectives of all the stakeholders). Secondly, risks or
variability in the variables (either positive or negative) are
not included and the underlying logic is rather determin-
istic. Another issue is the difficulty of defining the objec-
tives: ID projects are usually complex, i.e. many internal
and external variables must be considered. Hence, the
objectives are particularly difficult to identify and formal-
ize. Moreover, given the high number of stakeholders
involved, with their complex relationships and differ-
ent interests, the project’s objectives may be mixed or
conflict with individual objectives. Similarly, changes in
projects may give rise to a difference between ‘official’
and ‘actual’ objectives.

Given the complexity of ID projects, it is likely that,
in the course of a project, several changes will occur in
terms of activities and in terms of the desired outcomes.
The LF appears to be rigid to the objectives and activities
stated at the outset because it lacks a ‘time perspective’.
Furthermore, the LF has been criticized because it does
not always straightforwardly identify a causality effect
among its levels. Moreover, relationships may change
during the project’s execution. Also, unclear terminology
can cause problems: many users may find it difficult to
understand the differences among the various levels (e.g.
purpose, goal, and outcome). Another issue is the lack of
integration with other project management tools, since
PCM and LF do not substitute traditional project man-
agement tools (e.g. work breakdown structure, Gantt
chart). A final limitation is the lack of accountability: the
LF does not show who is responsible for what.
All these potential problems may induce NGOs to use
the LF only when necessary or compile it because it is
mandatory, but not using it as a key project management
tool, thus missing its potential. Because of that, beside
the analysis of how spread the LF is among NGOs, we
aim to understand its extent of adoption as a key tool in
the different project phases. Therefore, the first research
question is:
RQ1: To what extent is the LF adopted as a key project
management tool by the NGOs to manage ID projects?
The LF is considered one of the critical tools needed
to fulfill the project objectives and ensuring long-term
sustainable impact. Still, the simultaneous presence of
pros and cons in the use of the LF, led us thinking that
its adoption can vary according to different situations.
From this point of view, we took a contingent approach
on analyzing the extent of the LF’s adoption. Specifically,
there is no theoretical contribution stating what contex-
tual elements may influence LF adoption, but this could
be very relevant for research and practice. By under-
standing the drivers, in fact, it is easier to identify the
contexts in which the LF is less adopted and which can
be enabling factors. For this reason, we drew mainly on
the literature that has considered contingency theory
in other fields of research (e.g. project management,
operations management, organizational theory) (Powell
1995; Delery and Doty 1996; Sousa and Voss 2001, 2008;
Figure 2. Variables included in this study and their relationship.
IMPACT ASSESSMENT AND PROJECT APPRAISAL  147
Ketokivi and Schroeder 2004; Zotteri and Kalchschmidt
2007; Kalchschmidt 2012), and on this basis we defined
specific variables. Specifically, we considered three main
groups of variables: first NGO objectives, then project
characteristics, and finally the external context.
Therefore, our second research question concerned
the drivers that induce greater adoption of the LF:
RQ2: How do NGO’s objectives, project characteristics,
and contextual factors influence the adoption of the
LF?
Finally, the existence of advantages and disadvan-
tages to the LF raises the question of whether it posi-
tively contributes to a project’s performance. Hence our
last research question was:
RQ3: How does the adoption of the LF affect perfor-
mance (impact) of the project?
The three research questions can be summarized in
the theoretical framework shown in Figure 2. The fig-
ure highlights that the main purposes of our research
were to verify the level of adoption of the LF (RQ1), to
identify what the related drivers were (RQ2) (i.e. context
variables, project variables, and project objectives), and
to determine how the performance indicators were
affected (RQ3).
3. Methodology
To address these questions, we adopted a survey
approach. This approach was consistent with our
research goals since we could leverage on already
defined constructs and we were interested in a widely
diffused methodology. We decided to design a survey
tool to cover project managers working for NGOs in the
development sector throughout the world and thus cre-
ate a global data-set. The questionnaire was designed to
obtain quantitative, well-comparable, and manageable
answers. Besides English, the survey was also translated
into Spanish (using a double parallel translation proce-
dure). Therefore, our results were not restricted to a spe-
cific geographical area that might have specific cultural
and context biases. This improved the generalizability
of the results. To control for the quality of the results,
we selected only respondents with at least two years of
expertise in managing projects.
148  R. GOLINI ET AL.

Project managers operating in NGOs were contacted Table 1. Descriptive NGO statistics.
directly using information found in public databases Tot. Income
(mainly the Directory of Development Organizations4) Continent % N. Employees % (USD) %
and associations of NGOs (e.g. MESA de articulation,5 Africa 26 1–20 54 <1.000 15
America 11 21–50 17 1.000–10.000 6
CONCORD6). It is important to remind that the organi- Asia 20 51–100 10 10.000–100.000 12
zations listed in these databases do not necessarily have Europe 40 101–500 9 100.000–1.000.000 25
Oceania 2 >500 5 >1.000.000 26
ID projects in their portfolio. Because of this, we clearly Missing 1 Missing 5 Missing 16
stated at the beginning of the questionnaire that the Total 100% Total 100% Total 100%
survey was aimed to organizations dealing with ID pro-
jects and, to make it sure, we also introduced a control
question in the survey. Table 2. Descriptive project statistics.
Contacts with NGOs’ networks were very important Average
duration
to increase the size of the sample and to avoid sample Principal aim % (months) % Average size (USD) %
biases. NGOs were contacted by email and phone, when Delivery and 2 0–6 14 <100.000 38
possible, to increase the response rate. Contacts were integration
of tools
searched and managed globally to avoid limiting our Construction 6 7–12 29 100.000–200.000 24
considerations to a specific area or country. The survey of infra-
was administered in 2012 and the data-gathering pro- structure
Development 57 13–18 9 200.000–500.000 17
cess lasted for about four months. Eventually, 496 organ- of services
izations provided information useful for the research. Awareness 22 19–24 18 500.000+ 19
raising
Around 30,000 email and phone contacts were made to Other 12 24+ 24
construct the sample, and a response rate of 1.6% was Missing 1 Missing 6 Missing 2
Total 100% Total 100% Total 100%
obtained. The response rate is quite low, even if aligned
with other works published using the same approach
(e.g. Martinsuo et al. 2006). One of the reasons for the low In the following subsections, we present the meas-
response rate can be also related to the fact that, since the ures used for each group of variables (already shown in
PM discipline is quite new for NGOs, the level of interest Figure 2).
in our study might have been quite low. Given this low
response rate, we contacted about 50 non-respondents
3.1. Logical framework adoption
by telephone, but we could not identify a reason for the
non-responses (e.g. lack of time, lack of interest in the The focal variable in our study was adoption of the LF. To
research, not dealing with ID projects, the NGO secre- evaluate this, we employed two measures related to the
tary did not forward the email to the project managers). extent and depth of use. As a matter of fact, the litera-
To support the validity of our sample, we also checked ture states that not only should the LF be used in every
possible biases due to late respondents and contacts project (extent of adoption) but also that its adoption
established through different channels (e.g. email, tele- should be pervasive throughout the project (i.e. it has to
phone), finding that our sample was not affected by such be a core tool, not just a formal one) (Gasper 1997, 2000;
biases. Further analyses indicated that the low response is Smith 2000; Couillard et al. 2009). The two measures are
mainly since many of the organizations contacted did not theoretically independent of each other because a PM
actually manage ID projects (about half of the contacts). can always employ the LF only because it is mandatory
Table 1 presents the main characteristics of the NGOs (e.g. the tender or the sponsor requires it), but in this case
for which the surveyed PMs worked. its adoption may be formal and superficial.
Table 1 shows that the majority of the NGOs in our As said, the extent of adoption measure was the
sample were European. However, without considering number of times that the LF was employed in projects.
Oceania, a considerable global distribution was achieved, To measure this amount, we asked the percentage of
as envisaged in our initial goals. Most of the NGOs were projects in which they adopted the LF on a 1–5 scale
small-sized organizations, with fewer than 20 employees (where 1 = 0%; 2 = 1–25%; 3 = 26–50%; 4 = 51–75%;
and incomes that grew proportionally with size. 5 = 76–100%). As the difference between 1 and 2 in
Besides information about NGOs, the survey also pro- the scale is different from the other points (3, 4, 5), we
vided data on other descriptive variables concerning the merged 1 and 2, so the final scale is from 1 to 4.
projects directly managed by the interviewees (Table 2). The depth of relates to the how the LF was used and
Most the latter (72%) had managed fewer than 10 pro- its importance in guiding the project. To obtain this
jects in the previous two years. Moreover, 40% of the measure, we used the average of a set of indicators
projects in our sample were of small scale, with a high measured on a 1–5 Likert scale (1: Strongly Disagree – 5:
variance in terms of duration. The principal aim of most Strongly Agree) representing the importance of the LF
of them (57%) was the development of services (educa- in the various phases of a project (Table 3). These indica-
tion, finance, health care, etc.). tors were averaged into a single construct (the statistical

Table 3. Measures for LF adoption (all items are measured on a
1–5 scale).
LF adoption Cronbach’s Std.
measures Items alpha Average dev
LF depth of LF helps in the initial 0.741 3.94 0.852
adoption part of the planning
phase
LF provides guidance for
the entire project
LF is frequently updated
during the project
LF is used in the evalua-
tion phase at the end of
the project
LF is used in the reports
and project meetings
LF extent of Percentage of projects – 3.47 0.055
adoption adopting LFa
a
Scale: 1 = 0–25%; 2 = 26–50%; 3 = 51–75%; 4 = 76–100%.
correctness of the procedure was supported by the very
high Cronbach’s alpha).
3.2. Drivers and performance
In accordance with the literature, the drivers that may
induce a greater or lesser adoption of the LF were
grouped into three categories: context variables, project
variables, and NGO objectives (Table A1 in Appendix).
Context variables were elements that described the con-
ditions in which projects were managed. They included
factors like the size of the NGO, the level of development
of the country in which the NGO operated, and the over-
all variability of the context. Because the PM and his/her
project was the unit of analysis, we included the total
income and size of the NGO among the context charac-
teristics. Project variables referred to the characteristics
of projects, such as size, the number of projects typically
managed by the NGO, the number of active stakeholders,
etc. Finally, NGO objectives concerned the attention of
the NGO to both long-term and short-term objectives.
Next, in line with Golini et al. (2015) a factor analysis
was used to divide the performance indicators consid-
ered into two groups: the first related to time and cost
performance, the second to the strategic project results
(e.g. satisfaction of the local community) (Table A2 in
Appendix). The project managers were asked to assess
these indicators on a 1–5 Likert scale (1 – ‘Very low’, 5 –
‘Very high’).
To answer RQ2 and RQ3, we performed several regres-
sion analyses to consider the extent and depth of adop-
tion of LF and internal (i.e. the NGO’s attitude to focus on
management and operational parameters) and external
project objectives (i.e. the NGO’s attitude to focus on
long-term and strategic impacts). For each regression,
we checked that multicollinearity was not present by
looking at the variance inflation factor (always below
2.75) and the condition indexes (always below 7.24). We
also checked that the residuals were normally distrib-
uted. We also controlled for common method bias. We
first guaranteed the respondents’ anonymity to avoid
IMPACT ASSESSMENT AND PROJECT APPRAISAL  149
the social desirability effect. We then performed the
Harman single factor test: by means of a factor analy-
sis that took account of all the variables included in the
study (LF adoption, performance and drivers), we found
that they did not pertain to a single factor.
4. Results and discussion
4.1. Adoption of the LF as a key PM Tool (RQ1)
Inspection of the statistics in Table 3 provides an imme-
diate answer to our first research question. The LF is a
widely adopted tool, with 93.4% of our interviewees stat-
ing that they used it. Moreover, the level of adoption was
very high, since 60% of the respondents reported using
the LF in 75% or more of the managed projects (extent
of adoption measure) and with a mean of 3.9 out of 5
in terms of depth of use. This analysis shows that the
situation has changed since the year 2000, when Abbasi
and Al-Mharmah (2000) pointed out the low diffusion of
the LF among NGOs and confirms the results of Golini
et al. (2015). A possible explanation for the LF’s wide-
spread use relates to the criticism made by Couillard et
al. (2009), Crawford and Bryce (2003) and Gasper (1997)
concerning its oversimplification and scant integration
with the other project management tools. This criticism
was not really shared by the project managers in our
sample. Using additional questions, we confirmed that
there was relatively little agreement with the criticism
of oversimplification (mean: 2.43 out of 5) and, on the
contrary, the respondents underlined that the LF is quite
easily integrated with other project management tools
(mean: 3.71 out of 5).
4.2. Drivers of LF adoption (RQ2)
Despite the LF’s average high adoption, a substantial
number of PMs still do not use it to any great extent.
Hence, the second research question concerned the fac-
tors that explain a higher adoption of the LF. Three kinds
of factors were considered: NGO objectives, project char-
acteristics, and the context (Table 4). We related these
factors to the LF’s adoption (both extent and depth). In
Table 4, the significant factors, with a p-value <0.05, are
highlighted in gray.
Starting with the context variables, to be noted is that
context variability relates to the depth of use of the LF.
This result seems reasonable, given that in highly vari-
able contexts it is necessary to review and, if necessary,
adapt the project objectives. Hence, the logical frame-
work needs to be used and updated in all the project
phases. Furthermore, the size of the NGO negatively
affects the LF’s extent of adoption. This result seems
rather counterintuitive and requires further investiga-
tion. Regarding project variables, note that the average
size of the project impacts positively on the extent of
adoption. This may be related to the fact that larger
150  R. GOLINI ET AL.

Table 4. Research model (principal aims: 1: delivery and integration of tools, 2: construction of buildings and infrastructure, 3 devel-
opment of services, 4: awareness raising). Income and size were measured on a logarithmic scale.

*Sig. < 0.10; **Sig. < 0.05; ***Sig < 0.01.

projects can have greater requirements – for instance, Table 5. Relationship between LF and performance indicators.
by the donors – also in terms of the tools to be used.
However, the project size does not impact on the depth
of adoption, so that there is the risk of a formal, rather
than substantial, use of the LF, even in larger projects.
Also, to be noted is that NGOs involved in construction
projects (principal aim = 2) use the LF less in terms of
depth. In fact, construction projects have more tangible
and clearer objectives than the others. As the literature
reports, these are ‘hard’ projects (Crawford and Pollack
2004), in contrast to ‘soft’ projects, which require more
careful definition of objectives in relation to the stake-
holders. This may explain the lesser adoption of the LF by
this group. Communication of the results is significantly
related to a greater extent and depth of use of the LF.
In fact, as highlighted in the literature, use of the LF in
the various phases of the project, and especially in the
concluding one, allows better communication with the
project stakeholders. Finally, the focus on internal objec-
tives induces a higher level of depth of adoption, even if
with a relatively low effect (sig. <0.10).
*Sig. < 0.10; **Sig. < 0.05; ***Sig < 0.01.
4.3. Impact of LF on performances (RQ3)
internal objectives, the better the performance achieved
The last research question regards the extent to which
in the corresponding performance area (i.e. time–cost or
LF adoption impacts on project performance. To answer
strategic). What is most important is that the LF’s depth
this question, we performed a regression similar to the
of adoption has an impact on strategic performance,
previous one (Table 5). In this case, we ran two regression
while the extent of adoption on time–cost performance.
models to check the effects on strategic and time–cost
This result has several implications. Firstly, the LF has a
performance. We also left the drivers as control variables
positive or neutral effect on performance, but not a detri-
in the regression to avoid spurious effects in the meas-
mental one. This supports the view that the LF is a robust
ure of the LF (Baron and Kenny 1986). Compared with
and useful tool. Secondly, the adoption of the logical
the previous analysis, we omitted results communication
framework seems to ensure better operational results,
because it was not significant and created some multi-
while only a widespread use throughout the project can
collinearity issues.
lead to superior strategic performance. This highlights
First to be noted is that the control variables are sig-
the importance of providing training to NGOs to show
nificant only for the NGO objectives. As expected, the
how the LF can be a core tool to be used and updated
greater the importance given by the NGO to external or
during the various stages of the project. A result, which

is almost a warning, is that the LF is used by NGOs that
are internally focused (Table 4), but its positive effects
are on strategic performance (Table 5). This means that
NGOs have probably not yet mastered the use of the LF,
so that its adoption does not yield the intended results.
5. Conclusions
Despite their importance and their poor performance in
terms of project management, International Development
projects have received limited attention in the literature
for what concerns methods and tools. Our research,
thanks to a survey of almost 500 PMs around the world,
sheds some light on this topic with a focus on the Logical
Framework. What emerges from the analysis is that the
situation has changed from the one depicted by the LF
diffusion analysis performed by Abbasi and Al-Mharmah
(2000). The LF is now widespread among NGOs (93.4% of
the respondents stated that they have used it), with a very
high level of adoption in terms both extent (i.e. number
of projects in which the LF is used) and depth (i.e. the LF
is used throughout the project). While an evaluation of
the extent of adoption was also presented by Golini et al.
(2015), in this paper we added the depth of adoption that
represents an important and original contribution. This
higher level of adoption may be explained by another
result of our survey: the project managers did not share
the criticisms commonly made of the LF (oversimplifica-
tion and scant integration with other project manage-
ment tools). On the contrary, they considered the LF easy
to integrate with other project management tools. As to
the reasons why the LF is adopted, it is used more in con-
texts characterized by high variability that require review
and adaptation of the plan and the project’s objectives.
Moreover, the LF is associated to a greater willingness to
communicate with the project stakeholders. This means
that organizations willing to communicate better their
results are facilitated by the LF that results in an effec-
tive tool for disclosing the achieved results. Interestingly,
we found that the average size of projects has a positive
impact only on the extent of use of the LF. This may be
because larger projects are subject to higher expectations
by donors, for instance, in terms of the tools to be used.
Instead, the project’s size does not impact on the depth
of adoption, indicating the risk of a formal, rather than
substantial, use of the tool, even in larger projects. Finally,
the analysis shows that LF has a positive or neutral effect
on performance, supporting the view that the LF is a use-
ful tool for NGOs. However, this result also shows that its
impact could be higher, or that other instruments could
be introduced to increase project success. This highlights
the importance of supporting the diffusion of the tool
and knowledge about it, but also the need for further
research on its use and limits.
IMPACT ASSESSMENT AND PROJECT APPRAISAL  151
This work provides also some suggestions to man-
agers in NGOs actively involved in ID projects. First, we
suggest considering properly the purpose for which they
decide to adopt the LF. Besides being a useful tool, in the
case of complex and large projects it may be not effec-
tive as managers may expect since main stakeholders
(i.e. donors) may expect or require more complex and
structured tools. Thus, we suggest to critically evaluate
case by case whether to use or not the tool. Second, in
case LF is considered a proper tool, we suggest adopt-
ing it extensively also for communicating both internally
and externally the project goals and progress. Our results
suggest in fact to managers that an extensive use of LF
can favor coordination and control of the project, with
positive impacts on project’s performance. It is however
important for managers to properly communicate the
benefits of the tool and to train people to its effective
use, in particular to adopt it in conjunction with other
project management systems.
Finally, we wish to highlight some limitations of this
study. Firstly, we collected a global sample but with
limited focus on the differences among practices in
different countries. This comparison was not possible
since the sample did not allow comparison of what
NGOs in different countries are doing. We argue that
this analysis would be beneficial for NGOs willing to
benchmark their operations, and that future research
should address this issue. Secondly, the context varia-
bles considered were derived from other studies and
were not verified in the field. A case-based analysis
would likely be of help in determining whether other
contextual variables are important in this case, thereby
making the theoretical model presented here more
complete and reliable.
Notes
1. 
DAC Members’ Net Official Development Assistance
i n 2 0 1 3 ( h t t p s : / / w w w. o e c d . o r g / d a c / s t a t s /
documentupload/ODA%202013%20Tables%20
and%20Charts%20En.pdf ).
2. 
h ttps://www.globalhumanitarianassistance.org/
wp-content/uploads/2012/04/Private-funding-an-
emerging-trend.pdf.
3. 
Ernesto Sirolli, September 2012, https://www.ted.com/
talks/ernesto_sirolli_want_to_help_someone_shut_
up_and_listen?language=it. David Damberger, Arpil
2011, https://www.ted.com/talks/david_damberger_
what_happens_when_an_ngo_admits_failure.
4. 
https://www.devdir.org.
5. 
https://mesadearticulacion.org/.
6. 
https://www.concordeurope.org/.
Acknowledgments
The authors gratefully acknowledge financial support
received by the PMI Sponsored Research Program.
152  R. GOLINI ET AL.
Disclosure statement
No potential conflict of interest was reported by the authors.
Funding
This work was supported by ID Project Management: a survey
of nongovernmental organizations.
ORCID
Ruggero Golini http://orcid.org/0000-0002-8632-2793
Paolo Landoni http://orcid.org/0000-0001-5079-1101
Matteo Kalchschmidt http://orcid.org/0000-0001-7997-8535
References
Abbasi GY, Al-Mharmah H. 2000. Project management practice
by the public sector in a developing country. Int J Project
Manage. 18:105–109.
Ahonen P. 1999. General considerations and user experiences
of project cycle management in a small donor-transitional
country context. Impact Assess Project Appraisal. 17:97–105.
Ahsan K, Gunawan I. 2010. Analysis of cost and schedule
performance of international development projects. Int J
Project Manage. 28:68–78.
Akroyd D. 1999. Logical framework approach to project
planning, socio-economic analysis and to monitoring and
evaluation services: a smallholder rice project. Impact Assess
Project Appraisal. 17:54–66.
Baron RM, Kenny DA. 1986. The moderator-mediator variable
distinction in social psychological research: conceptual,
strategic, and statistical considerations. J Pers Soc Psychol.
51:1173–1182.
Bell S. 2000. Logical frameworks, Aristotle and soft systems: a
note on the origins, values and uses of logical frameworks,
in reply to Gasper. Publ Adm Dev. 20:29–31.
Coleman G. 1987. Logical framework approach to the
monitoring and evaluation of agricultural and rural
development projects. Project Appraisal. 2:251–259.
Couillard J, Garon S, Riznic J. 2009. The logical framework
approach-millennium. Project Manage J. 40:31–44.
Cracknell BE. 2000. Evaluating development aid: issues,
problems and solutions. Thousands Oaks, CA: Sage.
Crawford L, Pollack J. 2004. Hard and soft projects: a framework
for analysis. Int J Project Manage. 22:645–653.
Crawford P, Bryce P. 2003. Project monitoring and evaluation: a
method for enhancing the efficiency and effectiveness of aid
project implementation. Int J Project Manage. 21:363–373.
Delery JE, Doty DH. 1996. Modes of theorizing in strategic
human resource management: Tests of universalistic,
contingency, and configurational performance predictions.
Acad Manag J. 39:802–835.
Diallo A, Thuillier D. 2004. The success dimensions of
international development projects: the perceptions of
African project coordinators. Int J Project Manage. 22:19–31.
Diallo A, Thuillier D. 2005. The success of international
development projects, trust and communication: an African
perspective. Int J Project Manage. 23:237–252.
Durning B. 2014. Benefits of coupling environmental
assessment and environmental management to aid disaster
risk reduction and management. J Environ Assess Policy
Manage. 16:1450029.
Ebrahim A. 2003. Accountability in practice: mechanisms for
NGOs. World Dev. 31:813–829.
Esteves AM, Franks D, Vanclay F. 2012. Social impact assessment:
the state of the art. Impact Assess Project Appraisal. 30:34–
42.
Gasper D. 2000. Evaluating the ‘logical framework approach’
towards learning oriented development evaluation. Publ
Adm Dev. 20:17–28.
Gasper DR. 1997. Logical frameworks: a critical assessment:
managerial theory, pluralistic practice. ISS Working Papers-
General Series.
George C. 2001. Sustainability appraisal for sustainable
development: integrating everything from jobs to climate
change. Impact Assess Project Appraisal. 19:95–106.
Golini R, Corti B, Landoni P. 2017. More efficient project
execution and evaluation with logical framework and
project cycle management: evidence from international
development projects. Impact Assess Project Appraisal.
35:128–138.
Golini R, Kalchschmidt M, Landoni P. 2015. Adoption of
project management practices: the impact on international
development projects of non-governmental organizations.
Int J Project Manage. 33:650–663.
Golini R, Landoni P. 2014. International development projects
by non-governmental organizations: an evaluation of the
need for specific project management and appraisal tools.
Impact Assess Project Appraisal. 32:121–135.
Gore T, Fischer TB. 2014. Uncovering the factors that can
support and impede post-disaster EIA practice in developing
countries: the case of Aceh Province, Indonesia. Environ
Impact Assess Rev. 44:67–75.
Ika LA. 2012. Project management for development in Africa:
why projects are failing and what can be done about it.
Project Manage J. 43:27–41.
Ika LA, Diallo A, Thuillier D. 2012. Critical success factors for
World Bank projects: an empirical investigation. Int J Project
Manage. 30:105–116.
Ika LA, Lytvynov V. 2011. The “management-per-result”
approach to international development project design.
Project Manage J. 42:87–104.
Ika L, Saint-Macary J. 2012. The project planning myth in
international development. Int J Manage Projects Bus. 5:5–5.
Kalchschmidt M. 2012. Best practices in demand forecasting:
tests of universalistic, contingency and configurational
theories. Int J Prod Econ. 140:782–793.
Ketokivi MA, Schroeder RG. 2004. Strategic, structural
contingency and institutional explanations in the adoption
of innovative manufacturing practices. J Oper Manage.
22:63–89.
Khang DB, Moe TL. 2008. Success criteria and factors for
international development projects: a life cycle based
framework. Project Manage J. 39:72–84.
Landoni P, Corti B. 2011. The management of international
development projects: moving toward a standard approach
or differentiation? Project Manage J. 42:45–61.
Lovegrove N, Gebre B, Lee T, Kumar R. 2011. McKinsey-Devex
survey results: practitioners see need for new approaches
to system-wide reform. McKinsey-Devex. http://www.devex.
com/en/news/mckinsey-devex-survey-results-practitioners-
see/77026
Martinsuo M, Hensman N, Artto K, Kujala J, Jaafari A. 2006.
Project-based management as an organizational innovation:
drivers, changes, and benefits of adopting project-based
management. Project Manage J. 37:87.
McCarthy J, Zakrajšek E. 2007. Inventory dynamics and business
cycles: what has changed? J Money Credit Bank. 39:591–613.
Newcomer K, Baradei LE, Garcia S. 2013. Expectations and
capacity of performance measurement in NGOs in the
development context. Publ Adm Dev. 33:62–79.
 IMPACT ASSESSMENT AND PROJECT APPRAISAL  153

Powell TC. 1995. Total quality management as competitive
advantage: a review and empirical study. Strateg Manag J.
16:15–37.
Saad M, Cicmil S, Greenwood M. 2002. Technology transfer
projects in developing countries–furthering the Project
Management perspectives. Int J Project Manage. 20:617–
625.
Smith P. 2000. A comment on the limitations of the logical
framework method, in reply to Gasper, and to Bell. Publ Adm
Dev. 20:439–441.
Solem RR. 1987. The logical framework approach to project
design, review and evaluation in AID: genesis, impact,
problems, and opportunities, AID. Working Paper No.
99. Washington Center for Development Information &
Evaluation Agency for International Development.
Sousa R, Voss CA. 2001. Quality management: universal or
context dependent? Prod Oper Manage. 10:383–404.
Sousa R, Voss CA. 2008. Contingency research in operations
management practices. J Oper Manage. 26:697–713.
Steinfort P. 2010. Understanding the antecedents of project
management best practice-lessons to be learned from
aid relief projects [PhD]. Melbourne: School of Property,
Construction and Project Management, RMIT University.
Youker R. 1999. Executive point of view: managing international
development projects-Lessons learned. Project Manage J.
30:6–7.
Zhai L, Xin Y, Cheng C. 2009. Understanding the value of project
management from a stakeholder’s perspective: case study
of mega-project management. Project Manage J. 40:99–109.
Zimmermann F, Smith K. 2011. More actors, more money, more
ideas for international development co-operation. J Int Dev.
23:722–738.
Zotteri G, Kalchschmidt M. 2007. Forecasting practices:
empirical evidence and a framework for research. Int J Prod
Econ. 108:84–99.

Appendix

Table A1. Measures for the drivers.

Group Variable Measure
Context variables Context variability 1–5 Likert scale variables measuring the agreement with the following sentences (1: Strongly disagree; 5:
strongly agree):
1.The context of my projects is characterized by a high variability (prices, laws, availability of personnel,
suppliers, …
2.During the execution/implementation phase my projects are significantly changed (due to unexpect-
ed constraints, local needs, etc.)

Correlation of 0.391 (sig. 0.000)

Size of the NGO The NGO’s number of employees
Country development Binary variable identifying NGOs based in developed (0) or developing countries (1)
Project variables Average project The average duration (in month) of the projects managed by the interviewee
duration
Average size of the The average size (in USD) of the projects managed by the interviewee
project
Number of projects The number of projects managed by the interviewee in the last two years
Average number of The average number of stakeholders involved in the managed projects
stakeholders
Principal aim of the Nominal variable identifying the type of the project:
project 1.Delivery and integration of products and tools (e.g. medical, IT, energy equipment)
2.Construction of buildings or infrastructures
3.Development of social, economic, health, and environmental services (education, finance, health-
care, …)
4.Awareness raising and campaigning (human rights, environment, …)
5.Other

Results communication 1–5 Likert scale variable measuring the agreement with the following sentence: At the end of the project
we evaluate and communicate the results to the stakeholders (1: Strongly disagree; 5: strongly agree)
NGO objectives External objectives The NGO’s attitude to focus on long-term and strategic impacts
Likert scale items indicating the importance of the following objectives:
1.Obtain long-term project impact (outcome/goal)
2.Stakeholder/partners involvement
3.Extension of the project’s ownership to the local community
4.Economic sustainability after the end of the project
5.Satisfaction of the local community
Cronbach’s alpha = 0.874
Internal objectives The NGO’s attitude to focus on time and cost. Average of 1–5 Likert scale items indicating the importance
of the following objectives:
1.Comply with the budget
2.Comply with the expected time
Correlation: 0.681 (sig. 0.000)
154  R. GOLINI ET AL.

Table A2. Project performance measures.

Type of performance Performance measures Factor loadings Cronbach’s alpha
Time–cost performance Comply with the budget 0.887 0.772
Comply with the time schedule 0.862
Strategic performance Obtain long-term project impact (outcome/goal) 0.507
Stakeholder/partners involvement 0.616
Extension of the project’s ownership to the local community 0.814 0.752
Economic sustainability after the end of the project 0.645
Satisfaction of the local community 0.739
Minimum eigenvalue: 1.13
Total variance explained: 73%