Journal of Public Health | Vol. 32, No. 1, pp. 14 –15 | doi:10.

1093/pubmed/fdq004

Commentary
Healthy trees make a healthy wood
Douglas G. Manuel1,2,3,4,5, Jeffrey C. Kwong4,5,6,7
1
The Clinical Epidemiology Program, Ottawa Hospital Research Institute, Room 1-008 Administrative Services Building, 1053 Carling Ave., Ottawa, ONT, Canada K1Y 4E9
2
Statistics Canada, Ottawa, ONT, Canada
3
The Departments of Family Medicine and Epidemiology and Community Medicine, University of Ottawa, Ottawa, Canada
4
The Institute for Clinical Evaluative Sciences, Toronto, ONT, Canada
5
Dalla Lana School of Public Health, University of Toronto, Toronto, ONT, Canada
6
The Department of Family and Community Medicine, University of Toronto, Toronto, ONT, Canada
7
Ontario Agency for Health Protection and Promotion, Toronto, ONT, Canada
Address correspondence to Douglas G. Manuel, E-mail: dmanuel@ohri.ca

Downloaded from http://jpubhealth.oxfordjournals.org/ by guest on June 16, 2016

Keyword public health

Global warming, health inequities, infectious disease pan-
demics, obesity: many of the world’s most important
health problems are complex, as are interventions pro-
posed to attenuate their harmful effects. With this in
mind, Smith and Petticrew’s call for broader evaluations
of public health interventions is welcomed.1 However,
seeing the need for ‘macro-evaluations’ is the easy task.
The hard task, as the authors acknowledge, is actually per-
forming such studies.
Despite a decade of similar advocacy, few macro-
evaluations have been performed.2 Smith and Petticrew
recommend that public health consider new methods,
embracing collaboration with other disciplines, because the
traditional micro-approach of public health is too narrow
for the task. While we agree that public health needs to
broaden its toolkit, we suggest there is much to learn
from successful macro-evaluations studies already per-
formed. We review two of our favourite studies and ident-
ify potential lessons related to the strength of evidence;
scope of evaluation (extent of ‘macro-ness’ as defined by
Smith and Petticrew); and dependence on leadership and
stakeholder engagement. We hope others will discuss
lessons learned from other studies.
Whether the field is public health or other disciplines,
most evaluations fall into one of the following categories: (1)
experimental, where individuals or populations are randomly
assigned by the investigators to receive the intervention; (2)
observational, where the intervention is not determined by
the investigators; and (3) modeled, where investigators simu-
late the introduction of an intervention or a combination of
interventions, and various inputs can be manipulated to
predict and examine a range of potential outcomes.
Our first favourite is an experimental study of a Mexican
incentive-based welfare program called ‘Oportunidades’ that
provided investments in nutrition, health and education for
young children living in low-income families.3 The program
(or intervention) consisted of micronutrient-fortified food
for women and children, cash transfers to families that were
conditional on attendance at school, health care appoint-
ments, and a mandatory nutrition and health education
session. The study by Rivera et al. 3 described the nutritional
impact in a subgroup of 347 communities that were ran-
domized to the intervention immediately or after a 1-year
delay.
We learned two lessons and noted one drawback from
this study. First, that it is possible to incorporate a high-
quality intervention trial into a multi-component social
program that is delivered at a massive scale;4 by 2004, the
Oportunidades program covered 4.5 million families.
Second, leadership from the highest levels of government is
prerequisite for implementing innovative health policy that
spans multiple ministries of government as well as the
Douglas G. Manuel, Senior Scientist
Jeffrey C. Kwong, Assistant Professor of Epidemiology

14 # The Author 2010, Published by Oxford University Press on behalf of Faculty of Public Health. All rights reserved.

private sector.5,6 Therefore, we postulate that support from
high-level leadership was instrumental in the Oportunidades
intervention study. The drawback is that the study is still a
‘micro-evaluation’ from Smith and Petticrew’s perspective.
The intervention was essentially a single cause-effect and the
main outcomes were biomedical markers of health (chil-
dren’s height and anaemia).
Our second favourite is a modeling study by Woodcock
et al. 7 that assessed urban transportation and the environ-
ment. This study estimated the health effects of alternative
urban land transport scenarios—lower-carbon-emission
vehicles versus increased active travel versus a combination
of the two. The authors examined the impact of these
hypothetical policies on physical activity, air pollution and
the risk of road traffic injury. Although only health out-
comes were reported, the study included provisions to
evaluate non-health outcomes such as economic growth.
The lesson here is that modeling studies are ideally suited
for macro-evaluation. Woodcock et al.’s7 study has most of
the characteristics of a macro-evaluation, such as multiple
sectors, disciplines and causal pathways. Modeling studies
often require the involvement of ‘untraditional bedfellows’
that Smith and Petticrew encourage us to collaborate with,
and are the cornerstone of many different disciplines’ effort
to describe the natural history and likely outcome of events,
notably fields such as ecology, environmental sciences,
engineering and economics.
Macro-evaluative studies almost by definition require a
wide range of study types and data, and it is helpful to look
for ways to use existing studies and data to support this
complex work. Modeling studies, such as the example of
urban transportation, need data encompassing multiple
viewpoints to describe population risk exposure, hazards or
transitions from different health states, population and econ-
omic outcomes, physical and social structures and inter-
actions, and so forth. However, modeling studies are only as
robust as the evidence that goes into building the models;
therefore, it is critical that models incorporate evidence from
experimental and high-quality observational studies of indi-
vidual policies or interventions. Thus, modeling studies
combine individual micro-evaluations into a macro-
evaluation.
In this way, our most important lesson becomes apparent
when you examine these studies together. Experimental
studies generally provide the strongest evidence, and are
important for building a case for policy effectiveness,2 but
they tend to be micro- rather than macro-evaluations and
are most dependent on leadership and stakeholder engage-
ment. In contrast, modeling studies are often viewed as
HE ALTH Y TR E ES MAK E A H EALT HY WOOD 15
providing the weakest evidence but have the greatest poten-
tial to be macro-evaluations. Observational studies, in most
cases, lie somewhere between the two extremes on these
three dimensions of evidence, leadership and ‘macro-ness.’
All three study types are up for the task of macro-
evaluation. More often, we should look at the wood, but a
healthy wood is made from sound trees.
Funding
Dr. Manuel holds a Chair in Applied Public Health from
The Canadian Institute for Health Research and the Public
Health Agency of Canada. Dr. Kwong is supported by a
Career Scientist Award from the Ontario Ministry of Health
and Long-Term Care and a Research Scholar Award from
Downloaded from http://jpubhealth.oxfordjournals.org/ by guest on June 16, 2016
the Department of Family and Community Medicine,
University of Toronto.
Disclaimer
The opinions, results and conclusions are those of the authors, and no
endorsement by funding agents or the Ontario Ministry of Health and
Long-Term Care or by the Institute for Clinical Evaluative Sciences is
intended or should be inferred.
References
1 Smith R, Petticrew M. Public health evaluation in the 21st century:
time to see the wood as well as the trees. J Public Health 2009.
2 Craig P, Dieppe P, Macintyre S et al. Developing and evaluating
complex interventions: the new Medical Research Council guidance.
BMJ 2008;337:a1655.
3 Rivera JA, Sotres-Alvarez D, Habicht JP et al. Impact of the Mexican
program for education, health, and nutrition (Progresa) on rates of
growth and anemia in infants and young children: a randomized
effectiveness study. JAMA 2004;291:2563– 70.
4 Zwarenstein M, Treweek S, Gagnier JJ et al. Improving the reporting
of pragmatic trials: an extension of the CONSORT statement. BMJ
2008;3:1223 – 6.
5 Yach D, McKee M, Lopez AD, Novotny T. Improving diet and phys-
ical activity: 12 lessons from controlling tobacco smoking. BMJ
2005;330:898– 900.
6 Manuel DG, Creatore M, Rosella L et al. What does it take to make a
healthy province? A benchmark study of jurisdictions in Canada and
around the world with the highest levels of health and the best
health behaviours, 2009. Toronto, Institute for Clinical Evaluative
Sciences. ICES Investigative Report.
7 Woodcock J, Edwards P, Tonne C et al. Public health benefits of
strategies to reduce greenhouse-gas emissions: urban land transport.
Lancet 2009;374:1930 – 43.