271

Article

Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
Elevating the Scientific and Public Policy
Discourse on Crime Prevention: Taking
Stock of the ‘What Works’ Report’s
Influence 20 Years on
Brandon C. Welsh and Andrea B. Wexler

Abstract In 1997, the Office of Justice Programs published Preventing Crime: What Works, What Doesn’t, What’s
Promising (Sherman, L. W., Gottfredson, D. C., MacKenzie, D. L., Eck, J. E., Reuter, P., and Bushway, S. D. (1997).
Preventing Crime: What Works, What Doesn’t, What’s Promising. Washington, DC: Office of Justice Programs). The
report was commissioned by the US Congress and was prepared by a team of criminologists from the University of
Maryland. It aspired to be a methodologically rigorous and comprehensive review of the effectiveness of crime
prevention programmes, ranging from prenatal home visits to community policing to parole. This 20-year review
of the ‘what works’ report finds that it has been influential in elevating both the scientific and public policy discourse
on crime prevention. It did this on three main fronts. First, it reaffirmed that not all evaluation designs are equally
valid and made clear that only designs that provide confidence in observed effects should contribute to the evidence
base. Secondly, it advanced the equally important task of assessing research evidence and, despite some limitations,
adopted a more rigorous method for this purpose. Thirdly, undergirding all of this was the report’s commitment to the
communication of science for the benefit of all parties: policymakers, practitioners, researchers, and the public.
Implications for policy—with special reference to evidence-based policing—and research are discussed.

Introduction rigorous and comprehensive review of crime pre-

It is not an exaggeration to say that the ‘what works’
report—officially titled Preventing Crime: What
Works, What Doesn’t, What’s Promising (Sherman
et al., 1997)—was a landmark publication in the
field of criminology and criminal justice (Gest,
2001; Greenwood, 2006). Commissioned by the
US Congress, it aspired to be a methodologically
vention programmes, ranging from prenatal home
visits to community policing to parole. The report
concluded that there was sufficient scientific evi-
dence to establish a provisional list of programme
modalities that worked, did not work, and were
inconclusive but showed promise in reducing
crime. In many respects, the report represented

School of Criminology and Criminal Justice, Northeastern University, Churchill Hall, 360 Huntington Avenue, Boston, MA
02115, USA. E-mail: b.welsh@northeastern.edu
The authors wish to thank the journal editor and the anonymous reviewer for insightful comments.

Advance Access publication: 6 February 2019

Policing, Volume 13, Number 3, pp. 271–285
doi:10.1093/police/paz006
ß The Author(s) 2019. Published by Oxford University Press. All rights reserved.
For permissions please e-mail: journals.permissions@oup.com
272 Policing Article B. C. Welsh and A. B. Wexler

the beginning of the US interest and the interest of 1998). The aims of the programme were two-fold:

Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
many other Western countries, in an evidence- (a) ‘to better inform policymakers on the import-
based approach to crime policy (Fagan and ance of basing crime prevention programs on sci-
Buchanan, 2016; Farrington, 2000). entific evidence’ and (b) ‘to advance the body of
Like other influential policy works before it and knowledge upon which sound crime prevention de-
since, timing had something to do with its influ- cisions should be made’ (Sherman et al., 2006, p. 3).
ence. Published in February 1997, the report fol- Intriguingly, the most consequential immediate
lowed passage of the 1994 Violent Crime Act and influence of the report took place outside of the
became part of the lingering, vitriolic debate on USA. In 1997, the newly elected British government
prevention versus punishment in the USA initiated a comprehensive spending review of all
(Currie, 1994); it appeared as the US crime drop government services, including criminal justice
of the 1990s was in full swing (Blumstein and and other responses to crime. The review by the
Wallman, 2006); and it arrived on the cusp of the Home Office, including its report, titled Reducing
evidence-based movement in the social and beha- Offending (Goldblatt and Lewis, 1998), was mod-
vioural sciences (Chalmers, 2003). elled on the Preventing Crime report—down to how
Whatever role timing may have played, it is what the research was organized and assessed. This
the report said that carried far more weight. At review led directly to the government’s ‘Crime
center stage was the report’s focus on the science Reduction Programme’, a three-year (1999–2002),
of crime prevention and its relevance for public £400 million (approximately $640 million at the
policy, and this translated into an immediate influ- time) initiative. According to Dhiri et al. (2001, p.
ence on several fronts. This began with coverage in 181), it represented the ‘biggest single investment in
the press. For example, the New York Times called an evidence-based approach to crime reduction to
the report ‘[t]he most comprehensive study ever of have ever taken place in any country’.2
crime prevention’ (Butterfield, 1997, p. A20). U.S. As important as some of these events were, they
News & World Report referred to the report as ‘au- were rather short-lived and far from impactful on
thoritative’ and stated that it ‘makes the fervent the scientific and policy understanding of the pre-
ideological battles over anticrime policy seem vention of crime. When viewed over a longer time
beside the point’ (Gest, 1997, p. 38).1 The report frame, however, a much different view of the influ-
was the subject of four Congressional hearings, and ence of the report takes shape. It is the thesis of this
its authors were invited on many occasions to brief article that the ‘what works’ report proved influen-
policymakers and other leaders across the USA and tial in helping to elevate both the scientific and
abroad. It also attracted the attention of charitable public policy discourse on crime prevention. It
foundations and private donors interested in sup- did this on three main fronts.
porting a continuation of the report’s work. These First, the report reaffirmed that not all evaluation
grants and donations were used to establish the designs are equally valid and made clear—in no
Crime Prevention Effectiveness Program at the uncertain terms—that only designs that provide a
University of Maryland’s Department of high degree of confidence in observed effects (i.e.
Criminology and Criminal Justice (Sherman et al., experimental and high-quality quasi-experimental
1
Gest (1997, p. 38) added that, ‘Dud programs are scattered across the spectrum, from summer jobs for youths (a liberal
favorite) to military-style boot camps for lesser offenders (a centrist-conservative favorite).’
2
It is important to note that the Crime Reduction Programme was criticized for failing to deliver on its primary objective of
using a ‘research-driven’ approach to guide policy and practice on what works best to reduce crime (Maguire, 2004). This had
a great deal to do with poor government management and problems with the implementation of programs at the local level
(Nutley and Homel, 2006).
Scientific and public policy discourse on crime prevention
designs) should contribute to the evidence base.
These views were by no means original (Cook and
Campbell, 1979). What was original was the au-
thors’ use of an objective and understandable meas-
ure, known as the Scientific Methods Scale (SMS),
to rate evaluation designs on overall internal valid-
ity. The bottom line is that evaluation designs need
to be guided by the research question at-hand, not
the other way around.
Secondly, for the equally important task of assess-
ing the research evidence (i.e. all of the included
evaluation studies), the report moved beyond the
traditional narrative review and adopted an empir-
ical review method. The report also established rules
for drawing conclusions about what works, what
does not work, and what is promising. Crucially,
as part of this process, the report called attention
to the provisional nature of science: scientific con-
clusions drawn today can be altered—even up-
ended—with the results of new studies over time.
Third, and undergirding all of this, was the re-
port’s commitment to the communication of sci-
ence for the benefit of all parties: policymakers,
practitioners, researchers, and the public. As noted
by Shonkoff and Bales (2011, p. 17), ‘There is a
widespread assumption that science has a role to
play in advising policymakers on crafting responses
to complex social problems . . . There are also many
countervailing forces that impede the fulfillment of
that role.’ The report’s efforts to engage in commu-
nicating science took many different forms, and this
may very well be one of its most important legacies.
For example, elements of this have been central to
the advancement of evidence-based policing (Lum
and Koper, 2017; Sherman, 2013).
In an effort to review the potential influence of
the ‘what works’ report, we consulted a wide range
of published documentation, including peer-re-
viewed journal articles, essays, books, and reports.
This began with the original report (Sherman et al.,
1997; see also Sherman et al., 1998) and subsequent
related works ( MacKenzie, 2000; Sherman et al.,
2002, 2006). It is important to note that the report
and subsequent related works were not without
Article Policing 273
detractors, and the criticisms covered many fronts
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
(Haggerty, 2008; Stafford and Silverlock, 2000). We
reviewed as many critiques as we could locate, and
these are discussed throughout the article. Relatedly,
we were also interested in more general critiques of
the evidence-based approach (Sampson, 2010;
Sampson et al., 2013), and these were consulted.
Lastly, we drew upon the growing body of crimino-
logical literature as it pertains to the evidence-based
approach. This includes writings on theory, methods
of experimentation and systematic review, and re-
search utilization (Fagan and Buchanan, 2016;
Mears, 2007, 2010; Petrosino and Lavenberg, 2007;
Weisburd et al., 2016; Zane and Welsh, 2018).
The article is organized around the three afore-
mentioned major areas of influence of the report:
(a) rating evaluation designs; (b) assessing research
evidence; and (c) communicating science. We
begin with some background on the report.
Background: the ‘what works’
report
In documenting the beginnings of the ‘what works’
report, one is struck by the short yet intense period
of time leading up to its public release in February
1997. The starting point was in April 1996, when
the 104th US Congress enacted a conference report
that required the Attorney General to provide
Congress with an independent review of the effect-
iveness of state and local crime prevention pro-
grammes funded by the federal justice
department. To help achieve these ends, the
review was required to use ‘rigorous and scientific-
ally recognized standards and methodologies’ (U.S.
House of Representatives, 1996, p. 103).
At the direction of the Assistant Attorney
General for the Office of Justice Programs, the
National Institute of Justice (NIJ) was tasked with
issuing a ‘competitive solicitation for proposals’.
This was done in coordination with several other
federal agencies, including the Bureau of Justice
Assistance, the Office of Juvenile Justice and
274 Policing Article
Delinquency Prevention, and the Executive Office
of Weed and Seed (Sherman, 1997a, chap. 1, p. 4).
In June 1996, the NIJ released the solicitation, and
within short order, a panel of experts appointed by
the agency began the process of evaluating pro-
posals. By August 1996, the panel had completed
its review of proposals and selected the one sub-
mitted by a team of six scholars from the
University of Maryland’s Department of
Criminology and Criminal Justice.3 The deadline
for completing the report was 27 January 1997.
With only 6 months to complete the report, the
University of Maryland team set in motion a rather
Herculean task, which included reviewing more
than 500 evaluations of crime prevention pro-
grammes and producing a 565-page report. In add-
ition to the six scholars, the project was aided by at
least 22 graduate students and a scientific advisory
panel of distinguished scholars from criminology
and affiliated fields.
The scope of the report was defined as such: ‘a
critical assessment, based on a growing body of sci-
ence, of the effectiveness of a wide range of crime
prevention strategies, operated at the local level,
with and without the support of Federal funds’
(Sherman, 1997a, chap. 1, p. 5). Within this
scope, four key features served as the foundation
of the report and guided its conclusions: definition
of crime prevention; classification of crime preven-
tion; science of crime prevention; and methods for
searching for studies and assessing effectiveness. An
overview of each of these features follows.
Definition
For the purposes of the report, crime prevention was
defined in its widest sense by its outcome: the pre-
vention of a future criminal act. By not considering
the character or function of prevention (Foxcroft,
2014), this definition allowed for the inclusion of
3
The team included all six authors of the eventual report: Lawrence Sherman, Denise Gottfredson, Doris MacKenzie, John
Eck, Peter Reuter, and Shawn Bushway.
4
External validity is often included as a fourth criterion. In the present context of the validity of a single study, it is not
possible to make an assessment of the generalizability of the study’s observed effects. This requires at least a second test or
replication of the study.
B. C. Welsh and A. B. Wexler
the full range of techniques or measures that have
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
been used to try to prevent crime—from prenatal
home visits to community policing to parole.
Herein, the authors argued that ‘[t]he value of a
broad framework for analyzing crime prevention
policies is its focus on the whole forest rather than
on each tree’ (Sherman, 1997b, chap. 2, p. 4).
Classification
Rather than draw upon one of the more traditional
taxonomies that have been used to classify or or-
ganize the wide array of crime prevention pro-
grammes ( Brantingham and Faust, 1976; Tonry
and Farrington, 1995; van Dijk and de Waard,
1991), the authors elected to focus on the institu-
tional setting or domain in which programmes take
place. Altogether, seven major institutional settings
were articulated: families; schools; communities;
labor markets; places (e.g. urban centers, homes);
police agencies; and courts and corrections. In
acknowledging that the definitions of these settings
were quite broad and that there was some overlap
across some settings, the authors seemed content in
finding that ‘much of the crime prevention litera-
ture fits quite neatly into [these] seven major insti-
tutional settings’ (Sherman, 1997b, chap. 2, p. 8).
Scientific basis
Perhaps the most important feature of the report is
that it was firmly anchored in the science of pro-
gramme evaluation. Important to this is that there
are ‘widely agreed-upon rules for assessing the level
of certainty that a conclusion in any one test is
correct’ (Sherman et al., 1998, p. 3). These rules
pertain to the methodological quality of the evalu-
ation design, which depends on three main criteria:
statistical conclusion validity; internal validity; and
construct validity (Cook and Campbell, 1979;
Shadish et al., 2002).4 ‘Validity refers to the
Scientific and public policy discourse on crime prevention
correctness of inferences about cause and effect’
(Shadish et al., 2002, p. 34). All things being
equal, designs with higher validity provide more
confidence in the observed effects. In recognition
that not all evaluations are equally valid and want-
ing to include only the highest quality designs, the
authors developed an instrument to rate evaluation
designs: the SMS. (The SMS and validity are dis-
cussed in more detail in the section on rating evalu-
ation designs.)
Methodology
The research methods used in the report relate to
two key functions: searching for studies and assess-
ing the research evidence. A fair amount is known
about the process that was used to identify studies.
This began with consulting recent secondary re-
views. If the authors determined the secondary
review to be reliable, it was used to make a ‘deter-
mination of the strength of the evidence about the
category [or program type]’ (Sherman and
Gottfredson, 1997, appendix, p. 3). If no secondary
review was available, a ‘primary evidence analysis’
was carried out. This involved an effort to identify
‘every evaluation it [the chapter research team] can
locate from all sources’ (Sherman and Gottfredson,
1997, appendix, p. 3). Much less is known about the
search techniques that were used to identify studies
as part of the primary evidence analysis. For ex-
ample, were electronic bibliographic databases
used? Were the references of eligible studies used
to identify additional studies? Was forward
searching used with eligible studies—a way to iden-
tify more recent follow-ups? These are some of the
search techniques that are required in the conduct
of systematic reviews (Petrosino and Lavenberg,
2007). What is known is that the search for studies
was limited mostly to the published literature,
which raises the additional concern of publication
bias (Wilson, 2009).
On the matter of assessing the research evidence
(i.e. the more than 500 studies collected) to allow
5
A fourth category, what is unknown, was also discussed, but it received far less systematic attention.
Article Policing 275
for conclusions to be made about what works, what
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
does not work, and what is promising5 in the seven
settings, there was an effort to make the approach
somewhat robust. At its core, the authors relied on
the vote-count review method, which tallies-up the
‘number of studies with statistically significant
findings in favor of the hypothesis and the
number contrary to the hypothesis’ (Wilson,
2001, p. 73). Two elements helped to improve
upon the vote-count review: (a) it integrated the
SMS and (b) a clear set of rules was established
for the different conclusions (i.e. what works and
so on). For conclusions about what works and what
does not work, this had the effect of providing some
measure of external validity. (These rules are dis-
cussed in the section on assessing research
evidence.)
Rating evaluation designs
One of the most noteworthy features of the ‘what
works’ report was its use of a scale to rate the meth-
odological quality of each study’s evaluation design.
Known as the SMS, the authors set out to devise a
tool that would (a) provide a sound measure of
overall internal validity and (b) easily be commu-
nicated to stakeholders. The scale is as follows, ran-
ging from level 1 (weakest on overall internal
validity) to level 5 (highest):
Level 1: Correlation evidence: low of-
fending correlates with the intervention.
Level 2: Nonequivalent control group
or one-group pre-post design: treat-
ment group compared with nonequi-
valent control group; treatment group
measured before and after the interven-
tion (with no control group).
Level 3: Equivalent control group
design: treatment group compared
with comparable control group,
276 Policing Article
including pre-post and treatment-con-
trol comparisons.
Level 4: Control of extraneous vari-
ables: treatment group compared with
control group, with control of extrane-
ous influences on the outcome (e.g., by
matching, prediction or propensity
scores, or statistical controls).
Level 5: Randomized experiment: units
assigned at random to treatment and
control groups.
With respect to overall internal validity, the SMS
is heavily influenced by the pioneering theoretical
and empirical work on experimentation by Donald
Campbell and Thomas Cook, especially their classic
book (Cook and Campbell, 1979; see also Campbell
and Stanley, 1966; Shadish et al., 2002). As one ex-
ample, level 3 is considered to be the minimum
interpretable design by Cook and Campbell
(1979). This is because it rules out many threats
to internal validity, including history, maturation/
trends, instrumentation, testing, and mortality. It
was also regarded by the authors of the ‘what works’
report as the minimum design that was adequate
for drawing conclusions about what works and
what does not work.
While the SMS focused exclusively on internal
validity, it is important to note that all evaluation
projects were also rated on statistical conclusion
validity and on construct validity. Specifically, the
following four aspects of each study were rated: ‘For
statistical conclusion validity: (a) Was the statistical
analysis appropriate? (b) Did the study have low
statistical power to detect effects because of small
samples? (c) Was there a low response rate or dif-
ferential attrition? For construct validity: (d) What
was the reliability and validity of measurement of
the outcome?’ (Farrington et al., 2002, pp. 17–18).
The authors also opted for this simple five-point
scale to facilitate ease of communication. It was
intended that each point on the scale should be
understandable. This was in sharp contrast to
many other scales of methodological quality that
B. C. Welsh and A. B. Wexler
were in use in the medical and social and beha-
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
vioural sciences, which relied on a summation of
scores (e.g. 0–100) on a number of specific criteria
(Brounstein et al., 1997; Juni et al., 1999; Moher
et al., 1995). As noted by Farrington et al. (2002,
p. 15), one of the concerns with these types of scales
is that ‘it is hard to know what meaning to attach to
any score, and the same score can be achieved in
many different ways’.
The SMS has been the subject of some criticism.
For example, Hope (2005), focusing only on the
chapter on community crime prevention, argues
that the ‘research designs that score highly on the
SMS are likely to suffer from Type II Error—that is,
they are likely to be interpreted as inferring errone-
ously that treatment effects do not exist when in
fact they do’ (p. 276, emphasis in original). The
core of his argument is that key threats to internal
validity (i.e. selection, history), of which the experi-
mental method attempts to control, are the ‘social,
collective “mechanisms” that they [the community
interventions] activate in order to bring about out-
comes’ (Hope, 2005, p. 276). A rebuttal to this view
could take on any number of issues, including what
appears to be a dismissal of sophisticated experi-
mental and quasi-experimental designs to account
for some of the unique challenges confronting the
evaluation of area-based studies (e.g. mixed units of
analysis, small sample size) and oversight to evi-
dence of an indirect relationship between research
design and study outcomes in criminological stu-
dies (Weisburd et al., 2001; Welsh et al., 2011).
Another criticism of the SMS was directed at the
threshold—the minimum research design (i.e. level
3)—that was used to include studies in the report.
Hough (2010) contends that it is too generic and
should be adjusted in consideration of the body of
work being reviewed: ‘The threshold for inclusion
of studies should be set individually for each review,
in light of available evidence.’ No mention is made
if this pertains to the level of methodological rigor
of the studies under review, the quantity of the
studies, or both.
Scientific and public policy discourse on crime prevention
In the context of reviews of the literature, includ-
ing systematic reviews carried out under the aus-
pices of the Campbell Collaboration, Farrington
and Petrosino (2001, p. 42) note that the criterion
of methodological quality that is used for including
studies is likely the ‘most important and controver-
sial’. It is important because it speaks to the degree
of confidence that one can have in reported effects
of individual studies and conclusions drawn from
the aggregation of those studies. It is controversial
because setting the bar too high may end up exclud-
ing all or many studies on a topic, leaving nothing
to say and giving the impression that the studies are
flawed (Farrington and Petrosino, 2001). This is
especially problematic for area-based studies in
general, which are less likely to use designs that
equate to levels 4 and 5 on the SMS (Farrington
and Welsh, 2006).
Even more problematic in using what amounts
to a rigid threshold approach (e.g. only including
randomized experiments) is that it dismisses the
strength of the causal inference that can be achieved
from a wide range of other rigorous designs (i.e.
levels 3 and 4 on the SMS), whether it be for indi-
vidual- or area-based studies (Nagin and Weisburd,
2013). In the only known empirical study on the
subject, Stockard and Wood (2017) found that use
of a rigid ‘threshold approach’ (i.e. only including
randomized experiments), when compared with an
‘inclusive approach’ (i.e. in accordance with the
work of Campbell and Cook), resulted in the exclu-
sion of a large proportion of identified studies,
leading to the concern that ‘policy recommenda-
tions [would be] based on only part of the available
literature’ (p. 472). A further analysis showed that
effect sizes from the inclusive approach were ‘more
precise and closer to those of other reviews’
(Stockard and Wood, 2017, p. 471).
In the case of the ‘what works’ report, there was a
clear trade-off between the level of certainty in the
answers that can be given about programme effects
and the level of useful information that can be
gleaned from the available science. By adopting
Article Policing 277
level 3 as the threshold of methodological quality,
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
Sherman and his colleagues took a ‘middle road
between reaching very few conclusions with great
certainty and reaching many conclusions with little
certainty’ (MacKenzie, 2000, p. 461).
Assessing research evidence
Just as it is crucial to use the highest quality evalu-
ation designs to investigate the effects of crime pre-
vention programmes, it is also important that the
most rigorous methods be used to assess the avail-
able research evidence. There are several different
types of review methodology, with the main ones
being narrative, vote-count, and systematic/meta-
analytic. The systematic review, which often in-
cludes a meta-analysis, is considered to be the
most exacting method and is recommended above
all others (Petrosino and Lavenberg, 2007;
Petticrew and Roberts, 2006). As noted by
Petrosino et al. (2001, p. 2), ‘the foremost advan-
tage of systematic reviews is that when done well
and with full integrity, they provide the most reli-
able and comprehensive statement about what
works’. In short, the systematic review uses rigorous
methods for locating, appraising, and synthesizing
evidence from prior evaluation studies, and it is
exhaustive in its coverage of the literature (pub-
lished and unpublished).
The ‘what works’ report used the vote-count
method, not the systematic review method, to
make an assessment of the research evidence. The
main reason was the relatively short time frame that
was available to carry out the review. The vote-
count review method falls short of the systematic
review on a number of grounds, but in the present
circumstances, the most pressing matter had to do
with the limited depth in the coverage of the litera-
ture. Sherman and Gottfredson (1997, appendix,
p. 1) capture this point with a view to the larger
mission of the project: ‘Our approach strikes a
compromise between breadth and depth, without
any compromise in scientific integrity.’
278 Policing Article B. C. Welsh and A. B. Wexler

Importantly, the vote-count review offers some Critiques of the report’s efforts to assess the

advantages over the traditional narrative review,
the latter being the most commonly used method
at the time and the method the authors were
attempting to improve upon. For instance, the
vote-count incorporates a quantitative element by
considering statistical significance. There is also
some effort to overcome researcher bias. This
can include the adoption of a methodology for
searching for studies, which can help mitigate
self-selection. Furthermore, as noted earlier, two
elements helped to improve upon the vote-count
method. One was the integration of the SMS.6 The
other was the establishment of a set of rules for
drawing conclusions about the research evidence,
with all programme types classified into one of four
categories: what works, what does not work, what is
promising, and what is unknown.7
The end product was a list showing that 15 pro-
gramme types were effective, 23 programme types
were ineffective, 30 programme types were promis-
ing, and as many as 68 programme types were of
unknown effectiveness (Sherman et al., 1997,
1998). Importantly, the list was considered provi-
sional, which was, in the words of the authors, ‘just
as all scientific knowledge is provisional’ (Sherman
et al., 1998, p. 3). This acknowledgement stands out
as one of the key contributions of the report.
Clearly, providing a list of programme types that
work, for example, is compelling and readers want
to know this information. But the list itself is far less
important than all of the considerations—the
process—that went into deriving the list.
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
research evidence center on methodological limita-
tions of the vote-count review and concerns
about the generalizability of the conclusions.
Perhaps the most important methodological limi-
tation of the vote-count review has to do with its
focus on statistical significance. The main problem
with using statistical significance is that it depends
partly on sample size and partly on effect size. For
example, a significant result may reflect a small
effect in a large sample or a large effect in a small
sample. As noted by Wilson (2001, p. 73), at the
root here is that ‘[l]arger studies, all else being
equal, provide more precise estimates of the rela-
tionship of interest and thus should be given greater
weight in a review.’
With respect to concerns about the generalizabil-
ity of the conclusions, Tilley (2000, p. 124) takes
issue with the seemingly infinite conditions for pro-
gramme effectiveness (‘population demographics,
program elements, and social context’), leading
him to conclude that the four categories used to
classify programme types are ‘highly misleading’.
He goes on to say that they should be replaced
with more accurate renderings: ‘What has been
found to work somewhere’ (for what works);
‘What has been found not to work somewhere’
(for what does not work); ‘What seems to have
worked somewhere’ (for what is promising); and
‘What may work somewhere’ (for what is un-
known) (Tilley, 2000, p. 124). Stafford and
Silverlock (2000, p. 92) take this criticism one
step further, arguing that the report’s ‘findings are

6
This had the effect of addressing a serious limitation: equal weight given to all studies irrespective of methodological quality.
7
What works: These are program types that prevent crime. Program types coded as working must have at least two level-3 to
level-5 evaluations with tests of significance showing effectiveness and the preponderance of all available evidence supporting
the same conclusion. What does not work: These are program types that fail to prevent crime. Program types coded as not
working must have at least two level-3 to level-5 evaluations with tests of significance showing ineffectiveness and the
preponderance of all available evidence supporting the same conclusion. What is promising: These are program types
where the level of certainty from available evidence is too low to support generalizable conclusions but where there is
some empirical basis for predicting that further research could support such conclusions. Program types were coded as
promising if they were found to be effective in significance tests in one level-3 to level-5 evaluation and in the preponderance
of the remaining evidence. What is unknown: Any program type not classified in one of the other three categories is defined as
having unknown effects.
Scientific and public policy discourse on crime prevention
of limited use to the practitioner’. This is because
there is no recipe for ‘how’ to replicate the findings.
The authors were deliberate in acknowledging
concerns about the generalizability of the report’s
conclusions for what works and what does not
work:
The weakest aspect of this classification
system is that there is no standard
means for determining external valid-
ity: exactly what variations in program
content and setting might affect the
generalizability of findings from exist-
ing evaluations. In the current state of
science, that can be accomplished only
by the accumulation of many tests in
many settings with all major variations
on the program theme. None of the
programs reviewed for this report
have accumulated such a body of
knowledge so far. (Sherman et al.,
1998, p. 6)
With this health warning in mind, and no appar-
ent solution available, it does raise questions about
the public policy utility of the lists; in addition, it
brings us back to the provisional nature of the lists.
Indeed, it would take the findings of systematic re-
views of the Campbell Collaboration and others
over the next two decades to bring clarity to some
of the conclusions reached by the report (Weisburd
et al., 2016).
Communicating science
The communication of science has been defined as
the ‘exchange of information and viewpoints about
science to achieve a goal or objective such as foster-
ing greater understanding of science and scientific
methods or gaining greater insight into diverse
public views and concerns about the science related
to a contentious issue’ (National Academies of
8
National Academies of Sciences, Engineering, and Medicine (2017, pp. 40–43) delineates three main approaches to science
communication in a policy-making context: (a) aggregation and translation, (b) brokering, and (c) partnerships.
Article Policing 279
Sciences, Engineering, and Medicine, 2017, pp. 1–
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
2). In their effort to communicate the science of
crime prevention to policymakers, practitioners,
researchers, and the public, the authors of the
‘what works’ report adopted two structured
approaches.8 The first, known as ‘aggregation and
translation’, involves ‘one-way communication’
(National Academies of Sciences, Engineering,
and Medicine, 2017, p. 40). This took the form of
the report itself, a summary of the report published
by the NIJ (Sherman et al., 1998), a government
website from which the report could be down-
loaded, and numerous Congressional hearings
and meetings. Admittedly rather straightforward,
this approach was about getting the information
out there and in multiple understandable formats
for the different consumers.
A second approach used by the authors of the
report to communicate science is referred to as
‘brokering’. As described by the National
Academies of Sciences, Engineering, and Medicine
(2017, p. 41), a ‘broker in this context is an inter-
mediary (an organization or a person) that bridges
science and policy making by providing informa-
tion while developing relationships within net-
works and linking those who produce knowledge
with those who may use it’. It seems safe to say that
this approach was a product of the widespread
interest in the report, something that happened in
the immediate aftermath of its release. Initially, this
took the form of the Crime Prevention
Effectiveness Program, which was established at
the authors’ university. While the original concep-
tion of the programme was never truly realized,
part of its mission—to advance an evidence-based
approach to preventing crime—continued for
many years after in the form of an annual meeting
that brought together researchers and policy-
makers. Known as the Jerry Lee Crime Prevention
Symposium, with financial backing from the phil-
anthropist Jerry Lee, the annual meeting initiated
280 Policing Article
academic and policy research and policy briefings
with elected officials and legislative staff on Capitol
Hill. Also, from the first meeting (in April 2001), it
established a close partnership with the newly
formed Campbell Collaboration, the leading inter-
national organization dedicated to the advance-
ment of evidence-based social policy (Farrington
and Welsh, 2001).
Beyond the approaches used, the communica-
tion of science was at the heart of the mission of
the project; specifically, it was central to the core
components of the report: rating evaluation designs
and assessing research evidence. For the rating of
evaluation designs—through the use of the SMS—
it was imperative that the scale be understandable
to target audiences outside of academia. As was
later noted, ‘the SMS can be criticized, but it has
the virtue of simplicity. It can be improved, but at
the cost of simplicity’ (Farrington et al., 2002, p.
21). In short, the key was not to dilute the central
message: ‘What is essential is to use some kind of
methodological quality scale in order to communi-
cate to scholars, policymakers, and practitioners
that not all research is of the same quality, and
that more weight should be given to higher quality
evaluation studies’ (Farrington et al., 2002, p. 21).
The report’s effort to communicate scientific
concepts in a straightforward and understandable
way to the public and other stakeholders is by no
means unique. Other projects, which may take the
form of reviews or larger platforms, continuously
grapple with striking a balance between communi-
cating scientific information in an accurate and
simple way. For example, Harvard University’s
Center on the Developing Child’s long-term project
to communicate the science of child development
to policymakers and the public is guided by the
following principle: ‘science can be served credibly
and the public can be better informed without ser-
ious distortions of complex concepts even as they
are shortened and translated into lay terms’
(Shonkoff and Bales, 2011, pp. 17–18).
With respect to the assessing research evidence
component of the ‘what works’ report, here the
B. C. Welsh and A. B. Wexler
communication of science was about much more
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
than generating lists of what works, what does not
work, and so on. In the context of programme
evaluation, the authors saw it as vital to convey
the provisional nature of science; that is, knowledge
is ‘always becoming more refined, and therefore no
conclusion is permanent’ (Sherman et al., 1998, p.
3). While the scientific community understands
this principle, it is often overlooked in reviews of
the literature, whereby, conclusions can be touted
as final. More importantly, this principle is not
well-understood outside of the scientific commu-
nity and can be a cause for concern with science in
general. To these latter points, the words of
Bromme and Beelmann (2018, p. 351) are instruct-
ive: ‘The provisional nature of scientific findings is
not an exception or fault within the progress of
science. Revising the present state of knowledge
is . . . the everyday business of researchers.’
Moreover, in explaining the provisional nature
of science, it was done neither to temper the fact
that the vote-count review had some deficiencies
nor that there were some concerns about external
validity. Instead, the authors were clear-eyed and
transparent about the provisional nature of science,
noting in the 1998 summary of the report: ‘All con-
clusions reported in this Research in Brief reflect
the state of scientific knowledge as of late 1996.
By the time this Research in Brief is published [in
July 1998], new research results may be available
that would modify the conclusions’ (Sherman
et al., 1998, p. 3).
Discussion and conclusions
It has been 20 years since the publication of the
‘what works’ report by Sherman et al. (1997). The
report aspired to be a methodologically rigorous
and comprehensive review of the effectiveness of
crime prevention programmes, and it had an im-
mediate influence in several areas. In the interven-
ing years since its release, the report’s relevance has
not appeared to wane. For example, the SMS
Scientific and public policy discourse on crime prevention
continues to receive widespread application by
scholars and organizations (e.g. Campbell
Collaboration, Washington State Institute for
Public Policy) engaged in research contributing to
evidence-based policy. Also, as of August 2018, the
Web of Science shows that the report (along with
the 1998 summary) has been cited more than
12,000 times in over 9,000 different published
works. Furthermore, the ‘what works’ report has
been recognized as the beginning of interest in an
evidence-based approach to crime policy in the
USA and abroad.
This article set out to take stock of the ‘what
works’ report’s potential influence over the last
two decades. On the basis of a review of a wide
range of documentation on the report and in
related subject areas, we argue that the report
proved influential in elevating both the scientific
and public policy discourse on crime prevention
and that it did this on three main fronts. First, it
reaffirmed that not all evaluation designs are
equally valid and made clear that only designs
that provide confidence in observed effects should
contribute to the evidence base. Secondly, it
advanced the equally important task of assessing
research evidence and, despite some limitations,
adopted a more rigorous method for this purpose.
Thirdly, undergirding all of this was the report’s
commitment to the communication of science for
the benefit of all parties: policymakers, practi-
tioners, researchers, and the public.
Each of these three points plays a central role in
current efforts in advancing an evidence-based ap-
proach to crime policy, and it is in this context that
we explore implications for policy and research. On
the matter of advocating for the use of the highest
quality designs to evaluate programme effects, sev-
eral key issues warrant attention. One is the
acknowledgement that this has nothing to do
with only using randomized controlled experi-
ments. The ‘what works’ report and, more broadly,
the body of work by Campbell, Cook, and other
leading experimentalists in the social and beha-
vioural sciences say nothing to this effect
Article Policing 281
(National Research Council, 2005). In fact, this
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
view flies in the face of the contribution to policy
made by high-quality quasi-experimental methods
(Nagin and Weisburd, 2013).
High-quality designs to evaluate programme ef-
fects are also crucial for informing decisions about
spending on crime prevention. The reason is that an
economic analysis (i.e. the technique used to meas-
ure economic efficiency) is an extension of an out-
come evaluation and is only as defensible as the
evaluation on which it is based. It has long been
recommended that economic analyses be limited
to programmes that have been evaluated with an
“experimental or strong quasi-experimental de-
sign” (Weimer and Friedman, 1979, p. 264).
Increasingly, this is becoming a fixture in economic
analyses with a direct bearing on public policy. For
example, the use of high-quality designs is a criter-
ion of the Washington State Institute for Public
Policy’s benefit-cost modelling work (Lee et al.,
2012), one of the most rigorous and consequential
evidence-based policy initiatives of the last 15 years.
Another issue has to do with the need for greater
attention to challenges facing programme imple-
mentation (Fagan, 2017; Fixsen et al., 2009; Telep
and Weisburd, 2016). The importance of imple-
mentation to the evidence-based model is best cap-
tured by the following: ‘Evidence-based policing
assumes that experiments alone are not enough.
Putting research into practice requires just as
much attention to implementation as it does to
controlled evaluations’ (Sherman, 1998, p. 7).
On the matter of assessing research evidence, it is
important to note that the ‘what works’ report did
not in any way influence the wave of systematic
reviews of the literature that would shortly follow
under the auspices of the Campbell Collaboration
or those conducted by other researchers. But it can
be argued that the report demonstrated the need for
a narrower focus and greater rigor in the reviewing
process. Despite all of the efforts to improve the
vote-count method (i.e. by integrating the SMS
and establishing a set of rules for the conclusions),
it stands out as the weakest component in an
282 Policing Article
otherwise rigorous report. We think the main mes-
sage here is that future reviews of the literature need
to adopt the methodology of systematic reviews. In
the case of intervention research, this includes a
much narrower focus. Long gone are the days of
sweeping reviews of what works, unless they are
founded on systematic reviews (Weisburd et al.,
2016).
There is also a need for systematic reviews to be
updated in a timely manner. The Campbell
Collaboration, for example, recommends that au-
thors update reviews every 5 years. In addition to
attending to the provisional nature of science, this
has much to do with making reviews relevant to the
times. If findings of systematic reviews are to have
any influence on policy, they need to report on the
latest research.
When it comes to the communication of science,
the ‘what works’ report seems to have been ahead of
its time or, more correctly, the authors were well
aware of the necessity to convey scientific informa-
tion in a way that is understandable to all the key
stakeholders. It is clear that the report was not writ-
ten to simply publicize the research findings; it was
also written to communicate the science of prevent-
ing crime in order to foster more effective policy
and practice moving forward.
The importance of this approach has given rise to
several key developments. In the context of evi-
dence-based policing, this has helped to foster
increased receptivity of police officers to this poli-
cing philosophy (Telep, 2017; Telep and Lum,
2014). It has also fostered the establishment of nu-
merous professional societies dedicated to the ad-
vancement of evidence-based policing, both on an
international (Society of Evidence-Based Policing)
and national (Australia/New Zealand, Canada, and
USA) stage (Martin and Mazerolle, 2015).
Another key development in communicating the
science of preventing crime has been crime preven-
tion registries. These registries are designed to as-
semble, analyse, and promote effective crime
prevention programmes. One of the registries,
Blueprints for Healthy Youth Development
B. C. Welsh and A. B. Wexler
(blueprintsprograms.com; formerly Blueprints for
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
Violence Prevention), was started around the same
time of the release of the ‘what works’ report.
Another registry, Crime Solutions (crimesolu-
tions.gov/), was started in 2011 and draws heavily
on the report. A recent critical review of these and
other registries finds that there is large-scale vari-
ability across registries, with special concerns about
discrepancies in the criteria used to identify what
works and in the lack of agreement on the pro-
grammes that get selected for dissemination
(Fagan and Buchanan, 2016). The review authors
call for a number of changes to improve on this
state of affairs, and these changes should be heeded.
At the same time, there needs to be a programme
of research dedicated to the science of communi-
cating the science of crime prevention. As noted by
Shonkoff and Bales (2011, p. 30), ‘knowledge trans-
fer from research to policy and service delivery’ is a
complex process that merits study in its own right.
Other fields in the social, behavioural, and natural
sciences have embarked on this task (National
Academies of Sciences, Engineering, and
Medicine, 2017; Shonkoff and Bales, 2011). Fagan
and Buchanan’s (2016) critical review is an import-
ant step for the prevention of crime, and we think it
is high time for criminology and criminal justice
more broadly to follow suit.
References
Blumstein, A. and Wallman, J. (eds). (2006). The Crime
Drop in America, Rev. edn. New York: Cambridge
University Press.
Brantingham, P. J. and Faust, F. L. (1976). A Conceptual
Model of Crime Prevention’. Crime & Delinquency 22(3):
284–296.
Bromme, R. and Beelmann, A. (2018). Transfer Entails
Communication: The Public Understanding of (Social)
Science as a Stage and a Play for Implementing
Evidence-Based Prevention Knowledge and Programs’.
Prevention Science 19(3): 347–357.
Brounstein, P. J., Emshoff, J. G., Hill, G. A., and Stoil, M. J.
(1997). ‘Assessment of Methodological Practices in the
Evaluation of Alcohol and Other Drug (AOD) Abuse
Prevention’. Journal of Health and Social Policy 9(2): 1–9.
Scientific and public policy discourse on crime prevention
Butterfield, F. (1997). Most Efforts to Stop Crime Fall Far
Short, Study Finds. New York Times, 16 April 1997, p.
A20.
Campbell, D. T. and Stanley, J. C. (1966). Experimental and
Quasi-Experimental Designs for Research. Chicago: Rand
McNally.
Chalmers, I. (2003). ‘Trying to Do More Good Than Harm
in Policy and Practice: The Role of Rigorous, Transparent,
Up-To-Date Evaluations’. Annals of the American
Academy of Political and Social Science 589: 22–40.
Cook, T. D. and Campbell, D. T. (1979). Quasi-
Experimentation: Design and Analysis Issues for Field
Settings. Chicago: Rand McNally.
Currie, E. (1994). ‘What’s Wrong with the Crime Bill’. The
Nation 258(4): 118–121.
Dhiri, S., Goldblatt, P., Brand, S., and Price, R. (2001).
Evaluation of the United Kingdom’s ‘Crime Reduction
Programme’: Analysis of Costs and Benefits. In Welsh,
B. C., Farrington, D. P., and Sherman, L. W. (eds),
Costs and Benefits of Preventing Crime. Boulder, CO:
Westview Press, pp. 179–201.
Fagan, A. A. (2017). ‘Illuminating the Black Box of
Implementation in Crime Prevention’. Criminology &
Public Policy 16(2): 451–455.
Fagan, A. A. and Buchanan, M. (2016). What Works in
Crime Prevention? Comparison and Critical Review of
Three Crime Prevention Registries’. Criminology &
Public Policy 15(3): 617–649.
Farrington, D. P. (2000). ‘Explaining and Preventing
Crime: The Globalization of Knowledge’. Criminology
38(1): 1–24.
Farrington, D. P., Gottfredson, D. C., Sherman, L. W., and
Welsh, B. C. (2002). The Maryland Scientific Methods
Scale. In Sherman, L. W., Farrington, D. P., Welsh B.
C., and MacKenzie, D. L. (eds), Evidence-Based Crime
Prevention. New York: Routledge.
Farrington, D. P. and Petrosino, A. (2001). ‘The Campbell
Collaboration Crime and Justice Group’. Annals of the
American Academy of Political and Social Science 578:
35–49.
Farrington, D. P. and Welsh, B. C. (eds). (2001). ‘What
Works in Preventing Crime? Systematic Reviews of
Experimental and Quasi-Experimental Research’. Annals
of the American Academy of Political and Social Science
578: 1–173.
Farrington, D. P. and Welsh, B. C. (2006). ‘A Half-Century
of Randomized Experiments on Crime and Justice’. Crime
and Justice: A Review of Research 34: 55–132.
Fixsen, D. L., Blase, K. A., Naoom, S. F., and Wallace, F.
(2009). ‘Core Implementation Components’. Research on
Social Work Practice 19(5): 531–540.
Foxcroft, D. R. (2014). ‘Can Prevention Classification Be
Improved by Considering the Function of Prevention?’.
Prevention Science 15(6): 818–822.
Article Policing 283
Gest, T. (1997). A Taxpayer’s Guide to Crime and
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
Punishment. U.S. News & World Report, 21 April 1997,
p. 38.
Gest, T. (2001). Crime and Politics: Big Government’s Erratic
Campaign for Law and Order. New York: Oxford
University Press.
Goldblatt, P and Lewis, C. (eds). (1998). Reducing Offending:
An Assessment of Research Evidence on Ways of Dealing
with Offending Behaviour. London: Home Office.
Greenwood, P. W. (2006). Changing Lives: Delinquency
Prevention as Crime-Control Policy. Chicago: University
of Chicago Press.
Haggerty, K. D. (2008). ‘Book Review of Evidence-Based
Crime Prevention’. Theoretical Criminology 12(1): 116–
121.
Hope, T. (2005). ‘Pretend It Doesn’t Work: The “Anti-
Social” Bias in the Maryland Scientific Methods Scale’.
European Journal on Criminal Policy and Research
11(3–4): 275–296.
Hough, M. (2010). ‘Gold Standard or Fool’s Gold? The
Pursuit of Certainty in Experimental Criminology’.
Criminology & Criminal Justice 10: 11–22.
Juni, P., Witschi, A., Bloch, R., and Egger, M. (1999). ‘The
Hazards of Scoring the Quality of Clinical Trials for Meta-
Analysis’. Journal of the American Medical Association 282:
1054–1060.
Lee, S., Aos, S., Drake, E. K. et al. (2012). Return on
Investment: Evidence-Based Options to Improve Statewide
Outcomes. Olympia, WA: Washington State Institute for
Public Policy.
Lum, C. M. and Koper, C. S. (2017). Evidence-Based
Policing: Translating Research into Practice. New York:
Oxford University Press.
MacKenzie, D. L. (2000). ‘Evidence-Based Corrections:
Identifying What Works’. Crime & Delinquency 46:
457–471.
Maguire, M. (2004). ‘The Crime Reduction Programme in
England and Wales: Reflections on the Vision and the
Reality’. Criminal Justice 4(3): 213–237.
Martin, P. and Mazerolle, L. (2015). ‘Police Leadership in
Fostering Evidence-based Agency Reform’. Policing 10:
34–43.
Mears, D. P. (2007). ‘Towards Rational and Evidence-Based
Crime Policy’. Journal of Criminal Justice 35(6): 667–682.
Mears, D. P. (2010). American Criminal Justice Policy: An
Evaluation Approach to Increasing Accountability and
Effectiveness. New York: Cambridge University Press.
Moher, D., Jadad, A. R., Nichol, G. et al. (1995). ‘Assessing
the Quality of Randomized Controlled Trials: An
Annotated Bibliography of Scales and Checklists’.
Controlled Clinical Trials 16(1): 62–73.
Nagin, D. S. and Weisburd, D. (2013). ‘Evidence and Public
Policy: The Example of Evaluation Research in Policing’.
Criminology & Public Policy 12: 651–679.
284 Policing Article
National Academies of Sciences, Engineering, and
Medicine (2017). Communicating Science Effectively: A
Research Agenda. Washington, DC: National Academies
Press.
National Research Council (2005). Improving Evaluation of
Anticrime Programs. Washington, DC: National
Academies Press.
Nutley, S. and Homel, P. (2006). ‘Delivering Evidence-Based
Policy and Practice: Lessons from the Implementation of
the UK Crime Reduction Programme’. Evidence and
Policy 2(1): 5–26.
Petrosino, A., Boruch, R. F., Soydan, H., Duggan, L., and
Sanchez-Meca, J. (2001). ‘Meeting the Challenges of
Evidence-Based Policy: The Campbell Collaboration’.
Annals of the American Academy of Political and Social
Science 578: 14–34.
Petrosino, A. and Lavenberg, J. (2007). ‘Systematic Reviews
and Meta-Analyses: Best Evidence on “What Works” for
Criminal Justice Decision Makers’. Western Criminology
Review 8: 1–15.
Petticrew, M. and Roberts, H. (2006). Systematic Reviews in
the Social Sciences: A Practical Guide. Oxford, UK:
Blackwell.
Sampson, R. J. (2010). ‘Gold Standard Myths: Observations
on the Experimental Turn in Criminology’. Journal of
Quantitative Criminology 26(4): 489–500.
Sampson, R. J., Winship, C., and Knight, C. (2013).
‘Translating Causal Claims: Principles and Strategies for
Policy-Relevant Criminology’. Criminology & Public
Policy 12: 587–616.
Shadish, W., Cook, T. D., and Campbell, D. T. (2002).
Experimental and Quasi-Experimental Designs for
General Causal Inference. Boston: Houghton-Mifflin.
Sherman, L. W. (1997a). Introduction: The Congressional
Mandate to Evaluate. In Sherman, L. W., Gottfredson, D.
C., MacKenzie, D. L., Eck, J. E., Reuter, P., and Bushway
S. D. Preventing Crime: What Works, What Doesn’t,
What’s Promising, chap. 1. Washington, DC: Office of
Justice Programs, pp.1–22.
Sherman, L. W. (1997b). Thinking about Crime Prevention.
In Sherman, L. W., Gottfredson, D. C., MacKenzie, D. L.,
Eck, J. E., Reuter, P., and Bushway S. D. Preventing Crime:
What Works, What Doesn’t, What’s Promising, chap. 2.
Washington, DC: Office of Justice Programs, pp.1–32.
Sherman, L. W. (1998). Evidence-Based Policing.
Washington, DC: Police Foundation.
Sherman, L. W. (2013). The Rise of Evidence-Based
Policing: Targeting, Testing, and Tracking. In Tonry, M.
(ed), Crime and Justice in America: 1975–2025. Crime and
Justice: A Review of Research, vol. 42. Chicago: University
of Chicago Press, pp. 377–451.
Sherman, L. W., Farrington, D. P., Welsh, B. C., and
MacKenzie, D. L. (eds). (2002). Evidence-Based Crime
Prevention. New York: Routledge.
B. C. Welsh and A. B. Wexler
Sherman, L. W., Farrington, D. P., Welsh, B. C., and
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
MacKenzie, D. L. (eds). (2006). Evidence-Based Crime
Prevention, rev. edn. New York: Routledge.
Sherman, L. W. and Gottfredson, D. C. (1997). Appendix:
Research Methods. In Sherman, L. W., Gottfredson, D.
C., MacKenzie, D. L., Eck, J. E., Reuter, P., and Bushway
S. D. Preventing Crime: What Works, What Doesn’t,
What’s Promising. Washington, DC: Office of Justice
Programs, pp. 1–8.
Sherman, L. W., Gottfredson, D. C., MacKenzie, D. L., Eck,
J. E., Reuter, P., and Bushway, S. D. (1997). Preventing
Crime: What Works, What Doesn’t, What’s Promising.
Washington, DC: Office of Justice Programs.
Sherman, L. W., Gottfredson, D. C., MacKenzie, D. L., Eck,
J. E., Reuter, P., and Bushway, S. D. (1998). Preventing
Crime: What Works, What Doesn’t, What’s Promising.
Research in Brief, July. Washington, DC: National
Institute of Justice.
Shonkoff, J. P. and Bales, S. N. (2011). ‘Science Does Not
Speak for Itself: Translating Child Development Research
for the Public and Its Policymakers’. Child Development
82(1): 17–32.
Stafford, J. and Silverlock, L. (2000). What Works and What
Makes What Works Work? In Ballintyne, S, Pease, K., and
McLaren, V. (eds), Secure Foundations: Key Issues in
Crime Prevention, Crime Reduction and Community
Safety. London: Institute for Public Policy Research, pp.
91–101.
Stockard, J. and Wood, T. W. (2017). ‘The Threshold and
Inclusive Approaches to Determining “Best Available
Evidence”: An Empirical Analysis’. American Journal of
Evaluation 38(4): 471–492.
Telep, C. W. (2017). ‘Police Officer Receptivity to Research
and Evidence-Based Policing: Examining Variability
within and across Agencies’. Crime & Delinquency 63:
976–999.
Telep, C. W. and Lum, C. M. (2014). ‘The Receptivity of
Officers to Empirical Research and Evidence-Based
Policing: An Examination of Survey Data from Three
Agencies’. Police Quarterly 17(4): 359–385.
Telep, C. W. and Weisburd, D. (2016). Policing. In
Weisburd, D., Farrington, D. P., and Gill, C. E. (eds),
What Works in Crime Prevention and Rehabilitation:
Lessons from Systematic Reviews. New York: Springer,
pp. 137–168.
Tilley, N. (2000). The Evaluation Jungle. In Ballintyne, S.,
Pease, K., and McLaren, V. (eds), Secure Foundations: Key
Issues in Crime Prevention, Crime Reduction and
Community Safety. London: Institute for Public Policy
Research, pp. 115–130.
Tonry, M. and Farrington, D. P. (1995). Strategic
Approaches to Crime Prevention. In Tonry, M. and
Farrington, D. P. (eds), Building a Safer Society:
Strategic Approaches to Crime Prevention. Chicago:
University of Chicago Press, pp. 1–20.
Scientific and public policy discourse on crime prevention
U.S. House of Representatives (1996). Conference Report
104-378. 104th Congress, 1st Session, 27 April 1996.
van Dijk, J. J. M. and de Waard, J. (1991). ‘A Two-
Dimensional Typology of Crime Prevention Projects:
With a Bibliography’. Criminal Justice Abstracts 23:
483–503.
Weimer, D. L. and Friedman, L. S. (1979). Efficiency
Considerations in Criminal Rehabilitation Research:
Costs and Consequences. In Sechrest L, White, S. O.,
and Brown, E. D. (eds), The Rehabilitation of Criminal
Offenders: Problems and Prospects. Washington, DC:
National Academy of Sciences, pp. 251–272.
Weisburd, D., Farrington, D. P., and Gill, C. E. (eds).
(2016). What Works in Crime Prevention and
Rehabilitation: Lessons from Systematic Reviews. New
York: Springer.
Weisburd, D., Lum, C. M., and Petrosino, A. (2001). ‘Does
Research Design Affect Study Outcomes in Criminal
Article Policing 285
Justice?’. Annals of the American Academy of Political
Downloaded from https://academic.oup.com/policing/article-abstract/13/3/271/5308392 by Pontifica Universidad Catolica de Peru user on 21 July 2020
and Social Science 578: 50–70.
Welsh, B. C., Peel, M. E., Farrington, D. P., Elffers, H., and
Braga, A. A. (2011). ‘Research Design Influence on Study
Outcomes in Crime and Justice: A Partial Replication
with Public Area Surveillance’. Journal of Experimental
Criminology 7(2): 183–198.
Wilson, D. B. (2001). ‘Meta-Analytic Methods for
Criminology’. Annals of the American Academy of
Political and Social Science 578: 71–89.
Wilson, D. B. (2009). ‘Missing a Crucial Piece of the Pie:
Simple Document Search Strategies Inadequate for
Systematic Reviews’. Journal of Experimental
Criminology 5(4): 429–440.
Zane, S. N. and Welsh, B. C. (2018). ‘Toward an “Age of
Imposed Use”? Evidence-Based Crime Policy in a Law
and Social Science Context’. Criminal Justice Policy
Review 29(3): 280–300.