See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/326990671

From policy analysis to policy analytics

Chapter · May 2018

DOI: 10.1332/policypress/9781447334910.003.0018

CITATION READS

1 694

2 authors:

Justin Longo Kathleen Mcnutt

University of Regina University of Regina
30 PUBLICATIONS   165 CITATIONS    36 PUBLICATIONS   316 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Citizen Science View project

Global Matrix on the Physical Activity of Children and Youth View project

All content following this page was uploaded by Justin Longo on 21 December 2018.

The user has requested enhancement of the downloaded file.

page 367

1 EIGHTEEN
2
3
4 From policy analysis to
5
6
policy analytics
7
8
Justin Longo and Kathleen McNutt
9
10
11
12
13 Introduction
14
15 Public policy decisions that are based on rigorous analysis and the best available
16 evidence should better address public problems than policy based on anecdote,
17
belief, or inaccurate or partial data (Quade, 1975). The Policy Sciences (Lerner &
18 Lasswell, 1951) stands as a starting point for the modern policy analysis movement,
19
offering an integrated, multidisciplinary approach to the study of public problems
20
and the development of rational solutions based on careful analysis of evidence.
21
22
Harold Lasswell’s original conceptualization of the policy sciences sought to
23
distinguish analysis from political decision-making and to position policy analysis
as a foundation for good governance (Lasswell, 1951). From those origins, policy
24
25 analysts have been traditionally tasked with providing scope and precision to the
26 definition of policy problems, collecting and analyzing evidence, supporting
27 decision-making with fearless advice, faithfully implementing decisions, and
28 objectively overseeing the evaluation of how effective policy interventions are.
29 Despite significant advances during the first half-century of the policy analysis
30 movement, coming of age in the late 1960s and early 1970s, debates over the
31
real, perceived, and proposed role of policy analysis have coloured the profession’s
32
latter years (Dryzek, 1990[[not in references, is 1994]]; Stone, 1988). While
33
technical policy analysis rooted in quantitative methods became increasingly
34
sophisticated during the 1970s and 1980s, high-profile failures exposed the
35
36
limits of positivist policy analysis (May, 1992). Coupled with the perceived
37 inability of quantitative policy analysis to solve complex public problems, critics
38 of positivism argued that the attempt to model social interactions on the natural
39 sciences was a misguided form of technocracy (Amy, 1984), that policy wisdom
40 should be seen as more than the results of data impressively distilled (Meltsner,
41 1976; Prince, 2007; Wildavsky, 1978), and that positivism was fundamentally
42 incapable of dealing with complex problems in a democracy (Fischer, 1995). The
43 “malaise of the policy sciences” can be traced to an overemphasis of positivist,
44 neo-classical economics for understanding human behaviour, the increasing
45 complexity of policy problems that policy analysis has been incapable of solving,
46
and the divergence of policy analysis technocracy and democratic values (deLeon,
47
48
1994, p. 82). The implementation problem, where the intentions and objectives

367
page 368 Policy analysis in Canada

1 of policymakers fail to materialize at the point of delivery or enforcement, has

2
highlighted how positivist models do not adequately accommodate autonomous
3
human behaviour and situational judgement (Pressman & Wildavsky, 1973).
4 Yet countervailing forces might be emerging that can finally answer some of
5 these critiques of positivism and situate quantitative policy analysis firmly in the
6 service of improved policymaking. If part of the explanation for the diminished
7 dominance of positivist policy analysis can be found in a lack of data and limited
8
technological capacity (Morçöl, 2001), then policy analysis might begin to live
9
10
up to its promise if data systems were more robust and computational power was
11 more widely available to analysts and decision-makers. Recent technological
12 advances in our ability to capture, store, and process data—in volumes previously
13 unimaginable, from ubiquitous sources, with continuous flow, observed through
14 multiple channels—coupled with increased capacity to manage, analyze, and
15 understand these new data sources, have the potential to revive the positivist
16 tradition in policy analysis and, through it, how data and policy analysis can
17
better support policymaking (Bankes, 1993[[not in references]]; Longo, 2015).
18 Those same technologies can also serve to connect policy formulation with
19
implementation and evaluation processes in a continuous and real-time cycle of
20
ideas, experimentation, learning, and reformulation (Pirog, 2014). While referred
21
22
to elsewhere inter alia as “big data in public affairs” (Mergel et al., 2016[[page
23
no?]]), “computational social sciences” (Lazer et al., 2009[[page no?]]), and
“policy informatics” (Johnston, 2015[[page no?]]), we use the term “policy
24
25 analytics” (Daniell et al., 2016; De Marchi et al., 2016; Tsoukias et al., 2013) to
26 refer to the combination of new sources and forms of policy-relevant data with
27 the use of new analytics techniques and capacity to support policymaking that
28 better addresses public problems.
29 This chapter is an attempt to situate policy analytics within the policy analysis
30 tradition, arguing that we are now at a technology inflection point that offers
31
a response to some post-positivist critiques and a path towards an enhanced
32
policy analysis model. In the next section, we briefly describe the policy analysis
33
movement and its post-positivist reformation, and then sketch the significant
34
technological changes that provide the foundation for the policy analytics
35
36
approach. This is followed by some emergent examples of policy analytics in
37 practice, and potential applications in the future. We conclude with some caveats
38 that stand between the potential of policy analytics and its widespread adoption.
39
40
From policy analysis to policy analytics
41
42 Lasswell’s original conceptualization for the policy sciences distinguished
43 policy analysis from political analysis, stressing that good analysis could serve
44 as a foundation of good governance without becoming enmeshed in political
45 calculation (Lasswell, 1951). As an integrated, multidisciplinary approach to the
46
study of public problems and their potential solutions, policy analysis embodies
47
48
a rational, scientific approach to governance and aims, in the context of its

368
page 369 From policy analysis to policy analytics

1 subservience to democratic decision-making, to contribute to better decisions

2
than would be made in the absence of such analysis (Quade, 1975). Combining
3
social science theory with an emphasis on quantitative methods extracted from
4 economics and operations research, policy analysis developed in the positivist
5 tradition. During the first quarter century of the policy analysis movement,
6 techniques derived from this tradition—for example, mathematical modelling,
7 descriptive statistics, statistical inference testing, cost–benefit analysis, linear
8
programming, stochastic modelling, Bayesian analysis, quasi-linearization,
9
10
invariant embedding, and general systems theory—became staples of the theory
11 and practice of policy analysis (Radin, 2000[[not in references, is 2006]]).
12 Theoretical and applied advances continued to be made into the 1980s using
13 mathematical modelling and computer programming (Quade, 1980). Later
14 developments in systems dynamics (Forrester, 1971; Meadows et al., 1972[[not
15 in references, is Meadows and The Club of Rome]]), Integrated Assessment
16 Models (IAMs) for integrating science with policy (Parson & Fisher-Vanden,
17
1997), and increasingly sophisticated simulation tools (Wolfson, 2015) further
18 advanced the positivist approach to policy analysis. By the 1980s, we were
19
“arguably in a golden age of computer modelling for policy analysis” based on
20
advances in computational power, computer modelling, and simulations (Bankes,
21
22
[[not in references]]1993, p. 435).
23
Despite these theoretical, technical, and professional advances in the science of
the policy sciences, post-positivist critiques of positivist policy analysis began to
24
25 emerge around this quarter-century mark with questions about the capacity of
26 the positivist approach to adequately operate within the context of a democracy.
27 Positivism also came under closer scrutiny with the increased awareness of the
28 implementation problem (Pressman & Wildavsky, 1973). As a practical concern,
29 implementation focuses on actions by civil servants that serve to achieve the
30 objectives identified in the policy formation process. The challenge of the
31
implementation process highlighted that there is not always a clear connection
32
between the stated intentions and objectives of policymakers and the realization
33
of that intent in delivery or enforcement. Judgement was also found to be critical,
34
especially at the level of the “street level bureaucrat” (Lipsky, 1971) where a
35
36
contextual appreciation of the policy environment was more important for solving
37 problems than theoretical elegance or ideological ambitions. The implementation
38 challenge has also been extended to the discretionary actions of individuals in
39 society, where cumulative action might cause a policy objective to be missed
40 if compliance is difficult to detect or enforce at the level of the individual, but
41 significant when aggregated across many actors (Dobell et al., 2001).
42 As questions arose as to the relevance and track record of positivist policy analysis
43 for understanding questions involving human behaviour, values, and complexity,
44 post-positivists argued that technical mastery should be balanced with softer skills
45 (Fischer, 1980; 2003). Based on an evolving appreciation of the actual work of
46
the policy analyst, and the difficulty of ensuring alignment between policy goals
47
48
and individual behaviour, a revised set of core policy analysis skills began to

369
page 370 Policy analysis in Canada

1 emerge. These included case study methods, interviewing and qualitative data
2
analysis, organizational culture analysis, political feasibility analysis, participatory
3
design, stakeholder and citizen engagement, and small-group facilitation (Radin,
4 2000[[not in references, is 2006]]).
5
6
7
The emergence of big data
8
An additional, separate critique of quantitative policy analysis that has shadowed
9
10
the movement since its inception—a critique shared by both positivists and post-
11 positivists—was the frequent lack of robust and complete data (Morgan et al.,
12 1992). With the advent and growth of the internet, however, and the spreading
13 ubiquity of information and communication technologies (ICTs), policy analysis
14 now faces a data deluge in the form of “big data” that has increased the volume,
15 speed, and range of policy-relevant evidence available to policymakers (Kitchin,
16 2014a). These data accumulate from a variety of sources, including the signals
17
that individuals generate through their everyday activities using communication
18 devices such as smartphones (through both passive and active data generation)
19
(Laurila et al., 2012), consumer products connected to the internet of everything
20
(IoE) (Chan et al., 2008; Gubbi et al., 2013), personal wearable technology
21
22
(Estrin & Sim, 2010), electronic transaction cards (Hasan et al., 2013; Zhong
23
et al., 2015), sensors and public infrastructure (Chen et al., 2012; Zaslavsky et
al., 2013), web search queries (Choi & Varian, 2012), web traffic (Adamic &
24
25 Glance, 2005; Watts, 2007; Adar et al., 2008), and social media (Lewis et al.,
26 2008; Tufekci, 2014). Massive amounts of data are now generated daily through
27 the activities of individuals, from their interactions with web services and social
28 media platforms, purchasing behaviour and service usage revealed through
29 electronic transaction cards, movement and interaction captured through mobile
30 smartphones, behavioural choices measured through IoE consumer products, a
31
range of measurements captured by in situ and personal sensors, satellite remote
32
sensing, counters and smart meters, and interactions with devices and control
33
technology. The accumulation of these data, and associated metadata such
34
as geolocation information and time and date stamps, results in a previously
35
36
unimaginable amount of data, measured with phenomenal precision, taken from
37 multiple perspectives and captured continually in real-time. Advances in data
38 storage technologies now make it possible to preserve increasing amounts of data,
39 and faster data transfer rates allow for cloud storage at low cost.
40 Corporations (beyond the technology firms that stand to profit from increased
41 sales of collection, storage, and analysis capacity) are using these uncountable
42 bits of data to better understand people’s behaviour, for example, to encourage a
43 user to return to a webpage, to understand how content read online might affect
44 someone’s emotional state, or what motivates people to post new social media
45 content, click on an ad, buy a product (and a subsequent product), purchase a
46
service, or watch a movie because they watched a similar one (McAfee et al.,
47
48
2012; Kramer et al., 2014). New store location decisions, airline ticket pricing,

370
page 371 From policy analysis to policy analytics

1 and stock market trades are all being influenced by data analytics. Big data are also
2
being used to judge who is a worthy credit risk, who would be a good person to
3
hire, and who would make an ideal mate (Mayer-Schönberger & Cukier, 2013).
4
5
6
Policy analytics
7 As the combination of big data and data analytics is promoted outside of
8
government, the growing volumes of digital data increasingly available provide
9
10
a platform for policy analysis based on precise, continuous, dynamic data,
11 generated from observed behaviour. Policy analysis traditionally involves a
12 process of extrapolating from data collected from some representative samples,
13 or developing an aggregate picture of an average person and then developing
14 policies or interventions based on that composite. Policy analytics, by taking
15 advantage of this new data and computational power, can provide a much richer
16 and more precise view of a system and the individual agents in it. Even when
17
traditional policy approaches had the benefit of data on nearly everyone—from
18 the census, for example—that data provided only a point-in-time snapshot,
19
with a delay between the time of data capture and the release of the statistics.
20
Policy analytics, alternatively, can be based on massive amounts of continually
21
22
updated, real-time data from multiple sources. Traditional data approaches do
23
not tell us how people are dynamically interacting with their world, whereas
policy analytics offers the promise of revealing how agents and systems react to
24
25 changes in environmental conditions and variables, extending to the possibility of
26 experimental manipulation of policy instrument variables in a continual loop of
27 policy formulation, implementation, and evaluation. Traditional data collection
28 largely relies on respondents to cooperate with researchers, raising challenges
29 such as low response rates and respondent bias, while policy analytics eliminates
30 this researcher/respondent interaction by focusing on the direct observation of
31
behaviour. Lastly, while data collection in traditional opinion research centres
32
on what respondents prefer or believe, big data measures what people actually
33
do, revealing unfiltered insights into policy-relevant human behaviour (Pentland,
34
2012a).
35
36
Governments, since there have been government, have always collected,
37 generated, and compiled vast amounts of data. In doing the things that governing
38 entails—for example, collecting vital statistics, administering the tax system,
39 recording government operations activity, managing public infrastructure and
40 natural resources, surveying and recording public and private lands, processing
41 regulatory requirements, or managing social service delivery—a wealth of data
42 is amassed (Cate, 2008). Through census and survey work by public statistics
43 agencies, or the monitoring of system conditions across a range of policy domains
44 (from the natural environment to health systems), data collection serves to fuel
45 policy-oriented research (Fellegi, 1996). Governments have always collected
46
immense quantities of data through these administrative and operational actions.
47
48
Throughout the past 65 years of the policy movement, a period that coincides with

371
page 372 Policy analysis in Canada

1 the development and spread of digital ICTs (Longo, 2015), these data collection
2
and analysis activities in governments have increasingly been computerized, and
3
in recent years most new data is “born digital,” supplemented by the ongoing
4 digitization of previously collected data (Rogers, 2014).
5 With this growth in the volume of digital data and capacities for mining insights
6 from it, calls for governments to provide open, easy-to-use and largely free-of-
7 charge access to public data have grown in recent years (Lathrop & Ruma, 2010;
8
Ginsberg, 2011). These calls have been propelled by Sir Tim Berners-Lee, the
9
10
inventor of the World Wide Web, who challenged governments to share their data
11 repositories through an open, linked architecture in an often-cited presentation
12 at the TED (Technology, Entertainment, Design) [[correct?]] conference
13 (Berners-Lee, 2009). When used by those outside government, open government
14 data can yield third-party-developed citizen services such as public transit route
15 planners, expanded policy networks for knowledge creation in external policy
16 advocacy organizations, and great transparency and public accountability (Longo,
17
2011). When used in the internal-to-government policy system, this increased
18 data volume can become the basis of an enhanced policy analytic capacity within
19
government (Longo, 2015).
20
Governments now have the opportunity to combine the rapidly accumulating,
21
22
new sources of big data with traditional forms of highly structured administrative
23
data, to provide a clear, real-time, and dynamic picture of human behaviour in
the policy environment. This expanded data availability, coupled with increased
24
25 data analytics capacity, provides the foundation for policy analytics. Many of these
26 new analytic methods—such as complex computational methods that emphasize
27 correlations and patterns rather than the more traditional scientific processes of
28 developing substantive theories through hypothesis testing—have the potential to
29 improve public welfare by providing evidence-based analysis on emergent issues
30 and challenges. Policy analytics could help to reduce uncertainty in decision-
31
making, identify cost savings and new revenue streams, improve service delivery,
32
and unlock new opportunities for interventions. Big data and policy analytics
33
could be used for decision support, improving public services, and engaging
34
citizens and stakeholders. Big data technology can supplement government’s
35
36
existing toolbox and provide new analytical approaches for managing public
37 issues, engaging in policy micro-experimentation, monitoring performance across
38 programs, and improving service delivery outcomes. Predictive analytics promise
39 data-driven forecasts for the early identification of trends and opportunities for
40 intervention. Decision support systems analyze real-time information to identify
41 previously hidden patterns and correlations. The granularity of big data allows
42 for the personalization of public services, from healthcare and social services to
43 power consumption, to better meet client needs (Hondula et al., 2017).
44
45
46
47
48

372
page 373 From policy analysis to policy analytics

1 Approaches to policy analytics

2
3
Policy analytics can take a range of approaches. Perhaps the easiest, first line of
4 analysis lies in social media monitoring or listening to analyze and respond to
5 citizen’s preferences, experiences, articulated values, and behaviours (Grubmüller
6 et al., 2013). Social listening involves searching and monitoring social media for
7 words, phrases, hashtags, or mentions of government accounts or persons. Simple
8
searches can be conducted through the interface of social media platforms like
9
10
Twitter or tools such as twXplorer (https://twxplorer.knightlab.com/). This
11 approach is becoming increasingly popular with governments seeking to gauge
12 public opinion, as it provides a more nuanced appreciation of citizen attitudes and
13 how these attitudes change over time (Paris & Wan, 2011). With approximately
14 8 million Twitter accounts in Canada sending more than 10 million tweets per
15 day, governments in Canada can use social media as a form of citizen engagement
16 either through passive listening and detection of sentiment, or through active
17
promotion of topics (Longo, forthcoming[[details/published yet?]]). Further
18 analysis can centre on determining deeper sentiment and meaning, and clustering
19
opinion to reveal network properties. Till et al. (2014) studied how the policy
20
analyst could attempt to make sense of public opinion as represented in the
21
22
massive amounts of free-form unstructured text in social media using latent
23
semantic analysis and self-organizing maps for clustering similar ideas. Social
listening and sentiment analysis reveal the real potential for advanced forms of
24
25 public engagement. While work around sentiment analysis provides some insight
26 into citizens’ interests, preferences and values (Schintler & Kulkarni, 2014), this
27 research is in early stages of development. However, the emergence of social
28 collaborative technologies such as social networking sites, blogs, and wikis has
29 spawned numerous virtual communities that facilitate interaction between
30 individuals with similar experiences and interests (McNutt, 2014). Social media
31
websites are increasingly positioned as a link between citizen and government,
32
clients and organizations, and consumers and merchants. This type of engagement
33
increases government’s ability to allow citizens to participate in decision-making
34
in concrete ways, thus increasing accountability and allowing government to
35
36
become more responsive to citizens’ preferences.
37
38 Predictive analytics
39
40 A deeper transformation of the policy analysis tradition into policy analytics
41 starts with the new wealth of big data and combines it with the science of data
42 analytics (LaValle et al., 2011), placed in the context of public policymaking.
43 As much as technology firms use the term “big data” as a marketing ploy, with
44 no clear distinction between “big data” and “a lot of data,” there is not a clear
45 distinction between “data analysis” and “data analytics.” However, data analytics
46
carries connotations of working with data that is not well ordered in rows and
47
48
columns or relational databases, and is of sufficient volume, velocity, and variety

373
page 374 Policy analysis in Canada

1 to be beyond traditional data analysis techniques (Davenport et al., 2012). In

2
terms of techniques applied in that analysis, whereas descriptive and inferential
3
statistics are the standards of data analysis, data analytics includes methods such
4 as text mining for sentiment analysis, social media analytics to assess content and
5 network structure, audio and video analytics to reveal patterns of behaviour or
6 predictions of responses to particular stimuli, and predictive analytics (Gandomi
7 & Haider, 2015). Additional advances in data visualization can yield valuable new
8
insights and serve as helpful supports for decision-making, taking complex data
9
10
and revealing patterns and connections (Lindquist, 2015).
11 Predictive analytics can serve as an input into framing a policy problem before
12 it is appreciated as such, indicating where a need is being unmet, or where an
13 emerging problem might be countered early. It involves assessing past performance
14 to reveal a probable outcome that can be expected if everything else remains
15 constant. Also referred to as forecasting, “predictive analytics” has become the
16 more common term due largely to the high-profile use of “predictive policing,”
17
which involves the use of predictive analytic techniques in law enforcement to
18 identify potential crimes, offenders, and victims, and to direct police resources
19
proactively (Perry et al., 2013). Similar concepts such as credit scores have long
20
been used to predict an individual’s risk of defaulting on a loan. Predictive analytics
21
22
involves the application of quantitative techniques including modelling, machine
23
learning, and data mining to identify likely targets for policy interventions, to
prevent possible future policy problems, or understand previous events (for
24
25 example, detecting income tax fraud). By identifying trends and patterns in
26 historical and contemporary data, and modelling relationships among variables,
27 predictions about future risks and opportunities can be forecast as a guide for
28 decision-making for uncertain outcomes. This is the function of policy analysis
29 in general, but predictive analytics involves finer granularity at the level of the
30 individual.
31
32
33 Real-time policy experimentation
34
In traditional policy analysis, framed using the policy cycle, problem definition,
35
36
analysis, solution identification, decision-making, implementation, and evaluation
37 (cycling back to future analysis) happen in discrete stages, with explicit time lags
38 between them to allow for the accumulation of enough evidence upon which to
39 base subsequent analysis. Data are collected so as to better understand the policy
40 problem. Evidence is collected to inform the analysis and solution choice. Once
41 a policy choice is determined, it is implemented. Data collection then follows
42 the implementation of the policy to evaluate whether it had the desired effect. A
43 real-time experimental policy analytics cycle, in contrast, would take advantage of
44 new big data sources, coupled with data analytics techniques, bringing together
45 the discrete stages of the policy cycle into one continuous process. While a
46
policy problem is being observed, interventions could also be underway using the
47
48
same devices that are used to collect the data, with their impact on the problem

374
page 375 From policy analysis to policy analytics

1 becoming part of the evidence base for further modifying the policy variables.
2
These further modifications would also be observed for their impact as the
3
system response to the policy intervention moved closer to the policy target or
4 equilibrium (Esperanza & Dirk, 2014). Because the implementation environment
5 is directly linked to the formulation process, and the technology infrastructure
6 for communicating new policy signals to the system significantly reduces the
7 time lag, adaptation to the policy signals will show up immediately in the data
8
collected from the system. Public policy should be improved as governments
9
10
manipulate input variables in law, markets, architecture, social norms, and
11 information (Lessig, 2006; Johnston & Hondula, 2015), measuring with fine-
12 grained accuracy the impacts correlated with those changed variables in order
13 to propose, pilot, test, evaluate, and redesign policy interventions (Haynes et al.,
14 2012; Paquet, 2009). Policy experimentation eliminates the lag between policy
15 idea and feedback and addresses the critique that policy research is too slow to
16 be relevant to policymakers (Isett et al., 2016).
17
This micro-experimental approach also provides an opportunity to reduce the
18 scale of the policy intervention, to avoid system shocks and gradually calibrate
19
the policy to achieve the target or equilibrium. The traditional approach to
20
policy formation has been oriented towards big initiatives, where governments
21
22
analyze, assess, consider, think, decide, and then announce. Rather than focus
23
on implementing large-scale policy changes based on our best understanding of
the system, a real-time experimental policy analytics cycle offers the possibility of
24
25 small-scale pilot interventions whose effects can be precisely observed in real time.
26 Small-scale experimentation allows for small mistakes in the service of learning
27 as the policy is scaled up. It also addresses the implementation challenge, noted
28 above, by linking in time and space the intention of the policy and its effect on
29 the ground—reducing the distance between Washington and Oakland, as it were
30 (Pressman & Wildavsky, 1973).
31
Policy experimentation is not a new approach; it builds on the idea of policy
32
incrementalism (Lindblom, 1959) with examples of trials, experiments, and pilots
33
of varying degrees of ‘scientificness’ (Breckon, 2015). Some recent experiments
34
show its potential for success (Bond et al., 2012; Cabinet Office, 2012). But
35
36
earlier experiments had a number of limitations: the experimental field was far
37 removed from the policy interface, and the experimental conditions were difficult
38 to control (Berk et al., 1985). What is different in the policy analytics era is the
39 technology that connects the policy analytic system (as a locus of analysis and
40 policy signal) and the citizen (as generator of policy relevant data and target of
41 those policy signals).
42 Policy experimentation does raise ethical concerns. While medical randomized
43 control trials (RCTs) have developed good guidelines around informed consent
44 and the ethics of placebos, deception and blinding (Edwards et al. 1998), the
45 politics of policy experimentation in a democracy raise legal and ethical questions
46
about the unequal treatment of citizens in a true RCT. Many have questioned
47
48
the fairness of a democratic society giving a new intervention to some but not

375
page 376 Policy analysis in Canada

1 to others—whether providing a benefit or imposing a cost (Breckon, 2015).

2
The concept of equipoise—a state of not knowing which intervention is best,
3
or even whether any intervention is better than doing nothing—is the key
4 ethical requirement (London, 2009). This technical argument, however, does
5 not address the political fallout that may accompany news that a government
6 was offering a benefit to some regions and not others, or imposing a cost only
7 on some individuals.
8
9
10 Emergent examples and potential applications
11
12 What do these new approaches to policy analytics and real-time policy
13 experimentation look like in practice? New sources of data have the potential
14 to significantly change the study of social and political phenomena (Savage &
15 Burrows, 2007; Lazer et al., 2009). The combination of digital signals and new
16 analytic techniques can help to understand and predict behaviour in contexts
17
such as transportation, crime, energy use, migration, food safety, urban planning,
18 and public health (Rogers, 2013; Rogers et al., 2015).
19
Google Flu Trends (GFT) is perhaps the first, best-known example of a policy
20
analytic approach using web search “big data” for predicting a policy-relevant
21
22
event, in this case the public health implications of influenza-like illness (McAfee
23
et al., 2012). GFT was designed to predict flu prevalence rates by data mining
records from flu-related internet searches, specifically Google searches related to
24
25 influenza-like symptoms. First launched in 2008 but using data from as early as
26 2003, GFT analyses user searches of terms such as “flu,” “fever,” and “cough”
27 to predict the presence of influenza-like illness in a population compared to a
28 baseline level for a specific region. These predictions would then help public health
29 authorities plan for a possible impending influenza pandemic. Early estimates
30 suggested that the analysis could predict regional outbreaks of the flu up to ten
31
days before traditional methods used by the United States Centers for Disease
32
Control and Prevention (CDC) (Helft, 2008), and were 97 per cent accurate
33
compared to CDC data (Ginsberg et al., 2009). While initially successful, GFT
34
also illustrates potential downsides in taking a policy analytics approach. In 2013,
35
36
the tracking system drastically overestimated the peak flu level compared to the
37 CDC’s prediction of doctor visits which was much more accurate (Butler, 2013).
38 One central problem was that user searches containing the word “flu” were
39 assumed to relate to influenza-like illnesses, when in fact people are not very good
40 at self-diagnosing influenza (Boulos et al., 2010). This leads to a large volume
41 of user-generated data that are not valid and reliable, as compared to directly
42 measured data. A second problem is that, because Google search is a commercial
43 product continually refined by its engineers, their algorithm modification efforts
44 aligned with Google’s business model and not the public health objectives of
45 GFT (Lazer et al., 2014).
46
Policy analytics has also been applied in select locations in an evolution from
47
48
high-occupancy vehicle (HOV) lanes to high-occupancy toll (HOT) lanes. HOV

376
page 377 From policy analysis to policy analytics

1 lanes take a policy analysis approach to rationing access to special highway lanes
2
to manage congestion, imposing static conditions for entry (for example, having
3
more than one passenger in a vehicle) regardless of demand. Fewer cars meet the
4 conditions required to use the HOV lanes, so traffic in those lanes usually moves
5 more quickly than in regular highway lanes. As drivers in the regular lanes see the
6 disparity in traffic flow, they are ostensibly incentivized to meet the conditions
7 required to use the HOV lane: for example, arranging to carpool with another
8
commuter. Tolls are another method for managing congestion on a roadway (in
9
10
addition to raising revenue to help offset its cost), though toll rates are usually
11 static. If tolls do change based on time of day or direction of traffic, these are
12 usually based on an a priori calculation involving expected demand, willingness
13 to pay, and political acceptance (a classic policy analysis approach).
14 HOT lanes, alternatively, take a policy analytic approach by taking in real-time
15 data from the highway environment to assess actual congestion and dynamically
16 adjusting the price to access the lane for a driver who does not meet the HOV
17
conditions. Earlier HOT lane approaches imposed a static toll (Burris and
18 Stockton, 2004), 1 but technology advances now mean that prices can respond
19
to changing demand and actual conditions on the road (perhaps suggesting a
20
more accurate acronym—HOST—for “high-occupancy smart toll” lanes). In Los
21
22
Angeles, a pilot project that saw the first use of onboard occupancy transponders
23
combined data from the toll and traffic systems with parking data, public transit
ridership data (riders earn toll charge credits), a violation processing system,
24
25 traffic overview cameras, dynamic messaging signs, and tolling point notification
26 lights to compute congestion and pricing recommendation analytics every five
27 minutes. Drivers are presented with dynamic pricing for entering the HOT
28 lanes, and decide whether they want to do so—in essence, revealing the demand
29 curve for access to the HOT lane (Cohen et al., 2016). Dynamic road pricing
30 has been successful in decreasing congestion and has been widely accepted by
31
users, and is now a permanent feature in Los Angeles (Schroeder et al., 2015). In
32
a similar experiment in San Diego, dynamic tolls lead to statistically significant
33
improvements in both peak-period utilization and volume distribution and were
34
superior to the previous fixed-fee structure (Supernak et al., 2003).
35
36
Additional examples are being experimented with, and stand as potential
37 opportunities for applied policy analytics. Smart electricity meters can incentivize
38 conservation behaviour and reduce peak-load demand by informing consumers of
39 differential electricity rates that are calculated in response to fluctuating demand
40 (Newsham and Bowker, 2010; Blumsack and Fernandez, 2012). An intriguing
41 possibility is in on-demand local public transportation services where, instead of
42 pre-determined routes and schedules, public transportation is arranged in response
43 to rider requests for carriage (Murphy, 2016). Big data approaches to monitoring
44 behaviour in teams, and adapting team dynamics and leadership to improve
45 performance, continues to be investigated (Pentland, 2012b). The principles
46
of nudge theory are being applied in dynamic ways that take advantage of the
47
48
powerful devices ubiquitously moving around us to measure the environment,

377
page 378 Policy analysis in Canada

1 along with individual behaviour and health conditions, and intervening by sending
2
information to the individual via devices such as their smartphone in order
3
to change a behaviour (Bert et al., 2014). Other areas of potential application
4 include financial services regulation (to give regulators an opportunity to spot
5 problems such as the mortgage-backed securities catastrophe earlier than they
6 did; Groll et al., 2015), monetary policy (including new approaches to measuring
7 the consumer price index and the velocity of money) (Armah, 2013), national
8
statistics collection (including a continuous “big data census”) (Struijs et al.,
9
10
2014), education (Williamson, 2016), healthcare (Murdoch and Detsky, 2013;
11 Andreu-Perez et al., 2015), disaster response (Huang and Xiao, 2015), and natural
12 hazards management (Hondula et al., 2017).
13
14
Conclusion
15
16 While the specific policy areas, analytical techniques, and policy interventions
17
of policy analytics are still emerging, the movement away from traditional data
18 collection and policy analysis methods towards the concept of policy analytics
19
as described above requires careful consideration. Much has been devoted to
20
the technical aspects of big data analytics. However, little attention has been
21
22
given as yet to the consequences for public policy and governance (Michael &
23
Miller, 2013). The ability of real-time data to provide critical information to
decision makers is enormous. Yet as governments consider investments in high-
24
25 performance computing and expand their policy analytics capacity, fundamental
26 political, technical, ontological, and methodological questions remain about the
27 appropriate use of big data and analytics in support of policymaking.
28 Much of the emerging commentary on the possible future of policy analytics
29 in government has come from two camps: optimism from private sector
30 technology firms with a vested interest in promoting investments in big data
31
collection, storage and analytic technology; and both enthusiasm and scepticism
32
from academia, civil society, and journalism on the possibilities and pitfalls in a
33
data-rich future. The supply of data will continue to grow as the presence and
34
reach of devices and sensors increases. The opportunities to work with that data
35
36
will expand as more vendors offer solutions and capacity. What remains unclear
37 is what governments should do in the face of these opportunities. The ability to
38 take advantage of this new wealth of data will require the building of capacity in
39 the public service, as well as the use of outside consultants, to manage, process,
40 analyze, and interpret it, and make it useful to support decision-making (Mergel
41 et al., 2016). Building this capacity will require training for current and future
42 public service policy analysts (Longo, 2015; Mergel, 2016). Decision makers in
43 government must become intelligent consumers, balancing their embrace of new
44 technology opportunities with appropriate scepticism of the claims of vendors and
45 service providers. Governments will face these pressing investment decisions in the
46
coming years even as more significant philosophical, even existential, questions
47
48
about the nature of political decision-making, policy advice, and governance

378
page 379 From policy analysis to policy analytics

1 continue to be debated. Will politicians demand more precise insights from their
2
advisors, and the electorate increasingly expect that evidence-supported decision-
3
making be applied to solving public problems? Or will previous versions of the
4 rejection of technocracy emerge in new forms (Parson, 2015), with evidence-
5 based policymaking further marginalized in our “post-truth” era of political-based
6 evidence-making (Harsin, 2015)?
7 Even if the capacity to use big data exists, getting access to the data will still
8
be a challenge for governments. It is a paradox of the big data era that privacy
9
10
concerns limit what data governments can collect on citizens, just as private sector
11 firms are expanding their data collection. Much of the data being generated
12 and collected through mobile and IoE devices, social media, web searches and
13 traffic, and transaction cards is the closely guarded property of the companies
14 that facilitate their use (Klein and Verhulst, 2017). Users appear willing to give
15 up significant amounts of their data to private companies in exchange for “free”
16 web services, loyalty points, and value-added data (Van Dijck, 2014), but object
17
to the collecting of private data by governments even when aimed at improving
18 public policy and service delivery (Kitchin, 2014b; Malomo & Sena, 2016[[not
19
in references, is 2017]]). These perceived privacy concerns present an even
20
greater challenge when real legal limits on what governments can collect further
21
22
constrain governments’ capacities to make use of big data sources (Cate, 2008).
23
These limits on the use of new sources of big data are occurring at the same
time that data specifically collected for policy analysis has declined in recent years
24
25 (Groves, 2006). For example, Canada has seen vigorous debate in recent years
26 over the right of the state to collect private information on citizens through the
27 national census (Green & Milligan, 2010). These concerns would certainly follow
28 suggestions for data collection efforts that venture into new platforms and devices
29 (McNeely & Hahm, 2014). One avenue that would allow for data collection
30 by governments at a massive scale, while reducing privacy concerns, involves
31
the deployment of government-controlled sensors that measure policy-relevant
32
phenomenon without identifying individuals (Kim et al., 2014).2
33
While the politics of policy analytics are being sorted out, deeper ontological
34
issues remain. Two critiques of policy analytics derive from earlier critiques
35
36
of positivism: that the “fact/value dichotomy” is not as clearly delineated as
37 positivists contend (Putnam, 2002); and that the empiricism of policy analytics is
38 impressive but is based on a still incomplete view of the world. While more, and
39 more accurate, evidence can improve our understanding and form the basis for
40 better policy, the means for collecting and interpreting evidence should never be
41 assumed to be value neutral. Rather, our evidence-gathering and interpretation
42 systems reflect choices that privilege what we care about and ignore what we
43 consider unimportant (6, 2004). Our choices in what we measure are influenced
44 by our values, and in turn that evidence influences what we value. As for the
45 completeness of the data, we should not conflate the size of the databases with
46
the representativeness of the data. Consider that part of policy-relevant data that
47
48
is collected from mobile smartphones, IoE devices, and transaction cards. It is

379
page 380 Policy analysis in Canada

1 a world populated by those who own and use those devices and make those
2
transactions. Despite the mesh of sensors, card readers, cell towers, cables, and
3
servers that act as the collection net for a range of data reflecting the choices,
4 actions, and behaviours of people in society, those without the right devices may
5 be rendered “digitally invisible” (Longo et al., 2017). As a result, policy based
6 primarily on device-derived data will be biased towards those owning the devices,
7 failing to reflect the reality of those not revealed in the data. The enthusiasm for
8
ubiquitous data and policy analytics rests on the widespread belief that large data
9
10
sets offer a higher form of intelligence, revealing objective and accurate truth.
11 However, if this movement fails to acknowledge some of the limitations of new
12 forms of data collection and analytics, the core critiques of post-positivism will
13 go unanswered (boyd and Crawford, 2012; Hitchcock, 2013).
14 Important methodological questions are also raised by the movement toward
15 policy analytics. Traditional public policy and public administration data
16 acquisition involves consistent data collection methods derived from the social
17
sciences. Whether quantitative or qualitative, data gathering is largely conducted
18 through carefully designed instruments. As a result, traditional public-sector
19
data sets—such as the census, and taxation, health, and education records—aim
20
to be both reliable and valid. Policy analytics proposes a very different research
21
22
proposition: mining accumulated data, as opposed to collecting data for analysis.
23
Instead of designing a research instrument to test a hypothesis, data mining
seeks to identify relationships (Wigan & Clarke, 2013). Apophenia—the seeing
24
25 of patterns in random data)—can lead analysts to identify correlations due to
26 the sheer size of the data, especially if they are unaware of the context (boyd &
27 Crawford, 2012). This leads some policy researchers to worry that enthusiasm
28 for big data will lead to approaches that place the availability and quantity of data
29 over effective research design and the application of substantive theory (Cook,
30 2014). The rise of big data and analytics has led some pundits to herald the big
31
data era as the end of theory (Graham, 2012), or at least the end of the scientific
32
method (Anderson, 2008). These bold pretensions have little grounding and
33
present a dangerous formula for social science research, representing a move
34
away from actually understanding phenomena to simply indicating associations.
35
36
Ambitions around evidence-based policy analysis always include using deduction
37 to arrive at the optimal (variably defined) solution to a problem. Evidence,
38 however, is both contestable and extremely diverse (Dicks et al., 2014; Head,
39 2008) with different stakeholder preferences, interests, values, and judgements
40 (Fischer, 2003; Sabatier, 1991) shaping the advice offered to decision-makers.
41 This poses a significant challenge for policy analytics, which tends to illustrate
42 cross-correlations and complex patterns rather than causality and is unable to
43 account for the discursive nature of policy development. While the combination
44 of big data and data analytics represents a powerful force for the reformulation
45 of policy analysis into a modernized policy analytics approach to policymaking,
46
governance and public policy will still require judgement and decision-making.
47
48

380
page 381 From policy analysis to policy analytics

1 Policy analytics can support better decision-making, if we understand both its

2
potential and its weaknesses.
3
4 Notes
5 1 The province of Ontario introduced a high-occupancy toll (HOT) lane permitting in September
6 2016 for a section of highway near Toronto, though these lanes are standard HOV lanes, with
7 an option for a limited number of commuters to pay a flat fee for a permit to use the HOV
8 lane without meeting the vehicle occupancy requirement. See www.ontario.ca/HOTlanes.
9
Since the permit fee is not dynamically priced, this approach does not qualify as policy analytics
10
under our definition.
2 For example, the City of Montréal, in late 2016, deployed sensors at a number of locations
11
that detected the Bluetooth MAC address of smartphones in passing cars, going beyond simple
12
traffic counts to understand how cars move about the city as an aid to traffic planning and policy
13
(Etherington, 2016).
14
15
16 References
17
6, P. 2004. E-Governance: Styles of Political Judgement in the Information Age Polity.
18 London: Palgrave Macmillan.
19
Adamic, L., and N. Glance. 2005. [[need article title]]In Proceedings of the 3rd
20
International Workshop on Link Discovery (LINKDD 2005): [[need location and
21
22
name of publisher]] 36–43.
23
Adar, E., J. Teevan, and S.T. Dumais. 2008. “Large Scale Analysis of Web
Revisitation Patterns.” In Proceedings of the SIGCHI conference on Human Factors
24
25 in Computing Systems. [[need location and name of publisher, and ACM
26 in full]] ACM, 1197–1206.
27 Amy, D.J. 1984. “Towards a Post-Positivist Policy Analysis.” Policy Studies Journal
28 13: 207–211.
29 Anderson, C. 2008. “The End of Theory: The Data Deluge Makes the Scientific
30 Method Obsolete.” Wired Magazine 16(7): http://archive.wired.com/science/
31
discoveries/magazine/16-07/pb_theory.
32
Andreu-Perez, J., C.C. Poon, R.D. Merrifield, S.T. Wong, and G.Z. Yang.
33
2015. “Big Data for Health.” IEEE [[in full]] Journal of Biomedical and Health
34
Informatics 19(4): 1193–1208.
35
36
Armah, N.A. 2013. “Big Data Analysis: The Next Frontier.” Bank of Canada
37 Review (Summer): 32–39.
38 Berk, R.A., R.F. Boruch, D.L. Chambers, P.H. Rossi, and A.D. Witte. 1985.
39 “Social Policy Experimentation a Position Paper.” Evaluation Review 9(4):
40 387–429.
41 Berners-Lee, T. 2009. “The Next Web.” What’s Next in Tech session, TED
42 (Technology, Entertainment, Design) [[correct?]] Conference. [Video file], www.
43 ted.com/talks/tim_berners_lee_on_the_next_web.html. Long Beach, CA,
44 February.
45 Bert, F., M. Giacometti, M.R. Gualano, and R. Siliquini. 2014. “Smartphones
46
and Health Promotion: A Review of the Evidence.” Journal of Medical Systems
47
48
38(1): 9995.

381
page 382 Policy analysis in Canada

1 Blumsack, S., and Fernandez, A. 2012. “Ready or Not, Here Comes the Smart
2
Grid!” Energy 37(1): 61–68.
3
Bond, R.M., C.J. Fariss, J.J. Jones, A.D. Kramer, C. Marlow, J.E. Settle, and
4 J.H. Fowler. 2012. “A 61-Million-Person Experiment in Social Influence and
5 Political Mobilization.” Nature 489(7415): 295–298.
6 Boulos, M.N.K., A.P. Sanfilippo, C.D. Corley, and S. Wheeler. 2010. “Social
7 Web Mining and Exploitation for Serious Applications: Technosocial Predictive
8
Analytics and Related Technologies for Public Health, Environmental and
9
10
National Security Surveillance.” Computer Methods and Programs in Biomedicine
11 100(1): 16–23.
12 boyd, d., and K. Crawford. 2012. “Critical Questions for Big Data: Provocations
13 for a Cultural, Technological, and Scholarly Phenomenon.” Information,
14 Communication & Society 15(5): 662–679.
15 Breckon, J. 2015. Better Public Services through Experimental Government. London:
16 Alliance for Useful Evidence, www.capire.org/capireinforma/scaffale/Final.pdf.
17
Burris, M.W., and B.R. Stockton. 2004. “Hot Lanes in Houston-six Years of
18 Experience.” Journal of Public Transportation 7(3): 1.
19
Butler, D. 2013. “When Google Got Flu Wrong.” Nature 494(7436): 155.
20
Cabinet Office. 2012. Applying Behavioural Insights to Reduce Fraud, Error and
21
22
Debt. London: Behavioural Insights Team, Cabinet Office, https://www.
23
gov.uk/government/uploads/system/uploads/attachment_data/file/60539/
BIT_FraudErrorDebt_accessible.pdf.
24
25 Cate, F.H. 2008. “Government Data Mining: The Need for a Legal Framework.”
26 Harvard Civil Rights and Civil Liberties Law Review 43(2): 435–489.
27 Chan, M., D. Estève, C. Escriba, and E. Campo. 2008. “A Review of Smart
28 Homes: Present State and Future Challenges.” Computer methods and programs
29 in biomedicine 91(1): 55–81.
30 Chen, H., R.H.L. Chiang, and V.C. Storey. 2012. “Business Intelligence and
31
Analytics: From Big Data to Big Impact.” MIS [[in full]]Quarterly: Management
32
Information Systems 36(4): 1165–1188.
33
Choi, H., and Varian, H. 2012. Predicting the present with Google Trends.
34
Economic Record, 88(s1): 2–9.
35
36
Cohen, P., R. Hahn, J. Hall, S. Levitt, and R. Metcalfe. 2016. Using Big Data to
37 Estimate Consumer Surplus: The Case of Uber w22627. Cambridge, MA: National
38 Bureau of Economic Research, www.nber.org/papers/w22627.pdf.
39 Cook, T.D. 2014. “‘Big Data’ in Research on Social Policy.” Journal of Policy
40 Analysis and Management 33(2): 544–547.
41 Daniell, K.A., A. Morton, and D. Rios Insua. 2016. “Policy Analysis and Policy
42 Analytics.” Annals of Operations Research 236(1): 1–13.
43 Davenport, T.H., P. Barth, and R. Bean. 2012. “How Big Data is Different.”
44 Massachusetts Institute of Technology (MIT) Sloan Management Review 54(1): 43.
45 DeLeon, P. 1994. “Reinventing the Policy Sciences: Three Steps Back to the
46
Future.” Policy Sciences 27(1): 77–95.
47
48

382
page 383 From policy analysis to policy analytics

1 De Marchi, G., G. Lucertini, and A. Tsoukiàs. 2016. “From Evidence-Based

2
Policy Making to Policy Analytics.” Annals of Operations Research 236(1): 15–38.
3
Dicks, L.V., I. Hodge, N.P. Randall, J.P. Scharlemann, G.M. Siriwardena, H.G.
4 Smith, R.K. Smith, and W.J. Sutherland. 2014. “A Transparent Process for
5 ‘Evidence-Informed’ Policy Making.” Conservation Letters 7(2): 119–125.
6 Dobell, R., J. Longo, J. Cavender-Bares, W.C. Clark, N.M. Dickson, G.
7 Dinkelman, A. Fenech, P.M. Haas, J. Jäger, R. Pleune, F.L. Tóth, M.A. Schreurs,
8
and J. van Eijndhoven. 2001. “Implementation in the Management of Global
9
10
Environmental Risks.” In W.C. Clark, J. Jäger, J. van Eijndhoven, and N.M.
11 Dickson (eds.) Learning to Manage Global Environmental Risks, Volume 2: A
12 Functional Analysis of Social Responses to Climate Change, Ozone Depletion and
13 Acid Rain. Social Learning Group. Cambridge, MA: Massachusetts Institute of
14 Technology (MIT) Press.
15 Dryzek, J.S. 1994. Discursive Democracy: Politics, Policy, and Political Science.
16 Cambridge: Cambridge University Press.
17
Edwards, S.J., R.J. Lilford, D.A. Braunholtz, J.C. Jackson, J. Hewison, and J.
18 Thornton. 1998. “Ethical Issues in the Design and Conduct of Randomised
19
Controlled Trials.” Health Technology Assessment 2(15): i–vi.
20
Esperanza, L., and P. Dirk (eds.). 2014. Making Innovation Policy Work: Learning
21
22
from Experimentation. Paris: OECD
23
Estrin, D., and I. Sim. 2010. “Open mHealth Architecture: An Engine for Health
Care Innovation.” Science 330(6005): 759–760.
24
25 Etherington, D. 2016. “Here’s how Montréal is turning to Bluetooth to
26 address traffic congestion.” TechCrunch, 19 December, https://techcrunch.
27 com/2016/12/19/heres-how-montreal-is-turning-to-bluetooth-to-address-
28 traffic-congestion/.
29 Fellegi, I.P. 1996. “Characteristics of an Effective Statistical System.” Statistical
30 Journal of the United Nations Economic Commission for Europe 13(2): 89–117.
31
Fischer, F. 1980. Politics, Values, and Public Policy: The Problem of Methodology.
32
Boulder, CO: Westview Press.
33
Fischer, F. 1995. Evaluating Public Policy. Chicago, IL: Nelson-Hall
34
Fischer, F. 2003. Reframing Public Policy: Discursive Politics and Deliberative Practices.
35
36
New York: Oxford University Press.
37 Forrester, J.W. 1971. World Dynamics. Cambridge, MA: Wright-Allen Press.
38 Gandomi, A., and M. Haider. 2015. “Beyond the Hype: Big Data Concepts,
39 Methods, and Analytics.” International Journal of Information Management 35(2):
40 137–144.
41 Ginsberg, J., M.H. Mohebbi, R.S. Patel, L. Brammer, M.S. Smolinski, and L.
42 Brilliant. 2009. “Detecting Influenza Epidemics Using Search Engine Query
43 Data.” Nature 457(7232): 1012–1014.
44 Ginsberg, W.R. 2011. “The Obama Administration’s Open Government
45 Initiative: Issues for Congress.” Congressional Research Service (CRS) Report for
46
Congress 7-5700 R41361, 28 January. Washington, DC: United States CRS.
47
48

383
page 384 Policy analysis in Canada

1 Graham, M. 2012. “Big Data and the End of Theory?” Guardian, 9 March.
2
London: Guardian.
3
Green, D.A., and K. Milligan. 2010. The Importance of the Long Form Census
4 to Canada. Canadian Public Policy 36(3): 383–388.
5 Groll, T., S. O’Halloran, S. Maskey, G. McAllister, and D. Park. 2015. Big Data
6 and the regulation of banking and financial services. Bank Financial Services Policy
7 Report, 34(12): 1–10.
8
Groves, R.M. 2006. Nonresponse rates and nonresponse bias in household surveys.
9
10
Public opinion quarterly, 70(5): 646–675.
11 Grubmüller, V., K. Götsch, and B. Krieger. 2013. Social media analytics for
12 future oriented policy making. European Journal of Futures Research, 1(1): 1–9.
13 Gubbi, J., R. Buyya, S. Marusic, and M. Palaniswami. 2013. Internet of Things
14 (IoT): A vision, architectural elements, and future directions. Future Generation
15 Computer Systems, 29(7): 1645–1660.
16 Harsin, J. 2015. “Regimes of Posttruth, Postpolitics, and Attention Economies.”
17
Communication, Culture & Critique 8(2): 327–333.
18 Hasan, S., C.M. Schneider, S.V. Ukkusuri, and M.C. González. 2013.
19
“Spatiotemporal Patterns of Urban Human Mobility.” Journal of Statistical Physics
20
151(1–2): 304–318.
21
22
Haynes, L., B. Goldacre, and D. Torgerson. 2012. Test, Learn, Adapt: Developing
23
Public Policy with Randomised Controlled Trials. London: UK Cabinet Office–
Behavioural Insights Team, http://dx.doi.org/10.2139/ssrn.2131581.
24
25 Head, B.W. 2008. “Three Lenses of Evidence-Based Policy.” Australian Journal of
26 Public Administration 67(1): 1–11.
27 Helft, M. 2008. “Aches, a Sneeze, a Google Search.” New York Times. 11
28 November, A1, www.nytimes.com/2008/11/12/technology/internet/12flu.
29 html.
30 Hitchcock, T. 2013. “Big Data for Dead People: Digital Readings and the
31
Conundrums of Positivism.” Historyonics, http://historyonics.blogspot.
32
ca/2013/12/big-data-for-dead-people-digital.html
33
Hondula, D.M., E.R. Kuras, J. Longo, E.W. Johnston. 2017. “Toward Precision
34
Governance: Infusing Data into Public Management of Environmental Hazards.”
35
36
Public Management Review, http://dx.doi.org/10.1080/14719037.2017.1320043
37 Huang, Q., and Y. Xiao. 2015. “Geographic Situational Awareness: Mining
38 Tweets for Disaster Preparedness, Emergency Response, Impact, and Recovery.”
39 ISPRS[[in full]] International Journal of Geo-Information 4(3): 1549–1568.
40 Isett, K.R., B.W. Head, and G. Van Landingham. 2016. “Editors’ Introduction.”
41 Public Administration Review 76: 542.
42 Johnston, E.W. 2015. “Conceptualizing Policy Informatics.” In Erik W. Johnston
43 (ed.) Governance in the Information Era: Theory and Practice of Policy Informatics.
44 New York: Routledge, 3–22.
45
46
47
48

384
page 385 From policy analysis to policy analytics

1 Johnston E.W., and D.M. Hondula. 2015. “The Intentional Use of Information
2
to Enable Self-Governance.” MacArthur Foundation Research Network on Opening
3
Governance, 29 May, www.opening-governance.org/blog/2015/5/29/the-
4 intentional-use-of-information-to-enable-self-governance.
5 Kim, G.H., S. Trimi, and J.H. Chung. 2014. “Big-Data Applications in the
6 Government Sector.” Communications of the [[in full]]ACM 57(3): 78–85.
7 Kitchin, R. 2014a. The Data Revolution: Big Data, Open Data, Data Infrastructures
8
and Their Consequences. London: Sage.
9
10
Kitchin, R. 2014b. “The Real-Time City? Big Data and Smar t
11 Urbanism.” GeoJournal 79(1): 1–14.
12 Klein, T., and S. Verhulst. 2017. “Access to New Data Sources for Statistics:
13 Business Models and Incentives for the Corporate Sector.” Discussion paper
14 10, March. Paris21 (Partnership in Statistics for Development in the 21st Century),
15 www.paris21.org/sites/default/files/Paper_new-data-sources_final.pdf.
16 Kramer, A.D., J.E. Guillory, and J.T. Hancock. 2014. “Experimental Evidence
17
of Massive-Scale Emotional Contagion Through Social Networks.” Proceedings
18 of the National Academy of Sciences 111(24): 8788–8790.
19
Lasswell, H.D. 1951. “The Policy Orientation.” In D. Lerner, and H.D. Lasswell
20
(eds.) The Policy Sciences. Palo Alto, CA: Stanford University Press, 3–15.
21
22
Lathrop, D., and L. Ruma (eds.). 2010. Open Government: Collaboration,
23
Transparency and Participation in Practice. Sebastopol, CA: O’Reilly Media, Inc,
http://oreilly.com/catalog/9780596804367.
24
25 Laurila, J.K., D. Gatica-Perez, I. Aad, O. Bornet, T.M.T. Do, O. Dousse, J. Eberle,
26 and M. Miettinen. 2012. “The Mobile Data Challenge: Big Data for Mobile
27 Computing Research.” Pervasive Computing EPFL-CONF-192489.[[publishing
28 details? Web address?]]
29 LaValle, S., E. Lesser, R. Shockley, M.S. Hopkins, and N. Kruschwitz. 2011.
30 “Big Data, Analytics and the Path from Insights to Value.” Massachusetts Institute
31
of Technology (MIT) Sloan Management Review 52(2): 21.
32
Lazer, D., R. Kennedy, G. King, and A. Vespignani. 2014. “The Parable of Google
33
Flu: Traps in Big Data Analysis.” Science 343(6176): 1203–1205.
34
Lazer, D., A.S. Pentland, L. Adamic, S. Aral, A.L. Barabasi, D. Brewer, N.
35
36
Christakis, N. Contractor, J. Fowler, M. Gutmann, T. Jebara, G. King, M.
37 Macy, D. Roy, and M.V. Alstyne. 2009. “Computational Social Science.” Science
38 323(5915): 721–723.
39 Lerner, D., H.D. Lasswell (eds.). 1951. The Policy Sciences. Stanford, CA: Stanford
40 University Press.
41 Lessig, L. 2006. Code Version 2.0. New York:Basic Books.
42 Lewis, K., J. Kaufman, M. Gonzalez, A. Wimmer, and N. Christakis. 2008.
43 “Tastes, Ties, and Time: A New Social Network Dataset using Facebook.com.”
44 Social Networks 30(4): 330–342.
45 Lindblom, C.E. 1959. “The Science of ‘Muddling Through’.” Public Administration
46
Review 19(2): 79–88.
47
48

385
page 386 Policy analysis in Canada

1 Lindquist, E. 2015. “Visualization Meets Policy Making: Visual Traditions, Policy

2
Complexity, Strategic Investments.” In Erik W. Johnston (ed.) Governance in the
3
Information Era: Theory and Practice of Policy Informatics. New York: Routledge,
4 62–82.
5 Lipsky, M. 1971. “Street-Level Bureaucracy and the Analysis of Urban Reform.”
6 Urban Affairs Review 6(4): 391–409.
7 London, A.J. 2009. “Clinical Equipoise: Foundational Requirement or
8
Fundamental Error.” In B. Steinbock (eds.) The Oxford Handbook of Bioethics.
9
10
Oxford: Oxford University Press.
11 Longo, J. 2011. “#OpenData: Digital-Era Governance Thoroughbred or New
12 Public Management Trojan Horse?” Public Policy and Governance Review 2(2):
13 38–51.
14 Longo, J. 2015. “The Future of Computer-Supported Policy Analysis: Openness,
15 Collaboration, Collective Intelligence and Competition.” In Edward A. Parson
16 (ed.) A Subtle Balance: Expertise, Evidence, and Democracy in Public Policy &
17
Governance, 1970–2010. Montreal: McGill-Queen’s University Press,183–206.
18 Longo, J. forthcoming. “Citizen and Stakeholder Engagement into the Digital
19
Era: From Spicer to #Hashtags.” Canadian Public Administration.[[details yet]]
20
Longo, J., E.R. Kuras, H. Smith, D.M. Hondula, and E.W. Johnston. 2017.
21
22
“Technology Use, Exposure to Natural Hazards, and Being Digitally Invisible:
23
Implications for Policy Analytics.” Policy & Internet 9(1): 76–108.
McAfee, A., E. Brynjolfsson, T.H. Davenport, D.J. Patil, and D. Barton. 2012.
24
25 “Big Data. The Management Revolution.” Harvard Business Review 90(10):
26 61–67.
27 McNeely, C.L., and J. Hahm. 2014. “The Big (Data) Bang: Policy, Prospects,
28 and Challenges.” Review of Policy Research 31(4): 304–310.
29 McNutt, K. 2014. “Public Engagement in the Web 2.0 Era.” Canadian Public
30 Administration 57(2): 49–70.
31
Malomo, F., and V. Sena. 2017. “Data Intelligence for Local Government?
32
Assessing the Benefits and Barriers to Use of Big Data in the Public Sector.”
33
Policy & Internet 9(1): 7–27.
34
May, P.J. 1992. “Policy Learning and Failure.” Journal of Public Policy 12(4):
35
36
331–354.
37 Mayer-Schönberger, V., and K. Cukier. 2013. Big Data: A Revolution That Will
38 Transform How We Live, Work, and Think. Boston, MA: Houghton Mifflin
39 Harcourt.
40 Meadows, D.H., and The Club of Rome. 1972. The Limits to Growth: A Report
41 for the Club of Rome’s Project on the Predicament of Mankind. New York: Universe
42 Books.
43 Meltsner, A.J. 1976. Policy Analysts in the Bureaucracy. Berkeley, CA: University
44 of California Press.
45 Mergel, I. 2016. “Big Data in Public Affairs Education.” Journal of Public Affairs
46
Education 22(2): 231–248.
47
48

386
page 387 From policy analysis to policy analytics

1 Mergel, I., R.K. Rethemeyer, and K. Isett. 2016. “Big Data in Public Affairs.”
2
Public Administration Review 76(6): 928–937.
3
Mesarovic, M., and E. Pestel. 1974. Mankind at the Turning Point: The Second
4 Report to the Club of Rome. New York: EP Dutton and Co.
5 Michael, K., and K.W. Miller. 2013. “Big data: New opportunities and new
6 challenges” [guest editors’ introduction]. Computer 46(6): 22–24.
7 Morçöl, G. 2001. “Positivist Beliefs Among Policy Professionals: An Empirical
8
Investigation.” Policy Sciences 34: 381–401.
9
10
Morgan, M.G., M. Henrion, and M. Small. 1992. Uncertainty: A Guide to Dealing
11 with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge: Cambridge
12 University Press.
13 Murdoch, T.B., and A.S. Detsky. 2013. “The Inevitable Application of Big Data
14 to Health Care.” Journal of the American Medical Association 309(13): 1351–1352.
15 Murphy, C. 2016. Shared Mobility and the Transformation of Public Transit (TCRP
16 J-11/TASK 21). Chicago, IL: American Public Transportation Association
17
(APTA)/Shared-Use Mobility Center (SUMC), www.apta.com/resources/
18 reportsandpublications/Documents/APTA-Shared-Mobility.pdf
19
Mustafaraj, E., and P. Metaxas. 2010. “From Obscurity to Prominence in Minutes:
20
Political Speech and Real-Time Search.” Proceedings of the WebSci10: Extending
21
22
the Frontiers of Society On-Line, 26–27 April, Raleigh, NC.
23
Newsham, G.R., and B.G. Bowker. 2010. “The Effect of Utility Time-Varying
Pricing and Load Control Strategies on Residential Summer Peak Electricity
24
25 Use: A Review.” Energy Policy 38(7): 3289–3296.
26 Paquet, G. 2009. Crippling Epistemologies and Governance Failures: A Plea for
27 Experimentalism. Ottawa: University of Ottawa Press.
28 Paris, C., and S. Wan. 2011. “Listening to the community: social media
29 monitoring tasks for improving government services.” In CHI’11 Extended
30 Abstracts on Human Factors in Computing Systems May. [[need location and
31
name of publisher and ACM in full]] ACM, 2095–2100.
32
Parson, E.A. 2015. “Expertise and Evidence in Public Policy: In Defence of
33
(a Little) Technocracy.” In Edward A. Parson (ed.) A Subtle Balance: Expertise,
34
Evidence, and Democracy in Public Policy & Governance, 1970–2010. Montreal:
35
36
McGill-Queen’s University Press, 42–50.
37 Parson, E.A., and K. Fisher-Vanden. 1997. “Integrated Assessment Models of
38 Global Climate Change.” Annual Review of Energy and Environment 22: 589–628.
39 Pentland, A. 2012a. “Reinventing Society in the Wake of Big Data.” Edge –
40 Conversation: Technology, 30 August, www.edge.org/conversation/reinventing-
41 society-in-the-wake-of-big-data
42 Pentland, A. 2012b. “The New Science of Building Great Teams.” Harvard Business
43 Review 90(4): 60–69.
44 Perry, W.L., B. McInnis, C.C. Price, S. Smith, and J.S. Hollywood. 2013. Predictive
45 Policing: The Role of Crime Forecasting in Law Enforcement Operations. Santa Monica:
46
RAND, www.rand.org/content/dam/rand/pubs/research_reports/RR200/
47
48
RR233/RAND_RR233.pdf

387
page 388 Policy analysis in Canada

1 Pirog, M.A. 2014. “Data Will Drive Innovation in Public Policy and Management
2
Research in the Next Decade.” Journal of Policy Analysis and Management 33(2):
3
537–543.
4 Pressman, J.L., and A.B. Wildavsky. 1973. Implementation: How Great Expectations
5 in Washington are Dashed in Oakland: Or, Why It’s Amazing that Federal Programs
6 Work At All, This Being a Saga of the Economic Development Administration as Told
7 by Two Sympathetic Observers Who Seek to Build Morals on a Foundation of Ruined
8
Hopes. Berkeley, CA: University of California Press.
9
10
Prince, M. 2007. “Soft Craft, Hard Choices, Altered Context: Reflections on
11 Twenty-Five Years of Policy Advice in Canada.” In Laurent Dobuzinskis, Michael
12 Howlett, and David Laycock (eds.) Policy Analysis in Canada: The State of the
13 Art. Toronto: University of Toronto Press.
14 Putnam, H. 2002. The Collapse of the Fact/Value Dichotomy and Other Essays.
15 Cambridge, MA: Harvard University Press.
16 Quade, E.S. 1975. Analysis for Public Decisions. New York: Elsevier.
17
Quade, E.S. 1980. “Pitfalls in Formulation and Modeling.” In Giandomenico
18 Majone, and Edward S. Quade (eds.) Pitfalls in Analysis. IIASA[[in full]] and
19
John Wiley & Sons
20
Radin, B.A. 2006. Challenging the Performance Movement: Accountability Complexity
21
22
and Democratic Value. Washington, DC: Georgetown University Press.
23
Rogers, R. 2013. Digital Methods. Cambridge, MA: Massachusetts Institute of
Technology (MIT) Press.
24
25 Rogers, R. 2014. “Political Research in the Digital Age.” International Public
26 Policy Review 8(1): 73–87.
27 Rogers, R., N. Sánchez-Querubín, and A. Kil. 2015. Issue Mapping for an Ageing
28 Europe. Amsterdam: University of Amsterdam Press.
29 Sabatier, P.A. 1991. “Toward Better Theories of the Policy Process.” PS: Political
30 Science & Politics 24(2): 147–156.
31
Savage, M., and R. Burrows. 2007. “The Coming Crisis of Empirical Sociology.”
32
Sociology 41(5): 885–899.
33
Schintler, L.A., and R. Kulkarni. 2014. “Big Data for Policy Analysis: The Good,
34
The Bad, and The Ugly.” Review of Policy Research 31(4): 343–348.
35
36
Schroeder, J., R. Klein, T. Smith, K. Turnbull, K. Balke, M. Burris, et al. 2015.
37 Los Angeles Congestion Reduction Demonstration ExpressLanes Program: National
38 Evaluation Report (No. FHWA-JPO-14-126).[[need location and name of
39 publisher]]
40 Stone, D.A. 1988. Policy Paradox and Political Reason. Reading, MA: Addison-
41 Wesley Longman.
42 Struijs, P., B. Braaksma, and P.J. Daas. 2014. “Official Statistics and Big Data.”
43 Big Data & Society 1(1), doi:10.1177/2053951714538417.
44 Supernak, J., D. Steffey, and C. Kaschade. 2003.” Dynamic Value Pricing as
45 Instrument for Better Utilization of High-Occupancy Toll Lanes: San Diego
46
I-15 case.” Transportation Research Record: Journal of the Transportation Research
47
48
Board 1839: 55–64.

388
page 389 From policy analysis to policy analytics

1 Till, B.C., J. Longo, A.R. Dobell, and P.F. Driessen. 2014. “Self-Organizing Maps
2
for Latent Semantic Analysis of Free-Form Text in Support of Public Policy
3
Analysis.” WIREs Data Mining & Knowledge Discovery 4(1): 71–86.
4 Torgerson, D. 1985. “Contextual Orientation in Policy Analysis: The
5 Contribution of Harold D. Lasswell.” Policy Sciences 18(3): 241–261.
6 Tsoukias, A., G. Montibeller, G. Lucertini, and V. Belton. 2013. “Policy Analytics:
7 An Agenda for Research and Practice.” EURO Journal on Decision Processes
8
1(1–2): 115–134.
9
10
Tufekci, Z. 2014. “Big Questions for Social Media Big Data: Representativeness,
11 Validity and Other Methodological Pitfalls.” arXiv preprint, arXiv:1403.7400.
12 Van Dijck, J. 2014. “Datafication, Dataism and Dataveillance: Big Data Between
13 Scientific Paradigm and Ideology.” Surveillance & Society 12(2): 197–208.
14 Watts, D.J. 2007. “A Twenty-First Century Science.” Nature 445(7127): 489–489.
15 Wigan, M.R., and R. Clarke. 2013. “Big Data’s Big Unintended Consequences.”
16 Computer 46(6): 46–53.
17
Wildavsky, A. 1978. “Policy Analysis is What Information Systems Are Not.”
18 Accounting, Organizations and Society 3(1): 77–88.
19
Williamson, B. 2016. “Digital Education Governance: Data Visualization,
20
Predictive Analytics, and ‘Real-Time’ Policy Instruments.” Journal of Education
21
22
Policy 31(2): 123–141.
23
Wolfson, M. 2015. “Tales of Quantitative Analysis and Public Policy.” In Edward
A. Parson (ed.) A Subtle Balance: Expertise, Evidence, and Democracy in Public Policy
24
25 & Governance, 1970–2010. Montreal: McGill-Queen’s University Press, 87–111.
26 Zaslavsky, A., C. Perera, and D. Georgakopoulos. 2013. “Sensing as a Service
27 and Big Data.” arXiv preprint, arXiv:1301.0159.
28 Zhong, C., E. Manley, S.M. Arisona, M. Batty, and G. Schmitt. 2015. “Measuring
29 Variability of Mobility Patterns from Multiday Smart-Card Data.” Journal of
30 Computational Science 9: 125–130.
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

389

View publication stats