Policy: Theories and Evaluation

Organization, Policy, and Market

Open-system theory

All organizations are different—in part because

of the different environments in which they
operate—and that they should be organized to
accommodate different issues and
opportunities.

Consistent with this open-system theory,

exogenous factors become important to
consider in public policy so that public policy
encourages the achievement of greater goals
for all elements in the system.
Tragedy of the commons (Hardin,
1968)
Common Pool Resources
• Rivalry
• Non excludable
• Limited for finite capacity
• Collective action problem
• Need for cooperation and governance
• Important for livelihoods and well being
• Free-rider problem
 Governing the Commons

Clear boundaries

Collective decision making

Monitoring and accountability

Conflict resolution mechanism

Local rights and norms

Flexibility in governance

Collaboration
Nobel Laureate in Economics,
2009
Markets
Public Policy

PUBLIC
POLICY
Unintended Intended
Consequences Outcome
(Objectives)
Organizations
 Policy Approach
Structures must ensure that policymaking
resources are more effectively linked with
goal priorities, as well as that proper
processes are followed.

Policymaking culture will need to Controls should restore the balance while
adjust, which will need to evaluate speeding up the process. The main problem
some basic principles such as is to implement these additional protections
innovative and adaptive techniques. without undermining the relationship
Keeping track of how the systems they between policy maker and civil servant.
govern are evolving. Continuous,
iterative, long-term approach to
challenges. More practical, adaptive
action than policy documents and
speeches output is

Policymaking is inherently political, and

Having access to the right skills and ministers/executive leaders play a critical role. The
experience is essential for good political and practical should combined in good policy.
policymaking. The new environment will Good relationships between ministers and bureaucrats
necessitate the acquisition of new abilities. are required to achieve such a mix. The ideal, according
to both parties, is 'guided exploration,' in which ministers
are clear about their political and policy objectives and
then willing to engage in an open, iterative dialogue about
how to best achieve them.
Source: Hallsworth, M., & Rutter, J. (2011). Making policy better: Improving Whitehall's core business. Institute for Government.
 The Approach: Evidence-Based Policymaking

§ Evidence-based policymaking give

comprehensive perspective: PRINCIPLES

Ø To achieve the policy objectives

Ø To anticipate and mitigate the unintended effect of the Build and compile rigorous evidence about what works,
policy including costs and benefits

Monitor program delivery and use impact evaluation to

measure program effectiveness
§ Policymaking is a political activity. Evidence has
key role to gain wider common understanding Use rigorous evidence to improve programs, scale what
among policy makers which has different works, and redirect funds away from consistently
ineffective programs
assumption and judgment about the policy.

Encourage innovation and test new approaches

Source: Evidencecollaborative.org (2016) Principles of

evidence-based policymaking

This Photo by Unknown Author is

licensed under CC BY-SA
Policy is not created in a vacuum

• Politics affects policy

• Multiple interests and objectives
• Vested interest
• Hidden agenda
• Power play
• Leaders need to decide what represents the
best interests of public
• For the greater good
Measuring Policy Effectiveness
• Cost-Benefit Analysis
• Experiment
• Survey/Polling
• Program Evaluation
• Media Tracking
• Sentiment Analysis
• Media Framing
• Relevance, Efficiency, Effectiveness, Impact, Sustainability
Institutions/Corporations and Public Policy

• Challenges of detailed regulation

and legislation to protect workers,
consumers, investor, and public
welfare

• Corporations need a strategic,

forward-looking, and balanced
approach to government and
public affairs
Public policy deals with public goods
Group work

Please perform:
• Cost-benefit analysis for any energy policy
that you prefer, make sure you incorporate
monetary value in the analysis. Use your
assumption if necessary
• A design of experiment in any policy that you
know
Corporations and Public Policy Agenda

• Politics
• Intervention
• Culture
• Public vs private
interests
Corporations approach to public policy

• What is the desirable public policy that the

corporation should formulate?

• What is the feasible policy which corporation

can get enacted?

• How can the corporation ensure that the

chosen public policy is implemented?
 Choose your arenas
•Problems
•Issues
•Challenges
Pick your team wisely
 Policy experts and political experts

Policy experts Political experts

• Key team members located at • Located in the political capitols
headquarters with business and • Their clients are the decision-
corporate unit leaders makers in the legislature, the
• Have domain knowledge in executive, and the regulatory
specific industries or in general agencies
subjects
• Help lead the annual review
process which designs
desirable policies in the context
of corporate and societal trends
Managing Various Interest groups

• Economic/corporate groups
• Labor groups
• Professional associations
• Citizen or public interest groups
• Ideological groups
• Public sector groups
The CEO Activism

More and more CEOs are They are frustrated with the
taking a stand on divisive growing political turmoil
social issues-a dramatic and paralysis in the
departure from traditional government
role of CEOs

“Our jobs as CEOs now include driving what we think is

right. It is not exactly political activism, but it is action on
issues beyond business”

Bank of America’s CEO, Bryan Moynihan

Tactics of CEO Activists

Raising Awareness
Making public statements, can be
through Twitter

Leveraging Economic
Power
Using company’s power to put
pressure or influence people
How CEOs respond
From Design to Effective Implementation

CONCURRENT HOLISTIC
From Design To Effective
Implementation

• Begin With The End In Mind

• The Process Link
• Plan – Design – Develop – Test – Trial –
Socialize – Implement – Enforce – Feedback –
Learn – Modify
• Domains Of Public Services
• Health, Education, Connectivity, Mobility,
Justice, Civil Service, Safety, Security,
Livelihood, Energy, Knowledge
• Policy, National, Sub National, Community
• Tools and Enablers
• Actors and Shifting Roles
Pitfalls of Public Service
Design
• Misreading Scope And Scale Of The ‘End
In Mind’
• Incomplete Analysis of Concurrencies
• Undermining Extensiveness Need Of The
Design Processes
• Socialization Instead of Consultation
• Haste
• Neglect and Ignorance of Related
Context/Externalities
• Political Bias
ENTER
GOVERNMENT
AS A
PLATFORM
 Why is it Becoming a Trend?

• In Line With The Rise

Of Industry 4.0
• Unstoppable
Technology
Disruption
• Acceleration of
Interconnectivity of
Economies/Countries
• Threat of Widening
Inequality
Theories of Public Policy
Uses of Models
• Simplify and clarify our thinking
• Identify important aspects of policy problems
• Help us to communicate with each
• Direct our efforts to understand public policy
better by suggesting what is important and
unimportant
• Suggest explanations for public policy and
predict its consequences
 Institutio
nalism

Rationalism
Public
choice

System Elite
theory Theory

Policy
Models

Process Group
model Theory

Increme Game
ntalism Theory
Garbage
can
Which Theory is Appropriate?

1. Who participates in policy making?

2. How are policy decisions made?
3. What are the underlying assumptions of the
theory/model?
4. What are the consequences for the general
public of policy decisions made in accordance
with the particular theory/model?
1. Institutionalism
Public policy as institutional output . Emphasizes formal and legal aspects of
government

• Who: executive, legislative, and judicial branches

• How: policy is authoritatively determined, implemented, and enforced by these

institutions (legitimacy, universality, and coercion)

• Implications/assumptions: individuals have little impact; structure/design

affects outcomes
Inclusive Institutions

• Encouraging investment (because of

well-enforced property rights)
• Harnessing the power of markets
(better allocation of resources, more
efficient firms)
• Generating broad-based participation
2. Process Model
Public policy as political activity, political response to demands

• Who: voters, interest groups, legislators, presidents, bureaucrats, judges

• How: Identify problem, set agenda, formulate policy proposals, legitimate policies,
implement policies, evaluate policies

• Implications/assumptions: who participates has a critical or determinant impact on

the process
3. Group Theory
Public policy as group equilibrium, competing interests for an equilibrium

• Who: interest groups, their allies in government

• How: struggle among interest groups with legislature/executive as referee to manage
group conflict and establish rules of the game
• Implications/assumptions: groups will always join to press for particular
issues, all interests will have an opportunity for representation
 4. Elite Theory
Public policy as elite preference, wealthy and policy-planning
insiders influence values and preferences

• Who: elites that have power, ability to allocate value

• How: implementation of the preferences and values of the

governing elite; public officials merely carry out policies decided on
by the elites

• Implications/assumptions: public is apathetic elites agree upon

norms; political action is merely symbolic; protects the status quo
5. Rationalism
Public policy as maximum social gain
• Who: decision makers (all social, political, economic values sacrificed or
achieved by a policy choice) irrespective of dollar amount (Bentham, Mills)
• How: select policy alternative(s) that allows gains to society to exceed
benefits by the greatest amount
• Implications/assumptions: assumes that the values preferences of the
society as a whole can be known and weighted
• Bounded rationality
Basic Human Thinking System
System 1 System 2
Automatic Slow, but reliable
Quick operations More deliberate and
Impulsive effortful form of
Intuitive thought thinking
Gullible and biased to Controlled operations
believe Requires self-control
Detects simple Make deliberate choices
relations between options
Continuously In charge of doubting
monitors what is and unbelieving
going on outside Sometimes lazy and busy
and inside the Impose logical analysis
mind. Capable of reasoning
Fast but more errors Self-critisism
6. Incrementalism
Public policy as variations on the past, builds on past decisions and grows slowly and
steadily
• Who: policy makers, legislators, others with a stake in ongoing programs or problems
• How: continuation of past government activities with only incremental modifications
• Implications/assumptions: accepts the legitimacy of established programs; fear of
unintended consequences; sunk costs in other programs may minimize the
opportunities for radical change
7. Game Theory
Game theory is concerned with the strategic choices made by
participants or players during a competition, in which each participant
or player attempts to maximize profits and minimize losses.
• Who: players/decision makers who have choices to make and the
outcome depends on the choice made by each (assumes rationality
in making choices)
• How: each player has goals and resources, a strategy developed
given possible moves of opponent, and payoff values that
constitute the outcomes of the game
• Implications/assumptions: repeated plays should lead to better
policy outcomes
8. Public Choice
Public policy as collective decision making by self-interested
individuals
• Who: rational self-interested individuals will in both politics and
economics cooperate to achieve their goals
• How: individuals come together in politics for their own mutual
benefit; government must respond to market failures
• Implications/assumptions: individuals have sufficient
information to know what is in their best interest
9. Systems Theory
Public policy as system output
• Who: individuals, groups, or nations depending upon the scope of the
problem
• How: environment may stimulate inputs into political system, producing
outputs and feedback
• Implications/assumptions: systems implies an identifiable set of institutions
and activities in society that functions to transforms demands into authoritative
decisions requiring the support of the whole society; implies that the elements
of the system are interrelated, that the system can respond to forces in its
environment, and that it will do so to preserve itself
10. Kingdon-Garbage Can Model
• Who: participants inside and outside government
• How: choice opportunity is a garbage can into which various
kinds of problems and solutions are dumped by participants as
they are generated; policy outcomes are a function of the mix
of the garbage: problems, solutions, participants, and
participant resources in the can and how the can is processed
• Implications/assumptions: each of the actors and processes
can operate either as an impetus or as a constraint; streams
operate largely independent of one another
PROGRAM EVALUATION
What is Program Evaluation?

• Practical endeavor, not an academic exercise, and is

not primarily an attempt to build theory or necessarily
to develop social science knowledge
• The mission of program evaluation is to provide
information that can be used to improve the programs
• “…the systematic assessment of the operation and/or
outcomes of a program or policy, compared to a set of
explicit or implicit standards as a means of
contributing to the improvement of the program or
policy…”
 Research vs. Evaluation?

Systematic
Research Evaluation
Methods

• Production of generalizable • Knowledge intended for use

knowledge • Program- or funder-derived
• Researcher-derived questions questions
• Paradigm stance • Judgmental quality
• More controlled setting • Action setting
• Clearer role • Role conflicts
• Published • Often not published
• Clearer allegiance • Multiple allegiances

51
 “Research seeks to prove,
evaluation seeks to improve…”

5
2
 Surveillance & Monitoring
vs. Program Evaluation

• Surveillance - tracks disease or risk behaviors

• Monitoring - tracks changes in program
outcomes over time
• Evaluation - seeks to understand specifically
why these changes occur
5
3
What Can be Evaluated?
• Direct service • Laboratory diagnostics
interventions • Communication
• Community campaigns
mobilization efforts • Infrastructure-building
• Research initiatives projects
• Surveillance systems
• Training and educational
• Policy development
activities services
• Outbreak • Administrative systems
investigations

54
When to Conduct Evaluation?

Planning a Assessing a Assessing a Assessing a

NEW program DEVELOPING STABLE, MATURE program after
program program it has ENDED

Conception Completion

The stage of program development

influences the reason for program
evaluation.
Why Evaluate Programs?

• To gain insight about a program and its operations –

to see where we are going and where we are coming
from, and to find out what works and what doesn’t
• To improve practice – to modify or adapt practice to
enhance the success of activities
• To assess effects – to see how well we are meeting
objectives and goals, how the program benefits the
community, and to provide evidence of effectiveness
• To build capacity - increase funding, enhance skills,
strengthen accountability
 Framework for Program Evaluation

Steps
Engage
stakeholders

Ensure use Describe

and share Standards the program
lessons learned
Utility
Feasibility
Propriety
Accuracy Focus the
Justify
Evaluation
conclusions
design

Gather credible
evidence

cdc.gov
Identifying Stakeholders

Who are the stakeholders?

Persons involved in program operations
Persons served or affected by the program
Intended users of evaluation findings

What is their interest in the program?

Do they support the program?
Are they skeptical about or antagonistic toward the
program?

58
Identifying Stakeholders

Persons Involved in Program

Operations
Staff and Partners
Persons affected or served by the
program
Clients, their families and social networks, providers
and community groups
Intended users of the evaluation findings
Policy makers, managers, administrators, advocates,
funders, and others

Be sure to include supporters and skeptics!

60
 Which Stakeholders Matter Most?

Who is…
Affected by the program?
Involved in program operations?
Intended users of evaluation findings?

Who do we need to…

6
1 Enhance credibility?
Implement program changes?
Advocate for changes?
Fund, authorize, or expand the program?
Engaging Stakeholders
Stakeholders should be involved in…
• Describing program activities, context, and
priorities
• Defining problems
• Selecting evaluation questions and methods
• Serving as data sources
• Defining what constitutes the “proof” of success
• Interpreting findings
• Disseminating information
• Implementing results
Data Sources

Qualitative/
Focus Group Experiment
Data
Cust Satisfaction Monitoring
Surveys Data

Interview Data Answers Chart Review

Data

Financial Encounter
Data Cust Outcome Data
Surveys
Ethics and Politics of Program Evaluation
Ethical Issues in Program Evaluation
Voluntary participation.
• Free from coercion
• Competent to understand choices
• Free to withdraw from study
• Consent forms (in most cases)
No harm to participants.
• Voluntary participation is based on a full understanding of
possible risks.

Anonymity and confidentiality.

Participants should be given sufficient information about

risks and benefits
Research with Special (Vulnerable) Populations

• Children – must have parental consent

• People with diminished mental capacity (e.g.
dementia, mental retardation, severe psychosis)
• May require consent from family member or guardian
• Prisoners – must undergo full IRB review
 Example: T h e i n t e r v i e w e e s

IR. JERO WACIK, S.E. PROF. DR.-ING. IR. RUDI RUBIANDINI R.S. BARNABAS SUEBU, S.H.

M E C H A N I C A L E N G I N E E R I N G I T B , 1 9P 7E 4T R O L E U M E N G I N E E R I N G I T B , 1 9
F8A 5C U L T Y O F L A W C E N D R A W A S I H ,

FACULTY OF ECONOMICS UI, 1983 TU CLAUSTHAL, 1991

Diverse Cultures and Groups

• Must be culturally sensitive, not denigrating

of culture’s norms and values
• Excluding minorities without cause against
guidelines for federal funding
• Learn as much as possible about culture
being studied
• Can lead to ethical dilemmas
Ethical Controversy: Stanley Milgram

• Study of human obedience.

• Subjects had role of "teacher" and
administered a shock to "pupils".
• Pupils were actually part of the
experiment.
Ethics and Politics of Program Evaluation

• Ethics deals mostly with methods used in

research.
• Politics deals with the substance and use
of research.
• There are no formal codes of accepted
political conduct.
Politics in Perspective

Three issues:
• Science is not untouched by politics.
• Science does not proceed in the midst of
political controversy.
• Awareness of ideologies enriches the
study and practice of social research
methods.
THANK YOU