2017-Foreign Policy Analysis

Methods of Foreign Policy Analysis

Philip B.K. Potter, Woodrow Wilson Department of Politics, University of Virginia
https://doi.org/10.1093/acrefore/9780190846626.013.34
Published in print: 01 March 2010
Published online: 30 November 2017
Summary
Foreign policy analysis (FPA) is the study of how states, or the individuals that lead them,
make foreign policy, execute foreign policy, and react to the foreign policies of other states.
This topical breadth results in a subfield that encompasses a variety of questions and levels of
analysis, and a correspondingly diverse set of methodological approaches. There are four
methods which have become central in foreign policy analysis: archival research, content
analysis, interviews, and focus groups. The first major phase of FPA research is termed
“comparative foreign policy.” Proponents of comparative foreign policy sought to achieve
comprehensive theories of foreign policy behavior through quantitative analysis of “events”
data. An important strand of this behavioral work addressed the relationship between trade
dependence and foreign policy compliance. On the other hand, second-generation FPA
methodology largely abandoned universalized theory-building in favor of historical methods
and qualitative analysis. Second-generation FPA researchers place particular emphasis on
developing case study methodologies driven by social science principles. Meanwhile, the
third-generation of FPA scholarship combines innovative quantitative and qualitative methods.
Several methods of foreign policy analysis used by third-generation FPA researchers include
computer assisted coding, experiments, simulation, surveys, network analysis, and prediction
markets. Ultimately, additional attention should be given to determining the degree to which
current methods of foreign policy analysis allow predictive or prescriptive conclusions. FPA
scholars should also focus more in reengaging foreign policy analysis with the core of
international relations research.
Keywords: foreign policy analysis, methodological approaches, comparative foreign policy, events data
analysis, case study methodologies, network analysis, prediction markets, foreign policy behavior
Subjects: Foreign Policy
Introduction
The periodic reassessment of research methods is important to the vitality of any academic
discipline, but it has particular salience for a relatively young field such as foreign policy
analysis (FPA). Hudson and Vore (1995:221) acknowledge as much in their review of the FPA
literature, noting that, “in the study of foreign policy decision-making, the issues are not
theoretical but methodological.” I define foreign policy analysis as the study of how states, or
the individuals that lead them, make foreign policy, execute foreign policy, and react to the
Page 1 of 28
Printed from Oxford Research Encyclopedias, International Studies. Under the terms of the licence agreement, an individual user
may print out a single article for personal use (for details see Privacy Policy and Legal Notice).
date: 17 January 2022
foreign policies of other states. This topical breadth results in a subfield that encompasses a
variety of questions and levels of analysis, and a correspondingly diverse set of
methodological approaches. This essay surveys FPA’s methodological development from its
inception to the present and, in the process, outlines the body of existing methodological
practice and identifies opportunities for future progress. The objective is to provide both an
indication of the role that various quantitative and qualitative methods play in the FPA
literature and an entryway for contemporary researchers seeking to apply these approaches
to future work. Where appropriate, the reader is directed to more specific guides to the
intricacies and execution of each method.
For the sake of organizational clarity, this review follows a stylized format roughly based on
Neack, Hey, and Haney’s (1995) concept of “generational change” in foreign policy analysis.
The section that immediately follows is partially archeological, that is, it surveys methods of
events data analysis that were important to the early development of FPA, but in some cases
have fallen out of widespread usage. The second section, which surveys qualitative methods,
most closely reflects the current state of the art in the discipline. The third and final section
addresses both cutting-edge and underutilized approaches.
The Methodological Origins of Foreign Policy Analysis
The unique historical context and intellectual environment of the early 1950s – specifically,
the Cold War and the behavioral revolution – crucially shaped the early methodological
development of foreign policy analysis. These origins have proven central to the
methodological arc of the sub-discipline.
FPA was born of the opportunities presented by the largely atheoretical nature of historically
oriented diplomatic analysis and the exclusion of political leadership and decision-making
from the prevailing theories espoused by mainline international relations. Prior to the advent
of FPA as a distinct subfield, the study of foreign policy relied on traditional methods and had
long been the domain of political historians and diplomatic strategists in the tradition of
thinkers such as Thucydides and Machiavelli. Early FPA researchers saw this longstanding
tradition as part of their heritage, but, inspired by the methodological imperatives of the
behavioral revolution, believed that systematizing the study of foreign policy would lead to
progress in the form of generalizable and cumulative findings. Thus, from its inception, FPA
was an explicitly theoretical exercise aimed at uncovering the systematic elements of foreign
policy interactions, and the methods deployed reflected this.
Simultaneously, in response to the near monopoly of system-level theory in international

relations, the pioneers of FPA argued that individual leaders or groups of decision makers are
often the primary drivers of outcomes in international interactions (Snyder et al. 1954). Thus,
at the very core of FPA’s intellectual identity lies a revisionist methodology (vis-à-vis
diplomatic history) applied to a revisionist conception of the basic unit of analysis (vis-à-vis
mainline international relations).
Page 2 of 28
The strategic environment, specifically the position of the US in the early Cold War, also
figured prominently in the early development of FPA methods. In the face of this protracted
geopolitical conflict, American political leaders became unusually involved in the FPA
academic endeavor. The promise of concrete conclusions and general enthusiasm for
“scientific” approaches to political problems that stemmed from the success of the Manhattan
Project led the US government to invest large sums in early FPA efforts. With funding came
the expansion of major research centers such as the Rand Corporation and the Brookings
Institution that were instrumental to the maturation of FPA as a subfield and methodological
approach in international relations. However, the money and attention from the policy
community came with strings attached – most notably, an expectation for immediately
relevant research. Over time this requirement became increasingly difficult to reconcile with
the relatively high uncertainty surrounding quantitative estimates of foreign policy
phenomenon.
The first major phase of FPA research that emerged from this crucible is termed “comparative
foreign policy.” Proponents of comparative foreign policy argued that controlled comparison
of the domestic sources of external conduct across different countries could produce
comprehensive theories of foreign policy behavior. Methodologically speaking, these scholars
sought to achieve these ends primarily through quantitative analysis of “events” data, which I
describe in detail in the section that follows. However, this transition to quantitative analysis
was, at least in part, a refinement of even earlier attempts to develop a more robust
understanding of the foreign-policy decision making process. Snyder, Bruck, and Sapin’s
(1954) classic essay was arguably the first to encourage international relations scholars to
reopen the “black box” of the state in order to study the actions of individual leaders. A
significant body of early qualitative case study research flowed from this call to arms. To take
just two examples, Paige (1968) took a decision making approach to understanding the origins
of the Korean War, while Allison (1971) followed along similar lines with his well-known study
of the Cuban Missile Crisis.
The decision making school provided a useful groundwork, particularly by identifying the
leader as a crucial unit of analysis, a tradition that has persisted in FPA ever since. However,
the developments of the behavioral revolution eventually overtook the primarily qualitative
methods of these early FPA scholars. An increasing premium was placed on the
generalizability garnered by operationalizing foreign policy interactions numerically and
analyzing them quantitatively. This transition gave rise to the comparative foreign policy
literature, which maintained an emphasis on decision making and scientific analysis, but
moved away from case study analysis in favor of events data.
Comparative Foreign Policy and Events Data Analysis

The demand for foreign policy research that was scientific, generalizable, and policy relevant
caught nascent foreign policy analysts unprepared. Where other areas of political science
could respond to the challenge presented by the behavioral revolution with numerical data
Page 3 of 28
already at their disposal, the traditional fodder for diplomatic analysis – histories, documents,
interviews, biographies, and memoirs – were less easily reduced into the sort of data
necessary for rigorous, quantitative hypothesis testing.
This reality set foreign policy analysis somewhat behind other areas of political science
because it had to overcome two distinct obstacles. First, new data had to be collected that
was better suited to statistical analyses. Second, methods had to be developed with which to
analyze these data within a behavioral framework. Among others, Rosenau (1966; 1968),
McClelland (1970), and Brecher et al. (1969) took up these early challenges.
These early foreign policy analysts sought to develop a quantifiable unit of foreign policy
interaction. McClelland conceived of this core unit of analysis as the foreign policy “event,”
which is simply a formalized observation of a conflictual or cooperative interaction between
states. McClelland’s intention was to fill the gap between the traditional narrative approach to
foreign policy analysis and empirical techniques that relied upon discrete quantifiable data
that could be explored in statistical analyses (Schrodt 1994). In effect, the foreign policy event
takes a qualitative observation of foreign policy interaction and reduces it to a numerical or
categorical form suited for statistical analysis.
The process of generating events data was and is time-consuming and costly. It is most
commonly accomplished through the content analysis of thousands of newspaper reports on
the interactions among nations in light of a previously defined set of criteria or codebook.
Each observation uncovered in this way is then assigned some numerical score or a
categorical code, which can then be analyzed quantitatively (Schrodt 1994). This potentially
lengthy process requires that the researcher accomplish some or all of the following: identify
sources, identify a period of analysis, create or borrow a coding scheme, train coders,
generate the data, and check for reliability.
Foreign policy scholars have generated a significant number of important events datasets that
remain central to quantitative methods of foreign policy analysis. The best of them are
impressive collections offering decades-long periods of analysis, coverage of many countries
(if not the entire international system) and standards of intercoder reliability well in excess of
80 percent (Burgess and Lawton 1972). The paragraphs that follow describe a subset of the
available data. Particular attention is given to projects that were seminal to the
methodological development of the field and those that generated datasets still widely used by
contemporary scholars.
The World Event/Interaction Survey (WEIS)

The World Event/Interaction Survey Project began at the University of Southern California
under the direction of Charles McClelland as a research project on the characteristics and
processes of the international system (McClelland and Hoggard 1969). The initial WEIS
dataset records the flow of action and response between countries (as well as non-
governmental actors such as NATO and the United Nations) captured from a daily content
analysis of the New York Times from January 1966 through December 1978. This reliance on
Page 4 of 28
the New York Times produces a well-known bias toward western perspectives, which was
acknowledged from the outset by McClelland and his co-authors. However, they argued that
by using a single source they were better able to remove the “noise” surrounding
observations. Furthermore, the inclusion of non-state actors raises important methodological
issues with regard to the basic unit of analysis. This question has taken on increased salience
with the rise in concern about terrorist activities by non-state international entities.
The basic unit of analysis in the dataset is the interaction, which is simply a verbal or physical
exchange between nations ranging from agreements to threats to military force. Each of these
observations is coded to identify the actors, target, date, action category, and arena. The
WEIS databank also provides brief qualitative textual descriptions of each event. These
narratives provide context, which facilitates the process of identifying and understanding
outliers and applying statistical findings back to political reality – both important for
successful events analysis. The initial WEIS effort has been continuously updated and is
presently current through 1993 (Tomlinson 1993). Other projects, such as the Kansas Event
Data System, have applied WEIS coding rules to new research.
WEIS data has been widely used in the FPA literature, both by McClelland and his students
and by outsiders who took advantage of these public domain data to test their own questions.
The applications are diverse, underlining the versatility of well-designed events datasets.
Several early examples are noted by Rummel (1979): Tanter (1974) used these data to
understand the dynamics of the two major Berlin crises of the Cold War (1948–1949 and
1961); Kegley et al. (1974) explored patterns of international conflict and cooperation; while
many others began the ongoing process of understanding the relationships among key
contextual variables such as relative development, size, and political system, on international
conflict, cooperation, and systemic stability (Rosenau 1974). Applications continue to this day.
For example, Reuveny and Kang (1996b) utilized WEIS data in their exploration of causality in
the relationship between international trade and conflict.
The Conflict and Peace Data Bank (COPDAB)

The COPDAB project was designed by Azar and colleagues (Azar 1980; 1982; Azar et al. 1972)
as a longitudinal dataset of international and domestic events developed through content
analysis of daily newspapers. In an advance over WEIS methods, COPDAB data is drawn from
a wide variety of international and regional media outlets, thereby avoiding some potential
bias issues.
COPDAB coders scored each event on a 16-point ordinal scale ranging from cooperative
interactions to full-scale violence. The resulting dataset covers the interactions of 135
countries from 1948 to 1978 and can be analyzed at levels of aggregation ranging from the
day to the year. Each record includes nine variables: date of event, actor initiating the event,
target of the event, issue area(s), contextual information about the incident, and the source of
Page 5 of 28
the information about the event. The COPDAB dataset is particularly useful for those
interested in the interactions between interstate and civil conflict and cooperation, as
complementary datasets exist for both international and domestic events.
While the WEIS and COPDAB datasets are clearly conceptually related, scholars have
disagreed about their compatibility (Howell 1983; Vincent 1983; Goldstein and Freeman
1990). The underlying definitions of conflict and cooperation are quite similar; however,
coding differences introduce the potential for inconsistencies. Reuveny and Kang (1996a)
explore this issue in detail with a series of statistical tests and time-series analyses. They
argue that COPDAB and WEIS are indeed compatible for the overlapping period between
1966 and 1978. Building on this logic, they combine the WEIS and COPDAB series to create a
larger events dataset covering the period from 1948 to 1993 that is potentially useful for
scholars interested in working with a longer period of analysis.
International Crisis Behavior Project (ICB)

Although the final two projects outlined here (the International Crisis Behavior project and
the Correlates of War [COW] project) are often excluded from discussions of foreign policy
analysis, they are clearly a continuation of events research and are among the most frequently
updated and widely used events datasets. The distinctive feature of the ICB and COW datasets
is that they primarily focus on international conflict and therefore lack the range of conflictual
and cooperative events that characterize the data projects discussed to this point.
Researchers should note, however, that the ICB project does provide some indirect data on
cooperation.
Michael Brecher and Jonathan Wilkenfeld launched the International Crisis Behavior project in
1975 with the goal of creating a comparative resource for those studying the concept of
“international crisis.” There are two defining conditions for a crisis, which are built on work
done by Azar (of the COPDAB project): “(1) a change in type and/or an increase in intensity of
disruptive, that is, hostile verbal or physical, interactions between two or more states, with a
heightened probability of military hostilities; that, in turn, (2) destabilizes their relationship
and challenges the structure of an international system – global, dominant, or
subsystem” (Brecher and Ben-Yehuda 1985).
The ICB project is congruent with many of the core concepts in FPA – for example, in the
operationalization of key elements of decision maker perception. This is perhaps unsurprising,
as many of the ICB’s primary researchers are steeped in the FPA tradition. To take one
example, Michael Brecher’s (1974) book on Israeli foreign policy decisions, which pre-dates
his work on the ICB project, is often cited as a seminal contribution to FPA that seeks to
characterize a nation’s psychological and cultural environment as an access point to an
understanding of its foreign policy.
Page 6 of 28
As of January 2009, the core systemic dataset that results from this definition codes 452
incidents from the end of World War I through 2006 (version 9.0). Each crisis is coded for a
number of variables, ranging from characterizations of decision maker perception to
operationalizations of structural and environmental factors as well as crisis characteristics
and outcomes.
The ICB project is unusual in that it proceeds simultaneously at multiple complementary

levels. There are independent actor and system level datasets that allow the researcher to
explore distinctions between systemic and national level explanations for crisis emergence
and behavior. In addition, the project provides qualitative data in the form of a brief narrative
description of each crisis, 9 in-depth volumes comprising 15 in-depth case studies; and 14
other unpublished studies. These serve as an aid to the researcher interested in adding
additional nuance to statistical findings generated from quantitative analysis.
Correlates of War Project (COW)

Like the ICB project, the COW project does not attempt to capture multiple tiers of conflict
and cooperation, but rather focuses on conflict. Two definitions were developed by the COW
project in the 1980s, namely, a “militarized interstate dispute” (MID), and a “militarized
interstate crisis” (MIC). The former is defined as: “[A] set of interactions between or among
states involving threats to use military force, displays of military force, or actual uses of
military force […] these acts must be explicit, overt, non-accidental, and government
sanctioned” (Gochman and Maoz 1984). This “evolves into a militarized interstate crisis when
a member of the interstate system on each side of the dispute indicates by its actions its
willingness to go to war to defend its interests or to obtain its objectives.” These are steps two
and three along a four-step ladder of growing belligerence, beginning with an “interstate
dispute” and culminating in an “interstate war” (Leng and Singer 1988).
The majority of scholars currently working with COW events data use the MID dataset. The
current version of the dataset contains 2331 militarized disputes from 1816 to 2001 coded for
duration, outcome, and level of fatality. In addition, there are several complementary datasets
on various metrics of international interaction (ranging from alliance to power to geography)
that are associated with the broader COW project and can be easily mapped onto the MID
dataset. This body of quantitative data is perhaps the most widely used at the present time –
particularly among scholars interested in conflict.
Trade Dependence and Foreign Policy Compliance

An important strand of the behavioral work of the 1970s and 1980s addressed the relationship
between trade dependence and foreign policy compliance. While this was far from the only
research question to draw on quantitative data, the methodological challenges that confronted
it were representative of those faced by quantitative FPA in general and are therefore worthy
of some attention. Several scholars working in this area (e.g., Richardson and Kegley 1980;
Moon 1983; 1985) argued that relatively smaller and weaker states adopt the foreign policies
Page 7 of 28
of their dominant trading partners. Thus, economic dependence severely constrains the
independent decision making of leaders in states that are economically reliant on larger
patrons. However, consensus on this conclusion was elusive, in large part because of how
difficult it is to measure the two key concepts – dependence and compliance. The inevitable
result was that discussion of the relationship became bogged down in issues of definition and
operationalization. This is symptomatic of a larger issue in the quantitative study of foreign
policy. Because the operationalization of the amorphous concepts in foreign policy
necessitates discretion from the researcher, it is easy to critique the underlying assumptions
that gave rise to the data, not to mention the model. Furthermore, if more than one scholar
takes on a question in FPA, they typically settle on different operationalizations of the same
underlying phenomenon. A high profile example of this trend can be found in the proliferation
of events datasets on conflict and cooperation that has already been discussed. The
unfortunate result is that many studies are not comparable or cumulative to the degree we
find in the hard sciences.
Events Data – Methodological Challenges

Events data analysis poses a number of methodological challenges that should be taken into
account by those analyzing foreign policy. The first of these issues relates to the very core of
the events data endeavor – that is, the idea that foreign policy incidents can be reduced to a
single quantifiable value. Despite the best efforts of the designers of the data projects
described here, it remains difficult to effectively accomplish a cardinal or even ordinal ranking
of disparate foreign policy events. However, many of the statistical approaches widely used in
political science require cardinal level data, or at least data spaced at even thresholds. As a
result, those seeking to generate statistical models of events data need to be particularly
careful to apply methods that rely upon defensible assumptions about the nature of the
underlying data.
Researchers should also be aware of methodological issues that may arise from the relative
sparseness of positive observations in events data. The degree to which this is a problem
depends on the type of model and the level of aggregation that is used, but if one considers
the daily probability of a foreign policy event it is apparent that null observations would
dominate the dataset. King and Zeng (1999; 2001) demonstrate that bias and inappropriately
inflated statistical significance may arise in models of zero-inflated data. This is particularly
problematic in instances where these null data contain no real information. There are several
potential solutions to this problem should it arise. Tomz, King, and Zeng (1999) suggest a rare
events correction for logistic analysis, which they have made available as an addition to the
widely used STATA software. A less sophisticated check for rare events bias is to simply drop a
random subset of null observations in order to confirm that findings derived from the
remaining sample are consistent with the original result.
The non-independence of foreign policy events presents an additional methodological

challenge. Non-independence simply means that positive foreign policy interactions tend to
contribute to future positive interactions, while negative events are associated with
Page 8 of 28
subsequent negative events. At first appearance this might seem obvious, but this reality
undercuts an assumption of independence that underpins most statistical models used in
quantitative foreign policy analysis. Beck, Katz, and Tucker (1998) did much of the work that
brought this problem to the attention of the discipline and they suggest a solution that entails
generating a natural cubic spline with knots at the first and second derivative.
FPA scholars working with events data should also guard against selection bias (sometimes
referred to as selection effects) when designing research, as inattention to this
methodological challenge can significantly skew findings from both quantitative and
qualitative tests. Selection bias typically arises from pre- or post-sampling that preferentially
includes or excludes a particular type of observation from the sample that is subsequently
used in testing. This is particularly easy to do when working with data on foreign policy
because it is relatively easy to identify events, but difficult to tease out non-events. The
trouble is that without an accurate characterization of non-events it is impossible to say
anything about the causes or incidence of the events. To take one prominent recent example
of this methodological challenge, Robert Pape’s recent work on the causes of suicide terror
(2005) has come under fire for “sampling on the dependent variable” (Ashworth et al. 2008).
Because Pape limits his sample to incidents of suicide terror, he effectively leaves out the
instances in which such attacks did not occur. As a result, his research design prevents him
from effectively speaking of when suicide terror does and does not occur.
Beyond issues related to the application of statistical methods to events data, there is an
additional conceptual concern regarding the unit of analysis that should command attention
from foreign policy researchers. Because FPA concerns the foreign policy of states, but sees
this policy as emerging from the actions of individuals, traditional units of analysis are
blurred. The foreign policy event is the result of the interaction and interplay between
leaders, organizations, institutions, and states; however, many of the microfoundational
theories that underpin the FPA endeavor are cast at the level of the individual decision maker.
As a result, events analysis brings with it the nascent challenge of explaining how individual
actions aggregate to the foreign policy actions of states. To put the issue more succinctly,
while FPA theories distinguish themselves from mainline international relations by opening
the black box of the state, the empirical data collected by scholars interested in events
analysis typically returned to the state as the central unit of analysis.
There are also very practical concerns to bear in mind – simple tasks related to data
manipulation remain some of the primary challenges confronting researchers interested in
working with events data. It can be a nontrivial task to gather and combine data on foreign
policy events with the various explanatory and control variables that are required for
regression analysis. Researchers confronted with these difficulties should be aware of the
EUGene software developed by Scott Bennett and Allan Stam (2000). EUGene is a basic data
management tool that simplifies quantitative analysis of foreign policy interactions. The
software offers several advantages. First, it allows for relatively easy transition between
commonly used units of analysis – country–year, dyad–year, and directed dyad–year. Second,
the software is capable of easily combining many of the events datasets discussed here with
basic demographic and geopolitical data including data uploaded by the user.
Page 9 of 28
Finally, there is the issue of collecting new events data. The substantial early investments in
projects like WEIS and COPDAB were made at the high point of governmental and
institutional enthusiasm for events datasets – both datasets were products of the National
Science Foundation’s well funded Data Development for International Research (DDIR)
project. However, DDIR funding and government and private support for events data
collection projects in general declined markedly by the mid-1990s. While this decline had
many causes, it was in part brought on by the difficulties that comparative foreign policy had
delivering on its early promise. It proved far more challenging than expected to build policy
relevant quantitative models with predictive capacity. The relative decline in interest on the
part of traditional funding sources raises the issue of how new events data might be
generated. Computer coding of electronically stored sources, which will be discussed in
greater detail later in this essay, has emerged as one way to address this dilemma.
Qualitative Methods of Foreign Policy Analysis
The behavioral revolution and Cold War politics proved fertile ground for the emergence of
FPA. However, the first major challenge for the young field also stemmed from this dual
heritage. The problem was that these divergent intellectual pedigrees gave rise to
methodological requirements that were at times mutually exclusive – on the one hand, an
imperative from behavioralism for generalizability, and, on the other, a low tolerance for error
on the part of Cold Warriors seeking to immediately inform policy with scientific findings. The
emerging recognition of this tension and the seemingly unavoidable high error terms in
quantitative models of foreign policy brought an end to the exuberance among academics and
the US government for quantitative, events-driven foreign policy research. Policy makers
backed away from direct involvement in the FPA endeavor, while academics tempered their
commitment to events data and quantitative methods. What emerged was a second generation
of FPA methodology, one that largely abandoned universalized theory-building in favor of
historical methods and qualitative analysis (Neack et al. 1995).
The primary weapon in the arsenal of second-generation FPA researchers is the case study.
However, this transition should not be viewed as a complete departure from that which came
before it. Many of these scholars place particular emphasis on developing case study
methodologies driven by social science principles, with the explicit goal of building techniques
that provide intellectual rigor comparable to that of quantitative approaches. The result has
been a robust discussion of the role and execution of qualitative methods.
It is admittedly artificial to divide methods of foreign policy analysis by “generation,” as this

implies clean transitions that are in reality far more blurred. While the concept of
generational change is useful for understanding the broad developments in the field, the
reader should be aware that there are many exceptions to the general rule. Alongside second
generation case studies were a wide range of quantitative approaches that, while often
abandoning the drive toward universalized theory that characterized previous work in
comparative foreign policy, stressed both the outputs and the outcomes of foreign policy
processes and actions. Similarly, careful qualitative analysis of foreign-policy decision making
Page 10 of 28
has always been an element of foreign policy analysis, and therefore cannot only be
considered to have followed sequentially on the quantitative work done in comparative foreign
policy (although it did take on renewed prominence).
Case Study Analysis

There is no shortage of examples of the excellent use of case study methodology in foreign
policy analysis. Graham Allison’s Essence of Decision (1971) is often cited as a seminal piece
of research in this area with an innovative methodological approach. While Allison’s volume is
concerned with a single incident – the Cuban missile crisis – the book is not a single case
study, but rather three. Allison explored the US decision making process in the context of
three competing explanatory theories: a rational actor model, an organizational process
model, and a government politics model. Each of these three explanatory models receives
independent analysis in a separate section of the book. Allison (1971:258) argues that these
three models combine to provide a clear understanding of decision making in the context of
the Cuban Missile Crisis: “Model I fixes the broader context, the larger national patterns, and
the shared images. Within this context, Model II illuminates the organizational routines that
produce the information, options, and action. Model III focuses in greater detail on the
individuals who constitute a government and the politics and procedures by which their
competing perceptions and preferences are combined.”
Another important strand of qualitative foreign policy analysis draws on work from political
psychology to theoretically inform case study analysis of the foreign policy decision making
process. These efforts began with “operational code analysis,” which involves determining
how decision makers’ core beliefs shape their foreign policy reactions (George 1969; Holsti
1970). Operational codes include decision makers’ beliefs about the likelihood of violence,
their ability to shape or prevent it, as well as leadership strategies and styles.
Robert Axelrod applies a related technique, termed cognitive mapping, to understand the
influence of leadership beliefs on foreign policy interactions. Cognitive mapping entails
defining a decision maker’s stated goals and then determining the causal linkages between
these goals as a way of predicting likely behavior. Several applications of this technique can
be found in an edited volume titled Structure of Decision (Axelrod 1976). A more recent
application of cognitive mapping can be found in Johnston’s (1995) work on Chinese–American
relations.
This early work developed into a substantial body of foreign policy analysis based more
broadly on the psychology of decision makers, a method that figures prominently in analyses
conducted at the individual level. Larson (1985) is a leading example of this sort of
scholarship. In her book, Origins of Containment, she traces the path of Cold War politics in
the context of the cognitive psychology of American policy makers.
A great deal of work has been done in recent years to improve and formalize case study
methodology. One such volume, King, Keohane, and Verba’s Designing Social Inquiry (1994),
has been influential (and controversial) enough that it is often referred to simply by the
Page 11 of 28
initials of its authors – KKV. King, Keohane, and Verba draw on their diverse methodological
backgrounds to argue that the core logic of causal inference and control should apply as much
to qualitative work as it does to quantitative research. They suggest that, by applying the logic
of statistics, it is possible to produce theoretically robust and generalizable results while
increasing certainty in the validity of qualitative findings.
Bennett and George’s (2005) more recent work on case study analysis has also emerged as an
important contribution to the development of robust qualitative methods. This book lays out
methods for designing case studies that are maximally useful for the formulation of policy,
which remains a fundamental goal of foreign policy analysis. Bennett and George suggest
greater emphasis on within-case analysis, process tracing, and theory building. While these
suggestions differ markedly from those of KKV, the underlying goal is quite similar – to create
scientific case studies from which lessons can be systematically drawn. In this sense, both
volumes speak convincingly to the aforementioned tension between nuance and
generalizability that plagues methods of foreign policy analysis.
This issue of generalizability has developed into the core methodological challenge
surrounding case study analysis both in foreign policy analysis and in political science more
generally. While systematic knowledge of foreign policy interactions does not necessarily
require the numerical comparability that comes with quantitative research, some degree of
generalizability remains important to the independent identity of foreign policy analysis, as it
is this forward-looking element that separates the sub-discipline from diplomatic history.
However, comparisons across cases are difficult for two reasons. First, case studies require
such a depth of knowledge and investment of time that it is unusual for a scholar to
accomplish more than a handful of them on any given question, though there are important
exceptions (e.g., Brecher 2008). Second, the comparatively loose structure of case studies can
hinder comparison, as many analyses fail to address the same subjects on the same terms.
One way that these challenges can be overcome is through collaboration within a consistent
framework.
An example of such collaboration can be found in a relatively recent volume edited by Beasley
et al. (2002). The volume brings together qualitative work from 15 independent researchers
systematically exploring the foreign policies of 13 states. Through the coordinating efforts of
the editor, the volume maintains a degree of comparability across the cases while drawing on
the deep knowledge of the individual contributors. As a result, the reader is able to engage in
comparative analysis within a coherent theoretical framework, allowing for the quick
identification of patterns and outliers. There are several examples of similarly structured
volumes, and they indicate an important role for collaboration as an approach to boosting the
sample sizes of qualitative analyses and thereby the generalizability of findings. The result is
“comparative foreign policy,” but of a qualitative variety.
Another interesting solution to the issue of case comparability is found in the qualitative
research that has emerged from the qualitative side of the International Crisis Behavior
Project, which was already mentioned in the context of events data analysis. These case
studies, though they were written over many years and appear in a variety of different outlets,
follow a similar format and concern themselves with a consistent set of issues. As a result,
Page 12 of 28
they are an explicitly cumulative effort. With each new case study, the body of comparable
knowledge increases and this expansion is accompanied by improvement in the robustness of
findings.
Gathering Qualitative Data

Those interested in applying case study methodology will need raw material with which to
build their analysis. For many questions, considerable ground can be covered using basic
library research techniques and secondary sources. However, some of the most fruitful case
studies (in terms of their contribution to the existing body of knowledge) bring new
information to light. There are several methods of obtaining original qualitative data. The
sections that follow will briefly discuss four methods that have become central in foreign
policy analysis: archival research, content analysis, interviews, and focus groups.
Archival Research
Original source material can be a crucial element of a quality case study. Typically, scholars
uncover such information through archival research. Relevant foreign policy materials are
commonly found in the document collections housed in presidential libraries, national
archives, and universities. While the basic concept behind archival research is self-
explanatory, the actual process of gaining access to collections and navigating their contents
can intimidate the neophyte. There are a number of guides to archival methods that can
alleviate such anxiety. Directions for identifying and searching appropriate archival sources as
well as tips for navigating the archives themselves can be found in Marc Trachtenberg’s
(2006) recent volume on methods – Appendix II will be of particular interest to those seeking
to utilize archival methods. Hill (1993), Larson (2001), and Lustick (1996) provide additional
detail on the nuances of archival research.
Content Analysis
Content analysis is a hybrid method that has played a longstanding and important role in
quantitative and qualitative foreign policy analysis. The section of this essay on events data
already discussed the ways in which content has been used to generate quantitative data for
statistical analysis. For example, some of the earliest approaches to events data generation
coded the content of elite communication (Winham 1969). However, more detailed content
analyses have also been used to generate the raw material for case studies or other
qualitative analyses. Ole Holsti (1969) was a pioneer of this method, while, more recently,
Steve Walker and his students at Arizona State have developed a typology and quantitative
content analysis scheme for operational code analysis (Walker et al. 1998). Those interested in
additional detail on the mechanics of content analysis should consult Weber (1985),
Neuendorf (2002), and West (2001).
Page 13 of 28
Interviews
Because the role of the individual figures so prominently in foreign policy analysis, interviews
can be a particularly valuable method for accessing information about the mechanics of the
decision making process. Interviews enable FPA scholars to delve deeply into the
idiosyncrasies of the foreign policy process, gleaning deep insights from decision makers and
those around them. Over time, FPA scholars have developed a robust set of interview methods
designed to enable researchers to maximize the acquisition of information without introducing
biases into findings.
There are a number of excellent examples of innovative interview methods in foreign policy
analysis, which can serve as models for those interested in interview research. Prime among
them are FPA classics such as Yuen Foong Khong’s Analogies at War (1992). Based on a series
of interviews with senior officials (and archival research), Khong argues that leaders routinely
reference the past when making foreign policy decisions and that this cognitive bias can
profoundly alter decision making. Schoutlz (1987) does similar interview work in the context
of US policy toward Latin America. More recently, Silber and Little (1995) draw on a series of
interviews to uncover the foreign policy interactions at play in the collapse of the former
Yugoslavia. Berg (2001), Brenner et al. (1985), McCracken (1988), Mishler (1986), and
Seidman (1998) provide useful, in-depth tutorials on interview methods.
Focus Groups
Focus group research is a derivative of interview methodology in which the researcher
attempts to facilitate an organized discussion among the participants. In foreign policy
analysis this typically takes the form of a meeting of experts in a particular foreign policy
area, or participants in a prior foreign policy decision. Focus group methods can be
particularly informative because the emerging consensus that comes from such discussions
pools the knowledge of the participating individuals and therefore can overcome some of the
potential biases of recollection and self-inflation that accompany individual interviews.
However, concerns arise as well, due to some of the very same pathologies that FPA scholars
have identified in the context of group decision making. For example, Janis’s (1972) concept of
groupthink can take hold in such settings, with focus group members avoiding controversy
and settling instead on a comfortable consensus, even if this consensus is out of step with
reality. Along similar lines, the value of elite focus groups can suffer due to deference to
higher-ranking participants and domination of the discussion by more talkative individuals
who might overshadow important contributions by those less inclined to assert themselves
(Krueger 2000).
Third Generation Methods of Foreign Policy Analyses
Neack, Hey, and Haney’s (1995) concept of generational change, to which this review has
adhered thus far, captures only part of the methodological richness of FPA. There have long
been methods of foreign policy analysis that fall outside this strict quantitative/qualitative
Page 14 of 28
divide, and there has been considerable recent growth in these alternative methods.
Meanwhile, the distinction between quantitative and qualitative approaches to FPA has
become increasingly blurred as the relative advantages of each approach have become more
widely recognized. These events auger the arrival of a third generation of FPA scholarship
that combines innovative quantitative and qualitative methods, thereby bridging the internal
contradictions that split the second wave from the first and unifying a variety of methods of
foreign policy analysis.
Several methods of foreign policy analysis are available to aspiring third generation foreign
policy analysts seeking to move beyond events data and case studies including: computer
assisted coding, experiments, simulation, surveys, network analysis, and prediction markets.
The sections that follow will briefly introduce each of these methods, though the list is by no
means exhaustive.
Machine Coding
Computer assisted coding of electronically stored information offers several advantages and
represents an important methodological innovation that is likely to play an increasingly
significant role in the future of foreign policy analysis. First, machine coding can be more
reliable than human coding simply because it removes the possibility of individual error and
the resulting questions of intercoder reliability. Second, machine coding is extremely rapid.
Where earlier events datasets were generated over periods of many years, computers can sift
through huge quantities of data in minutes. The result is that machine coding greatly reduces
the cost of events data generation – effectively bringing control over such data to the masses
(Gerner et al. 1994; Schrodt and Gerner, 1994). However, the obvious benefits of machine
coding are accompanied by two important caveats: the initial programming that creates the
coding rules must be accurate, and the raw data must exist in a machine readable format
(Gerner et al. 1994). Advocates of human coding often counter that the low cost and speed of
machine coding are accomplished at the expense of accuracy and nuance.
At present, the best example of a machine coded events project is the Kansas Event Data
System (KEDS). This project is among the most active events datasets, due in part to the
relatively low cost and speed of generating data in this manner. KEDS provides a computer
program that enables users to specify and create personalized events datasets with a variety
of output options. The researchers on the KEDS team use this software to code news reports
and generate political event data focusing on the Middle East, the Balkans, and West Africa;
however, this approach can be extended to other regions or the international system as a
whole.
The machine coding community, including members of the KEDS project, is particularly
interested in predictive models built on the unique capacities of this technology (Schrodt,
1979; 1994; Gerner et al. 1994; Schrodt and Gerner 2000; Shellman forthcoming). Machine
coding not only partially circumvents the need for large financial investments in events data
by reducing the required labor and time, but also has the potential to address some of the
concerns about the lack of predictive capacity that caused the decline in external funding in
Page 15 of 28
the first place. Because machine coding concentrates the researcher’s effort on developing
decision rules rather than on the coding itself, once underway these programs can generate
empirical data in real time. Such models that draw on continuously updated data effectively
serve as early warning systems capable of identifying when political phenomena of interest
are likely to occur. For example, Shellman and Stewart use machine coding to predict
incidents of forced migration, which they applied with some success in Haiti (Shellman and
Stewart 2007). This particular application of events data remains at the cutting edge of the
FPA literature and will likely continue to be a productive avenue for future research.
Experiments and Simulation in Foreign Policy Analysis

Like all social science, foreign policy analysis struggles methodologically with the issues of
control and causality. The quantitative and qualitative methods already discussed took hold in
foreign policy analysis in part because the gold standard of the scientific method –
experimental control – is typically off limits either for practical or for ethical reasons.
However, with careful attention to design and feasibility, there are applications for
experimental methods in the study of foreign policy, and where there are not, researchers
have begun to turn their attention to simulation, which can achieve some of the same
objectives. To take one recent example, Christensen and Redd (2004) examine how the
context of foreign-policy decision making affects choice and assess this relationship in a
controlled experiment conducted on undergraduates. They find that, at least in this context,
the way in which information is presented directly affects the decision maker’s evaluation.
In recent years the nuts and bolts of experimental methods have drawn increasing attention.
Along these lines, McDermott (2002) provides an interesting discussion of the origins and
practice of experimental methods in political science, as well as the unique challenges this
approach presents. One such challenge that should be considered carefully by those designing
experiments meant to speak to foreign policy behavior is the trade-off between internal and
external validity in experiments. Internal validity indicates that the proposed relationship
between the independent and dependent variables is the true causal one. When such validity
is high it means that extraneous variables and alternative explanations have been ruled out.
While typically very difficult to achieve in the social sciences, high internal validity results
from proper randomization in an experiment. External validity speaks to the degree to which a
proposed relationship is generalizable to a broader set of cases or the world at large. Thus,
experimental methods are powerful because they are high in internal validity; however, a leap
occurs when we attempt to generalize experimentally derived results to actual political
behavior.
This leap can be particularly worrisome when it is from an experimental finding generated
from a non-elite individual – for example, an undergraduate student as was the case in the
Christensen and Redd study – to a foreign policy decision maker. In such cases, the
assumption of external validity may not be reasonable. Mintz, Redd, and Vedlitz (2006)
explore this issue in detail, replicating an experiment on the subject of counterterrorism with
a group of college student and a group of military officers. The authors find significant
Page 16 of 28
differences between these groups, suggesting that experimental subjects cannot be expected
to play the role of foreign-policy decision makers without careful regard for their actual
background. However, while these scholars argue that average individuals can tell us very
little about the behavior of elites, they do find it more acceptable to use subjects like students
as a sample of the public at large.
Simulation, a close relative of experimental methods, has its roots in the longstanding
practices of war gaming and diplomatic analysis. However, recent efforts in this area draw
extensively on advances in computing power and the internet. Research in this area builds on
early work by the Inter-Nation Simulation (INS) project (Guetzkow et al. 1963), and slightly
later efforts by Hermann (1969) and Alker and Brunner (1969).
The International Communication and Negotiation Simulations (ICONS) project is an ongoing

extension of this early work that allows political practitioners and students to develop decision
making and foreign policy skills through computer aided interactive simulation. Jonathan
Wilkenfeld and Richard Brecht developed ICONS in the 1980s, building on Noël’s (1969) early
POLIS simulations. As presently formulated, the ICONS project is more about training than
research, but the technique presents an intriguing methodological opportunity for those
interested in testing theories of foreign policy interactions in a controlled environment.
Survey Research in Foreign Policy Analysis

When it is focused at the elite level, as it often is, survey research in foreign policy analysis
directly relates to the previously discussed interview methods. This stands in some contrast to
the way in which survey research is conducted in other areas of political science. For example,
in American politics there is a long tradition of survey research designed to pinpoint public
opinion on a myriad of topics. In order to accomplish this, researchers are obliged to reach as
representative a sample of the population as possible. In contrast, FPA’s focus on elite
perception and behavior as a determinant of foreign policy leads to the wider usage of elite
interviews.
While surveys lack the depth of an interview, they offer the corresponding advantage of
breadth. First, by aggregating information from a more significant number of sources, a
survey can minimize some of the idiosyncratic error that can plague interview methodology.
Second, in a survey analysis it is easier to control for secondary variables that might influence
the recollection or reporting of subjects. Finally, surveys can both contribute to qualitative
analysis, and serve to generate high-quality data for aggregate analysis.
Holsti and Rosenau’s (1979; 1980) work on post-Vietnam attitudes is an excellent example of
what can be accomplished in foreign policy analysis with elite surveys. Holsti and Rosenau
were interested in the degree to which historical experience altered the perceptions and
beliefs of opinion leaders and decision makers. Their expectation was that the Vietnam
conflict significantly altered the perspective of those who drew their primary experience from
that conflict rather than World War II. To answer this question they extensively surveyed
Page 17 of 28
groups that they believed to comprise the national leadership structure – military personnel,
foreign service officers, business executives, labor leaders, clergy, media, etc. – and found
significant differences between occupations and within generations.
Surveys can be particularly valuable when conducted repeatedly over several years, as this
allows for longitudinal analysis – something that is crucial if one is interested in changes over
time. Both the Chicago Council on Foreign Relations and the Pew Research Center for the
People and the Press conduct quadrennial surveys of government, academic, military,
religious, and scientific “influentials” in order to measure the content of and changes in elite
opinion. These surveys, and others that could be conducted along similar lines, are an
underutilized resource for foreign policy analysis. Presser et al. (2004) and Rea and Parker
(2005) are useful resources for those seeking additional detail on the mechanics of survey
research and questionnaire design.
Network Analysis
FPA scholars can also benefit from the recent explosion of interest among political scientists
in network analysis. Social network analysis, which is simply the mapping and measuring of
relationships among entities in a complex system, is a useful tool for modeling foreign policy
relationships because it incorporates both bilateral connections and wider connections among
the larger group. Because of this, the technique analysis allows FPA scholars to understand
relational data – the contacts, ties, connections, and transfers between decision makers that
cannot be cleanly reduced to properties of the leaders themselves (Scott 1991). Furthermore,
a network theoretic framework consistently captures the role of third parties in foreign policy
interactions, which prove to be crucial to understanding outcomes.
Relational approaches have long been an underlying element in the study of foreign policy.
For example, Anne-Marie Slaughter (2004) writes on the relationship between elite networks
and international conflict. However, quantitative social network analysis first began to make
significant inroads into political science in the 1990s primarily through the study of “policy
networks” (Marin and Mayntz 1992; Marsh and Rhodes 1992), though there are earlier,
pioneering examples (e.g., Eulau and Siegel 1981; Tichy et al. 1979). These studies, as well as
later work in international relations (e.g., (Hammarström and Birger 2002; Wilkinson 2002;
Montgomery 2005; Heffner-Burton and Montgomery 2006; Maoz 2006; Ward 2006), provide
models for future work with foreign policy networks. In short, relational thinking and social
network analysis have already contributed to the clarification of a number of puzzles in
political science and present a potentially powerful way of approaching foreign policy
analysis.
Prediction Markets
Prediction markets are information exchanges built to generate forecasts using a price
mechanism. Futures generated from predictions of upcoming events are traded, such that
their value is tied to a particular outcome. The result of this arrangement is that the market
Page 18 of 28
prices of these futures can be interpreted as the predicted probability of that outcome. There
is a significant body of research that establishes the ability of markets to reduce error in
predictions. By aggregating the bets of many individuals, these markets effectively use the
price setting mechanism to uncover the consensus about a future foreign policy event in much
the same way that the stock market predicts the economic performance of a company or oil
futures respond to the expected scarcity of that resource. Pennock et al. (2001) demonstrate
that in many cases prediction markets systematically outperform the estimates of even the
best individual analysts. There are only a few examples of longstanding prediction markets
that handle political futures. These include Intrade, which floats, among many other things, a
diverse group of political contracts, and the longer running Iowa Electronic Market, which is
an academically oriented project designed for evaluating the probability of election outcomes.
Prediction markets have been applied sparingly in international relations and foreign policy
analysis, but have tremendous potential for future application because they offer an
interactive mechanism with which individual foreign policy experts can aggregate their
knowledge and opinions. Interestingly, given the methodological diversity that characterizes
FPA as a sub-discipline, the method by which each expert who trades futures on a prediction
market reaches his or her own conclusion is irrelevant. Thus, a prediction market can provide
an alternative way to combine and generalize both deep qualitative knowledge and
quantitative findings. Furthermore, this approach presents a novel way of dealing with error
and uncertainty.
Prospective researchers in this area should note that some early applications of this approach
have not gone smoothly. The Defense Advanced Research Projects Agency (DARPA) recently
abandoned a promising plan to use a futures market to forecast the probability of important
foreign policy events such as regime change and terrorist attacks when the media picked up
on the program and it became controversial. Despite a robust literature on the efficacy of
such markets, politicians and segments of the public seized upon the effort as being unethical
or even nonsensical (Looney 2003). The unwanted attention led DARPA, which usually
operates well beneath the public radar, to cancel the project almost immediately. It remains
an open question whether this approach will become more politically feasible – seemingly a
necessity because these markets generally require a significant initial investment, presumably
by a government or university. However, private markets such as Intrade, which is a for-profit
enterprise, seem to be a plausible alternative. Foreign policy futures, such as the probability
of an Israeli attack on Iran, are traded regularly on Intrade and provide useful information
about expectations. Moreover, futures on the outcome of the last presidential election vied
with polling data for public and media attention in the lead up to the 2008 US presidential
election suggesting that familiarity with these markets may be rising.
Remaining Methodological Challenges
Methods of foreign policy analysis have developed markedly over the past few decades, but
challenges remain. An unavoidable tension persists between the accuracy needed for policy
relevance and the scope needed for generalizability. As the grand theories of foreign policy
Page 19 of 28
interaction motivated those who launched the FPA enterprise proved elusive, the discipline
increasingly turned to the nuanced examinations of cases. However, if taken too far this trend
is a threat to the unique identity of FPA because it blurs the distinction with longstanding
traditions of historical analysis. This survey of available methods suggests that a partial
solution to this dilemma lies in bringing quantitative analysis and underutilized “third
generation” methods back into the FPA fold by reintegrating them into the well-developed
qualitative tradition. The goal should be to develop a healthy mix of methods that applies each
approach to the questions which each is best equipped to address.
Additional attention should also be given to determining the degree to which current methods
of foreign policy analysis allow predictive or prescriptive conclusions. In recent years,
enthusiasm for FPA has been fueled in part by the failure of most international relations
scholarship to accurately foresee key events in the international system – specifically the
decline of the Soviet Union and the end of the Cold War. The argument is made that Cold War
politics, because they were in some sense stable or at equilibrium, were better suited to
elegant and parsimonious models of the systemic behavior of state actors. In contrast, the
more chaotic world we presently inhabit is characterized by fluidity driven by human agents
and therefore is best understood using the methods of foreign policy analysis (Hudson and
Vore 1995). This is a reasonable hypothesis; however, prediction is a difficult game in the
social sciences and it remains unclear whether FPA is indeed superior in this arena. In short,
with a few notable exceptions such as the KEDS project, methods of foreign policy analysis
lack predictive capacity and, when they are able to predict, are often unable to clearly state
the degree of certainty surrounding these forecasts. More can and should be done to improve
this capacity.
Foreign policy analysts should also give deeper consideration to the issues that accompany
the choice of the unit of analysis in their models. FPA derives much of its explanatory power
from its ability to speak to the individual’s role in the foreign policy process, but the
dependent variables that these efforts attempt to explain are often the interactions between
states. The result is a gap in our understanding of the process of aggregation by which the
behavior of leaders results in the actions and reactions of states. This aggregation problem is
widely noted, but additional work is required to complete our understanding of this element of
the foreign policy process. Improvements in this linkage between theory and test, as well as a
consistent unit of analysis (individual or foreign policy event) are particularly crucial for
robust quantitative analysis, as it is in part the inability of the subfield to resolve this basic
issue that stifled earlier research on events data.
Finally, more must be done to reengage foreign policy analysis with the core of international
relations research. FPA scholars typically claim the first and second image as their domain,
but fail to engage with those in mainline international relations who also work in this area. In
the lead essay of the first issue of Foreign Policy Analysis, Valerie Hudson (2005) convincingly
makes the case that FPA has the potential to reshape the entire discipline of international
relations by focusing attention on the workings of the fundamental unit of analysis – the
Page 20 of 28
political decision maker. However, despite the call to arms, more often than not FPA scholars
labor in relative isolation. Some of these divisions emerge from methodological issues and can
therefore be resolved.
In sum, the future of foreign policy analysis appears to be bright. There is reason to believe
that longstanding methodological battles that characterized it are drawing to a close with the
recognition that multiple methods have their place in the study of foreign policy. In addition,
new methods and questions are emerging that are likely to contribute to our understanding of
the foreign policy process.
References
Alker, H.R., and Brunner, R.D. (1969) Simulating International Conflict – Comparison of 3
Approaches. International Studies Quarterly 13 (1), 70–110.
Allison, G.T. (1971) The Essence of Decision: Explaining the Cuban Missile Crisis. Boston: Little,
Brown.
Ashworth, S., Clinton, J.D., Meirowitz, A., and Ramsay, K. (2008) Design, Inference, and the
Strategic Logic of Suicide Terrorism. American Political Science Review 102 (2), 269–73.
Axelrod, R. (1976) Structure of Decision: The Cognitive Maps of Political Elites. Princeton:
Princeton University Press.
Azar, E.E. (1980) The Conflict and Peace Data Bank (COPDAB) Project. Journal of Conflict
Resolution 24, 143–52.
Azar, E.E. (1982) The Codebook of the Conflict and Peace Data Bank (COPDAB). College Park:
Center for International Development, University of Maryland.
Azar, E.E., Cohen, S.H., Jukam, T.O., and McCormic, J.M. (1972) Problem of Source Coverage in
Use of International Events Data. International Studies Quarterly 16 (3), 373–88.
Beasley, R.K., Kaarbo, J., Lantis, J.S., and Snarr, M.T. (2002) Foreign Policy in Comparative
Perspective: Domestic and International Influences on State Behavior. Washington, DC: CQ
Press.
Beck, N., Katz, J.N., and Tucker, R. (1998) Taking Time Seriously: Time-Series-Cross-Section
Analysis with a Binary Dependent Variable. American Journal of Political Science 42 (4), 1260–
88.
Bennett, A., and George, A.L. (2005) Case Studies and Theory Development in the Social
Sciences. Cambridge: MIT Press.
Bennett, D.S., and Stam, A. (2000) EUGene: A Conceptual Manual. International Interactions 26,
179–204.
Page 21 of 28
Berg, B.L. (2001) Qualitative Research Methods for the Social Sciences, 4th edn. Boston: Allyn
and Bacon.
Brecher, M. (1974) Decisions in Israel’s Foreign Policy. London: Oxford University Press.
Brecher, M. (2008) International Political Earthquakes: Crises and Conflicts Before, During and
After the Cold War. Ann Arbor: University of Michigan Press.
Brecher, M., and Ben-Yehuda, H. (1985) System and Crisis in International Politics. Review of
International Studies 11 (1), 17–36.
Brecher, M., Steinberg, B., and Stein, J. (1969) A Framework for Research on Foreign Policy
Behavior. Journal of Conflict Resolution 13 (1), 75–94.
Brenner, M., Brown, J., and Canter, D. (eds.) (1985) The Research Interview. London: Academic
Press.
Burgess, P.M., and Lawton, R.W. (1972) Indicators of International Behavior: An Assessment of
Events Data Research. Beverly Hills: Sage.
Christensen, E.J., and Redd, S.B. (2004) Bureaucrats Versus the Ballot Box in Foreign Policy
Decision Making: An Experimental Analysis of the Bureaucratic Politics Model and the
Poliheuristic Theory. Journal of Conflict Resolution 48 (1), 69–90.
Eulau, H., and Siegel, J.W. (1981) Social Network Analysis and Political Behavior: A Feasibility
Study. The Western Political Quarterly 34 (4), 499–509.
George, A.L. (1969) Operational Code – Neglected Approach to Study of Political Leaders and
Decision-Making. International Studies Quarterly 13 (2), 190–222.
Gerner, D.J., Schrodt, P.A., Francisco, R.A., and Weddle, J.L. (1994) Machine Coding of Event
Data Using Regional and International Sources. International Studies Quarterly 38 (1), 91–119.
Gochman, C.S., and Maoz, Z. (1984) Military Interstate Disputes, 1816–1976: Procedures,
Patterns, and Insights. Journal of Conflict Resolution 28 (4), 585–615.
Goldstein, J.S., and Freeman, J. (1990) Three Way Street, Strategic Reciprocity in World Politics.
Chicago: University of Chicago Press.
Guetzkow, H., Alger, C.F., Brody, R.A., Noel, R.C., and Snyder, R. (1963) Simulations in
International Relations: Developments for Research and Teaching. Englewood Cliffs: Prentice-
Hall.
Hammarström, M., and Birger, H. (2002) The Diffusion of Military Intervention: Testing a
Network Position Approach. International Interactions 28, 335–77.
Heffner-Burton, E., and Montgomery, A.H. (2006) Power Positions: International Organizations,
Social Networks, and Conflict. Journal of Conflict Resolution 50 (1), 3–27.
Page 22 of 28
Hermann, C. (1969) Crises in Foreign Policy: A Simulation Analysis. Indianapolis: Bobbs Merrill.
Hill, M.R. (1993) Archival Strategies and Techniques. Beverly Hills: Sage Publications.
Holsti, O.R. (1969) Content Analysis for the Social Sciences and Humanities. Reading: Addison
Wesley.
Holsti, O.R. (1970) The “Operational Code” Approach to the Study of Political Leaders: John
Foster Dulles’ Philosophical and Instrumental Beliefs. Canadian Journal of Political Science /
Revue Canadienne de Science Politique 3 (1), 123–57.
Holsti, O.R., and Rosenau, J.N. (1979) Vietnam, Consensus, and the Belief Systems of American
Leaders. World Politics 32 (1), 1–56.
Holsti, O.R., and Rosenau, J.N. (1980) Does Where You Stand Depend on When You Were Born?
The Impact of Generation on Post-Vietnam Foreign Policy Beliefs. Public Opinion Quarterly 44
(1), 1.
Howell, L.D. (1983) A Comparative-Study of the WEIS and COPDAB Data Sets. International
Studies Quarterly 27 (2), 149–59.
Hudson, V.M. (2005) Foreign Policy Analysis: Actor-Specific Theory and the Ground of
International Relations. Foreign Policy Analysis 1 (1), 1–30.
Hudson, V.M., and Vore, C.S. (1995) Foreign Policy Analysis Yesterday, Today, and Tomorrow.
International Studies Quarterly 39, 209–38.
Janis, I.L. (1972) Victims of Groupthink. Boston: Houghton Mifflin Company.
Johnston, A.I. (1995) Cultural Realism: Strategic Culture and Grand Strategy in Chinese History.
Princeton: Princeton University Press.
Kegley, C., Jr, Salmore, S.A., and Rosen, D.J. (1974) Convergences in the Measurement of
Interstate Behavior. In P.J. McGowan (ed.) Sage International Yearbook of Foreign Policy Studies,
vol. 2. Beverly Hills: Sage Publications, pp. 309–39.
Khong, Y.F. (1992) Analogies at War: Korea, Munich, Dien Bien Phu, and the Vietnam Decisions
of 1965. Princeton: Princeton University Press.
King, G., Keohane, R.O., and Verba, S. (1994) Designing Social Inquiry. Princeton: Princeton
University Press.
King, G., and Zeng, L.C. (1999) Logistic Regression in Rare Events Data. At http:/
gking.harvard.edu/files/0s.pdf <http://gking.harvard.edu/files/0s.pdf>, accessed July 2009.
King, G., and Zeng, L.C. (2001) Explaining Rare Events in International Relations. International
Organization 55 (3), 693–715.
Page 23 of 28
Krueger, R. (2000) Focus Groups: A Practical Guide for Applied Research, 3rd edn. Thousand
Oaks: Sage Publications.
Larson, D.L. (2001) Sources and Methods in Cold War History: The Need for a New Theory-
Based Archival Approach. In C. Elman and M.F. Elman (eds.) Bridges and Boundaries: Historians,
Political Scientists, and the Study of International Relations. Cambridge: MIT Press, pp. 327–50.
Larson, D.W. (1985) Origins of Containment: A Psychological Explanation. Princeton: Princeton

University Press.
Leng, R.J., and Singer, J.D. (1988) Militarized Interstate Crises – the Bcow Typology and its
Applications. International Studies Quarterly 32 (2), 155–73.
Looney, R.E. (2003) DARPA’s Policy Analysis Market for Intelligence: Outside the Box or Off the
Wall? Strategic Insights 2 (9).
Lustick, I.S. (1996) History, Historiography, and Political Science: Multiple Historical Records
and the Problem of Selection Bias. American Political Science Review 90 (3), 605–18.
Maoz, Z. (2006) Network Polarization, Network Interdependence, and International Conflict,

1816–2002. Journal of Peace Research 43 (4), 391–411.
Marin, B., and Mayntz, R. (eds.) (1992) Policy Networks: Empirical Evidence and Theoretical
Considerations. Boulder: Westview Press.
Marsh, D., and Rhodes, R.A.W. (eds.) (1992) Policy Networks in British Government. New York:
Oxford University Press.
McClelland, C.A. (1970) Some Effects on Theory from the International Event Analysis
Movement. Working paper. University of Southern California.
McClelland, C.A., and Hoggard, G. (1969) Conflict Patterns in the Interactions Among Nations.
In J.N. Rosenau (ed.) International Politics and Foreign Policy. New York: Free Press, pp. 711–24
McCracken, G. (1988) The Long Interview. London: Sage.
McDermott, R. (2002) Experimental Methodology in Political Science. Political Analysis 10 (4),

325–42.
Mintz, A., Redd, S.B., and Vedlitz, A. (2006) Can We Generalize from Student Experiments to the
Real World in Political Science, Military Affairs, and International Relations? Journal of Conflict
Resolution 50 (5), 757–76.
Mishler, E.G. (1986) Research Interviewing: Context and Narrative. Cambridge: Harvard
University Press.
Montgomery, A.H. (2005) Proliferation Determinism or Pragmatism? How to Dismantle an

Atomic Bomb Network. International Security 30 (2), 153–87.
Page 24 of 28
Moon, B.E. (1983) The Foreign Policy of the Dependent State. International Studies Quarterly 27
(3), 315–40.
Moon, B.E. (1985) Consensus or Compliance? Foreign-Policy Change and External Dependence.
International Organization 39 (2), 297–329.
Neack, L., Hey, J.A.K., and Haney, P.J. (1995) Generational Change in Foreign Policy Analysis. In
L. Neack, J.A.K. Hey, and P.J. Haney (eds.) Foreign Policy Analysis. Contiguity and Change in its
Second Generation. Englewood Cliffs: Prentice Hall.
Neuendorf, K.A. (2002) The Content Analysis Guidebook. Beverly Hills: Sage.
Noël, R.C. (1969) The Polis Laboratory. American Behavioral Scientist 12 (6), 30–35.
Paige, G. (1968) Korean Decision, June 24–30, 1950. New York: Free Press.
Pape, R.A. (2005) Dying to Win: The Strategic Logic of Suicide Terrorism. New York: Random
House.
Pennock, D.A., Lawrence, S., Giles, C.L., and Nielsen, F.A. (2001) The Real Power of Artificial
Markets. Science 291 (5506), 987–8.
Presser, S., Rothgeb, J.M., Couper, M.P., et al. (2004) Methods for Testing and Evaluating Survey
Questionnaires. New York: Wiley.
Rea, L.M., and Parker, R.A. (2005) Designing and Conducting Survey Research: A
Comprehensive Guide. San Francisco: Jossey-Bass.
Reuveny, R., and Kang, H. (1996a) International Conflict and Cooperation: Splicing COPDAB and
WEIS Series. International Studies Quarterly 40 (2), 281–305.
Reuveny, R., and Kang, H. (1996b) International Trade, Political Conflict/Cooperation, and
Granger Causality. American Journal of Political Science 40 (3), 943–70.
Richardson, N.R., and Kegley, C.W., Jr (1980) Trade Dependence and Foreign Policy Compliance:
A Longitudinal Analysis. International Studies Quarterly 24 (2), 191–222.
Rosenau, J.N. (1966) Pre-Theories and Theories of Foreign Policy. In J. Rosenau (ed.) Scientific
Study of Foreign Policy. New York: Free Press, pp. 95–151.
Rosenau, J.N. (1968) Comparative Foreign Policy – Fad, Fantasy, or Field. International Studies
Quarterly 12 (3), 296–329.
Rosenau, J.N. (ed.) (1974) Comparing Foreign Policies: Theories, Findings, and Methods. New
York: Halsted Press.
Rummel, R.J. (1979) Understanding Conflict and War, vol. 4: War, Power, Peace. Beverly Hills:
Sage Publications.
Page 25 of 28
Schoutlz, L. (1987) National Security and United States Policy Towards Latin America.
Princeton: Princeton University Press.
Schrodt, P.A. (1979) Forecasting in International-Relations – Theory, Methods, Problems,

Prospects – Choucri, N., Robinson, T.W. American Political Science Review 73(4), 1208–10.
Schrodt, P.A. (1994) Event Data in Foreign Policy Analysis. In L. Neack, P.J. Haney, and J.A.K. Hey
(eds.) Foreign Policy Analysis: Continuity and Change in its Second Generation. New York:
Prentice Hall.
Schrodt, P.A., and Gerner, D.J. (1994) Validity Assessment of a Machine-Coded Event Data Set for
the Middle-East, 1982–92. American Journal of Political Science 38 (3), 825–54.
Schrodt, P.A., and Gerner, D.J. (2000) Cluster-Based Early Warning Indicators for Political
Change in the Contemporary Levant. American Political Science Review 94 (4), 803–17.
Scott, J. (1991) Social Network Analysis: A Handbook. London: Sage.
Seidman, I. (1998) Interviewing as Qualitative Research: A Guide for Researchers in Education

and the Social Sciences, 2nd edn. New York: Teachers College Press.
Shellman, S.M. (forthcoming) Machine Coding Nonstate Actors’ Behavior in Intrastate Conflict.
Political Analysis.
Shellman, S.M., and Stewart, B. (2007) Predicting Risk Factors Associated with Forced
Migration: An Early Warning Model of Haitian Flight. Civil Wars 9 (2), 174–99.
Silber, L., and Little, A. (1995) The Death of Yugoslavia. London: Penguin.
Slaughter, A.-M. (2004) A New World Order. Princeton: Princeton University Press.
Snyder, R.C., Bruck, H., and Sapin, B.M. (1954) Decision-Making as an Approach to the Study of
International Relations. Princeton: Princeton University Press.
Tanter, R. (1974) Modelling and Managing International Conflicts: The Berlin Crises. Beverly
Hills: Sage Publications.
Tichy, N.M., Tushman, M.L., and Fombrun, C. (1979) Social Network Analysis for Organizations.
The Academy of Management Review 4 (4), 507–19.
Tomlinson, R.G. (1993) Monitoring WEIS Event Data in Three Dimensions. In R.L. Merritt, R.G.
Muncaster, and D.A. Zinnes (eds.) International Event-Data Developments: DDIR Phase II. Ann
Arbor: University of Michigan Press, pp. 55–85.
Tomz, M., King, G., and Zeng, L.C. (1999) RELOGIT: Rare Events Logistic Regression (Version
1.1). Cambridge. At http:/gking.harvard.edu/stats.shtml <http://gking.harvard.edu/stats.shtml>,
accessed July 2009.
Page 26 of 28
Trachtenberg, M. (2006) The Craft of International History. Princeton: Princeton University

Press.
Vincent, J.E. (1983) Weis Vs Copdab – Correspondence Problems. International Studies

Quarterly 27 (2), 161–8.
Walker, S.G., Schafer, M., and Young, M.D. (1998) Systematic Procedures for Operational Code
Analysis: Measuring and Modeling Jimmy Carter’s Operational Code. International Studies
Quarterly 42 (1), 175–89.
Ward, H. (2006) International Linkages and Environmental Sustainability: The Effectiveness of

the Regime Network. Journal of Peace Research 43 (2), 149–66.
Weber, R.P. (1985) Basic Content Analysis. Beverly Hills: Sage.
West, M.D. (2001) Theory, Method, and Practice in Computer Content Analysis. Westport: Ablex.
Wilkinson, D. (2002) Civilizations as Networks: Trade, War, Diplomacy, and Command-Control.

Complexity 8 (1), 82–6.
Winham, G.R. (1969) Quantitative Methods in Foreign Policy Analysis. Canadian Journal of
Political Science 2 (2), 187–99.
Links to Digital Materials

International Crisis Behavior Project. At www.cidcm.umd.edu/icb/ <http://www.cidcm.umd.edu/icb/>,
accessed July 2009. The ICB project provides quantitative and qualitative data on international crises. The core
systemic dataset currently contains 452 incidents from the end of World War I through 2006. The link above provides
access to ICB data, codebooks, citations, and a variety of other useful materials.
Correlates of War Project. At www.correlatesofwar.org/ <http://www.correlatesofwar.org/>, accessed

July 2009. The COW project website provides widely used events data on militarized interstate disputes as well as
several other datasets that may be appropriate for FPA scholars. Data, documentation, and codebooks are available
through the link.
EUGene Software. At www.eugenesoftware.org/ <http://www.eugenesoftware.org/>, accessed July 2009.

The link provides access to the EUGene software package, as well as manuals and documentation (all free of charge).
EUGene allows for relatively easy transition between commonly used units of analysis – country–year, dyad–year, and
directed dyad–year. In addition, the software enables the researcher to combine many of the events datasets
discussed in this essay with basic demographic and geopolitical data including data uploaded by the user.
Kansas Event Data System. At http:/web.ku.edu/keds/ <http://web.ku.edu/keds/>, accessed July 2009.

KEDS provides a computer program that enables users to specify and create personalized events datasets. Detailed
descriptions of machine coding methods, as well as several datasets and codebooks, are available on the KEDS web
page.
Page 27 of 28
The Consortium on Qualitative Research Methods. At www.maxwell.syr.edu/moynihan/programs/cqrm/

index.html <http://www.maxwell.syr.edu/moynihan/programs/cqrm/index.html>, accessed July
2009. CQRM promotes qualitative research methods in the social sciences. They hold an annual training institute that
may be useful for FPA scholars interested in expanding their knowledge of qualitative methods. In addition, they
maintain a database of syllabae on qualitative methods.
Intrade. At www.intrade.com/ <http://www.intrade.com/>, accessed July 2009. Intrade is the leading

prediction market for political futures. The site has prediction markets for a wide range of political and financial
outcomes that may be of interest to FPA scholars.
Acknowledgments
I would like to acknowledge the helpful comments of Michael Glosny, Deborah Larson, Rachel Augustine
Potter, and two anonymous reviewers. All errors are my own.
Page 28 of 28

2017-Foreign Policy Analysis

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2017-Foreign Policy Analysis

Uploaded by

Copyright:

Available Formats

Methods of Foreign Policy Analysis

Methods of Foreign Policy Analysis

Subjects: Foreign Policy

The Methodological Origins of Foreign Policy Analysis

Simultaneously, in response to the near monopoly of system-level theory in international

Comparative Foreign Policy and Events Data Analysis

The World Event/Interaction Survey (WEIS)

The Conflict and Peace Data Bank (COPDAB)

International Crisis Behavior Project (ICB)

The ICB project is unusual in that it proceeds simultaneously at multiple complementary

Correlates of War Project (COW)

Trade Dependence and Foreign Policy Compliance

Events Data – Methodological Challenges

The non-independence of foreign policy events presents an additional methodological

Qualitative Methods of Foreign Policy Analysis

It is admittedly artificial to divide methods of foreign policy analysis by “generation,” as this

Case Study Analysis

Gathering Qualitative Data

Third Generation Methods of Foreign Policy Analyses

Experiments and Simulation in Foreign Policy Analysis

The International Communication and Negotiation Simulations (ICONS) project is an ongoing

Survey Research in Foreign Policy Analysis

Remaining Methodological Challenges

Janis, I.L. (1972) Victims of Groupthink. Boston: Houghton Mifflin Company.

Larson, D.W. (1985) Origins of Containment: A Psychological Explanation. Princeton: Princeton

Maoz, Z. (2006) Network Polarization, Network Interdependence, and International Conflict,

McCracken, G. (1988) The Long Interview. London: Sage.

McDermott, R. (2002) Experimental Methodology in Political Science. Political Analysis 10 (4),

Montgomery, A.H. (2005) Proliferation Determinism or Pragmatism? How to Dismantle an

Schrodt, P.A. (1979) Forecasting in International-Relations – Theory, Methods, Problems,

Scott, J. (1991) Social Network Analysis: A Handbook. London: Sage.

Seidman, I. (1998) Interviewing as Qualitative Research: A Guide for Researchers in Education

Trachtenberg, M. (2006) The Craft of International History. Princeton: Princeton University

Vincent, J.E. (1983) Weis Vs Copdab – Correspondence Problems. International Studies

Ward, H. (2006) International Linkages and Environmental Sustainability: The Effectiveness of

Weber, R.P. (1985) Basic Content Analysis. Beverly Hills: Sage.

Wilkinson, D. (2002) Civilizations as Networks: Trade, War, Diplomacy, and Command-Control.

Links to Digital Materials

Correlates of War Project. At www.correlatesofwar.org/ <http://www.correlatesofwar.org/>, accessed

EUGene Software. At www.eugenesoftware.org/ <http://www.eugenesoftware.org/>, accessed July 2009.

Kansas Event Data System. At http:/web.ku.edu/keds/ <http://web.ku.edu/keds/>, accessed July 2009.

The Consortium on Qualitative Research Methods. At www.maxwell.syr.edu/moynihan/programs/cqrm/

Intrade. At www.intrade.com/ <http://www.intrade.com/>, accessed July 2009. Intrade is the leading

You might also like