Polls, Polls, Everywhere

Unless you are one of those individuals without a radio or television, it is difficult
to go through a day in the United States without hearing the results of a poll
someone or an organization has conducted. Polls are as ubiquitous as personal
injury lawyers. They are everywhere, especially during presidential election years
and at any other time there is an important political race to be divided.

Not all polls are political, of course. Polls are frequently used in marketing
research, for example to ascertain the market for a particular product, or whether
more of us prefer ketchup or salsa with our sandwich. But regardless of the
purpose for which they are being used, all polls seek to gauge public opinion or
attitudes at a given time or period. Our focus here is political polls, or polls that
gauge public opinion on politics or political issues. But first, we must define public
opinion.

What is Public Opinion?

Pick up different American Government textbooks and you are likely to come
across different definitions of public opinion. There are as many definitions of
public opinion as many authors trying to define the concept. One textbook defines
public opinion as the term used to “denote the values and attitudes that people have
about issues, events, and personalities at any point in time.”

Okay. Seems straightforward enough, but do you see some problems with this
definition? Let’s see how many we can discover here. First the phrase “at any point
in time” implies that opinions on public issues change over time. Second, we are
assuming that people know what they are being asked and that polls accurately tap
their opinions. Is that a fair assumption? Do polls measure public opinion or
simply give a snapshot of collective opinion at a certain point in time? Are values
the same as attitudes?

Individual versus Collective Opinions

Public opinion or survey studies have consistently shown that individual
Americans are political uninformed and have inconsistent opinions or attitudes on
issues of public policy. However, the same studies also show that the collective
opinions of the American people are relatively stable and rational over time. This
has led scholars to speak of the “rational public.” Indeed, Americans have a high
degree of consensus on several core values that form the basis of American
political culture, e.g., liberty, equality, and democracy. But the question remains:
can the collective opinions of the American people be measured on any given
issue, and how accurate and reliable are these measurements?

1
Efforts to Measure Public Opinion
Opinions are usually measured by public opinion polls or what is also referred to
as survey research. Polls are interviews or surveys of a sample of citizens (it is too
expensive and time consuming to ask everyone!) used to estimate how the public
feels about an issue or set of issues.

Scientific polling or public opinion polling, as we know it today, developed in the

1930s. Pollsters used scientific methods to measure attitudes. Methods of gathering
and analyzing data improved over the years and survey data began to play an
important role in politics and social life.

Early Election Forecasting

As early as 1824, newspapers have tried to predict election winners. In 1886, the
Boston Globe used exit polls to predict the results of key elections. From 1920 to
1932, Literary Digest used straw polls and correctly predicted every presidential
election from 1920 to 1932. Its luck ran out when it predicted that Alf Landon
would beat Franklin Delano Roosevelt (FDR), the incumbent president. FDR won
by a landslide.

What Went Wrong?

Literary Digest used a type of polling called the straw poll. Straw polls simply
ask as many people as possible to a given set of questions. They do not use a
sample of people in a random and scientific manner so that the sample will
represent the population. Apart from trying to ask many people as possible about
their attitudes toward the presidential election of 1932, Literary Digest also made
three fatal errors:

1. First, their sample was drawn from sources (including mail surveys inserted
in the magazine) that over-sampled the upper middle class and Republicans
while underestimating the Democratic vote. In the middle of the Great
Depression, the people likely to own automobiles or phones or join clubs
were the upper middle class Republican voters.

2. Second, they mailed questionnaires in early September and opinions

changed as the election drew near.

3. Third, they committed the sin of what we now call self-selection. Only
highly motivated individuals responded, and so the survey over-sampled
wealthier and better educated electorate. As a general rule, people answering

2
 mail surveys are more likely to be wealthier and better educated, and they
tend to care more about issues than does the general electorate.

George Gallup, however, correctly predicted the results of the 1936 election, using
the random sampling method. His company, the Gallup Corporation continues to
be quite successful in predicting electoral outcomes. But how could a subset of the
population represent the views of everyone?

How We Measure Public Opinion Today

Public officials learn about public opinion in many ways, but the source they use
and rely on the most continues to be the random sampling method. The video clips
in this folder provide further explanation about this method.

In order for a poll to be reliable, it must have several characteristics:

1. Knowing the population to sample.

Knowing which population to study and which will produce an accurate
or reliable result is an integral part of a good poll. For example, a survey
of the general public’s opinions about US trade policy with Brazil would
be of limited value because most members of the public know too little
about this issue to make an informed opinion. A change to the survey’s
design might propose that the survey target a population that is informed
on the issue, such as economists, international relations professors, or
former state department officials.

2. Question Wording – Bad questions lead to bad results. An example of a

bad question would be “If the government takes our guns that we use to
protect our families away from us, only criminals will have guns and we
will all be in danger. Are you in favor of placing your family in greater
danger? Yes/No.” Another one: “No child should go hungry in the
United States. Do you favor cutting welfare programs for children or
not? Still another example: Do you favor helping the poor, or do you
favor cutting welfare? Why are these bad survey questions? Because
they are very subjective and lead to a preferred answer.

3. Sampling – In order for a poll to be reliable, the sample must be taken

accurately. The best method is a weighted or stratified random sampling,
a variation of random sampling method that is statistically weighted to
reflect the demographics of the population being polled. Such a sample
not only guarantees that each person in the population has the same

3
 statistical chance of being selected, but also assures that the proportion of
every key demographic group in the population being polled is reflected
in the sample. Most national surveys and commercial polls use samples
from 1,000 to 1,500.

Sampling Error – The margin of error (or sampling error as it is called)

will be quite small if the sample is carefully selected. All polls contain
some error, 3 to 5% is considered a reasonably small rate of error. A 3%
error rate means the poll is 97% accurate! These rates become extremely
important if a race is close. For example, a public opinion poll result
might read as follows:

George W. Bush 48%

John Kerry 52%
Margin of Error: 5%

Do these numbers tell you anything? No. The contestants are only 4
percentage points apart, given the error rate the real race could look like
this:

George Bush 53% (48% plus 5)

John Kerry 47% (52% minus 5)

4. Contacting Respondents – Once the methodology of polling is chosen,

the next question is how to contact those to be surveyed. Since most
Americans have phones, calling would be a valid method. Some surveys
are done in person as well, but many worry that the presence of
interviewer causes problems.

In general, you should never trust a poll that does not tell you the targeted
population, question wording, the sampling method, and the ways in which
respondents were contacted. Reputable and reliable pollsters will also tell you the
number of respondents (the “N”) and the error rate (+ or – 5%) so that you can
determine for yourself whether to believe the results. Any poll that tells you to call
555-9712 for yes and 555-9713 for no is unscientific and unreliable. The same is
true of Internet polls that say should register your opinion now, and of
telemarketing surveys in the mall. These are not random samples at all; they are
modern-day examples of straw polls.

4
 Here is a list of some polling websites!

The National Election Study at the University of Michigan (www.umich.edu/~nes)

offers regular polls on elections, voting behavior, and electoral issues.

The Gallup Organization (www.gallup.com) is one of the best known and well
respected polling agencies. The website offers access to reports, polling data, and
more about a variety of issues.

The Pew Center for People and the Press (www.people-press.org) is an

independent opinion research group that studies attitudes toward the press, politics
and public policy issues.

Media Research Center (www.mediaresearch.org) is a conservative watchdog

group that claims the media have a liberal bias. Offer links to conservative media
and political sites.

Media Matters Center (www.mediamatters.org) is a liberal watchdog group

looking for conservative biases in the media.

Fairness and Accuracy in Reporting or FAIR (www.fair.org) is a liberal watchdog

group looking for media bias.