Evaluating evidence

For each of the following passages, identify whether the evidence and reasons are relevant to the
conclusion.
a. Some people say that calling people names because of the way they look is offensive. My friend is very
overweight, and people call him names for being fat. He says he doesn't mind as he finds horrible things to
call back. This shows there is no harm in calling people names as they can just retaliate if they want to.
b. Is vegetarianism really healthier than eating meat? The answer is yes! There have been several studies
outlining the various advantages that being vegetarian has over eating meat regularly. Furthermore, just ask
vegetarians and they will quickly explain how their diet is superior to that of non-vegetarians. More
importantly, many restaurants and businesses are opening that focus on vegetarian options and menu items;
so clearly, vegetarianism is healthier than eating meat.
c. Einstein was a pacifist. Therefore, pacifism must be right.
d. The Earth supports living organisms. Europa, one of the moons of Jupiter, is like Earth in that both have
large oceans of liquid water. Therefore, Europa supports living organisms.
e. We all know that people should not drive while they are drunk. Texting while driving is like driving
while drunk in that both make people react more slowly to road hazards. Therefore, people should not text
while driving.
f. My car dealer recommends that I pay $300 to rustproof my car. He should know; I guess I’d better do it.
g. These testimonials are from Simple Green Smoothies.

1
 Problems with surveys/questionnaires
Identify the problems in these survey questions
1. How often do you exercise?
A. Rarely
B. Regularly
C. Occasionally
D. Usually
2. Do you agree that acquired immunodeficiency syndrome (AIDS) can be transmitted by shaking hands
with a person with AIDS or through other means of physical contact?

3. How often does the computer system go down?

A. Less than once per month
B. Once per month
C. Once per week
D. More than once per week
4. How many cigarettes do you smoke per day?
A. None
B. 5 or less
C. 5-25
D. 25 or more
5. Do you do physical exercise, such as cycling?
6. Do you think the school board should stop spending taxpayers’ money on non-essential arts programs in
elementary schools?
7. In the late 1990’s Scotland was considering independence from England. An opinion poll showed that
51% of Scots favored "independence." Another poll taken at the same time showed that only 34% favored
being "separate" from England. What could be the reasons for the different result?

2
 Problems with sampling
Try to identify the source of bias in each scenario and speculate on the d irection of the bias
(overestimate or underestimate).
Glossary:

 Biased wording of survey questions can cause people to favor certain responses over others.
 Response bias: people are systematically dishonest when answering a question.
 Undercoverage: the researcher systematically excludes members of the population from being in
the sample.
 Nonresponse: people chosen for the sample cannot be reached or refuse to participate.
 Convenient sampling: the sample is taken from a group of people easy to contact or to reach.
 Voluntary response sampling: a sample made up of volunteers who often have strong opinions on
a topic on either direction.

Scenario 1
David hosts a podcast, and he is curious how much his listeners like his show. He decides to start with an
online poll. He asks his listeners to visit his website and participate in the poll.
The poll shows that 89% percent of the 200 respondents "love" his show.

1. What is the most concerning source of bias in this scenario?

A. Response bias
B. Undercoverage
C. Voluntary response sampling
2. Which direction of bias is more likely in this scenario?
A. 89% is probably an underestimate of the percentage of all listeners that love the show.
B. 89% is probably an overestimate of the percentage of all listeners that love the show.
C. 89% is probably an unbiased estimate.

Scenario 2
David hosts a podcast, and he is curious how much his listeners like his show. He decides to poll the next
100 listeners who send him fan emails. They don't all respond, but 94 of the 97 listeners who responded said
they "loved" his show.

1. What is the most concerning source of bias in this scenario?

A. Convenience sampling
B. Nonresponse
C. Voluntary response sampling
2. Which direction of bias is more likely in this scenario?
A. The results are probably an underestimate of the percentage of all listeners that love the show.
B. The results are probably an overestimate of the percentage of all listeners that love the show.
C. The results are probably an unbiased estimate.

Scenario 3
3
A senator wanted to know about how people in her state felt about internet privacy issues. She conducted a
poll by calling 100 people whose names were randomly sampled from the phone book (note that mobile
phones and unlisted numbers aren't in phone books). The senator's office called those numbers until they got
a response from all 100 people chosen.
The poll showed that 42% of respondents were "very concerned" about internet privacy.

1. What is the most concerning source of bias in this scenario?

A. Convenience sampling
B. Undercoverage
C. Voluntary response sampling
2. Which direction of bias is more likely in this scenario?
A. 42 percent is probably an underestimate of the percentage of people in her state concerned about internet
privacy.
B. 42 percent is probably an overestimate of the percentage of people in her state concerned about internet
privacy.
C. 42 percent is probably an unbiased estimate.

Scenario 4
A senator wanted to know about how people in her state felt about internet privacy issues. She conducted a
poll by calling people using random digit dialing, where computers randomly generate phone numbers so
unlisted and mobile numbers can still be reached. They called over 1000 random phone numbers—most
people didn't answer—until they had reached 100 respondents.
The poll showed that 46% of respondents were "very concerned" about internet privacy.

1. What is the most concerning source of bias in this scenario?

A. Convenience sampling
B. Nonresponse
C. Voluntary response sampling
2. Which direction of bias is more likely in this scenario?
A. 46 percent is probably an underestimate of the percentage of people in her state concerned about internet
privacy.
B. 46 percent is probably an overestimate of the percentage of people in her state concerned about internet
privacy.
C. 46 percent is probably an unbiased estimate.

Scenario 5
A high school wanted to know what percent of its students smoke cigarettes. During the week when students
visited the counselors to schedule classes, they asked every student in person if they smoked cigarettes or
not.
The data showed that 5 percent of students smoked cigarettes.

1. What is the most concerning source of bias in this scenario?

4
A. Biased wording
B. Voluntary response
C. Biased response
2. Which direction of bias is more likely in this scenario?
A. The results probably underestimate the percentage of students who smoke cigarettes.
B. The results probably overestimate the percentage of students who smoke cigarettes.
C. The results are probably an unbiased estimate.

Scenario 6
A high school wanted to know what percent of its students smoke cigarettes. Counselors selected a random
sample of students to take a survey on drug use. One of the questions reads, "If you are under the age of 18
years, do you illegally smoke cigarettes?"
The data showed that 5 percent of students smoked cigarettes.

1. What is the most concerning source of bias in this scenario?

A. Biased wording
B. Voluntary response
C. Convenience sampling
2. Which direction of bias is more likely in this scenario?
A. The results probably underestimate the percentage of students who smoke cigarettes.
B. The results probably overestimate the percentage of students who smoke cigarettes.
C. The results are probably an unbiased estimate.

Scenario 7
Four animal charities wished to know the views of the public on whether pets taken overseas should be held
in quarantine before being allowed to re-enter the country. Each one selected the sample in a different way.
Which sample is the most accurate?
Sample 1
Charity 1 chose 1000 dog-owners from across the nation. The survey was balanced to ensure that roughly
equal numbers were interviewed in every part of the country.
Sample 2
Charity 2 chose 1000 dog-owners from across the nation. The survey was balanced to ensure that more
people were included in the survey in parts of the country which had large populations, and fewer
representatives were questioned if the population was low.
Sample 3
Charity 3 chose 1000 pet-owners from across the nation. The sample was chosen to ensure that a broad
range of pet-owners were included, including owners of snakes, budgies and tropical spiders.
Sample 4
Charity 4 chose 1000 people, representing a variety of pet-owners and people who do not own pets. The
sample was selected from every county, weighted to include more people from heavily populated areas.

5
Scenario 8
Medical marijuana could be a huge step toward more effectively treating cancer patients. While some people
argue that legalizing medical marijuana will increase recreational use of the drug, I argue, “Where is the
proof for such an assertion?” Last month, 75 people were surveyed in Detroit, Michigan, and were asked if
they believe that legalizing medical marijuana will increase recreational use of the drug in their state.
Ninety-three percent responded that they did not believe legalizing medical marijuana would increase
recreational use of the drug. Therefore, our national lawmakers should pass a law legalizing medical
marijuana.

Scenario 9
People who join online dating services tend to succeed in finding a good match. Researchers conducted an
online survey of 229 people, aged 18 to 65, who had all used an Internet dating site. The survey asked
respondents about their primary relationship from the online experience. The research showed that 94
percent of those surveyed saw their “e-partners” again after first meeting them, and the relationships lasted
for an average of at least seven months.