Educ Stud Math

DOI 10.1007/s10649-016-9691-x

Transfer of solutions to conditional probability problems:

effects of example problem format, solution format,
and problem context

Alan F. Chow 1 & James P. Van Haneghan 2

# Springer Science+Business Media Dordrecht 2016

Abstract This study reports the results of a study examining how easily students are able to
transfer frequency solutions to conditional probability problems to novel situations. University
students studied either a problem solved using the traditional Bayes formula format or using a
natural frequency (tree diagram) format. In addition, the example problem and the target
problem presented for solution had matched or mismatched context. The results found better
performance when students studied frequency solutions and when the content of the study
problem and target problem were matched. Beyond solution to the target problem, students did
not show any differences in performance on a subsequent multiple choice task. Results are
discussed in light of Gigerenzer and Hoffrage’s Psychological Review, 106, 425–430 (1999)
claims about the effectiveness of using frequency solutions to help teach Bayesian conditional
probability concepts.

Keywords Conditional probability . Frequency . Bayes

1 Introduction

Methods of delivering instruction on probability concepts can influence student levels of

learning. Differences between teaching approaches between teachers covering the same
materials in identical classes can have differing learning outcomes. Even and Kvatinsky

* Alan F. Chow
alchow@southalabama.edu

James P. Van Haneghan

jvanhane@southalabama.edu

1
Department of Management, Mitchell College of Business, University of South Alabama, 5811 USA
Dr S RM 346, Mobile, AL 36688, USA
2
Department of Professional Studies, College of Education, University of South Alabama, UCOM
3623, Mobile, AL 36688, USA
 A.F. Chow, J.P. Van Haneghan

(2010) found that teachers covering the same material in Bidentical^ classes had different
levels of student understanding. They attributed this difference to the differing teaching
methods of the teachers. One teacher focused on the mechanics of the problem solving,
without much emphasis on alternative methods or underlying concepts. The other teacher
focused more on the understanding and provided alternative methods of solution. Students
having the teacher focused on understanding performed better than those having the teacher
who simply focused on the mechanics. Pantziara and Philippou (2012) studied the learning of
students using procedural approaches of instruction of fractions against the learning of students
using conceptual approaches of instruction of fractions. They found that teaching methods using
both words and figures together can help students to have a better understanding of fractions.
Just as studies have shown that differences in teaching approaches can influence learning,
there is a large amount of evidence (e.g., Gigerenzer, 2002) that people have difficulties with
conditional-probability problems. Gigerenzer suggests that the format of the problem makes a
difference in people’s abilities to solve these types of problems. Problems in a frequency
format appear to be easier to solve than problems presented as probabilities. Gigerenzer
suggests and provides evidence that teaching individuals to think about conditional
probabilities as frequencies can help them better grasp these problems. In one study,
Sedlmeier and Gigerenzer (2002) found that a computer tutorial that facilitated transforming
conditional-probability problems into natural frequencies was effective in improving student
performance better than teaching students the Bayes formula. However, Brase (2007) and
Brase, Cosmides, and Tooby (1998) suggest that the facilitative impact of natural frequencies
depends upon the ability of the individual to recognize its applicability. Brase et al. suggest that
the evolution of human capabilities to address frequencies was linked to situations where the
appropriate environmental affordances would easily activate counting schemes. In situations
where such affordances were lacking, Brase et al. found that the use of natural frequencies was
less intuitive.
While statisticians have been applying the concept of conditional probabilities since Bayes
(1763), students, and many practitioners, have had difficulty grasping the concept (Hoffrage,
Lindsey, Hertwig, & Gigerenzer, 2000). The concept of conditional probability can be
summarized verbally as the probability that event 2 will occur given that event 1 has already
occurred and can be expressed as the ratio of the probability of event 1 and event 2 both
occurring, to the probability of event 1 occurring. For some time, researchers have investigated
various methods of introducing conditional probabilities, with interest in determining whether
methods of presentation make any difference (e.g., Gigerenzer & Hoffrage, 1995). Some
researchers have focused on the presentation methods, while others have focused on the
intervention approach attempting to maintain the advantages of relating information in terms
of probability (Brase et al., 1998).
Gigerenzer (2002) leads a group of psychological theorists whose basic belief is that
participants’ ability to adequately interpret conditional probabilities lies in an evolutionally
selected frequentist understanding of probability. The presentations of Gigerenzer and his
colleagues provide support to the theory that a frequency based presentation approach is not
only the best method to solve conditional probabilities, but also leads to a better understanding
by participants of the concepts of conditional probability.
In a similarly designed study, Brase, Fiddick, and Harries (2006) found that some partic-
ipants had better interpretations and understandings of the conditional-probability problems
when the frequency format solution was applied. They suggest that one reason may be the type
of participants participating in the experiment. They concluded that participants with higher
Transfer of solutions to conditional probability problems

learning and education, as well as those compensated for their participation, might outperform
those with lower levels of education and those uncompensated for their efforts in the study.
Studies into methods of presenting conditional probabilities have taken differing ap-
proaches. Several researchers have taken to testing the concept that using frequency represen-
tations helps participants better assimilate and understand the information for statistical
reasoning (Gigerenzer & Hoffrage, 1995). In these studies the researchers typically are
focusing their efforts directly on the results of utilizing frequency (tree diagram) formats
versus Bayes formula formats of the presented information. One variation is the study of the
performance of participants after an intervention where they are shown how to translate the
probabilities into frequencies in the solution of a conditional-probability problem. Sedlmeier
and Gigerenzer (2002) created a 2-h computer tutorial intervention to train participants to
translate probabilities into frequency formats in order to correctly solve conditional-probability
problems. The results of their study concluded that this type of intervention was effective in
improving the performance of students in their solution of conditional-probability problems.
Practitioners from various disciplines and backgrounds have investigated the problem of
conditional probability formats and their presentations, with differing results. Barbey and
Sloman (2007) provided a thorough review of studies that deal with what they refer to as
base-rate neglect (Kahneman & Tversky, 1973). This concept involves the inability to
determine the actual meaning of probabilistic information due to an underlying inability to
adequately interpret the conditional probability when confronted with conditional situations
and information.
Base-rate neglect (Kahneman & Tversky, 1973) is how researchers explain why decision-
makers often leave out relevant information resulting in misinterpretation in statistical infer-
ence. More simply, it is a neglect of prior probability information when trying to make a
decision The main area where base-rate neglect is studied and debated is in psychological
studies relating to conditional probabilities and application of Bayes’ rule. Gigerenzer and
Hoffrage (1995) showed that while prior research on base-rate neglect often lead to conclu-
sions that the human mind is less capable of processing appropriate cognitive algorithms for
making conditional inference, when the Bayes algorithms are expressed in frequency format
even statistically naïve participants are capable of properly processing and correctly identifying
as many as half of the Bayes algorithms presented. Their findings suggest that improvement in
the performance of participants on conditional probability interpretations and understanding
can be achieved by using and teaching students to translate probability problems into a
frequency format.
Richland, Stigler, and Holyoak (2012) conducted an analysis of psychological and
educational research in an effort to determine if classroom instruction methods can be
related to improved transfer of mathematical concepts and understanding. Novick and
Holyoak (1991) reported their findings of two experiments and a verbal protocol study,
identifying four major findings related to analogical transfer. First was that mapping the
elements of the presented example problem and the target problem, mapping the diagram or
parts of the equation from one problem to another, is different from adapting the solution
process to be used in the target problem. Part of the challenge is mapping, but it is more than
that, similar thinking needs to be done.
Second, that schema plays a role in the process of transfer. Third, that time worked on a
target problem has an influence on transfer and accuracy. Finally, they reported that
mathematical expertise does not predict success in transfer. Day and Goldstone (2011) reported
that participants had success in transferring strategies when presented with a simulated physical
 A.F. Chow, J.P. Van Haneghan

system to dissimilar content and appearance. They suggest Bthat analogical transfer between
overtly dissimilar cases maybe more common—and much more relevant to our cognitive
processing^. These studies suggest that transfer based on using a concrete, structured method,
may be likely even when presented with a target problem of dissimilar or differing context.
Perkins and Salomon (2012) in a review of recent studies on transfer of learning note that
most recognize that transfer of learning situations does not always or often come in the form of
one problem example to one problem solved. They suggest that in a broader aspect, transfer of
learning would include the ability to take a lecture and discussion on a broad concept or topic
and later apply that concept in a real world situation to develop a deeper level of understand-
ing. They suggest three bridges of transfer of learning calling them Bdetect, elect, and
connect^. Here, the student must first detect a link from the concept to the application, must
elect to cognitively analyze the link, and finally must make the connection between the
concept and the application.
There is evidence that conditional-probability problems are better understood using fre-
quency presentation, but it is unlikely that the traditional presentation for conditional proba-
bilities will be abandoned in statistics courses. Hence, it is important to know how frequency-
formatted (tree diagram) solutions or problems map into the Bayes formula conditional-
probability problem. Hence, in the present set of studies, we were interested in how people
mapped frequency solutions onto traditional problems.

2 Study

This study intended to increase the statistical power of Van Haneghan and Chow (2007), by
increasing the number of participants and increasing the measures evaluated. In the earlier
study, students were presented with one of three different conditions in an analogical transfer
design set up to examine how students exposed to worked example solved using frequency
formats solved conditional probability problems posed in the traditional Bayes formula. One
group of students was given a worked example in a probability format solved with the
conditional probability formula. The next group was given a worked example in probability
format that had been translated into a frequency solution. The third group was given a worked
example problem with a solution in frequency format. Following review of their worked
examples, the students were asked to solve a single problem in a probability format.
Another concept for consideration was whether the context of the problems and examples
presented had an influence on the participants’ ability to understand the problem information
and adequately incorporate that information into a proper solution. Bassok and Holyoak
(1989) suggested that transfer may be sensitive to context, and that more novice participants
may be more sensitive to context than expert participants. While it is of importance to find the
underlying cognitive reasons as to why introductory learners would find one method of
problem solving more easily adaptable and useable, within this study the focus is more directly
on which classroom practice of introduction to conditional probability is more suitable for the
student learner.
In other related studies on format presentation of conditional-probability information (e.g.,
Gigerenzer & Hoffrage, 1995), researchers have predominately focused on contextual
examples and problems related to the medical field and medical examples. One of the
reasons for utilizing this type of context and problems is that many of the experimental
participants have been medical professionals, medical students, or psychology students.
Transfer of solutions to conditional probability problems

Hoffrage et al. (2000) use examples in both medical and legal context in studying the effects of
applying natural frequencies in the instruction of statistics. Their conclusion lends to a
recommendation to teach or train participants with constructions of representation rather than
applications of rules, suggesting we train participants on how to Btranslate probabilities into
natural frequencies.^ The interest in testing sensitivity to context is to see if the findings of the
other studies (Brase, 2007; Gigerenzer & Hoffrage, 1995 in particular) will be replicated with a
participant pool having a different background and knowledge base. The participants used in
our study were students taking introductory level undergraduate courses where they were
introduced to the basic concepts of probability but most had no previous formal exposure to
the subject. A main difference between the participants in our study and those used in the
Gigerenzer and Hoffrage (1995) study is that their participants were paid, while ours were
volunteers.
When teaching large sections it can be challenging for faculty to provide an appropriate
assessment method to adequately evaluate student learning. Appaji and Kulkarni (2012)
acknowledged that using MCQs must often be relied upon for assessing such large groups.
They also suggest that this type of assessment can be structured and worked in a way to
adequately assess critical thinking as well as the levels of Bloom’s taxonomy of knowledge,
comprehension, application, and analysis. Having the ability to use MCQs for its ease of
testing large numbers of students, but being able to assess more than the ability to merely
regurgitate factual knowledge would be an important tool for those required to teach large
sections. The inclusion of the MCQs in this study had an aim to evaluate the transfer of
learning from an example problem solution to a target problem solution.

2.1 Methods

The object of the study was to provide further knowledge of how frequency and probability
representations are linked. We used a classical analogical transfer design (similar to Gick &
Holyoak, 1983) to examine how students who were exposed to worked examples treated by
using frequency formats managed to solve conditional-probability problems posed in the
traditional format. We compared three conditions. In one condition, the students received a
worked example in a probability format solved using the traditional Bayesian probability
formula. In the second condition, they studied a problem in a probability format that was
translated into a frequency solution. After studying the worked example, they solved a target
problem with either matching context or mismatched context. An additional condition in this
experiment was to add a set of multiple choice questions to each participant’s packet of
material. After attempting to solve the target problem, the participants attempted to answer five
multiple choice questions, some with mixed context and some with mismatched context.
Based on Gigerenzer’s ideas about frequency solutions being more easily understood, we
initially hypothesized that the second condition would yield better performance than the
traditional formulaic solution. We also looked to the multiple choice problems to investigate
if these additional problems could be answered correctly using the format of the provided
examples.

2.1.1 Participants

The participants for this study were undergraduate students enrolled in introductory level
mathematics and statistics courses. The students were selected because they had no more than
 A.F. Chow, J.P. Van Haneghan

a limited level of exposure to statistics and little or no prior statistical training. Several
instructors agreed to participate in the experiment. Summer enrollment limited the number
of participants to n = 121. The gender of the participants consisted of 44 males and 77 females,
with 13 freshmen, 36 sophomores, 42 juniors, and 28 seniors (2 reported no classification).
The students were from the following majors. : 50 Business, 9 Nursing, 9 Education, 52 Other
(including those not responding) As can be seen, the largest group of majors are in business
related fields. However, their enrollment in introductory statistics courses and random assign-
ment to conditions lessens any potential impact of major on the results of the study.
As noted above, the participants in this study were selected based on their enrollment in the
introductory courses. Each participant was presented a sealed packet which included the
testing instruments. Each course section was assigned an equal number of packets representing
each of the experimental configurations to reduce the risk of sectional bias. Packets were
randomly assigned to students within the sections, and each student returned the testing
materials to the packet upon completion.
These participants were enrolled in mathematics and statistics courses required of their
respective majors. In each case, the course covered some level of probability, including an
introductory level coverage of conditional-probability. In all cases, the participants in each of
these courses had received some instruction on probability, but had not received instruction on
conditional probability within the course itself.

2.1.2 Materials

The materials for this study were the solved example problems, the target problems, and the
multiple choice questionnaire (Appendix 1, 2, 3, 4, and 5). The solved example problem had
medical context, and was presented in probability format, with either a traditional formulaic
formatted solution method or a frequency formatted solution method. A step-by-step narrative
of the solution process for the example problem was given. The solved example problems had
detailed solutions provided based on the algorithm suggested by Gigerenzer and Hoffrage
(1995).

2.1.3 Instrumentation

Each participant was provided with one of the two solved example problems. The solved
example problems were in one of the experimental formats:

1) Presented in Bayes formula probability format with medical context, but solved using
natural frequency (tree diagram) format; or
2) Presented in Bayes formula probability format with medical context, but solved using
Bayes formula probability format.

The instrumentation used for this study was a target conditional-probability problem that
each participant was asked to solve, followed by a series of multiple choice questions (MQCs).
The two target problems (Appendix 3) had either medical context or business context, and
were presented in probability format. Participants were instructed to solve the target problem
using the solution method presented in the example problem.
Scoring for each participant on the target problem was based on the level of correctness in
his or her solution. A scoring rubric was constructed so that frequency and formulaic solutions
Transfer of solutions to conditional probability problems

could both be captured. The scoring was similar for each solution type. Scores involved rating
six different aspects of the solution with a maximum of 5 points per element. The scoring
rubric was tested for reliability by having three different instructors evaluate the same sample
(n = 30) of problems and assigning points according to the rubrics. Consistency in the scoring
validated the reliability of the rubric across multiple assessors. Pearson Correlations between
reviewers ranged between 0.65 and 0.75. The total points awarded for each solution to the
target problem were the sum of the points awarded for each of the tasks based on the rubric
criteria. The total points awarded ranged from a high (perfect) score of 30 to a low score of 0.
Points for the solution of the target problem were used in the analysis of the data.
The five multiple-choice format problems had context that was either matched to the
medical context of the example problem or unmatched (business context). Each of the
MCQs had only one correct answer. The three other choices were foils, or distracters, which
were developed based on common and expected solution and calculation errors.
For the MCQs, participants received two points for providing the correct response, one
point for selecting the most reasonable incorrect answer, and zero points for providing either of
the other choices. Using the MCQs and awarding points in this manner allowed for more
parametric testing to increase the statistical power of the experiment. Denyer and Hancock
(2012) suggested that using MCQs and rewarding Bclose^ answers with partial credit can also
lead to a level of deeper learning.

2.2 Procedure

We provided brief instructions for the participants, focusing on the specific requirements of the
study. Students received a packet with the example problem, the target problem, and MCQs.
Students studied the solution to the example problem. When students completed the target
problem and questionnaire, they returned all items to the envelope and turned in the study
packet.
Participants were instructed to review the solution of the example problem for 10 to 15 min,
paying particular attention to the process used in the solution. When comfortable with the
review, the participants attempted to solve the target problem presented. Participants were
allowed and encouraged to go back to the solution of the example problem for guidance. After
attempting to solve the target problem participants were directed to estimate the correct answer
to five MCQs.

2.3 Results

The results of the study focused on three research questions. First, was whether the
format of the example solution had any influence on participant performance in
correctly completing the target problem and MCQs, or not. The second question
was whether the context of the target solution problem matching the context of the
example solution had any influence on participant performance. The final question
was whether an interaction effect was present between the format of the problems and
the context match of the problems.
Running the analysis separately for the target problem score using the scoring rubric and the
MCQs provided further insight into the performance resulting from the experimental condi-
tions. The ANOVA Tests of Between-Subject Effects for MCQs and Target Problem Scoring is
displayed in Table 1.
 A.F. Chow, J.P. Van Haneghan

Table 1 ANOVA tests of between-subject effects for MCQs and target problem scoring

Source Dependent variable SS df MS F Sig. Partial eta squared

Format MC Points 16.84 1 16.84 2.42 .122 0.02

Target Points 1335.52 1 1335.52 14.44 .000 0.11
Context MC Points 9.27 1 9.27 1.33 .251 0.01
Target Points 1247.13 1 1247.13 13.48 .000 0.10
Format × Context MC Points 1.06 1 1.06 0.15 .697 0.00
Target Points 35.81 1 35.81 0.39 .535 0.00
Error MC Points 814.02 117 6.96
Target Points 10,820.92 117 92.49
Total MC Points 2567.00 121
Target Points 49,553.00 121
Corrected total MC Points 840.98 120
Target Points 13,418.45 120

The predictions of the research questions were that students presented with solved example
problems in the form of natural frequencies (tree diagram) would score higher on conditional-
probability problem performance than participants presented with solved example problems in
the form of Bayes formula probabilities, and that participants presented with matching context
in the solved example and the target solution problems would score higher on the target
solution problems. ANOVA procedure analyzed collected data in order to determine if a
significant difference in the performance on MCQs and target problem exists between
participants presented data and problems in the two different example formats. Table 2
provides the ANOVA Descriptive Statistics for MCQs and Target Problem Scoring. The
results of the study showed that as predicted, the formatting of the example solution had a
significant influence on the performance of the participants on the target solution problem.
Matching the context of the target solution problem with the context of the solved example
problem also had significant influence on the participants’ performance on the target problems.
The study did not find significant interaction effects between the solution format and the
context.
While the MCQs for the matching context had a higher mean that that of the MCQs for the
mismatched context, this difference was not statistically significant. Likewise, the MCQs for
the frequency formatting had a higher mean than that of the MCQs for the probability
formatting, with the difference also being insignificant.

2.4 Discussion

Based on the results of scoring on the target solution problems, the participants receiving
frequency formatting solution presented did outperform those receiving probability formatting
solution presented. In contrast, when reviewing the results of the scoring of participant
performance on the MCQs, the results did not support the earlier findings, indicating that
the effect of the formatting of data on the performance on MCQs of conditional probability is
not significant.
There are several possible explanations as to why the results indicate non-significance on
the MCQs. The learning in this short-term intervention could be merely procedural learning,
Transfer of solutions to conditional probability problems

Table 2 ANOVA descriptive statistics for MCQs and target problem scoring

Format Context M SD n

MC points Frequency Mismatched 3.97 2.41 30

Matched 4.33 2.77 30
Total 4.15 2.58 60
Bayesian Probability Mismatched 3.03 2.31 30
Matched 3.77 2.99 31
Total 3.41 2.68 61
Total Mismatched 3.50 2.39 60
Matched 4.05 2.87 61
Total 3.78 2.65 121
Target points Frequency Mismatched 17.93 10.45 30
Matched 23.27 10.03 30
Total 20.60 10.50 60
Bayesian Probability Mismatched 10.20 7.09 30
Matched 17.71 10.46 31
Total 14.02 9.66 61
Total Mismatched 14.07 9.67 60
Matched 20.44 10.54 61
Total 17.28 10.57 121

where the participants are simply learning the steps to follow in order to solve problems in the
same format as the example problem. This could help explain why the results based on the
scoring of the target solution problems were significant yet the results based on the MCQs
were not. Participants may have been able to follow the procedural steps in solving the
problem in the same manner as the example, yet could not make the transition to the different
type of problem presented in the form of the MCQs The instructions were to estimate the
answers to the MCQs. Those instructions may have focused the participants on thinking about
plausible and implausible answers rather than working through the problems using the
procedures they worked through to solve the target problem. Perhaps they used flawed
conceptual knowledge rather than procedurally creating solutions. The effort to create a
solution may have led to less focus on the sensibility of their answers to the target problem.
Another possibility for the lack of significance in the MCQs could be related to the findings
of Hartman and Lin (2011) who studied student performance on MCQs in a general chemistry
course. Using MCQs based on algorithmic problems, they found that students’ success
decreased with each additional step it takes to solve the algorithm. With this in mind, students
solving problems with numerous steps may lose track of their place when trying to solve and
select the best option in MCQs. Tiemeier, Stacy, and Burke (2011) also found that students in a
Therapeutics sequence in Pharmacy school performed better on the MCQs toward Bloom’s
level of recall than on those at the level of application. They also saw an additional decrease in
performance when targeting the analysis domain. These findings may also explain our lack of
significant difference on the MCQs of our study.
The results of the MCQs suggest that the format and context of the presented problems may
not be of influence on the transfer of learning in this format. Further study on this matter may
look to include an additional element of the MCQs employed by Faize, Dahar, and Niwaz
 A.F. Chow, J.P. Van Haneghan

(2010). In their study, physics students were asked to select the best choice of the options, but
were also instructed to write a brief verbal or mathematical justification of their selection. The
results of their study showed that participants required to justify their selection performed
significantly better than participants who only had to select a choice. Using a similar approach,
Fisher (2008) found that having participants in a study involving medical students to write
justification for their selection in MCQs helped protect against ambiguously worded questions
and options. This clarification can help increase the performance by allowing students to better
exhibit their level of learning of the material.
The results of the study indicate that participants had improved success in solving the target
solution problems when they were presented with examples and target problems that had a
matched context. Participants who received the matching medical context problems
outperformed those who had the mismatched business context in the target solution problem
scores. As one participant reported, it was difficult when presented with terms that they were
less familiar to make a connection between the previous solution method and the proposed
target problem of a different context. As Reed (1989) points out, finding the appropriate level
of abstraction to use in linking across problems can create a significant burden to students
trying to bridge from one problem context to another. In the context of the present study, the
problem contexts did vary in ways that might have made the linking of the study to target
problem more complex in the mismatched context. For example, the medical target problem
looked a naturally dichotomous item (presence versus absence of a disease), whereas the
business target problem looked at a more amorphous dichotomy (financing for 6 years versus
financing for other lengths of time). The medical test in the study problem had 100 % accurate
prediction of cases, whereas the analogous component in the business problem was the 20 %
default rate for 6 year loans. These two differences may have made the mapping of compo-
nents from one problem to the other more difficult. It may also have made the business
problem a slightly more difficult one (although solvable by the same procedures). Whether the
differences between the performance on the medical and business problems can be attributed
to difficulties finding the appropriate conceptual match between components, the slightly
greater complexity of the business problem, or both is an open question for further research.
Regardless of whether problem difficulty played a role, given the focus on using the study
problem to solve the target problem, it is likely that the differences in context played a role.
The findings are also similar to findings from Novick and Holyoak (1991). Three pertinent
findings from that study are related to the findings of this experiment. First, they found that
successful mapping of the problem solution was not sufficient for successful transfer of
learning. In our experiment, it seems that even when participants were successful in solving
the target solution problem they were not likewise able to transfer the learning and mapping to
the MCQs. Given the instructions and procedural focus, they may not have gained sufficient
knowledge to estimate answers correctly on the MCQs. Second, Novick and Holyoak found
that the process of connecting the solution of one problem type to the solution of another
analogous one was related to the specific problems learned and with only one sample that
process would not lead to much transfer. The findings of our study suggest that participant
learning was limited and not readily transferrable. Finally, their study concluded that the level
of success in multiple solution problems relative to the analogical transfer and accuracy of the
solved problems was related to the sense of urgency or level of time pressure perceived by the
participants. In our experiment, there may have also been a time related factor influencing the
level of success on solving the MCQs, since the MCQs were presented after the participants
had attempted to complete the target solution problem.
Transfer of solutions to conditional probability problems

3 Conclusions

There are several conclusions to be drawn from this experiment. The first is that participants
showed superior performance on the solution of target problems when the example presented
had data in the form of natural frequencies as opposed to Bayesian probabilities. While this
supports the findings reported by many, it does not attempt to delineate between the underlying
reason being the use of natural frequencies themselves, or the concept of the natural frequen-
cies as a subset of the larger concept of the part/whole relation, or nested sets.
The findings also contradict some of the findings presented by Brase (2007). From his
study, Brase concluded that short-term interventions could not improve performance. The
findings of this study do provide some support to the concept that a specific short-term
intervention can improve performance when applying the frequency-formatted data presenta-
tion rather than the Bayesian probability formatting.
A second conclusion from this study is that context of the presented data influences the
participants’ ability to solve conditional-probability problems when using a short-term inter-
vention. Having context in the target problem that matches the example provided an improved
performance in solving the target problem. This would indicate that when having the matching
context, the learner is not required to make the additional cognitive step of translating from one
context to another, and thus it reduces the cognitive load, enabling the learner to perform better.
Like previous research that determined that context based on prior knowledge and under-
standing improves learning (Anderberg, 2000; Bandura & Goldman, 1995), we found that in
the short-term intervention approach used in the study, participants can perform better when
they do not have to make the additional cognitive transfer of changing or interpreting
information context.
A third conclusion is that while the goal of learning should include the knowledge and
understanding of the underlying theoretical basis, a more practical goal is that the learning
leads to the ability to correctly apply the knowledge learned in an appropriate and useful
manner. While the intervention may not have dealt with the theoretical side of conditional
probability, it did provide for identifying a more effective method of presentation of material
for improved learner application.
The short-term intervention may provide insight into a more long-term approach to
presenting and teaching conditional probability concepts using both the natural frequency
approach and the relative context (matching and familiar context) approach. Based on this
study, a more in-depth study might provide additional understanding of how students learn
quantitative material, and how best to present learners with such materials so that they have the
best chance of successful learning.
Students can successfully translate traditionally presented conditional-probability problems
into frequency based solutions. Students sometimes get lost mapping a frequency solution
from a traditionally presented problem format, and as was shown by the lack of difference on
the multiple choice part. Obviously, better success in mapping between frequency and
traditionally presented problems would require more practice. Further, the ability to procedur-
ally map frequency solutions from traditionally presented problems still does not guarantee
that students gain a conceptual understanding of conditional probability. Hence, in order to
develop a flexible understanding of conditional probability there needs to be a focus on
concepts as well as a procedural mapping between formats.
Based on the findings, both format and context in the presentation of material is important
in helping students to understand and apply the concept of conditional probability. Our goal as
 A.F. Chow, J.P. Van Haneghan

researchers and educators is to find methods of instruction that take the students from a
functional level to a level of deeper understanding. Studying methods of instruction that
improve the students’ ability to transfer the understanding from one problem to another, and
from one problem type to another can lead to a better understanding and improve performance
on solving conditional-probability problems.

Appendix 1 Example 1

Below you will find an example problem with its completed solution. Please study the
example problem and its solution, focusing your attention on the methodology used in solving
this particular problem. After you have studied this example problem, you are asked to try to
solve the target problem on the next page. While attempting to solve the target problem feel
free to look back at this solved example problem for assistance.
Example Problem: Scientists have recently discovered a new disease, Disease ABC, which
infects 10 % of the population. A test used to detect the disease can accurately detect the
disease in those who have it, but have a false positive rate of 12 %. What percent of patients
who take the test will have the disease, given a positive test result?
Solution:
In solving this problem first recognize there are two conditions: contracting the disease and
testing for the disease. For each of these conditions there are two events possible. For the
condition of contracting the disease the events are contracting or not contracting. For the
condition of testing for the disease the events are a positive test or a negative test. For each
condition the events are mutually exclusive and collectively exhaustive. From this information
we have the following for probabilities:
P ðHasDiseaseÞ ¼ 10 % P ðNotHasDiseaseÞ ¼ 90 %
PðT est þ jHasDiseaseÞ ¼ 100% PðT est þ jHasDiseaseÞ ¼ 100%

using the following Bayesian algorithm for computing posterior probability

PðHasDiseaseÞPðT estþ jHasDiseaseÞ
PðHasDiseasejT estþ Þ ¼
PðHasDiseaseÞPðTest þ jHasDiseaseÞ
þ PðNotHasDiseaseÞPðTest þ jNotHasDiseaseÞ

ð10%Þð100%Þ 10 10
P ðHasDiseasejTest þ Þ ¼ ¼ ¼ ¼ 48 % :
ð10%Þð100%Þ þ ð90%Þð12%Þ 10 þ 10:8 20:8

Appendix 2 Example 2

Below you will find an example problem with its completed solution. Please study the
example problem and its solution, focusing your attention on the methodology used in solving
this particular problem. After you have studied this example problem, you are asked to try to
solve the target problem on the next page. While attempting to solve the target problem feel
free to look back at this solved example problem for assistance.
Example Problem: A person has a 10 % chance of contracting a particular disease. A newly
developed test to detect the presence of the infection in a person accurately identifies all persons
who have the disease. However, of those without the disease the test has a false positive rate
Transfer of solutions to conditional probability problems

(incorrectly identifies those without the disease with a positive test result) at the rate of 10 %. If a
person has a positive test result, what are the chances that the person does not have the disease?
Solution:
In solving this problem first recognize there are two conditions: contracting the disease and
testing for the disease. For each of these conditions there are two events possible. For the
condition of contracting the disease the events are contracting or not contracting. For the
condition of testing for the disease the events are a positive test or a negative test. For each
condition the events are mutually exclusive and collectively exhaustive. If we translate the
probabilities to frequency of occurrence, we get the following:
Has Disease 10 out of 100 Not Has Disease ¼ 90 out of 100
Testsþ Has Disease 10 out of 10 Testþ Not Has Disease ¼ 9 out of 90

Using the tree diagram below

Now to find the probability, we need to know the number of patients who have a positive test
and have the disease, and divide that by the number of patients who have a positive test. The
result is the 10 patients having the positive test and the disease divided by the 10 patients having
the positive test and the disease plus the 9 patients having the positive test without the disease.
10 10
¼ ≈0:53 ¼ 53%
10 þ 9 19

Appendix 3 Target problems

Target Problem 1:
The chance of contracting a particular disease is 15 %. A test to detect the infection of
this disease is accurate for those with the disease, but has a false positive rate (incorrectly
identifies those without the disease with a positive test result) of 15 %. What are the
chances of a person not having the disease, if they tested positive?
Target Problem 2:
Market research indicates that 24 % of people who finance a new car in the next year
will finance it for 6 years. Of those that finance the new car for 6 years, 20 % will default
on the loan and have the car repossessed. Twenty-five percent of the people who finance
their new car for duration other than 6 years will default on the loan and have their car
 A.F. Chow, J.P. Van Haneghan

repossessed. What are the chances that a person will have financed their new car for
6 years given that they defaulted on their loan?

Appendix 4 Multiple choice questions in medical context

1. The following is some current research related to disease Q and the efforts to test for this
infection:
Eight percent of people contract the disease. A newly developed test to detect the
presence of the infection in a person accurately identifies all persons who have the disease.
However, of those without the disease, the test has a false positive rate (incorrectly
identifies those without the disease with a positive test result) at the rate of 20 %. If a
person has a positive test result, what are the chances that the person does not have the
disease?

a) 8
b) 20
c) 70
d) 30
2. The chance of contracting a particular disease is 10 %. A test to detect the infection of this
disease is accurate for those with the disease, but has a false positive rate (incorrectly
identifies those without the disease with a positive test result) of 20 %. What are the
chances of a person not having the disease, if they tested positive?

a) 64
b) 36
c) 10
d) 18
3. Scientists have recently discovered a new disease, Disease XYZ, which infects 3 % of the
population. A test used to detect the disease can detect the disease in those who have it
90 % of the time, and has a false positive rate (incorrectly identifies those without the
disease with a positive test result) of 10 %. What percent of patients who take the test will
have the disease, given a positive test result?

a) 22
b) 90
c) 30
d) 78
4. A person has an 18 % chance of contracting a disease. A test to detect the presence of the
disease in a person accurately identifies 90 % of persons who have the disease. However,
of those without the disease, the test has a false positive rate (incorrectly identifies those
without the disease with a positive test result) at the rate of 15 %. If a person has a positive
test result, what are the chances that the person does not have the disease?

a) 43
b) 14
Transfer of solutions to conditional probability problems

c) 57
d) 18
5. The likelihood of contracting a particular disease is 6 %. A test to detect the disease is
accurate for all of those with the disease, but has a false positive rate (incorrectly identifies
those without the disease with a positive test result) of 25 %. What are the chances of a
person not having the disease, if they tested positive?

a) 25
b) 20
c) 80
d) 94

Appendix 5 Multiple choice questions in business context

1. Companies A and B provide mobile telephone service in your area. Thirty percent of
people have service with Company A, while the remainder has service with Company B.
15 % of the customers of Company A will cancel their service in the next year. Thirty
percent of the customers of Company B will cancel their service in the next year. Given
that a customer is going to cancel their service in the next year, what are the chances that it
will be a customer of Company A?

a) 60
b) 18
c) 30
d) 82
2. According to a national marketing survey of American households, 60 % of households
surveyed had incomes greater than $25,000. The survey also found that 80 % of
households with income greater than $25,000 in two cars, while households with income
less than $25,000 had two cars 40 % of the time. What is the probability that household
had income greater than $25,000 given that it had two cars?

a) 42
b) 75
c) 25
d) 69
3. A company has 2 factories that produce digital cameras, Factory A and Factory B. Factory
B produces 3 times as many digital cameras as Factory A does. Twenty-three percent of
digital cameras produced by factory A are defective. 8 % of digital cameras produced by
Factory B are defective. What is the probability that a digital camera was produced by
factory B given that it is defective?

a) 20
b) 10
c) 40
d) 51
 A.F. Chow, J.P. Van Haneghan

4. Restaurants A & B are the only places open for breakfast on Saturday mornings in Small
Town, USA. Sixty percent of the people in Small Town that go out for breakfast on
Saturday mornings go to restaurant A. Restaurant A has a customer satisfaction rating of
98 % (2 % unsatisfied). Restaurant B has a customer satisfaction rating of 95 %. What is
the probability that a customer ate breakfast at Restaurant A if they had an unsatisfactory
experience (unsatisfactory rating)?

a) 38
b) 62
c) 24
d) 40
5. Computer Company XYZ makes computers in two factories, Factory 1 and Factory 2.
Forty-five percent of all the computers made by XYZ Company are made in Factory 1.
The rate of defective computers (percent that are defective) are 20 % for Factory 1 and
10 % for Factory 2. If a customer buys a defective computer, what is the likelihood that it
was made in Factory 1?

a) 20
b) 62
c) 26
d) 38

Appendix 6

Table 3 Scoring rubric for probability based solution

Performance criteria Level 4 Level 3 Level 2 Level 1

(5 points) (3 points) (1 points) (0 points)

Identifies Correctly identifies Identifies one given Identifies one given Does not identify
conditions both given condition of the condition of the any of the
of problem conditions of problem and this problem but does conditions
the problem identifies other not attempt to given in the
condition provide other problem
condition
Identifies Correctly identifies Correctly identifies Correctly identifies Does not identify
events in each both events for both events for one event for any of the
condition each condition one condition or one condition events for either
one event for condition
each condition
Identifies event Correctly identifies Correctly identifies Correctly identifies Does not
probabilities all event both event one event identify any
given probabilities probabilities for probability for of the event
given one condition or one condition probabilities
one event
probability for
each condition
Identifies Correctly identifies Correctly identifies Correctly identifies Does not identify
conditional all conditional both conditional one conditional any of the
Transfer of solutions to conditional probability problems

Table 3 (continued)

Performance criteria Level 4 Level 3 Level 2 Level 1

(5 points) (3 points) (1 points) (0 points)

probabilities probabilities probabilities for probability for conditional

given given one event or one one event probabilities
conditional
probability for
each event
Incorporates data/ Correctly Correctly incorporates Correctly Does not
information incorporates all some data into incorporates incorporate
into formula data or information correct formula some data into any data into
into the correct for solution the incorrect any formula
formula for formula for for solution
solution solution
Calculates Correctly calculates Makes correct Makes incorrect Does not
requested requested calculations of calculation calculate
conditional conditional incomplete data resulting in any solution
probability probabilities resulting in incorrect
incorrect solution solution
Total score

Appendix 7

Table 4 Scoring rubric for frequency based solution

Performance criteria Level 4 Level 3 Level 2 Level 1

(5 points) (3 points) (1 points) (0 points)

Identifies Correctly identifies Identifies one Identifies one given Does not identify
conditions both given given condition condition of the any of the
of problem conditions of of the problem problem but does conditions
the problem and this identifies not attempt to given in the
other condition provide other problem
condition
Identifies Correctly identifies Correctly identifies Correctly identifies Does not identify
events in both events both events for one event for any of the events
each condition for each one condition or one condition for either
condition one event for each condition
condition
Translates event Correctly translates Correctly translates Correctly translates Does not translate
probabilities all event both event one event any of the event
given into probabilities probabilities for probability for probabilities into
frequencies given into one condition into one condition frequencies
frequencies frequencies or into frequencies
one event probability
for each condition
into frequencies
Translates Correctly translates Correctly translates Correctly translates Does not translate
conditional all conditional both conditional one conditional any of the
probabilities probabilities probabilities for probability for conditional
given into given to one event or one one event into probabilities into
frequencies frequencies conditional probability frequencies frequencies
for each event into
frequencies
 A.F. Chow, J.P. Van Haneghan

Table 4 (continued)

Performance criteria Level 4 Level 3 Level 2 Level 1

(5 points) (3 points) (1 points) (0 points)

Sketches tree Correctly sketches Correctly sketches Sketches tree Does not sketch
diagram and tree diagram and tree diagram and diagram incorrectly tree diagram
inputs frequency correctly inputs correctly inputs but correctly inputs correctly or
information all frequency some frequency some frequency input any
information information information frequency
information
correctly
Calculates Correctly Makes correct Makes incorrect Does not calculate
requested calculates calculations of calculation
conditional requested incomplete data resulting in any solution
probability conditional resulting in incorrect solution
probabilities incorrect solution
Total score

References

Anderberg, E. (2000). Word meaning and conceptions. An empirical study of relationships between students’
thinking and use of language when reasoning about a problem. Instructional Science, 28, 89–113.
Appaji, A., & Kulkarni, R. (2012). Multiple choice questions as a teaching learning tool in addition to assessment
method. National Journal of Integrated Research in Medicine, 3(4), 91–95.
Bandura, M. M., & Goldman, C. (1995). Expanding the contextual analysis of clinical problems. Cognitive and
Behavioral Practice, 2, 119–141.
Barbey, A. K., & Sloman, S. A. (2007). Base-rate respect: From ecological rationality to dual processes.
Behavioral and Brain Sciences, 30, 241–297.
Bassok, M., & Holyoak, K. J. (1989). Interdomain transfer between isomorphic topics in algebra and physics.
Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 153–166.
Bayes, T. (1763). An essay toward solving a problem in the doctrine of chances. Philosophical Transactions of
the Royal Society of London, 53, 370–418.
Brase, G. L. (2007). The (in) flexibility of evolved frequency representations for statistical reasoning: Cognitive
styles and brief prompts do not influence Bayesian inference. Acta Psychologica Sinica, 39, 398–405.
Brase, G. L., Cosmides, L., & Tooby, J. (1998). Individuation, counting, and statistical inference: The role of
frequency and whole-object representations in judgement under uncertainty. Journal of Experimental
Psychology: General, 127, 3–21.
Brase, G. L., Fiddick, L., & Harries, C. (2006). Participant recruitment methods and statistical reasoning
performance. The Quarterly Journal of Experimental Psychology, 59, 965–976.
Day, S., & Goldstone, R. (2011). Analogical transfer from a simulated physical system. Journal of Educational
Psychology, 37, 551–567.
Denyer, G. & Hancock, D. (2012). Multiple choice questions to combat plagiarism and encourage conceptual
learning. Proceedings of The Australian Conference on Science and Mathematics Education (formerly
UniServe Science Conference) (pp. 31–36). Sydney: University of Sydney.
Even, R., & Kvatinsky, T. (2010). What mathematics do teachers with contrasting teaching approaches address in
probability lessons? Educational Studies in Mathematics, 74, 207–222.
Faize, F., Dahar, M., & Niwaz, A. (2010). Improving students performance on multiple-choice questions in
physics – Practice in justifying the options. International Journal of Academic Research, 2, 204–207.
Fisher, J. (2008). Mulliple choice: Choosing the best option for more effective and less frustrating law school
testing. Capital University Law Review, 37, 119–136.
Gick, M., & Holyoak, K. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15, 1–38.
Gigerenzer, G. (2002). Calculated risks. New York: Simon and Schuster.
Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency
formats. Psychological Review, 102, 684–704.
Gigerenzer, G., & Hoffrage, U. (1999). Overcoming difficulties in Bayesian reasoning: A reply to Lewis and
Keren (1999) and Mellers and McGraw (1999). Psychological Review, 106, 425–430.
Transfer of solutions to conditional probability problems

Hartman, J., & Lin, S. (2011). Analysis of student performance on multiple-choice questions in general
chemistry. Journal of Chemical Education, 88, 1223–1230.
Hoffrage, U., Lindsey, S., Hertwig, R., & Gigerenzer, G. (2000). Communicating statistical information. Science,
290, 2261–2262.
Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80, 237–251.
Novick, L. R., & Holyoak, K. J. (1991). Mathematical problem solving by analogy. Journal of Experimental
Psychology, 17, 398–415.
Pantziara, M., & Philippou, G. (2012). Levels of students’ Bconception^ of fractions. Educational Studies in
Mathematics, 79, 61–83.
Perkins, D., & Salomon, G. (2012). Knowledge to go: A motivational and dispositional view of transfer.
Educational Psychologist, 47(3), 248–258.
Reed, S. K. (1989). Constraints on the abstraction of solutions. Journal of Educational Psychology, 81, 532–540.
Richland, L., Stigler, J., & Holyoake, K. (2012). Teaching the conceptual structure of mathematics. Educational
Psychology, 47(3), 189–203.
Sedlmeier, P., & Gigerenzer, G. (2002). Teaching Bayesian reasoning in less than two hours. Journal of
Experimental Psychology: General, 130, 380–400.
Tiemeier, A., Stacy, Z., & Burke, J. (2011). Using multiple choice questiosn written at various Bloom’s
Taxonomy levels to evaluate student performance across a Therapeutics Sequence. Innovations in
Pharmacy, 2(2), 1–11.
Van Haneghan, J., & Chow, A. (2007). Transferring frequencies solutions to conditional probability problem.
Poster session presented at the annual meeting of the American Psychological Association, San Francisco,
CA.