Chapter 9 Confrey 2006

CHAPTER 9
The Evolution of Design Studies

as Methodology
Jere Confrey
In the NRC report, Scientific Research in Lagemann (2002) focused on the need for
Education (Shavelson & Towne, 2002), three more research that produces useable class-
broad types of research were discussed: room guidance. This review synthesizes the
trends, causal effects, and mechanism. current progress of the methodology and
Mechanism was described as research that identifies areas for future development.
answers the question, “how or why is it Design studies are defined as “entailing
happening”; the authors1 described “design both ‘engineering’ particular forms of learn-
experiments” as an “analytic approach for ing and systematically studying those forms
examining mechanism that begins with of learning with the context defined by the
theoretical ideas that are tested through means of supporting them. This designed
the design, implementation, and system- context is subject to test and revision. Suc-
atic study of educational tools (curriculum, cessive iterations that result play a role sim-
teaching methods, computer applets) that ilar to systematic variation in experiment”
embody the initial conjectured mechanism” (Cobb et al., 2003 , p. 9).
(p. 120). The Committee identified two A design study is an extended investiga-
products of such work as “theory-driven pro- tion of educational interactions provoked by
cess of designing” and “data-driven process use of a carefully sequenced and typically
of refining [instructional strategies]” (p. 121). novel set of designed curricular tasks study-
Both of these products can be viewed as ing how some conceptual field, or set of pro-
related to a class of research known as design ficiencies and interests, are learned through
studies, the focus of this chapter. interactions among learners with guidance.
Researchers across the country have rec- The study seeks to document what resources
ognized the need to strengthen the “instruc- and prior knowledge the students bring to
tional core” (Elmore, 1996) and to identify the task, how students and teachers inter-
effective “instructional regimes” (Cohen, act, how records and inscriptions are cre-
Raudenbush, & Ball 2003 ) as critical to ated, how conceptions emerge and change,
the improvement of education. Likewise, what resources are used, and how teaching is
135
Downloaded from https://www.cambridge.org/core. University of Cambridge, on 18 Jul 2018 at 22:44:00, subject to the Cambridge Core terms of use, available at
https://www.cambridge.org/core/terms. https://doi.org/10.1017/CBO9780511816833.010
136 the cambridge handbook of the learning sciences
accomplished over the course of instruction, of work rejects the view that one conducts
by studying student work, video records, and pure research in laboratory or experimen-
classroom assessments. tal settings and only later exports it to class-
Design researchers make, test, and refine rooms. Instead, one conducts the work in the
conjectures about the learning trajectory complexity of the classroom (Brown, 1992;
based on evidence as they go, often col- Greeno, 2003 ). Applied and pure research
laborating with or acting as the teacher, are merged.
and assembling extensive records on what Any developing field of research such as
students, teachers, and researchers learn the learning sciences has roots in a variety of
from the process. They then conduct fur- methods. Learning sciences emerged most
ther analysis after the fact to produce extensively from cognitive science, devel-
research reports and/or iterations of the opmental psychology, technological innova-
tasks, materials, and instrumentation. This tions, sociocultural theory, and mathemat-
class of research methods is also referred to ics and science education. Through these,
as “design research,” “design experiments,” one can also trace the developmental path
or “design-based research methods.” I prefer of design studies. The first part of this chap-
to use the term “design studies” in recogni- ter locates design studies in an evolution
tion of the need for programmatic and itera- from clinical interviews, teaching experi-
tive investigations over time. Design studies ments, and design experiments.
have been labeled an “emerging paradigm for A methodology must be situated clearly
the study of learning in context through the within a set of goals and theories: philoso-
systematic study of instructional strategies phers of science have warned us repeat-
and tools” (DBRC, 2003 , p. 5 ). They are dis- edly that method, in the absence of theory,
tinctive from curricular studies and evalua- becomes hollow and procedural (Elkana,
tion (Confrey & Stohl, 2004), pure discourse 1974; Lakatos & Feyerabend, 1999). Hence,
analyses of classrooms (Gee, 1999) instruc- this chapter will also describe the class of
tional design (Merrill, 2001; Reeves, 2000) theories related to the method, so as to pro-
and action research/lesson study (Fernandez mote a class of evidence-supported theories
& Yoshida, 2004). Design studies inherit that can guide instructional decision making
many features from clinical interview stud- towards the goal of improving student learn-
ies and teaching experiments, and are clearly ing (Cobb et al., 2003 ). The question of what
located within the larger field of the learning kinds of theories can guide the develop-
sciences; they seek both to provide system- ment of instructional decision-making will
atic and warranted knowledge about learn- be taken up in the second part of the
ing, and to produce theories to guide instruc- article.
tional decision making toward improved Finally, for design research to qualify
student learning. as a methodology, it must have criteria
Design studies most often involve novel for its conduct, a set of canons of evi-
treatments of curricular areas, such as the dence, and grounding for its warrant. In
introduction of new topics, new technolo- research, evidence of methodological rigor
gies, or novel forms of interaction. Exam- comes in three forms: (1) the experimenta-
ples of design studies include research on tion/investigation has itself been adequately
students’ understanding of statistical ideas conducted and analyzed; (2) the claims are
(Cobb, 2002; Lehrer & Pritchard, 2002), justified, robust, significant relative to the
modeling (Lehrer & Schauble, 2004; Lesh, data and the theory, and are subjected to
1999), student and teacher reasoning about alternative interpretations; and (3 ) the rele-
rational numbers (Confrey & Lachance, vance of the claims to the practices of educa-
2000; Simon, 2000), student representations tion is explicit and feasible. A third section
(diSessa, 2004) and students’ use of inno- briefly discusses design research in relation
vative technological tools (Collins, Joseph, to methods, evidence, praxis, and the basis
& Bielaczyc, 2004; Greeno, 1997). This kind for the warrant of its claims.
the evolution of design studies as methodology 137
The Evolution of Design Experiments tasks that lead towards cognitive reconstruc-
tions, conceptual changes (including over-
coming common “misconceptions,” or con-
Early Basis in Piaget, Vygotsky, Dewey,
sidering alternative conceptions) involving a
and Clinical Studies
coordination of the processes of assimilation
Design studies are rooted in an abiding and accommodation; and (3 ) for ideas to
interest in understanding children’s think- become viable, children must assess ideas’
ing. Piaget, Vygotsky, and Dewey all argued feasibility, utility, and durability through a
extensively that in the formation of student process of schema construction and reflec-
thought lay the genesis of the process that tive abstraction. Piaget’s famous dictum,
would be the key to characterizing learning. “one comes to know the world through
For Piaget, this basis was evident in his idea knowing oneself” (cited in von Glasersfeld,
of “genetic epistemology.” For Vygotsky, it 1982, p. 613 ), emphasizes his view that
lay in “genetic historic method.” For Dewey, while knowledge involves descriptions of an
the sources of the formation of student external world, it also invariably involves
thought were in the nature of inquiry and an interaction between the knower and the
its connections to pragmatism. These schol- known, and therefore knowledge is rela-
ars all recognized that understanding how tional (Ackerman, 1995 ; Kegan, 2000).
thoughts are formed is key to understanding Among Piaget’s methodological contribu-
what counts as knowledge. tions was the clinical method:
By contrast, more behavioristic tradi-
tions gauge learning primarily by measur- The clinical examination is thus experi-
able effects on performance on tests or mental in the sense that the practitioner
other direct measures of student outcomes, sets himself a problem, makes hypotheses,
emphasizing speed, efficiency, persistence, adapts the conditions to them and finally
and, occasionally, the transfer of thought to controls each hypothesis by testing it against
the reactions he stimulates in conversations.
new settings. In the learning sciences, these
But the clinical examination is also depen-
criteria are not sufficient, because they mea- dent on direct observation, in the sense
sure the outcomes of learning as fixed acqui- that the good practitioner lets himself be
sitions only, and in the face of weakness or led, though always in control, and takes
poor performance, cannot offer explanatory account of the whole of the mental context,
frameworks. In the learning sciences, key fea- instead of being the victim of “systematic
tures of knowledge include why one believes error” as so often happens to the pure exper-
something, how it came about, how it is imenter.
related to other ideas, and what it permits The good experimenter must, in fact,
one to do. Thus, methodologies in the learn- unite two often incompatible qualities; he
ing sciences are obliged to include a broader must know how to observe, that is to say, to
let the child talk freely, without ever check-
set of outcome measures that capture the
ing or side-tracking his utterance, and at the
process of learning, as well as the final state same time he must constantly be alert for
of the learner. something definitive, at every moment he
Among Piaget’s key assumptions was must have some working hypothesis, some
that a child is located in an environ- theory, true or false, which he is seeking to
ment and strives to make sense of that check. (Piaget, 1976, p. 9)
environment. Piaget contributed three pri-
mary insights that required new method- Piaget warned that “the greatest enemies of
ologies, including the “clinical method”: (1) the clinical method are those who unduly
children’s views are not congruent with simplify the results of an interrogatory,
adults’ views; (2) the process by which those who either accept every answer the
children gain cognitive proficiency requires child makes as gold or those on the other
that their understandings are progressively hand who class all as dross” (p. 9). For
refined through experience with a series of Piaget, the primary task of the clinical
interview was to reveal children’s’ liber- through interviews, but recognized that to
ated convictions, which “implies previously understand the links between formal and
formed schemas, tendencies of mind, intel- informal ideas “an urgent methodological
lectual habits, etc.” (p. 13 ). “The hypothe- problem confronting us is to find ways of
sis is that assertion that the child invents his studying real concepts in depth” (Vygotsky,
explanations in such a way as to reveal some- 1978, p. 91).
thing of the spontaneous tendencies of his Vygotsky further recognized that to
mind” (p. 14). Awareness of such schemas understand conceptual development, one
is essential to the design of successful needs to study teaching – the role of a “more
instruction. knowledgeable other” in learning. His notion
Vygotsky’s work has contributed pow- of the zone of proximal development sug-
erfully to the evolution of these method- gested that “what the child can do in coop-
ologies, emphasizing the individual’s devel- eration today, he can do alone tomorrow.
opment within a sociocultural setting. For Therefore, the only good kind of instruc-
Vygotsky, cultural activities are the primary tion is that which marches ahead of devel-
source of cognition, and they shape how opment and leads it. It must be aimed not so
one thinks in fundamental ways. This is much at the ripe as at the ripening functions”
summed up in his often-quoted statement, (Vygotsky, 1986, pp. 188–189). This is an
“Any function in the child’s cultural develop- early impetus for learning sciences to inves-
ment appears twice, or on two planes. First it tigate teaching within studies of develop-
appears on the social plane and then on the ment. Discourse, facilitated by inscriptions
psychological plane. First it appears between and representations, plays a key method-
people as an interpsychological category, and ological role in studying developmental
then within the child as an intrapsychologi- processes:
cal category” (Vygotsky, 1978, p. 5 7).
We believe that child development is a
Vygotsky’s work contributes to design
complex dialectical process characterized
studies through his attention to selecting the by periodicity, unevenness in the develop-
appropriate unit of analysis in experimenta- ment of different functions, metamorphosis
tion. He used the term “unit of analysis” to or qualitative transformation of one form
connect thought and language and “contain into another, intertwining of external and
in the most fundamental and elementary internal factors, and adaptive processes
form those properties that belong to verbal which overcome impediments that the child
thinking as a whole” (Vygotsky, 1986, p. 211). encounters. (Vygotsky, 1978, p. 73 )
In any methodology, the key unit of analysis The third major source of influence on
must be articulated. design studies methodology can be traced
A second methodological contribution to Dewey’s work, based in philosophy and
derives from Vygotsky’s examination of the epistemology as much as in psychology. Both
relationship between scientific and sponta- James (pragmatism) and Darwin (evolution)
neous concepts, of which he wrote, “System- fundamentally influenced his thinking.
atic reasoning, being initially acquired in the
sphere of scientific concepts later transfers A theory corresponds to the facts when it
its structure organization into spontaneous leads us to the facts which are its con-
concepts, remodeling them from above” sequences, by the intermediary of expe-
(Vygotsky, 1986, p. 172). Vygotsky saw sci- rience . . . but they are always subject to
being corrected by unforeseen future conse-
entific concepts essentially as cognitive tools,
quences or by observed facts that had been
with “tool” referring to the indirect func-
disregarded. Every proposition concerning
tion of an object as a means of accomplish- truths is really in the last analysis hypo-
ing activity. To capture the effects of tools thetical and provisional, although a large
on thought, he focused on the idea of signs number of these propositions have been so
and symbols as mediating cognitive tools. frequently verified without failure that we
Vygotsky studied conceptual development are justified in using them as if they were
absolutely true. But logically, absolute truth of the classroom not as deterministic, but as
is an ideal which cannot be realized, at complex and conditional. In these settings,
least not until all the facts have been regis- instructional guidance is based on affecting
tered, or as James says, “bagged” and until the likelihood of certain events and out-
it is no longer possible to make other observ- comes by adjusting the conditions of instruc-
ations and other experiences. (Dewey, 1981,
tion. One can liken instructional guidance to
pp. 49–5 0)
the notion of different realizations of a simu-
Pragmatism, as Dewey cast it, is a form lation – one cannot predict an outcome pre-
of empiricism. But rather than suggest that cisely, because each realization is, in effect,
experimentation produces facts whose links unique, but the multiple realizations yield
to hypotheses create truth, he recognized tendencies that can guide decision-making
that consequences produced through action and parameter setting.
and experience are but one possible set Based on these premises and rationales,
of outcomes. It is only as they are real- one cannot prescribe practices, but one can
ized through a variety of circumstances guide practice by means of explanatory
that they become stable and secure, even frameworks accompanied by data, evidence,
though always provisional. Pragmatism does and argument. An explanatory framework
not place theory on a shelf, useful only as is: (1) at best a model of likely outcomes;
a guide to pristine experimentalism, but (2) closely connected to its theories; (3 ) as
rather places it squarely into the world of robust as its links to evidence from multi-
action and experience. It engages with com- ple sources of interaction within ecologically
plexity, rather than striving to artificially authentic settings; (4) as rigorous as the doc-
reduce it. umentation and analysis that underlies items
This theoretical perspective guides 1, 2, and 3 ; and (5 ) as valid as it is useful to
Dewey’s emphasis on inquiry, defined as others who are familiar and experienced in
“the controlled or directed transformation similar contexts. The theory of design stud-
of an indeterminate situation into one that is ies incorporates this epistemological view of
so determinate in its constituent distinctions classroom praxis.
and relations as to convert the elements of
the original situation into a unified whole”
From Constructivism and Sociocultural
(Dewey, 1981, p. 226). Dewey’s emphasis
Theories to Teaching Experiments
on the transformation of a hypothesis in
practice through the lens of inquiry provides During the 1970s and 1980s, mathemat-
us both praxis and a means to study it. He ics and science education passed through
recognized that in the beginning, there is a “structure of the disciplines” phase,
only the indeterminate, which undergoes in which content specialists encountered
transformation through a problematic to a developmental theories (Duckworth, 1996;
hypothesis, which, by means of the activity Hawkins, 2002). In the late seventies and
of inquiry, is transformed to a determinate eighties, the movement examined issues
situation producing a set of knowledge of problem solving, matured toward “con-
claims. These claims will only reach the structivism” and comprised an orientation
status of truth if they are borne out in other toward understanding student thinking and
spheres of activity. That is, the claims must documenting the variety of ways of con-
meet the pragmatic test that they prove ceptualizing problems, heuristics, and strate-
useful in interpreting multiple situations gies. In the late eighties, the clinical inter-
over time. view matured into the teaching experiment,
The scholarship of Piaget, Vygotsky, and imported from pedagogical research in the
Dewey thus provides the theoretical ratio- Soviet Union (Kantowski et al., 1978), as
nale for selecting design studies as a method- sociocultural approaches increasingly influ-
ology to produce theories about instruc- enced math and science education. Inves-
tional guidance. Such studies support views tigators increasingly recognized the role of
structured or guided apprenticeship in sup- argued for complex, multidimensional, and

porting changes over time. longitudinal studies of practice.
The teaching experiment uncovered the Simon (2000) suggested methods of anal-
process by which students learn subject mat- ysis for teaching experiments. He identified
ter. It included longitudinal study, inter- key roles for emergent perspectives, con-
vention into student learning, an iterative ceptual framework, and framing of paradig-
process of data gathering and planning, matic cases. In working on teacher develop-
and qualitative rather than quantitative ment experiments, he framed his work in the
data. Thompson (1979) argued that the context of joint teacher-researcher interac-
teaching experiment could be adapted to tions and recognized a key role for “promot-
the constructivist movement in the United ing the development of teachers.” He wrote,
States to yield “a constructivist teaching “Teaching as a process of inquiry in which
experiment” in which a researcher builds the teacher is engaged in an ongoing cycle of
a model of a teacher interacting with a interaction and reflection has a great deal of
child. Steffe (1991) identified its charac- potential” (p. 3 5 9).
teristics. Researchers were to examine the Cobb (2000) discussed teaching experi-
mathematics of children (rather than tradi- ments in relation to emergent perspectives,
tional mathematics), viewed independently documenting their use in relation to inves-
of their background and training, and locate tigating students’ mathematical learning in
it in negotiations of interactive communi- alternative classroom contexts developed
cation (rather than import it from formal in collaboration with teachers. Teachers’
mathematics), to focus on the evolution learning is a second focus of his atten-
of their activities rather than the products, tion. Drawing on Gravemeijer (1995 ), Cobb
in order to examine the modifications in also expressed interest in the “development
their schemas, in relation to goal-directed of instructional sequences and the local,
activities. domain specific instructional theories that
underpin them” (p. 3 13 ). He further elab-
Because the teaching experiment involves orated the development and research cycle,
experimentation with the ways and means first noted by Simon (1995 ). In evaluating
of influencing children’s knowledge, it is the teaching experiment, Cobb discussed
more than a clinical interview. It is directed the role of generalizability, trustworthi-
toward understanding the progress children ness, and commensurability in retrospective
make over extended periods of time, and analyses.
one of the main goals is to formulate a In the same volume, Handbook of
model of learning the particular content Research Design in Mathematics and Science
involved. (Steffe, 1991, p. 178) Education, Confrey and Lachance (2000)
discussed their experience in a three-year
Lesh and Kelly (2000) identified an essen- teaching experiment. They coined the term
tial recursiveness in design, originally rec- “transformative teaching experiments” to
ognized in Cobb and Steffe (1983 ). The indicate the need to study more specula-
student is expected to build knowledge tive and innovative teaching practices. They
of mathematics, the teacher builds knowl- argued that teaching experiments should
edge of the student building knowledge of study conjectures rather than hypotheses
mathematics, and the researcher is build- (“assertions waiting to be proved or dis-
ing knowledge of both, as well as of the proved”). A conjecture is
interactions between the two. Hence, they
described the idea of a “multi-tiered teach- . . . a means to reconceptualize the ways
ing experiment.” They identified bases for in which to approach both the content
design including content quality, technology, and pedagogy of a set of mathematics
school-to-career transitions, equity, teacher topics . . . a strong conjecture should shift
development, and instructional design. They one’s perspective and bring new events,
previously insignificant or perplexing, into observe student work, and to listen to and
relief. At points in its evolution, the conjec- record student discourse on the topics. We
ture should feel like a grand scheme begin- (1) conducted mini-interviews of students’
ning to emerge from many, previously dis- work on tasks during seatwork periods, (2)
parate pieces, making them more cohesive. captured sets of small groups’ interactions
(Confrey & Lachance, 2 000, p. 2 3 5 )
around key tasks, and examined the partici-
pants’ work and statements, and (3 ) engaged
Confrey and Lachance linked the con- in interactive teaching, encouraging students
jecture to the work of Lakatos, who rec- to share their ideas and to respond to sugges-
ognized that theory is established “through tions from others. Most of the instruction
the incessant improvement of guesses by was accomplished through the introduction
speculation and criticism” (Lakatos, 1976, of carefully sequenced challenges selected to
p. 5 ). They identified two dimensions of the raise important topics for examination and
conjecture: one based in content, and the instruction. The introduction and closing of
other in pedagogy, classroom organization, class each day were critically important: the
and tasks, activities, tools, and resources. team presented summary statements of the
Their research questions focused on how day’s progress and solicited comments from
instruction would change if the conjecture the students. Frequent performance-based
were implemented in curriculum, classroom assessments were used to document varia-
interactions, teaching, and assessment. They tions in student progress.
outlined how one could evaluate the qual- After each class, I met with the research
ity of the experiment in terms of the face team and we discussed what had occurred,
validity and peer review quality of the con- and documented critical moments of activ-
jecture’s development, the coherence of the ity in which the students’ behaviors were
rational reconstruction, and the quality of surprising, needed strengthening, or played
the student voice in terms of its exten- out what had been conjectured. Materials
sion, authenticity, and representativeness of were revised in approximately two-week
students’ comments. Externally, they evalu- segments, in advance of instruction, in light
ated the teaching experiment as to whether of current classroom events, and again on a
its range of products (reports, curricular daily basis after examination of a single day’s
products, professional development mate- work. Team members presented alternative
rials, and policy documents) were feasi- interpretations, being required to substanti-
ble, sustainable, compelling, adaptable, and ate the claims with evidence linking these to
generative. a developing understanding of the conjec-
For example, in 1988, I articulated the ture. When disagreements arose, common
“splitting conjecture,” positing that the cog- assessments would often be developed to
nitive roots of multiplication, division, and help determine which interpretation(s) was
ratio are independent of the roots of count- more likely, compelling, or viable. Observa-
ing, addition, and subtraction (Confrey, tion and discussion notes were kept to high-
1988). I linked splitting to partitioning, scal- light these instances. The team mapped the
ing, similarity, and related ideas in geometry. students’ conceptual moves, documented
In 1992, I began a three-year teaching experi- alternative methods, created, transcribed,
ment with a group of third graders. For three and studied video excerpts, and prepared
years, I and my team taught the group of research reports. At the end of the project,
eighteen students. Each year, the team out- all students completed an interview on
lined a set of tasks that would strengthen their experiences in the program (Confrey,
students’ understanding of the interrela- Lachance, & Hotchkiss, 1996) and a posttest
tionships among multiplication, division, of common misconception items on ratio
and ratio. Each day of the experiment, reasoning (Scarano & Confrey, 1996).
we implemented the planned set of tasks, In shifting from the clinical interview to
which included substantial opportunities to the teaching experiment these researchers
retained, modified, and refined some aspects were involved in the earliest work on bring-
of the methodology while abandoning oth- ing new technologies into the classroom
ers, and adopted some new approaches. (Fisher, Dwyer, & Yoacam, 1996; Papert,
They maintained, and to some extent 1980; Pea, 1987). These early pioneers rec-
strengthened, their understanding of the ognized that by bringing technology into
importance of building a model of stu- the classroom, one can shift learning in fun-
dents’ mathematical thinking, and main- damental ways, both by improving student
tained their commitment to children as proficiency through stoking the fires of the
active constructive agents – who if provided individual mind, and by viewing learning as
appropriate tasks, challenges, and opportu- a distributed enterprise in which tools play
nities to explain, could create interesting, key mediating roles and in which knowledge
creative, and often productive solutions. The is built through collaborative activities.
researchers strengthened their own views on In 1992, Allan Collins first used the term
the importance of unleashing these aspects “design experiments,” which he related to
of active cognition to form viable models of Herb Simon’s classic book, The Sciences of
their capabilities and proclivities. Further- the Artificial (Simon, 1969), which drew
more, they strengthened their recognition a distinction between the natural sciences
of two bookends of the constructivist pro- and the sciences of the artificial or “design
cess – prior knowledge as influenced by the sciences.” In his seminal paper, “Toward
child’s experience and beliefs, and the crit- a Design Science of Education,” Collins
ical role of reflection in this activity of self- wrote,
emergence – in bounding student concep-
tions in secure webs and structures. What is different today is that some
of the best minds in the world are
A number of key elements have shaped
addressing themselves to education as
the evolution of teaching experiments. This
experimentalists: their goal is to compare
work was fostered in mathematics and sci- different designs to see what affects what.
ence education communities; the influence Technology provides us with powerful tools
of the Piagetian perspective of genetic epis- to try out different designs, so that instead of
temology has endured. The development of theories of education, may begin to develop
new computational and dynamic technolo- a science of education. But it cannot be an
gies has further propelled the work toward analytic science like physics or psychology;
fresh considerations of the nature of math- rather it must be a design science more
ematics and science. The choice to move like aeronautics or artificial intelligence. . . .
to the complex setting of whole classrooms Similarly, a design science of education
must determine how different designs of
has accelerated the evolution of methodol-
learning environments contribute to learn-
ogy by locating individuals’ thinking in a web
ing, cooperation, motivation, etc. (Collins,
of sociocultural forces that shape the devel- 1992 , p. 16).
opment of the ideas. Seeing knowledge as
distributed and as mediated by the available Collins listed eight decisions that per-
tools has meant an increased need to con- mit exploration of a given design space:
sider how to adequately describe the events (1) involving teachers as co-investigators,
unfurling under teachers’ guidance. (2) comparing innovations, (3 ) making
objective appraisals, (4) selecting promis-
ing innovations, (5 ) involving multidisci-
Early Development of Design
plinary expertise, (6) incorporating system-
Experiments
atic variation, (7) using frequent revisions,
Most people trace the beginnings of the use and (8) evaluating success using multiple cri-
of the term “design experiments” or “design teria. Optimistically, he wrote, “This design
research” to Allan Collins and Ann Brown in theory will attempt to specify all the vari-
1992. I would locate the movement toward ables that affect the success or failure of dif-
engineering further back, to those who ferent designs. Furthermore, it will attempt
to specify what values on these variables can emerge, and not to treat one as tem-
maximize chances for success, and how dif- porally prior to the other. She discussed
ferent variables interact in creating success- the choice of idiographic and nomothetic
ful designs” (Collins, 1992, p. 19). approaches, short and long time durations,
Also in 1992, Ann Brown published and qualitative and quantitative analyses.
“Design Experiments: Theoretical and She concluded with a preference for mixed
Methodological Challenges in Creating methods, citing in particular evidence of
Complex Interventions in Classroom Set- changes in student thinking over time.
tings” (Brown, 1992). She specified that However, she anticipated criticisms that,
design experiments were working wholes, in using methods that produce copious
organized around a working environment quantities of information, it is possible to be
for study, engineered to incorporate learning critiqued on the basis of a selection bias that
theory and dissemination and other forms could lead to misrepresentation of data.
of feasibility, and to possess input variables
(classroom ethos, teachers, students as
researchers, curriculum, technology), and
Recent Activity on Design
outputs (assessments and accountability).
She described her own professional journey,
Experiments and the Role of Theory
telling how her first grant application for
design experiments was unkindly labeled, Collins et al. (2004) reviewed the history
“Pseudo experimental work in quasi- of design experiments, summarizing how
naturalistic settings” (Brown, 1992, p. 15 2). the methodology had evolved into a means
She traced her new approach to a commit- to create a design science, to study learn-
ment to address the two diseases of learning: ing phenomena in the real world, to go
inert knowledge and passivity. A method- beyond narrow measures, and to derive
ological hurdle was the need to make covert research from formative findings. In the first
cognitive activity visible, first through inter- case, they commented that a design science
est in strategies, metacognition, and then in must provide enough specificity to avoid
social contexts through guided instruction “lethal mutations” during implementation
and reciprocal teaching. It was the transition (Brown & Campione, 1996), and yet recog-
from paired instructional strategies to a nized that:
community of learners that led to the
. . . any implementation of a design requires
development of design research. Brown rec- many decisions that go beyond the design
ognized the need to generate much richer itself. This occurs because no design can
data sources, including transcripts of group specify all the details, and because the
work, observations, student portfolios and action of the participants in the implemen-
other communications, and ethnographic tation requires constant decisions about
studies of interactions. Her article also how to proceed at every level. Designs in
represented her first attempts to contrast education can be more or less specific, but
methodologies: she compared her work on can never be completely specified. Evalua-
analogies in laboratory experiments and in tion of designs can only be made in terms
design research. She wrote, “Although in the of particular implementations, and these
can vary widely depending on the partic-
laboratory, this development from noticing
ipant’s needs, interest, abilities, interpreta-
to using, and from surface to deep, was tions, interactions and goals. (Collins et al.,
thought to be age-dependent, the classroom 2 004, p. 3 )
work suggests that the shift is knowledge-
based, occurring microgenetically within a They also developed the idea of “progres-
year as readily as cross-sectionally across sev- sive refinement” in design, referring to the
eral years” (p. 15 3 ). She urged researchers Japanese auto industry’s strategy of frequent
to view both laboratories and class- revisions for improved quality. Second, they
rooms as places where theoretical advances referred to a design as an “integrated system”
in which the elements must work together. They are all (1) design-based, (2) interven-
As a result, they recognized that design tionist rather than naturalistic, (3 ) theory
experiments rely on multiple dependent generative, (4) iterative, and (5 ) ecologically
variables and can only be understood by valid and practice-oriented.
characterizing rather than controlling the The role of theory in design studies
variables. In terms of theory, they contrasted merits further elaboration. Like Lesh and
the testing of hypotheses with “develop- Kelly’s (2000) work on multitiered teaching
ing a profile.” They specified that design experiments, and Confrey and Lachance’s
reports should include the goals and ele- (2000) work on conjecture, Cobb et al.
ments of the design, settings where it is (2003 ) acknowledged a need for different
implemented, descriptions of each phase, kinds of theory at different stages of the
outcomes found, and lessons learned. In a inquiry. They stressed that design experi-
final section, they identified the need to ments tend to emphasize an intermediate
develop summative versions of design exper- theoretical scope that orients the theory to
iments that (1) include climate variables such the particular circumstances of the experi-
as engagement, cooperation, risk taking, and ment, and have the potential for rapid pay-
student control; (2) learning variables such off. DiSessa and Cobb (2004) distinguished
as content knowledge, skills, dispositions, four uses of theoretical constructs in this
metacognitive strategies; and (3 ) systemic methodology: (1) grand theories (e.g. Piaget’s
variables such as sustainability, spread, scal- intellectual theories); (2) orienting frame-
ability, ease of adoption, and costs. works to define social affiliations among sci-
What becomes evident in current work is entists; (3 ) “domain specific instructional
a level of maturity absent from the previ- theories” which “entail the conceptual anal-
ous decade. Limitations in the specification ysis of a significant disciplinary idea,” which
of variables are acknowledged. Strict inter- “are of heuristic value”; and (4) ontologi-
pretations of hypothesis-testing are replaced cal innovations. The authors claimed that
with other means of characterizing the use domain specific instructional theories, which
of and evolution of theory. The necessity for “embody testable conjectures about both
more complex outcome measures is noted, learning processes and the means of ‘engi-
as well as the need to consider more cog- neering’ them,” typically rely on “established
nitive and affective factors to capture the theoretical constructs”, limiting their ability
breadth of effects. Finally, the design study to “scrutinize and reconceptualize aspects
approach is linked to practice, with spe- of competence in mathematics or to bet-
cific acknowledgment that when research is ter handle the complexity and diversity of
embedded in complex practices, it needs to settings in which we conduct experiments”
consider adoption, adaptation, spread, feasi- (p. 83 ). They proposed the idea of “onto-
bility, and sustainability (Collins et al., 2004; logical innovations,” which entails finding
Confrey, Castro-Filho, & Wilhelm, 2000; and validating a new category of existence
Fishman et al., 2004). in the world – such as metarepresentational
In “Design Experiments in Educational competence and socio-mathematical norms
Research,” Cobb et al. (2003 ) argued that (diSessa & Cobb, 2004).
the purpose of design studies is to “develop The strength of this treatment of theory
a class of theories about both the process is in directing the researcher to seek novel
of learning and the means that are designed theoretical constructs that can emerge from
to support that learning, be it the learn- instructional settings and novel tasks. Open-
ing of individual students, of a classroom ness to discovery, and to rethinking the con-
community, of a professional teaching com- tent, mediational means, and forms of praxis
munity, or of a school or school district has been a hallmark of design studies, and
viewed as an organization” (p. 10). Five cross- reflects its speculative character. But it is
cutting features of the methodology apply not clear why such innovations are labelled
to all these diverse types of experiments. “ontological” rather than “epistemological”;
it seems unnecessary, and appears to com- or indeterminate situation is akin to issuing

promise the pragmatist stance. What seems an invitation to entry. Anticipating students’
most important methodologically is to rec- responses and engaging in formative assess-
ognize that such theoretical constructs must ments create a set of landmarks that can help
provide explanatory power for a variety of to guide them through. Successful naviga-
classroom behaviors, utterances, and inter- tion of the conceptual corridor should pro-
actional patterns, should be shown to have duce conceptual tools that then assist a stu-
systematic effects on learning as measured dent in moving into and through the next
by a variety of outcome measures, must corridor. Instruction should strengthen stu-
exhibit reasonably high levels of predictabil- dents’ understanding of navigational tools:
ity, and must be examined for their stability, the problem or interest in a project; shar-
endurance, incubation period, and distribu- ing diverse methods; providing intermit-
tion across students. tent opportunities for closure and negoti-
At a broader grain size, I would argue that ated agreements; building representations,
the primary contribution of design studies inscriptions, or common meanings for lan-
is the articulation of domain-specific guid- guage; providing adequate practice; and
ance illuminating conceptual corridors for the developing explanation, argument, and dis-
learning of content. The aim of a design course patterns. If teachers are familiar with
experiment is the articulation of two related these navigational tools, they can antici-
concepts: a conceptual corridor and a con- pate how to provide appropriate scaffolding
ceptual trajectory. The conceptual corridor is for future instruction. Figure 9.1 represents
a theoretical construct describing the pos- the relationship between the trajectory and
sible space to be navigated successfully to corridor.
learn conceptual content. During any par- A key point is that the conceptual cor-
ticular set of episodes of teaching, that is, a ridor is never completely or rigidly speci-
design experiment, students will traverse a fied, because other ways to approach an idea
particular conceptual trajectory through the and to sequence and constrain it are possi-
corridor. Experimental design requires cap- ble, thus permitting the corridor to evolve.
turing data that document the nature of one Nonetheless, teachers need guidance on doc-
or more particular trajectories and their vari- umented ways to organize corridors, not
ants. Teachers need a sense of what possi- as rigid curricular sequences, but rather as
ble fruitful pathways a student can nego- intellectual spaces through which students
tiate, even though for any particular series progress. These are the methodological tools
of instructional episodes, only one trajec- for documenting findings on how to recog-
tory (with variations among students) is nize the importance of genetic epistemology
traversed. The goal of the experiment, how- and how to instantiate the genetic histori-
ever, is broader: to model the conceptual cor- cal method while recognizing a pragmatist’s
ridor, a description of all possible conceptual view of the complexity and uncertainty of
trajectories. practice.
From this theoretical perspective, effec- Design studies are by their nature itera-
tive instruction depends on how well one tive, as researchers document the changes a
engineers the conceptual corridor, such that particular group undergoes over time, and
the likelihood of fruitful trajectories is build a model of the trajectories and the cor-
increased. Design studies can be viewed as ridor. As a study is iterated, the similarities
engineering and identifying multiple means and differences in evolution and path are
for constructing that corridor. The design compared.
and sequence of tasks creates a set of con- Design studies should be complemented
straints. Knowing and building from stu- by other methods. A number of researchers
dents’ prior knowledge is a means of ori- have discussed how this is carried out in rela-
enting the students’ entry to the corridor. tion to clinical or lab-based studies (Barab
Creating the initial problematic challenge & Squire, 2004; Brown, 1992), whereas
Prior Class’s actual

knowledge conceptual
trajectory
Constraints
(borders of the
corridor)
Conceptual
corridor
Key
Obstacles
Learned Landmarks
ideas
Figure 9.1. The goal of design experiments.
others have examined ways the results may Is Design Research Scientific?
be linked to curricular evaluation, larger- Discussion of Issues of Warrant
scale assessments, or perhaps ethnogra-
phies (Confrey et al., 2000; Fishman et al., Design studies have come under consider-
2004). able criticism, especially in light of recent
Design studies have the potential to federal policy initiatives that place a higher
promote easier dissemination of research value on experimental research that aims to
into practice, due to the ecological valid- establish causal relationships as unequivocal
ity of the study setting. A number of as those strived for in some types of biologi-
researchers have warned that such valid- cal or physical science research. A statement
ity is limited, because classrooms typically such as the following: “Should we believe the
have less support, are more rigidly linked results of design experiments?” (Shavelson
into accountability systems, and lack the et al., 2003 ), derived from interpretations
access to professional development typical in of aspects of Scientific Research in Educa-
these experiments (Fishman et al., 2004). To tion, is actually nonsensical by the standards
strengthen transfer, researchers recommend established in that same report. Any partic-
that design studies anticipate these needs, ular study must be judged as scientific or
and address issues of usefulness, trustwor- not, depending on how it is connected to
thiness, and sustainability from the outset the question under investigation. The major-
of their work (Confrey & Lachance, 2000; ity of the principles for scientific studies
Fishman et al., 2004). (Shavelson & Towne, 2002) are easily met
by design study methodology: important provides a sense of what must be var-

questions are being addressed and are linked ied. The focus on daily logs of activities,
to and generative of relevant theory, the and discussions by research teams, to iden-
literature on those questions is evolving tify, interpret, and subsequently test alter-
in a systematic way, and the best of the native hypotheses provides such sources of
studies demonstrate a coherent and explicit principled variation. Formative assessments,
chain of reasoning. The question should designed to document representativeness
really be: if design research is intended to across students, must be systematically used.
produce knowledge claims about “a class of In response to the concern for an overabun-
evidence-supported theories that can guide dance of data, Dede was correct that at the
instructional decision-making towards the stage of rational reconstruction, that is, a
goal of improving student learning,” then study’s concluding analysis, the flexibility to
when should we believe the results, and how test alternative conjectures and interpreta-
should we interpret them? tions is more limited, but during this phase,
The most common critiques of design the key is to search out both confirming and
study methodology concern the represen- disconfirming events and to make explicit
tativeness of the work, the potential for the case for the explanatory power of pro-
replication, and questions about a design posed constructs, over time, across circum-
study’s generalizability and possible war- stance, and across students.
rants of cause or mechanism (Shavelson Finally, the emphasis on inquiry and mod-
et al., 2003 ). These valid concerns are typi- eling, with its underpinnings of how learn-
cally addressed by learning scientists in two ing best proceeds, is also a foundation for
ways. First, design studies are more akin to the warrant of the work. Design studies pro-
case studies and ethnographies, in that they duce models of students’ and teachers’ learn-
seek to provide levels of detail and specificity ing in rich interactional settings with a vari-
about complex interactions over extended ety of tasks and mediational means. In the
periods of time, rather than establishing complex settings of classrooms, understand-
broad and representative patterns. Second, ing, explaining, and gaining predictability
design studies are not typically scaled up are not about finding universal, immutable
to widespread practice; instead, they often laws but, rather, about creating models of
provide information that teachers find com- likely means of scaffolding that lead to suc-
pelling as a means to make sense of their cessful learning outcomes, by means of theo-
already rich experiences in practice. They are ries, materials, instructional approaches; and
often the basis for a staged series of investi- about providing instructional guidance on
gations which can be followed by curricu- domain-specific topics.
lar development and evaluation, or broader Sloane and Gorard (2003 ) offered an
forms of instructional efficacy studies. interesting criticism of design study method-
Representativeness and generalizability ology from a framework of model building.
are at work in many design studies. Dede Reiterating George Box’s 1978 maxim that
(2004) criticized design studies as “under- “All models are wrong, but some are useful”
conceptualized and over methodological- (cited in Sloane & Gorard, 2003 ), they iden-
ized,” and proceeded to explain that the tified three stages in model building: model
“freewheeling” and speculative part of design formulation or specification, estimation or
must be constrained by attention to prin- fit, and model validation. They outlined six
cipled variation. In addition, he suggested objectives of model formulation: parsimo-
that the failure to use most of the data nious description with adequate complexity,
produced (approximately at 5 percent) data set comparison, confirmation or refu-
meant that such work collects more than tation of theoretical relationships described
it can practically manage. These criticisms a priori, understanding of the error compo-
can be answered with reference to previ- nent, provision of a yardstick, and empirical
ously mentioned aspects of methodology. A understanding of underlying processes. They
clear specification of the initial conjectures suggested that design studies typically pay
attention to the model formulation and local cause-and-effect studies are the best way to
validation of the model, and tend to deem- improve classroom practice.
phasize fit of the data from the experi-
ments to the model, and ignore broader val- It is worth pointing out that the issues
debated by members of this committee are
idation. It could be argued that the fit of
not new, but have a long history in com-
data to model is inherent in design stud- plex fields where the limitations of the scien-
ies, in the iterations of the experiment, and tific method have been recognized for a long
that broader validation of a model requires a time. Ecology, immunology, epidemiology,
study of efficacy under a different method- and neurobiology provide plenty of exam-
ology. Future work might fruitfully examine ples where the use of alternative approaches
and explore how to apply these ideas more that include dynamical systems, game the-
completely to design studies. ory, large-scale simulations, and agent-
So far there is very little discussion of how based models have proved to be essential,
design studies might attend more adequately even in the design of experiments. We do
to issues of equity. I see this as a serious not live on a fixed landscape and, conse-
quently, any intervention or perturbation
oversight that resides in the fact that those
of a system (e.g., the implementation of new
engaged in the work lack sufficient diversity curricula) can alter the landscape. The fact
of background. In addition, Flyvbjerg (2001) that researchers select a priori specific lev-
provided a question to be asked of all social els of aggregation (often dictated by conve-
science efforts, namely, “who gains and who nience) and fail to test the validity of their
loses in power relations?” which could also results to such choices is not only common,
be asked of design studies. In addition, ques- but extremely limiting (validity). (Confrey
tions should be raised about how knowledge & Stohl, 2 004, p. 64)
is distributed across members of a class and
Design studies offer the field a method-
ways in which that knowledge is shared, par-
ology that seeks to recognize the complex-
titioned, authorized, and accessed (Confrey
ity of classroom practices and to provide the
& Makar, 2005 ).
kinds of insight needed to assist teachers in
accomplishing their challenging task. Their
success in doing so will depend in some mea-
Discussion and Conclusions sure on how well they can specify of what
kinds of knowledge they seek to produce,
I began this chapter citing by Scientific and the warrants for the knowledge they do
Research in Education and acknowledging produce.
both the promise and the relative immatu-
rity of design study methodologies. I close by
reiterating that the fundamental role of this Acknowledgments
work lies in its potential to create instruc-
tional theories at the level that holds the best The author wishes to acknowledge the assis-
possibilities to provide solutions to common tance in preparation and editing by Dr. Alan
ills in education. In the recent publication Maloney, Ms. Sibel Kazak, and Dr. Keith
by the National Research Council, On Eval- Sawyer.
uating Curricular Effectiveness,2 the authors
warn that an uncritical view of experimen-
talism may fail to recognize key elements Footnotes
of classroom instruction. They argue that
experimentalism rests on ideal conditions 1. The author of this chapter was a committee
and has a tendency to trade internal valid- member. However, this chapter reflects the
ity for external validity. In addition, they views of the author and not other members
warn that feedback is a critical element that of the committee or the NRC.
speaks to the adaptability of the process of 2. Confrey was the chair of the committee;
learning, and raises questions as to whether hence, there is less independence of the
comments here and the report than might Collins, A. (1992). Toward a design science of
appear at first glance. However, the use and education. In E. Scanlon & T. O’Shea (Eds.),
interpretation of the report are the product of New directions in educational technology (pp. 15 –
the author and should not be attributed to the 22). Berlin: Springer-Verlag.
committee, the NRC, or this report. Collins, A., Joseph, D., & Bielaczyc, K. (2004).
Design research: Theoretical and methodolog-
ical issues. The Journal of the Learning Sciences,
13 (1), 15 –42.
References Confrey, J. (1988, October). Multiplication and
splitting: Their role in understanding exponen-
Ackerman, E. (1995 ). Construction and trans- tial functions. Paper presented at the annual
ference of meaning through form. In L. P. meeting of the North American Chapter of
a. G. Steffe, J. (Ed.), Constructivism in edu- the International Group for the Psychology of
cation (pp. 3 41–3 5 4). Hillsdale, NJ: Lawrence Mathematics Education. DeKalb, IL.
Erlbaum Associates. Confrey, J., Castro-Filho, J., & Wilhelm, J.
Barab, S., & Squire, K. (2004). Design-based (2000). Implementation research as a means to
research: Putting a stake in the ground. The link systemic reform and applied psychology in
Journal of the Learning Sciences, 13 (1), 1–14. mathematics education. Educational Psycholo-
Brown, A. L. (1992). Design experiments: The- gist, 3 5 (3 ), 179–191.
oretical and methodological challenges in cre- Confrey, J., & Lachance, A. (2000). Transforma-
ating complex interventions in classroom set- tive teaching experiments through conjecture-
tings. The Journal of the Learning Sciences, 2 (2), driven research design. In A. E. Kelly & R. A.
141–178. Lesh (Eds.), Handbook of research design in
Brown, A. L., & Campione, J. (1996). Psycho- mathematics and science education (pp. 23 1–
logical theory and the design of innovative 265 ). Mahwah, NJ: Lawrence Erlbaum
learning environments: On procedures, princi- Associates.
ples, and systems. In L. Schauble & R. Glaser Confrey, J. (Producer/Writer), Lachance, A.
(Eds.), Innovations in learning: New environ- (Producer/Writer), & Hotchkiss, G. (Pro-
ments for education (pp. 289–298). Mahwah, ducer/Writer). (1996). In the voices of children.
NJ: Lawrence Erlbaum Associates, Inc. [Videotape]
Cobb, P. (2000). Conducting teaching experi- Confrey, J., & Makar, K. M. (2005 ). Critiquing
ments in collaboration with teachers. In A. E. and improving the use of data from high-stakes
Kelly & L. R. A. (Eds.), Handbook of research tests with the aid of dynamic statistics soft-
design in mathematics and science education ware. In C. Dede, J. P. Honan, & L. C. Peteres
(pp. 3 07–3 3 3 ). Mahwah, NJ: Lawrence Erl- (Eds.), Scaling up success: Lessons learned
baum Associates. from technology-based educational improve-
Cobb, P. (2002). Modeling, symbolizing, and tool ment (pp. 198–226). San Francisco: Jossey-
use in statistical data analysis. In K. P. E. Bass.
Gravemeijer, R. Lehrer, B. v. Oers & L. Confrey, J., & Stohl, V. (Eds.). (2004). On eval-
Verschaffel (Eds.), Symbolizing, modeling and uating curricular effectiveness: Judging the qual-
tool use in mathematics education (pp. 171–195 ). ity of k-12 mathematics evaluations. Washington,
Dordrecht: Kluwer Academic Publishers. DC: National Academy Press.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & DBRC. (2003 ). Design-based research: An
Schauble, L. (2003 ). Design experiments in emerging paradigm for educational inquiry.
educational research. Educational Researcher, Educational Researcher, 3 2 (1), 5 –8.
3 2 (1), 9–13 . Dede, C. (2004). If design-based research is
Cobb, P., & Steffe, L. P. (1983 ). The construc- the answer, what is the question? Commen-
tivist researcher as teacher and model builder. tary on Collins, Joseph, and Bielaczyc; diSessa
Journal for Research in Mathematics Education, and Cobb; and Fishman, Marx, Blumenthal,
14(2), 83 –94. Krajcik, and Soloway in the JLS special issue
Cohen, D. K., Raudenbush, S. W., & Ball, D. L. on design-based research. Journal of the Learn-
(2003 ). Resources, instruction, and research. ing Sciences, 13 (1), 105 –114.
Educational Evaluation and Policy Analysis, Dewey, J. (1981). The philosophy of John Dewey:
2 5 (2), 119–142. Volume I – the structure of the experience,
Volume II – the lived experience. Chicago: Uni- “Theorizing learning practice,” University of
versity of Chicago Press. Illinois, Champagne-Urbana.
diSessa, A. A. (2004). Students’ criteria for rep- Hawkins, D. (2002). The informed vision: Essays
resentational adequacy. In K. Gravemeijer, R. on learning and human nature. New York:
Lehrer, B. v. Oers, & L. Verschaffel (Eds.), Algora Publishing.
Symbolizing, modeling and tool use in mathemat- Kantowski, M. G., Steffe, L. P., Lee, K. S.,
ics education (pp. 105 –13 0). Dordrecht: Kluwer & Hatfield, L. H. (1978). The soviet “teach-
Academic Publishers. ing experiment”: Its role and usage in Ameri-
diSessa, A. A., & Cobb, P. (2004). Ontological can research. Annual Meeting of the National
innovation and the role of theory in design Council of Teachers of Mathematics (NCTM).
experiments. The Journal of the Learning Sci- San Diego, CA.
ences, 13 (1), 77–103 . Kegan, R. (2000). What “form” transforms? A
Duckworth, E. (1996). The having of wonderful constructive-developmental approach to trans-
ideas. New York: Teachers College Press. formative learning. In J. Mezirow & Associates
Elkana, Y. (1974). The discovery of the conser- (Eds.), Learning as transformation: Critical per-
vation of energy. London: Hutchinson Educa- spectives on a theory in progress (pp. 3 5 –69). San
tional, Ltd. Francisco: Jossey-Bass.
Elmore, R. F. (1996). Getting to scale with Lagemann, E. C. (2002). An elusive science: The
good educational practice. Harvard Educa- troubling history of education research. Chicago:
tional Review, 66(1), 1–26. University of Chicago Press.
Fernandez, C., & Yoshida, M. (2004). Lesson Lakatos, I. (1976). Proofs and refutations: The logic
study: A Japanese approach to improving math- of mathematical discovery. Cambridge: Cam-
ematics teaching and learning. Mahwah, NJ: bridge University Press.
Lawrence Erlbaum Associates. Lakatos, I., & Feyerabend, P. (1999). For and
Fisher, C., Dwyer, D., & Yoacam, K. (Eds.). against method. Chicago: University of Chicago
(1996). Education and technology: Reflections on Press.
computing in classrooms. San Francisco: Jossey- Lehrer, R., & Pritchard, C. (2002). Symbolizing
Bass. space into being. In K. Gravemeijer, R. Lehrer,
Fishman, B., Marx, R. W., Blumenfeld, P., Krajcik, B. v. Oers, & L. Verschaffel (Eds.), Symboliz-
J., & Soloway, E. (2004). Creating a framework ing, modeling and tool use in mathematics educa-
for research on systemic technology innovation (pp. 5 9–86). Dordrecht: Kluwer Academic
tions. The Journal of the Learning Sciences, 13 (1), Publishers.
43 –76. Lehrer, R., & Schauble, L. (2004). Modeling nat-
Flyvbjerg, B. (2001). Making social science matter: ural variation through distribution. American
Why social inquiry fails and how it can suc- Educational Research Journal, 41(3 ), 63 5 –679.
ceed again (S. Sampson, Trans.). Cambridge: Lesh, R. (1999). The development of representa-
Cambridge University Press. tional abilities in middle school mathematics:
Gee, J. P. (1999). An introduction to discourse The development of student’s representations
analysis: Theory and method. New York: during model eliciting activities. In I. E. Sigel
Routledge. (Ed.), The development of mental representation
Gravemeijer, K. P. E. (1995 ). Developing realistic (pp. 3 23 –3 49). Mahwah, NJ: Lawrence Erl-
mathematics instruction. Utrecht: Freudenthal baum Associates.
Institute. Lesh, R. A., & Kelly, A. E. (2000). Multitiered
Greeno, J. G. (1997). Theories and practices teaching experiments. In A. E. Kelly & L. R. A.
of thinking and learning to think: Mid- (Eds.), Handbook of research design in math-
dle school mathematics through applications ematics and science education (pp. 197–23 0).
project. American Journal of Education, 106(1), Mahwah, NJ: Lawrence Erlbaum Associates.
85 –126. Merrill, D. M. (2001). Toward a theoretical tool
Greeno, J. G. (2003 , November). Positioning, for instructional design. Instructional Science,
problematizing, and reconciling: Aspects of 2 9(4–5 ), 291–3 10.
productive cognition and learning in a situative Papert, S. (1980). Mindstorms: Children, comput-
perspective. Paper presented at a conference, ers, and powerful ideas. New York: Basic Books.
Pea, R. D. (1987). Cognitive technologies for Journal for Research in Mathematics Education,
mathematics education. In A. Schoenfeld 2 6(2), 114–145 .
(Ed.), Cognitive science and mathematics educa- Simon, M. A. (2000). Research on the develop-
tion (pp. 89–122). Hillsdale, NJ: Lawrence Erl- ment of mathematics teachers: The teacher
baum Assocates. development experiment. In A. E. Kelly &
Piaget, J. (1976). The child’s conception of the world. L. R. A. (Eds.), Handbook of research design in
Totowa, NJ: Littlefield, Adams, & Co. mathematics and science education (pp. 3 3 5 –
Reeves, T. C. (2000, April). Enhancing the worth of 3 5 9). Mahwah, NJ: Lawrence Erlbaum
instructional technology research through “design Associates.
experiments” and other developmental research Sloane, F. C., & Gorard, S. (2003 ). Exploring
strategies. Paper presented at the annual meet- modeling aspects of design experiments. Edu-
ing of the American Educational Research cational Researcher, 3 2 (1), 29–3 1.
Association. New Orleans, LA. Steffe, L. P. (1991). The constructivist teaching
Scarano, G., & Confrey, J. (1996, April). Results experiment: Illustrations and implications. In
from a three-year longitudinal teaching experi- E. von Glasersfeld (Ed.), Radical constructivism
ment designed to investigate splitting, ratio and in mathematics education (pp. 177–194). Boston:
proportion. Paper presented at the Annual Kluwer Academic Publishers.
Meeting of the American Educational Research Thompson, P. W. (1979). The constructivist
Association, New York, NY. teaching experiment in mathematics education
Shavelson, R. J., Phillips, D. C., Towne, L., & research. Paper presented at the Annual Meet-
Feuer, M. J. (2003 ). On the science of edu- ing of the National Council of Teachers of
cation design studies. Educational Researcher, Mathematics, Boston, MA.
3 2 (1), 25 –28. von Glasersfeld, E. (1982). An interpretation of
Shavelson, R. J., & Towne, L. (Eds.). (2002). Sci- piaget’s constructivism. Revue Internationale de
entific research in education. Washington, DC: philosophie, 3 6, 612–63 5 .
National Academy Press. Vygotsky, L. S. (1978). Mind in society: The devel-
Simon, H. A. (1969). The sciences of the artificial. opment of higher psychological processes. Cam-
Cambridge, MA: MIT Press. bridge, MA: Harvard University Press.
Simon, M. (1995 ). Reconstructing mathemat- Vygotsky, L. S. (1986). Thought and language (A.
ics pedagogy from a constructivist perspective. Kozulin, Trans.). Cambridge, MA: MIT Press.

Chapter 9 Confrey 2006

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 9 Confrey 2006

Uploaded by

Copyright:

Available Formats

CHAPTER 9

The Evolution of Design Studies

structured or guided apprenticeship in sup- argued for complex, multidimensional, and

it seems unnecessary, and appears to com- or indeterminate situation is akin to issuing

Prior Class’s actual

Figure 9.1. The goal of design experiments.

by design study methodology: important provides a sense of what must be var-

You might also like