Computers & Education 171 (2021) 104238

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

Toward a framework that connects individual TPACK and

collective TPACK: A systematic review of TPACK studies
investigating teacher collaborative discourse in the learning by
design process
Yi-Fen Yeh a, d, Kennedy Kam Ho Chan b, *, Ying-Shao Hsu c, d
a
College of Teacher Education, National Taiwan Normal University 162, Sec. 1, Heping E. Rd., Taipei, 106, Taiwan
b
The University of Hong Kong, Room 313, Runme Shaw Building, Pok Fu Lam, Hong Kong
c
Graduate Institute of Science Education, National Taiwan Normal University 88, Sec. 4, Ting-Chou Rd., Taipei, 116, Taiwan
d
Faculty of Education, University of Johannesburg, Johannesburg, South Africa

A R T I C L E I N F O A B S T R A C T

Keywords:
Technological pedagogical content knowledge
Learning technology by design
Teacher collaborative discourse
“Learning by design” is a widely used approach for developing teachers’ technological peda­
gogical content knowledge (TPACK) in the context of designing technology-enhanced instruction.
This approach offers teachers the opportunity to learn from colleagues with different areas of
expertise. Despite the critical importance of teachers’ collaborative discourse in mediating
teacher learning in the design process, there has not yet been a TPACK review to analyze how
researchers investigate this discourse. After a systematic literature search, we identified eleven
TPACK studies from seven geographical regions that investigated teachers’ collaborative
discourse, and then examined how the researchers had structured and analyzed this discourse and
integrated their findings. Our analysis identified four strategies used by the researchers to
structure and promote teachers’ collaborative discourse. We found that most studies relied on the
“coding and counting” data analysis method to reveal the distribution of TPACK (sub)sets
expressed in the collaborative discourse but not the dynamics of the knowledge construction
process or the progression of the discourse. Although the studies generally provided initial evi­
dence for the efficacy of the design process, this evidence was rarely based on changes in teachers’
TPACK as enacted during classroom instruction or on the quality of the design outcomes. We thus
propose a new conceptual framework that emphasizes the reciprocal knowledge exchange process
between the TPACK of individual teachers and the knowledge shared by teachers through
collaboration. This framework refocuses the attention of researchers on investigating and con­
necting teachers’ individual and collective TPACK and their development during the learning by
design process.

1. Introduction

Educational technology can support and enhance student learning, but successfully integrating technology into the teaching of

* Corresponding author.
E-mail addresses: yyf521@gapps.ntnu.edu.tw (Y.-F. Yeh), kennedyckh@hku.hk (K.K.H. Chan), yshsu@ntnu.edu.tw (Y.-S. Hsu).

https://doi.org/10.1016/j.compedu.2021.104238
Received 2 November 2020; Received in revised form 10 May 2021; Accepted 15 May 2021
Available online 21 May 2021
0360-1315/© 2021 Elsevier Ltd. All rights reserved.
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

specific subjects is a complex endeavor (Herring, Koehler, Mishra, Rosenberg, & Teske, 2016). Mishra and Koehler (2006) extended the
seminal work of Lee Shulman (1986, 1987) and argued that teachers should develop a complex, situated, and integrated body of
knowledge, known as technological pedagogical content knowledge (TPCK), or more recently as technological, pedagogical, and
content knowledge (TPACK) (Thompson & Mishra, 2007). Rather than simply combining the three constituent knowledge domains of
technology, content, and pedagogy, TPACK reflects the dynamic integration of the three domains and is essential in mediating the
enactment of context-specific, technology-enhanced (TE) instruction by teachers (Koehler & Mishra, 2008). The notion of TPACK
challenges teacher educators to go beyond merely teaching how to use technology and to instead develop teachers’ capacity to
integrate technology into the teaching of specific subject matter (Niess, 2005; Voogt, Fisser, Pareja Roblin, Tondeur, & van Braak,
2013). Developing teachers’ TPACK has become a major feature of teacher education and in-service teacher professional development
programs globally (Voogt et al., 2013; Wang, Schmidt-Crawford, & Jin, 2018). Teachers with high-quality TPACK are more effective in
planning (Harris & Hofer, 2011; Koehler & Mishra, 2005) and enacting (Hughes, 2005; Niess, 2005) TE instruction.
“Learning by design” (LBD) (Koehler & Mishra, 2008; Koehler, Mishra, Hershey, & Peruski, 2004) is a widely used approach that
engages teachers in collaborative design to develop teachers’ competences for technology integration in general (Voogt et al., 2011,
2015) and TPACK in specific (e.g., Agyei & Voogt, 2012; Baran & Uygun, 2016; Koehler, Mishra, & Yahya, 2007; Papanikolaou, Makri,
& Roussos, 2017). LBD is theoretically grounded in the situated view of teacher learning (e.g., Borko, 2004; Voogt, Westbroek,
Handelzalts, Walraven, Pieters, & De Vries, 2011). Specifically, LBD involves engaging teachers in teams of two or more to design TE
instruction through iterative cycles of design, redesign, and reflective practices (Voogt et al., 2015). The collaborative design process
during LBD enables teachers to share knowledge through collaborative discourse so that they can learn and integrate knowledge from
colleagues with different areas of expertise in a teacher community (Angeli & Valanides, 2005; So & Kim, 2009). Through the
collaborative design process, teachers can also practicalize their knowledge in specific contexts around an authentic design problem
related to technology integration (Koehler et al., 2007). The design process enhances the teachers’ agency and ownership of the
knowledge, repositioning them from recipients of knowledge to creators in their own specific contexts (Papanikolaou et al., 2017;
Voogt et al., 2015).
As teachers’ social and cognitive engagement frequently materializes in discourse within a teacher community (Csanadi, Eagan,
Kollar, Shaffer, & Fischer, 2018), teachers’ collaborative discourse is critical to their joint sense-making and knowledge building

Table 1
Summary of prominent reviews of TPACK and teacher collaborative design.
Authors Range of years Number of publications Focus of the review
reviewed

Abbitt (2011) 2005–2010 20 publicationsa • Instruments and methods used to measure TPACK and challenges,
purposes, and potential uses of these tools for TPACK-based evalua­
tion of pre-service teacher preparation experiences
Wu (2013) 2002–2011 24 journal articles • General characteristics of the published TPACK studies, including the
number of studies, sample groups, subject domains, and research
methods
Chai et al. (2013) Up to May 2011 74 journal articles • General characteristics of the studies (i.e., study site, publication
outlet, research methods) and how researchers designed their
program according to the TPACK framework
Voogt et al. (2013) 2005 to September 55 journal articles and • Theoretical basis (i.e., development of the concept, views on TK,
2011 1 book chapter development of TPACK as a concept in specific subject domains, and
TPACK and teacher beliefs) and practical use of TPACK (i.e.,
measuring TPACK and strategies for developing (student-) teachers’
TPACK)
Rosenberg and Koehler 2005–2013 74 journal articles • The extent to which context is included in TPACK research and the
(2015) meaning of context
Willermark (2018) 2011–2016 107 journal articles • General characteristics of recently published TPACK studies (e.g.,
research design and methods, research samples, subject domains,
etc.) and approaches to identifying teachers’ TPACK
Wang et al. (2018) 2006 January to 88 journal articles • Types of research methodologies for measuring pre-service teachers’
2015 September TPACK development, research findings on pre-service teachers’
TPACK development
Njiku, Mutarutinya, and 2009–2017* 28 journal articles • The nature of survey instrument items for measuring TPACK
Maniraho (2020)
Voogt et al. (2011) 1988–2009 9 journal articles • Processes in teacher design teams that contribute to teacher learning
Voogt, Pieters, et al. 2009–2015 14 doctoral theses • The processes and effects of collaborative design in teacher teams on
(2016) teacher learning and curriculum change
This review Up to February 2020 11 journal articles • How researchers structured and analyzed teachers’ collaborative
discourse during learning by design and the findings of these studies
Notes: * As the article did not specify the search period, the time period represents the years in which the studies were published.
a
These 20 publications were selected from 91 manuscripts, including 34 journal articles, 52 conference proceedings, 2 book chapters, 2 doctoral
dissertations, and 1 report.

2
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

processes (Lefstein, Louie, Segal, & Becher, 2020; Sherin, Linsenmeier, & van Es, 2009). A critical determinant of whether and how
teachers develop TPACK from the LBD process is the quality of their collaborative discourse during the collaborative design process (or
design talk) (Chai, Koh, & Tsai, 2016). To date, reviews of the role of teachers’ collaborative design process in mediating teacher
learning (Voogt et al., 2011; Voogt, Pieters, and Handelzalts, 2016; see Table 1) have not analyzed the teachers’ collaborative
discourse during the design process or interpreted the results through a TPACK lens.
Our review addresses this gap by focusing on how the various TPACK researchers have structured and analyzed teachers’
collaborative discourse, and then integrating the findings of these studies. As diverse methodological approaches and tools can be used
to analyze teachers’ collaborative discourse (Lefstein et al., 2020), an examination of the existing methods may reveal gaps in how
teachers’ collaborative discourse has been analyzed. With these foci, our review is thus distinct from and complementary to prior
reviews of TPACK studies (Table 1), which have largely focused on the TPACK of individual teachers and provide little detail on how
researchers investigate teachers’ TPACK in a collective setting (e.g., Chai, Koh, & Tsai, 2013; Voogt et al., 2013; Willermark, 2018).
Accordingly, our review makes at least three important contributions to the field. First, it identifies how researchers have structured
teachers’ collaborative discourse, an important mediator of teacher learning in the LBD process. Second, it reveals issues and gaps in
the focus and methods of previous studies of teacher’s collaborative discourse in the LBD context. Third, it proposes a new conceptual
framework to investigate teachers’ TPACK and its development during the LBD processes, which critically connects the investigation of
teachers’ TPACK at the individual level to TPACK at the collective level.

1.1. Theoretical perspectives

1.1.1. TPACK and TPACK development

TPACK refers to the professional knowledge teachers require for the effective design and implementation of technology-enhanced
(TE) instruction. Shulman (1986, 1987) regarded pedagogical content knowledge (PCK) as integrated knowledge, and Mishra and
Koehler (2006) similarly proposed TPCK (later called TPACK) as the overlap between three independent knowledge domains (i.e.,
technology, content, and pedagogy) and/or three mutually integrated subsets of knowledge (i.e., TPK, TCK, and PCK). Since its
introduction, TPACK has been widely embraced as a way of articulating the knowledge required by teachers to enact successful TE
instruction (AACTE Committee on Innovation and Technology, 2008; Herring, Koehler, & Mishra, 2016). Although many researchers
have embraced the notion of TPACK, each has conceptualized and interpreted TPACK differently (See a helpful review by Voogt et al.
(2013).). For example, some researchers have conceptualized TPACK as extended PCK (e.g., Cox & Graham, 2009; Niess, 2005) while
others have taken a transformative stance, viewing it as a distinct category of knowledge (e.g., Angeli & Valanides, 2009). Some re­
searchers have focused on specific technologies in their conceptualization (e.g., TPCK-W for the World Wide Web (Lee & Tsai, 2010)
and G-TPACK for geospatial (geographic) technologies (Doering & Veletsianos, 2007)), whereas others have foregrounded the
content-specific nature of TPACK (e.g., technological pedagogical science knowledge (TPASK; Jimoyiannis, 2010). Still others have
highlighted the importance of TPACK in teacher practices, such as lesson planning (Harris & Hofer, 2011) or practical teaching
experience (TPACK Practical or TPACK-P; Yeh, Hsu, Wu, Hwang, & Lin, 2014). Despite these variations, as Di Blas, Paolini, Sawaya,
and Mishra (2014) have pointed out, TPACK “has consistently been conceptualized as being a form of knowledge that is resident in the
heads of individual teachers” (p. 2457). Although researchers have acknowledged that a teacher’s TPACK is embedded in the social
context (Rosenberg & Koehler, 2015), only recently have TPACK researchers begun to suggest a more distributive and situative view of
TPACK, conceptualizing it as a form of knowledge shared by members of a teacher community (Carpenter et al., 2020; Di Blas et al.,
2014; Di Blas & Paolini, 2016; Jones, Heffernan, & Albion, 2015).
Koehler et al. (2007) stressed that TPACK development is “a multigenerational process, involving the development of deeper
understandings of the complex web of relationships between content, pedagogy, and technology and the contexts in which they
function” (Koehler et al., 2007, p. 740). Many approaches and professional development frameworks have been applied to enhance
teachers’ TPACK development in initial teacher education (De Rossi & Trevisan, 2018) and the professional development of in-service
teachers (Harris, 2016). Although not explicitly identified as such, many of these approaches are often grounded in a situative view of
teacher learning, which emphasizes learning as a social process occurring as individuals participate in the discourse and practices of a
particular learning community situated in particular physical and social contexts (Greeno, 2003; Lave & Wenger, 1991; Putnam &
Borko, 2000). Core aspects of this view of teacher learning are teachers’ day-to-day classroom practices, the distributed nature of
knowledge, and learning communities (Borko, 2004; Greeno, 2003; Putnam & Borko, 2004; Voogt et al., 2015). Engaging teachers in
collaboration in a teacher community to share their knowledge is deemed particularly important for developing teachers’ TPACK.
First, TPACK involves complex integration of the technology, content, and pedagogy. Engaging teachers in collaboration allows them
to share, learn from, and integrate the available knowledge that is distributed across colleagues with different expertise. Second, the
situated nature of TPACK means that teachers need to bring together a range of knowledge resources which are often shared by a team
of teachers.

1.2. LBD as a professional development approach

In their review of TPACK studies, Voogt et al. (2013) identified engaging teachers in designing a TE lesson or course as a major
strategy for developing their TPACK. This strategy has been used to enhance teachers’ TPACK in pre-service teacher education (e.g.,
Agyei & Voogt, 2012; Papanikolaou et al., 2017), in-service teacher professional development (Baran & Uygun, 2016; Jimoyiannis,
2010), and faculty professional training (Koehler et al., 2007; Rienties, Brouwer, & Lygo-Baker, 2013). Despite the variation in ter­
minology (e.g., “design-based learning” (Baran & Uygun, 2016), “collaborative design” (Voogt et al., 2015), “teacher design team”

3
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

(Voogt, Fisser, Tondeur, & van Braak, 2016a)) and slight differences in focus, the core of all of these strategies is learning by design
(LBD; Koehler & Mishra, 2008; Koehler et al., 2004). As Voogt, Fisser, Tondeur, and van Braak (2016) observed, “there seems [to be]
agreement that TPACK can best be developed through ‘learning by design’” (Koehler & Mishra, 2008, p. 33).
LBD was first proposed by Koehler and colleagues to avoid using traditional technocratic approaches in the development of
teachers’ technology-based pedagogy for specific content areas (Koehler et al., 2004). It is grounded in the perspective that teaching
can be regarded as a “design science”—a science teachers use to develop their teaching performance, guided by investigations based on
theory and evidence (Laurillard, 2012). LBD is an iterative technology-mapping process in which teachers consider tool affordances,
learner needs, and pedagogical concerns aligned with the important concepts of specific topics in meaningful and authentic contexts
(Angeli & Valanides, 2009). Theoretically, LBD aligns with the situated views of teacher learning (Borko, 2004; Greeno, 2003; Voogt
et al., 2015). First, LBD engages a group of teachers in mutual collaboration within a design community. Selecting and embedding the
technological tools that best support teaching and learning needs involves a complex decision-making process. Working in teams
enables teachers to exchange their knowledge, negotiate solutions for technology integration, and reflect deeply as practitioners
(McDuffie & Mather, 2009). Moreover, design-based activities not only provide a rich and authentic context for learning, but also lend
themselves to sustained inquiry and revision that help designers come away with the deep understanding needed to apply knowledge
in the complex domains of real-world practice connected to the teachers’ workplace (Mishra & Koehler, 2003; Vyas & Mishra, 2002).
During the co-design process, teachers develop their TPACK through engagement in authentic design problems involving the inte­
gration of technology to support their teaching and student learning (Yeh et al., 2014). Iterative cycles of design, redesign, and
reflective practices further enable teachers to individualize their own TPACK to be more coherently integrated and practically
transformed (Hong, Lin, Chai, Hung, & Zhang, 2019; Lee & Kim, 2014). Overall, this approach engages teachers in designing tasks and
repositions them from passive recipients of knowledge about technology-enhanced teaching to active designers of technology-rich
environments and creators of contextual and student-specific knowledge (Voogt et al., 2015).

1.3. Teacher collaboration and collaborative discourse

Teacher collaboration is an important feature of effective teacher professional development. As Desimone (2009) put it, “such
arrangements set up potential interaction and discourse, which can be a powerful form of teacher learning” (p. 184). Teacher
collaboration is critical for the successful implementation of innovative student-centered pedagogies (Timperley, Wilson, Barrar, &
Fung, 2007) and the improvement of student achievement (Ronfeldt, Farmer, McQueen, & Grissom, 2015). Teacher collaboration is
also advocated in professional development centered on technology integration (Kopcha, 2012; Macdonald, 2008; Schrum & Levin,
2013). Engaging teachers in collaboration benefits not only newcomers but also old-timers, who can learn from the technological
experience of the newcomers. Such collaboration can lead to a greater gain for the community that is larger than the sum of its parts
(Macdonald, 2008). However, engaging teachers in a design team does not necessarily result in meaningful collaboration (Pieters &
Voogt, 2016). Teacher collaboration can vary in depth and quality, ranging from superficial interaction to deeply meaningful
collaboration (Vangrieken, Dochy, Raes, & Kyndt, 2015). As the distribution of knowledge within the design teams ultimately takes
place through social interaction among the participants, the collaborative design process, including the interactions and dynamics
among the design team members during the LBD process, are critical to the success of the LBD approach (Koehler & Mishra, 2008;
Voogt et al., 2015). As Lefstein et al. (2020) pointed out, “discourse is a critical component of collaboration and community. It is the
medium through which members of a community of practice work together, make meaning, and negotiate roles and identity (p. 1). It
follows that the primary medium of interaction for the social distribution of knowledge is teachers’ collaborative discourse that occurs
during the collaborative design process. From a TPACK perspective, Chai et al. (2016) reminded us of the importance of teachers’
collaborative discourse during the design process, suggesting that as “the quality of the collaborative design processes is pivotal to the
constructed TPACK … researchers should explore ways of measuring design talk processes” (p. 102).
Taken together, it is the interactions and dynamics between the design team members that constitute the resources for teacher
learning during the LBD process. As the social and cognitive engagement of the design team members frequently materialize in
teachers’ collaborative discourse, it is critical to examine how researchers have structured and analyzed this discourse and to evaluate
their findings. Three research questions formed the basis of our review.

(1) How has the collaborative discourse of teachers been structured in TPACK studies?
(2) How was this discourse analyzed?
(3) What have these studies contributed to our knowledge of teachers’ collaborative discourse and TPACK and its development?

2. Methods

We used a systematic review process (Petticrew & Roberts, 2006) and began by querying two major databases, Web of Science
(WoS) and Scopus. We limited our search to peer-reviewed articles in the social sciences (search date: February 13, 2020). We followed
the TPACK review by Voogt et al. (2013) in using broad search terms to adequately survey the whole TPACK field. The keywords used
were “technological pedagogical content knowledge,” “technological pedagogical and content knowledge,” “TPACK,” and “TPCK.”
The initial search returned 1,203 and 1,430 hits from WoS and Scopus, respectively. After removing duplicates obtained from the
keyword search from each database, the numbers of articles were 707 and 819, respectively. After removing the duplicates found in
both databases, we obtained 978 articles. The authors then worked together to apply the following selection criteria:

4
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

1. Empirical studies (we excluded literature reviews, editorials, position papers, and commentaries);
2. A focus on teachers’ TPACK or its development;
3. Includes descriptions of the collaborative design process between teachers (either as an intervention or as part of the professional
development experience in initial teacher education or professional development activities);
4. Includes teachers’ collaborative discourse as part of the study; and
5. Sufficiently describes how the TPACK of teachers in the collaborative discourse process was analyzed.

Studies involving pre-service, in-service and university teachers and those undertaken in primary, secondary and tertiary settings
were included. The titles and abstracts were initially screened by the authors to identify articles for potential inclusion, with full-text
screening in cases of doubt. The three authors independently screened 20 articles, and there was no disagreement on the articles
selected for review. Each author then screened the abstracts of around 350 articles and around 40 full texts for further clarification (i.
e., 122 in total). After consulting with the research team in cases of doubt, we finally identified 11 articles for review. Fig. 1 summarizes
the details of the literature search process.
The research team worked collaboratively to develop an analytical framework. The first author read all the articles and developed
an initial coding scheme. The second and third authors then each read one half of the selected articles to refine the coding scheme.
After a series of research meetings, the authors developed the initial draft of the Article Analysis Form to focus the data analysis. The
second and third authors then each analyzed one half of the articles and completed the form independently. The Article Analysis Form
was iteratively refined (see the final version in Appendix 1). Summary tables based on the completed Article Analysis Form were then
compiled by the authors and discussed in research meetings. The completed forms were used to identify themes (see below for details).
We used investigator triangulation (Denzin, 2017) throughout the process to ensure reliability. The coding of the articles was
verified by at least two team members, and any difficulties were discussed, and the final coding negotiated until complete consensus
was reached. The completed tables were then verified and checked by the first author. The tables provide sufficient detail for readers to
see how our codes and themes were generated.

3. Findings

The basic information about the reviewed articles is summarized in Table 1. Two themes relevant to RQ1 were identified: the

Fig. 1. Literature search process.

5
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

organization of the collaborative discourse and the strategies used to promote it. The themes identified for RQ2 were the TPACK
models that guided the investigations into the collaborative discourse and the nature of the data and its analysis. Finally, we integrated
the findings related to the content of the teachers’ collaborative discourse and TPACK and its development.

(1) Study contexts

Table 2 provides the bibliographic information and research context of the studies reviewed. Of the 11 studies, all except #5 and #9
analyzed teachers’ collaborative discourse related to specific subject-matter teaching. Most were related to either first or second/
foreign-language teaching. In seven studies (#1, #2, #6, #7, #8, #10, #11), the data were collected from in-school teacher profes­
sional development workshops initiated in response to school-based initiatives or to facilitate the (re-)design of the curriculum/ma­
terial. The remainder were implemented in teacher education (#3, #4, #9) or faculty development (#5) courses. In terms of grade
level, most involved kindergarten or primary school teachers, with #5, #10, and #11 focusing on secondary school teachers or college
instructors.
Overall, the studies were conducted at diverse locations and educational levels, suggesting that LBD is widely recognized by re­
searchers around the world as a useful means of developing the TPACK of pre-, in-service, and university teachers. However, the
limited number of studies investigating the teachers’ collaborative discourse published in peer-reviewed journals signals a lack of
attention to the role of teachers’ collaborative discourse in empirical TPACK studies.

(2) Organization of teachers’ collaborative discourse

The reviewed studies differed in their design community composition, means of collaborative discourse, and design outcomes/artifacts
(Table 3). The number of design team members ranged from 3 to 7, except in #8 and #10. The members included pre-service teachers,
in-service teachers, teacher education instructors, and external experts, depending on the research purpose, context, and professional
development needs. In some studies, content experts (e.g., #1) or experienced teachers (e.g., #11) were involved in the design process.
For studies that were conducted alongside other school improvement plans, it was important that teachers teamed up based on their
needs, such as designing curricula or teaching collaboratively (#6, #7, #8).
Regarding the major means of collaboration, most studies engaged teachers mainly in face-to-face collaboration. Computer-
mediated communication (CMC) was also used to extend team members’ collaboration after their face-to-face interactions. These
included Facebook chats (#9) or email (#5). Some studies used web-conferencing systems (#3, #4, #10) or chat messaging platforms
(#11) to engage participants from different geographical locations in collaboration.
Teachers were engaged in designing TE materials and/or enacting TE instruction, but the design outcomes varied. The outcomes

Table 2
Summary of the reviewed studies.
Study Author(s) Journal Country Discipline Context

1 Boschman, Journal of Computer Assisted Netherlands Early literacy Kindergarten in-service teacher PD workshops
McKenney, Learning
Pieters and
Voogt (2016)
2 Boschman, Computers & Education Netherlands Early literacy Kindergarten in-service teacher PD workshops
McKenney, and
Voogt (2015)
3 Bueno-Alastuey Estudios Sobre Educación Spain Content and Teacher education course for pre-school/primary school
and García Language Integrated
Esteban (2016) Learning (CLIL)
4 Bueno-Alastuey, Technology, Pedagogy and Spain CLIL Teacher education course for pre-school/primary school
Villarreal, and Education
García Esteban
(2018)
5 Koehler et al. Computers & Education USA Educational Faculty development course: Learning technology by
(2007) Technology design
6 Koh (2019) International Journal of Science Singapore Mathematics School initiatives in primary school
and Mathematics Education
7 Koh and Chai Computers & Education Singapore English, School initiatives in primary school
(2016) Mathematics, Science
8 Koh, Chai, and Computers & Education Singapore (Not mentioned) Ministry-funded school-development project in a
Tay (2014) primary school
9 Nguyen and British Journal of Educational Australia ICT ICT in education course for pre-service primary-school
Bower (2018) Technology teachers
10 Tseng, Cheng, Computers & Education Taiwan English Distance learning project for senior secondary school
and Yeh (2019) students initiated by an in-service teacher with pre-
service teachers
11 Yan, Chai, and Australasian Journal of China Mathematics A development and evaluation study for inquiry-based
So (2018) Educational Technology learning in secondary school contexts

Note: Teachers of Grades 1 to 6 were categorized as primary teachers and those of Grades 7 to 12 as secondary teachers.

6
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

Table 3
Structure of the teachers’ collaborative discourse.
Study Composition of the design community Major means of Design outcomes/artifacts
collaborative discourse

Number & type of participants Number Number of Team composition Face- Online
of teams participants in to-
each team face

1 6 kindergarten teachers 2 5, 3 1 content expert in X PictoPal learning materiala

and 2 content (language) each team
experts
2 6 kindergarten teachers 1 7 All participants X PictoPal learning materiala
and 1 teacher coordinator
3 26 + 19 pre-service 10 N/A N/A X Analyzing an ICT-
teachers from 2 (BigBlueButton) integrated CLIL unit
universities design
4 28 + 17 pre-service 9 5–6 3–4 from 1 X (Skype) Sharing views in a
teachers from 2 university and 2 questionnaire and
universities from the other analyzing an ICT-
integrated CLIL unit
design
5 6 faculty members and 18 2 focal 4 1 faculty member X X (e-mail) A prototype of an online
education technology teams out and 3 Master’s courseb
Master’s students of 6 students
6 3 primary mathematics 1 3 3 in-service teachers X Technology-integrated
teachers and lessons (Topic: Averages;
Tool: Microsoft Excel)
7 27 primary teachers 5 5–6 English (2 teams), X Student-centered ICT
Math (2 teams), lessons for school
Science (1 team) curriculum
8 24 primary teachers 3 6, 7, 11 1 team each for X ICT lessons
Grades 1, 4, and 5,
respectively (1 lead
teacher in each
group)
9 9 pre-service teachers 3 3 3 pre-service X Moodle-based course
teachers
10 6 pre-service English 1 8 All participantse c
X (Adobe Online teaching materials
teachers, 1 in-service Connect) and activities (Adobe
English teacher, and 3 Connect, PowerPoint)
university professors
11 24 pre-service math 6 6 1 in-service teacher, X (QQ)d Technological pedagogical
teachers, 5 practicing 4 pre-service mathematics tools
secondary math teachers, 2 teachers, and 1 (TPMTs) with Flash script
university-based instructor technology for inquiry-
educational technology based learning
instructors

Notes.
a
PictoPal comprises computer activities that take place in a software-based learning environment and teacher-designed off-computer activities
(McKenney & Voogt, 2009).
b
The protocol comprises the syllabus, course structure, readings, student assignments, and assessment rubrics.
c
The article did not mention whether the discussion was conducted online or face-to-face. As the article described it as a post-teaching discussion,
we coded it as online.
d
The discussion was in the form of chat messages on the Tencent QQ platform (called QQ). It was unclear whether the participants met during
designated timeslots.
e
Three professors were involved in the larger project, but only one provided feedback near the end of the meeting.N/A indicates the information
was not available in the article.

mainly included creating technology-supported lessons (#5, #6, #7, #8, #9) and teaching or learning materials (#1, #2, #10, #11).
Some studies pre-specified the technological tools or platforms to be considered and implemented in the design outcomes (#1, #2, #6,
#9, #10, #11). The designated technologies were chosen mainly for their affordances for disciplinary learning needs, except in Studies
9 and 10, which used platforms catering to generic learning needs. For example, PictoPal, a platform supported by Clicker@ software,
facilitates student literacy learning through mapping written texts to pictures or text-to-speech functions (#1, #2). Study 11 required
pre-service teachers to create two-dimensional animations for selected math concepts using Adobe Flash, as they were familiar with the
C++ programming language. In the studies that did not predetermine the technology to be used (e.g., #3, #4, #5, #8), the teachers
had greater freedom to select the technology.
To summarize, the reviewed studies differed in design team composition, means of collaborative discourse, and design outcomes.
The distributive view of TPACK highlights the importance of carefully considering the design team composition. The design teams in

7
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

some studies involved only pre-service teachers or only in-service teachers as team members. As TPACK refers to integrating knowledge
relevant to designing and enacting TE instruction (Koehler & Mishra, 2008), in line with the distributive nature of TPACK, the teams
should ideally include content experts, technology experts (or teachers adept at using technology), and pedagogues (or experts with
expertise in more than one area). Thus, the various contributions of individual knowledge distributed across individuals can be shared
during the collaborative discourse, encouraging the development of TPACK within the design communities. Understandably, design
team composition in actual practice is highly dependent on the tasks and resources available. However, during initial teacher edu­
cation, design teams should preferably include teacher educators or experienced teachers who can provide the novices with ongoing
and process-oriented feedback during the design conversation. Such feedback is critical in helping novice teachers to learn how to
integrate technology into their lessons (Tondeur et al., 2012). Although some design teams did include external experts or course
instructors, they may or may not have facilitated the teachers’ collaborative discourse. Facilitators as discussion leaders should have a
repertoire of strategies, such as eliciting metacognitive and higher-order thinking from participants (Hmelo-Silver & Barrows, 2006,
2008) or prompting them to share experiences and reflections (González, Deal, & Skultety, 2016; van Es, Tunney, Goldsmith, & Seago,
2014). Facilitators play a critical role, as it is the process of teacher collaborative discourse that mediates the exchange of knowledge
rather than the mere presence of individuals with different forms of expertise. For example, facilitators should prioritize learning to
integrate technology to fulfill students’ standards-based or curriculum-structured learning needs by unpacking their pedagogical

Table 4
Strategies for promoting productive teacher collaborative discourse.
Strategies for developing collective TPACK

Study Providing inputs before Establishing a common goal in Engaging teachers in discussion and Facilitating reflection
discussion discussion analysis of artifacts

1. T: PictoPal Aligning on/off-computer Within-group (Implicitly embedded in revision tasks)

learning activities Pre-implementation design-revision
2. T: PictoPal Developing a series of PictoPal Within-group (None)
learning activities Pre-implementation design-revision
3. (None) Creating a TE CLIL unit in a Across-group Self-ratings of “how CLIL are you?” in
group Analyzing the CILI unit developed by questionnaire (before discussion)
Analyzing work from other another group
groups
4. (None) Creating a TE CLIL unit in a Across-group Offline written reflection on
group Analyzing and critiquing the CILI telecollaboration (after discussion)
Analyzing work from other unit developed based on a) lower/
groups higher-order thinking skills; and b)
cognitive skills progression
5. T: technologies exposure Designing a prototype for an Within-group Self-progress survey (periodic)
TP: effective web design online course Reaching consensus on design
decisions
6. TP: design of technology- Designing math lessons on Within-group Reflections at the end of the design
integrated, student-centered “averages” through TE inquiry Pre-implementation design-revision project
learning into authentic problems
P: meaningful learning (
Howland, Jonassen, &
Marra, 2013)
7. P: 21st-century competency Sharing ideas in designing and Within-group Presentations of a) findings from
frameworks implementing student-centered Design-revision-implementation students’ learning performances (based
P: meaningful learning ( ICT lessons in English, Math, and on pre-/post-tests); and b) their own
Howland et al., 2013) Science reflections (at the end of the school year)
Providing feedback for team
members on the difficulties
encountered in ICT lesson design
and implementation
8. (None) Discussing ICT lesson ideas, Within-group (None)
curricular matters, and logistical Pre-implementation design-revision
issues of the upcoming lessons
9. T: hands-on Moodle tutorial Designing a primary-level Within-group 600-word design justification
T: web-conferencing curriculum and related materials Reaching consensus on design
platform decisions
10. T: web-conferencing Creating PPT-based materials for Within-group Reflection on practice and redesign
platform distance learning Design-implementation-redesign by
P: visiting schools to learn Solving T and P problems in the pre-service teachers
students’ backgrounds; design process
designing the curriculum
11. TPACK: 3 theory-based Sharing topic-related materials Within-group (Implicitly embedded in revision tasks)
design principles of by practicing teachers Reaching consensus on design
technological pedagogical Designing flash-supported lesson decisions for TPMTs between pre-
mathematics tools (TPMTs) plans by pre-service teachers service and in-service teachers

Note: P = pedagogy, T = technology, TPACK = technological pedagogical content knowledge.

8
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

reasoning and decisions, rather than overemphasizing the technological affordances of tools (Harris & Hofer, 2011). However, the
reviewed studies did not always make clear the role of the facilitator, if present, in mediating the interactions and collaborative
discourses of the design teams.

(3) Strategies for promoting productive teacher collaborative discourse

Four strategies for promoting productive teacher collaborative discourse were identified (Table 4). The first was to provide the
teachers with the necessary knowledge before they engaged in collaborative discourse. These inputs were often the knowledge (sub)
sets that constitute TPACK. Workshops or training included topics on technology introduction and exposure (#1, #2, #5, #9, #10),
pedagogical foundations or student backgrounds (#7, #10), technological support for student learning (#5, #6), and using technology
to support students’ inquiry into subject content (#11). Content knowledge did not seem to be the focus of these workshops.
The second strategy was to establish a common goal in the collaborative discourse. The teachers were tasked with the collaborative
design of the TE curriculum. The design tasks were often guided by strategies fundamental to the TE curriculum, such as following
design cycles (#7, #11) or activity alignment (#1, #2). Some tasks even demanded that the TE curriculum be embedded with design
features such as supporting students’ mathematical inquiry (#6, #11) or distance learning (#5, #10), which challenged teachers to
become more deliberate and intentional in designing student-centered instruction. When given a common goal in designing TE tasks,
the teachers had opportunities to explicate their design reasoning within the collaborative discourse.
The third strategy involved engaging teachers in discussing and analyzing group design artifacts. All of the collaborative discourses
took place within the design teams, except for Studies 3 and 4, in which pre-service teachers in groups located in two universities
shared and analyzed the previously developed CLIL units. The researchers sometimes asked the teachers to offer constructive feedback
on the works created within their groups during the design process (#1, #2, #9, #11) or the final artifacts of other groups (#3, #4).
The feedback focus was sometimes peer support for difficulties in TE instruction (#7) or general comments on the TE curriculum (#1,
#2, #3, #5, #10). Interestingly, only in two studies were the teachers allowed to implement instruction using the materials designed
during the study (#7, #10). Subsequent collaborative discourse could help the teachers to optimize design outcomes and deal with the
difficulties they encountered during the enactment of TE instruction.
The fourth strategy was to engage teachers in sharing their reflections during their collaborative discourse, not only to develop the
professional knowledge of individual teachers, but also to promote the quality of collaborative discourse. All of the studies except #1,
#2, and #8 explicitly indicated that the teachers were prompted to reflect on the design process through the use of questionnaires or
written reflections. Nevertheless, this was mostly done after the design process was completed (#4, #6, #7, #9). Engaging teachers in
sharing their reflections during the design process can stimulate their collaborative discourse (#3). Ideally, the redesign should be
based on teachers’ reflections on practices (#10) so that they can share the issues and challenges they encountered during the
enactment of the TE materials or the design outcomes of the subsequent collaborative discourse, which could be oriented to support the
teachers’ in integrating technology into their instruction.
In summary, we identified four strategies TPACK researchers used to promote productive teacher collaborative discourse. It is
worth noting that these strategies were used either alone or in combination to promote teacher collaborative discourse, but it was
unclear whether specific strategies were required under certain conditions or whether more general approaches could improve the

Table 5
Research focus and TPACK model.
Study TPACK model What was investigated

TPACK/TPCK1 Factors

1 Self-defined model 7 (sub)sets1 of TPCK Design reasoningb

2 Self-defined model 7 (sub)sets1 of TPCK Design reasoningb, depth of inquiry in TPACK-related design talk
3 Techno-pedagogical-content skills ( T, P, C, TP (None)
Koehler & Mishra, 2009)
4 Mishra and Koehler (2006) TK, TPK, PCK, (None)
TPACK
5 Koehler et al. (2004); Mishra and 7 (sub)setsa of Group dynamics, social, and miscellaneous
Koehler (2006) TPACK
6 Technological pedagogical 7 (sub)sets1 of Design knowledge, context
mathematical knowledge (TPMK) TPMK
7 Cox and Graham (2009); Mishra and 7 (sub)sets1 of Design frames: idea development, design management, perception, enactment,
Koehler (2006) TPACK institutional, design scaffold, and interpersonal factors
8 Chai, Koh, and Tsai (2011) 7 (sub)sets1 of Contextual factors: physical/technological, cultural/institutional, intrapersonal,
TPACK interpersonal
9 Not committed to a single model Content, pedagogy, Learner characteristics, context, and teacher beliefs
technology
10 Mishra and Koehler (2006) 7 (sub)sets1 of Problems encountered in contexts and created solutions (implicitly analyzed through
TPACK the five contextual dimensions of Porras-Hernández & Salinas-Amescua, 2013)
11 Self-defined model 7 (sub)sets1 of TPCK The design stage in terms of design decisions, levels of technological tool integration

Notes.
a
Denotes the seven knowledge (sub)sets that underpin TPACK (TPCK): TK, PK, CK, TPK, TCK, PCK, and TPACK (TPCK).
b
Denotes practical concerns, existing orientations, and external priorities.

9
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

quality of discourse and enhance teachers’ TPACK. Two issues merit further discussion from the situative view of teacher learning.
First, none of the studies explicitly highlighted the importance of group norms, although these have been found to “promote supportive
yet challenging conversations about teaching” and can help to support teacher discussions or work sharing (Borko, 2004, p. 7). Second,
few studies (only #7 and #10) required teachers to enact teaching and then redesign the TE materials. Without implementing TE
instruction, the teachers could only envisage issues that might arise during actual enactment. More importantly, the teachers did not
have the chance to collect learning evidence from their students for feedback into their revision of the TE instruction/materials. Such a
feedback loop between students and teachers is critical in improving teachers’ TPACK, as it helps teachers learn from the design and
redesign cycles (Fishman, Marx, Best, & Tal, 2003).

(4) TPACK model(s) guiding the analysis of teacher collaborative discourse

Various theoretical TPCK/TPACK models were used to guide the study designs and data collection methods (Table 5). Mishra and
Koehler’s (2006) model was the most frequently used. Self-defined TPACK models proposed in the authors’ previous works were used
in four studies (#1, #2, #8, #11). Two studies combined multiple models to define TPACK (#6, #9). The researchers applied some/all
of the seven knowledge (sub)sets from the integrative conceptualization of TPACK (or TPCK) as the basis for characterizing the
composition of the collaborative discourse.
The quality of the collaborative discourse was also influenced by the teachers’ knowledge of instructional design. Their discourse
was coded by design-related factors such as design reasoning and contextual constraints. The teachers designed the teaching and
learning materials through frames, by stages, or in terms of problems encountered (#6, #7, #10, #11). The artifacts designed were
likely to be shaped by the contexts of the design teams and those that they were designed for (#1, #2, #5, #6, #8, #9).
Overall, the most common of analyzing the teachers’ collaborative discourse was to count and code the TPACK knowledge (sub)sets.
Some studies did not even focus on TPCK/TPACK. These studies implicitly operated from an integrative view of TPACK, assuming that
growth in any of the related constructs (i.e., content, technology, pedagogy) or TPACK subsets would automatically contribute to
growth in TPCK. However, this assumption has been severely questioned in the TPACK literature (Angeli & Valanides, 2009).
Questions then remain as to how the researchers could adequately analyze the development of the TPACK of the collective teacher
groups based on their collaborative discourse data. In the next theme, we zoom in on the details of the data analysis.

(5) Data collection and analysis of TPACK in teacher collaborative discourse

As Table 6 shows, investigations into teacher collaborative discourse ranged from using a single data source (e.g., video or audio
transcripts) to multiple sources such as questionnaires or design artifacts. Audio recordings were widely used, as they can help avoid
the potential distortion of interactions in teacher collaboration to which video-recording can lead. Multiple data sources revealed rich
information, and data triangulation was necessary for research quality (e.g., teachers’ pedagogical decisions/reasoning in # 9). Not all
of the studies clearly specified the data volume or collection timeframe (#3, #4). The collaborative discourse data were collected from
three to five design team meetings and spanned two-to three-month semesters (#1, #2, #7, #8, #10). The researchers did not
generally analyze the whole dataset but selected key stages related to the collaborative design (#5, #7, #10). They often offered
justifications of why and how data were collected and analyzed.
Most studies were small-scale qualitative studies, and all except #9 and #11 took the “coding and counting” approach to data
analysis (Chai et al., 2013). This involved 1) coding discourse units using the TPACK/TPCK knowledge (sub)sets and 2) frequency
counting. Boschman and colleagues reported that they coded the data deductively and then inductively (#1, #2), and a similar
approach, “hybrid thematic analysis,” was used in #9. Frequency counting enabled simple statistical comparisons of these categorical
coding results (e.g., frequency reports, percentages), indicating which TPACK knowledge (sub)sets dominated the discourse or
demonstrating possible knowledge transformation. Excerpts from the discourse were included to exemplify teachers’ knowledge ex­
changes. Some studies (#5, #7, #8) used mixed methods. Chi-square analysis was also used to compare the extent of the knowledge
(sub)sets represented in the discourse.
Units of analysis were used inconsistently across the reviewed studies. Boschman and colleagues used “utterances” and “episodes”
as the unit of analysis in #1 and 2, respectively. Different authors defined the same units rather differently. For example, “utterance”
was defined as a unit of speech in #1 and as an online chat message in #11. “Episode” was defined as a larger textual unit segmented by
topical turns in some studies (#2, #5, #6, #9, #10) but was not clearly defined in other studies (#3, #4). Koh and colleagues also
analyzed transcripts using sentences or paragraphs as units (#7, #8). The inconsistency in units may be due to the data volume in a
large range mentioned above. The differences in terminology and units across the studies make it difficult to compare and contrast
their findings.
Cohen’s K was the most commonly used method for calculating inter-rater reliability (#1, #2, #5, #6, #7), although in some
studies the reliability calculation was not explained (#5, #11), or the researchers only reported that the raters reached consensus (#4,
#8). In #9, data from different sources were coded and triangulated, and individual interviews were conducted to assess teachers’
design rationales. Otherwise, no validity tests, such as checking the interpretations with the participants, were apparent.
Two issues worth further elaboration. First, while it is legitimate for researchers to use different methods to analyze discourse data,
the unclear definition of unit of analysis and the idiosyncratic meaning of the same term (e.g., utterance) hamper meaningful com­
parison of findings across the studies. Second, the methodologies have been mostly limited to a coding and counting strategy. Although
this approach enabled the researchers to understand the composition of the TPACK embedded in the discourse, this method neglects
the temporal nature of verbal data (Csanadi et al., 2018) and the dynamics of knowledge construction. The forward and backward links

10
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

Table 6
Foci of data collection and data analysis for teacher collaborative discourse.
Study Data sources Number of Length of Unit of analysis No of Analysis Reliability
meetings and discussion units technique
duration of analyzed
data
collection

1 Transcripts (v) 3 workshop N/A Utterance 1,766 Content Inter-rater (K =

meetings over analysis 0.74)
3 months
2 Transcripts (v/a) 3 workshop About 150 min Episode 65 Content Inter-rater (K =
meetings over analysis 0.76)
3 months
3 Transcripts (a), written N/A N/A Episode 230 Content N/A
reflection, questionnaire analysis
4 Transcripts (v/a), written N/A N/A Episode 326 Content Inter-rating to reach
reflection analysis consensus
5 Field notes, emails, artifacts 15 weeks, 3 of N/A Episode Group 1 Content Inter-raters (90%)
and related notes, survey which were (26, 48, analysis
analyzed 48)
(beginning, Group 2
middle, and (43, 30,
end of 39)
semester)
6 Transcripts (a) 5 design N/A Episode ( 252 Content Inter-rater (K =
sessions over 2 Boschman et al., analysis 0.85)
months 2015)
7 Transcripts (a) 7 x 5 teams = 35 h Textual unit 12,598 Content Inter-rater for
35 sessions (7 based on analysis, chi- TPACK, DK and
design sessions sentences square design frame (K >
for each team) analysis 0.80)
across 2
semesters
8 Transcripts (a) 3 months 13 audio Paragraph 3,592 Content Inter-rating to reach
recordings (9 h analysis, chi- consensus
and 50 min) square
analysis
9 Transcripts (a), online 5 weeks 30 min face-to- Turns in the N/A Hybrid Investigator
resources during design, face discussion (5 topic of thematic triangulation
artifact screenshots, weeks) conversation analysis
researchers’ notes, individual
interviews
10 Transcripts (a) (post-teaching Any 3 30 min x 9 = 270 Episode 59 Content Re-coding one-
discussion, interview) discussion min analysis month after re-
sessions that coding
were
comparatively
richer and
lengthier were
chosen from
the early and
later stages
over 14 weeks
11 Chat messages on the Tencent 10 weeks (3 N/A Utterance (Stahl, N/A Content Inter-rater (86%)
QQ platform weeks between 2011) analysis
pre-service
and 7 weeks
between pre-
service and
practicing
teachers)

Notes:(a) = audio; (v) = video N/A = Information not available in the article.

between the design ideas were rarely analyzed in the studies reviewed. This is particularly problematic when the development of TPACK
of a group of teachers is the focus of investigation. Further analysis would have been useful (e.g., analyzing the collective consensus of
the group, the quality of the design outcomes (artifacts), the individual TPACK levels expressed, and the quality of the discourse using
theoretically derived rubrics). Visualizations of the interactions between team members can be a useful strategy and provide further
understanding of the progression of the discourse focus or knowledge construction process (#5).

(6) Discourse content, collaboration process, and implications for teachers’ professional/TPACK development

11
Y.-F. Yeh et al.
Table 7
Discourse content, collaboration process, and implications for teachers’ professional/TPACK development.
Study Discourse content
1 • Practical concerns (i.e., design-related
and implementation-related) were pre­
dominant, mostly related to TCK and
TPCK.
• Integrated TPCK emerged most often
when teachers envisioned actual
implementation.
2 • Most discussion time was spent on PCK
and TPCK; discussion of separate
subsets of knowledge (TK, PK, CK) was
rare.
• P was always explicated with other
knowledge subsets (e.g., TPK, PCK,
TPCK).
• Attention to TPCK gradually increased
across discussions.
• Practical concerns were dominant.
• External priorities hardly played a role;
teachers’ existing orientation (e.g.,
beliefs) featured more prominently in
discussion of deeper inquiry.
3 • T was articulated more when discussing
telecollaboration.
• PK and CK were mentioned more when
discussing the CLIL unit.
4 • PCK was the focus.
• TPK and TPACK were less discussed,
and the integration of TK into TPK and
TPACK needed time to evolve.
• Discourse developed first from the
individual knowledge subsets of TPACK
and later their intersections.
5 • Discourse moved from considering T, P,
and C as independent constructs
toward a more interdependent
construction.
6 • PCK, TPMK, and DK accounted for most
of the discourse episodes.
• PCK was adapted and deepened, but
TPMK was not.
7 • PCK was discussed more.
• TPK and TCK were less discussed.
• Seven design frames were identified, of
which only design management was
largely used by teachers.
8 • Cultural/institutional factors
dominated group discussions.
• The more talk focused on cultural/
institutional factors, the less TPACK
was discussed.
• Intrapersonal factors (e.g., teacher
beliefs) played a part in triggering
discussions related to TPACK.
9 • Pedagogy was rarely mentioned.
10
Collaboration process
• The content expert contributed to the
discussion by providing content-based ex­
planations and strategy suggestions.
• The researcher–facilitator provided
procedural support (e.g., dividing the
design work into three workshops with
open-ended tasks).
• Teachers’ collaboration was easily
hindered by technical constraints (e.g.,
problems with sound).
• Telecollaboration promoted collaboration,
as it made participation more equal and
showed potential for developing pre-
service teachers’ TPACK.
• The participants’ roles in the discussion
and the nature of their meaning-making in
the groups evolved over time.
• N/A
• N/A
• An experienced facilitator ensured that
TPACK was addressed during the
discussion.
• Tutor support, technological capabilities,
and group collaboration influenced the
technology-enhanced learning design
process.
• N/A
12
Computers & Education 171 (2021) 104238
Implications for teachers’ professional/TPACK
development
• Explicit attention should be given to
envisioning actual classroom
implementation, including practical
concerns related to both design and
implementation.
• Teachers require both procedural and
substantive (i.e., content) support to
achieve a deep level of inquiry in the
discussion. Content experts should
provide content input before and
during the design.
• Pre-service teachers’ discussion was
not deep, and did not result in changes
in the CLIL units. Tasks that extend
collaboration and reflection should be
included.
• A final task that engages pre-service
teachers in discussing more critical as­
pects (e.g., underlying learning the­
ories, the best type of ICT integration)
should be included.
• LBD is an approach that honors the rich
connections between C, T, and P.
• The five pathways to create teachers’
TPM provided implications on how to
promote its development.
• Multiple design frames should be used
as scaffolds to support teachers’ design
reasoning in student-centered ICT
lessons.
• Teachers need to develop
competencies to facilitate discourse
about design so that contextual
concerns can be turned into
opportunities to support pedagogical
improvement.
• Pre-service teachers should be
encouraged to articulate and reflect on
their beliefs about learning and
teaching and their design decisions
throughout the design process.
(continued on next page)
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

Table 7 (continued )
Study Discourse content Collaboration process Implications for teachers’ professional/TPACK
development

• PCK predominated, while TPK was • Contextual factors (e.g., technology

absent.
11 • Pre-service teachers articulated limited
PCK, and the deeper PCK of practicing
teachers provided material to further
develop pre-service teachers’ TK and
TCK.
Notes: P = pedagogy, C = content, T = technology, TP = technological pedagogy, TC = technological content, PC = pedagogical content, TPACK =
technological pedagogical content knowledge.
• Pre-service and in-service teachers in
different geographical areas were able to
work collaboratively using mobile devices
and social network services.
• Practicing teachers’ PCK played a crucial
role in refining the initial TPACK of pre-
service teachers.
quality, students’ learning
characteristics) should be tackled
through teachers’ collaborative
designs.
• Teachers should be encouraged to use
the well-tested design guidelines and
collaborative design processes to create
technological pedagogical mathe­
matics tools and students’ inquiry-
based learning.

Table 7 synthesizes the findings of the reviewed studies related to the three themes related to RQ3: discourse content, collaboration
processes, and implications for teachers’ development of TPACK. First, in terms of the content of the teachers’ collaborative discourse,
PCK was the predominant knowledge subset (#2, #4, #6, #7, #10), whereas TPK was rarely considered (#4, #7, #10). However, the
knowledge (sub)set represented in the discourse seems to be dependent on the topic discussed. For example, Bueno-Alastuey and
García Esteban (2016) found that when pre-service teachers discussed technology tools, T was mentioned more frequently, whereas PK
and CK were more commonly mentioned in discussions of the CLIL curriculum unit. Several studies reported that the teachers’ TPACK
represented in the group discourse became more complex as the knowledge subsets became integrated during the progress of
collaborative design (#2, #4, #5). Practical concerns (#1, #2, #7, #8) were often expressed during the discourse. Cultural/institu­
tional factors can dominate group discussions and hinder the teachers’ opportunities to share their TPACK (#8). Apart from external
factors, multiple studies (e.g., #2, #8) also pointed to the importance of intrapersonal factors, such as beliefs, in influencing teachers’
discourse content.
For the collaboration process, some studies found evidence that technological tools such as web conferencing (#3, #4) or mobile
devices (#11) provide affordances that promote teacher collaboration. These include promoting equal participation among the par­
ticipants (#4), connecting participants in different geographical locations (#3, #4, #11), and enabling sharing between teachers with
different levels of expertise (e.g., pre- and in-service teachers) (#11). However, technological constraints were also shown to hinder
collaboration (#3). Another important subtheme pertained to the importance of the presence of “more knowledgeable others” in the
collaborative process (#1, #9, #11). For example, #1 specifically mapped out the role of content experts in providing content-based
explanations and suggested strategies to promote collaborative discourse, and #11 stressed that practicing teachers sharing their PCK
was critical to the initial TPACK development of pre-service teachers.
The studies have a variety of implications for teachers’ professional development and TPACK development. Several of these studies
have implications for the content to be addressed during teachers’ collaborative discourse (#1, #9, #10). For example, teachers should
be directed to envision actual implementation (#1) and articulate their pedagogical reasoning and beliefs (#9) to promote their
TPACK development. The studies also suggested that to deepen the collaborative discourse, teachers may require subject-matter
support (#2), design guidelines (#11), and scaffolding (#7). Some studies (#2, #8) explicitly highlighted the role of facilitators in
orchestrating the discourse, to ensure that the discussion is focused on specific aspects conducive to collective TPACK construction.
Although these studies generally provided initial evidence of the efficacy of the LBD approach for promoting teachers’ TPACK
development (i.e., TPACK (sub)sets are evident in the teachers’ collaborative discourse), some also found that the discussion may lack
depth (#3, #4) or address TPK and TCK less often than expected (#7). Similarly, #10 reported that TPK was absent in the discussion.
As TPK and TPCK can be considered as the more critical knowledge subsets within the TPACK framework (Graham, 2011), these
findings raise questions about the actual effectiveness of the LBD approach in developing teachers’ TPACK. Studies #3 and #4 also
reported that the pre-service teachers’ CLIL units underwent no change as a result of the discussion, and suggested the inclusion in the
CLIL units of a final task that encourages pre-service teachers to reflect collaboratively. We found that only one study assessed the
quality of the design outcomes. Study 11 used a study-derived rubric comprising detailed criteria (i.e., the Technology Tools Inte­
gration Assessment Rubric; Harris, Grandgenett, & Hofer, 2010) to evaluate the design artifacts (i.e., technological pedagogical
mathematics tools) produced by the design teams. The evaluation was conducted by 30 practicing teachers. Surprisingly, none of the
studies evaluated the quality of the TE instruction enacted by the members of the design team.
In summary, the reviewed studies revealed the content of the teachers’ TPACK expressed in the teachers’ collaborative discourse
and the nature of the collaborative processes in that discourse. As TPACK was embedded in the teachers’ collaborative discourse, these
studies provide some evidence that teachers’ individual TPACK could benefit from meaningfully structured design processes or
collaboration. However, most of the studies reviewed did not provide evidence of how TPACK expressed in the collaborative discourse
influenced the TPACK enacted by individual teachers or the quality of the design outcomes. Moreover, most of the reviewed studies
provided no evidence of how teachers’ collective TPACK may influence the enactment of TPACK by individual teachers, or how the
teachers’ enacted TPACK in subsequent rounds of instruction may inform the TPACK constructed by collaborative discourse.

13
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

4. Discussion and conclusion

In line with the situative view of learning, which emphasizes the social and situated nature of learning (Borko, 2004; Greeno, 2003;
Putnam & Borko, 2000), LBD is widely recognized as a valuable approach to developing pre-/in-service/university teachers’ TPACK (e.
g., Agyei & Voogt, 2012; Koehler et al., 2007; Papanikolaou et al., 2017). Yet, despite the critical importance of teachers’ collaborative
discourse in mediating TPACK development (Chai et al., 2016), a surprisingly small number of LBD studies have included analysis of
teachers’ collaborative discourse, signaling that this critical aspect has been largely overlooked in empirical TPACK studies published
in peer-reviewed journals.
Several salient findings of our analysis are worth recapitulating. First, although researchers generally provided information about
how the teachers’ collaborative discourse was organized in terms of design team composition, means of collaborative discourse, and
design outcomes, they seldom described the critical role of facilitators in orchestrating the collaborative discourse or setting up and
maintaining group norms. Our review identified four strategies for promoting teachers’ collaborative discourse: (1) equipping teachers
with the necessary knowledge before engaging them in collaborative discourse; (2) establishing a common goal for the teachers’
collaborative discourse; (3) engaging teachers in discussing and analyzing group design artifacts, and (4) providing chances for
teachers to share their reflections during their collaborative discourse. Second, our analysis revealed that the methods used to analyze
the collaborative discourse were somewhat homogeneous, with most using the “coding and counting” approach to reveal the distri­
bution of TPACK (sub)sets expressed in the collaborative discourse. The studies seldom focused on the dynamics of the knowledge
construction process or the progression of the discourse. Finally, although the findings generally provided evidence for the efficacy of
the LBD approach, the evidence was rarely based on changes in teachers’ TPACK enacted in classroom instruction or the quality of the
design artifacts produced from the collaborative design processes. Based on these points, we considered how best to pursue further
investigation of TPACK and TPACK development in the context of collaborative discourse during the LBD process.
A group of science education researchers recently proposed the refined consensus model (RCM) for PCK (Carlson et al., 2019). This
model articulates three distinct realms of PCK: personal (at the individual level), collective (within a group of individuals), and enacted
(i.e., teachers’ personal PCK may be enacted during planning, teaching, and reflection). We believe that this notion can also be applied
to the field of TPACK, in which knowledge exchange may occur between teachers’ personal and collective TPACK during the LBD
process. The TPACK developed from the individual enactment of TE instruction may loop back to personal and collective TPACK, and
vice versa. To exemplify these knowledge exchange processes, consider Study #10, in which the design team comprised six pre-service
English teachers, one practicing teacher, and one university professor. The process started with the pre-service teachers designing
online teaching materials and activities and discussing the initial designs with the team. The pre-service teachers then implemented the
designed materials, discussed the problems they encountered during the enactment and reflected on their practices to redesign the
teaching materials and adjust their instruction. Extensive collaborative discourse associated with the design-teach-reflect-redesign
cycle enabled the pre-service and in-service teachers and university professors to exchange their individual TPACK and build new
collective TPACK. This process also promoted extensive knowledge exchange between different realms of TPACK, in which collective
TPACK was potentially offloaded to design artifacts or enacted by the pre-service teachers. The three TPACK realms interacted as the
design team shared their expertise and attempted to introduce technology into their implementation in authentic contexts. However,
this study only focused on the collective TPACK; it did not analyze either the changes in the individual TPACK of the pre-service
teachers during the LBD process or the quality of the design outcomes/enacted lessons resulting from the collaborative discourse.
Based on the RCM for PCK created by two dozen international scholars in the PCK field (Carlson et al., 2019) and our review, we
propose a conceptual framework, Collaboration-enriched TPACK framework (Fig. 2), which highlights the TPACK knowledge exchange
process during LBD with the goal of refocusing the attention of researchers on teachers’ collective and individual TPACK.

Fig. 2. Collaboration-enriched TPACK framework.

14
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

First, this framework highlights the need to examine teachers’ collaborative discourse. The nature and quality of the discourse, and
hence the quality of the collective TPACK (purple box), can potentially influence teachers’ personal TPACK (green box) and the quality
of the design artifacts (orange box) produced from the collaboration during the iterative design and redesign cycles of LBD (grey
arrows). Second, it highlights the need to trace the two-way knowledge exchange processes (pink arrows) between teachers’ collective
TPACK (purple box), personal TPACK (green box) and enacted TPACK (blue box). These two-way knowledge exchanges between the
three realms of TPACK are mediated through pedagogical reasoning (Carlson et al., 2019; Shulman, 1987), during which teachers
reason about their own teaching/design decisions and actions. Third, the framework highlights the purposeful structuring and
facilitation of teachers’ collaborative discourse so that it can expand organically and meaningfully to facilitate the development of
teachers’ TPACK. This requires strategic planning of the design team composition and design tasks, and organization of the discourse
(e.g., the presence of a skilled facilitator).
Self-reporting surveys have been the most frequent approach to determining individual teachers’ TPACK levels (Koehler, Shin, &
Mishra, 2012; Willermark, 2018). However, recent TPACK studies have revealed that the TPACK represented in self-reported surveys
does not always influence teachers’ instructional quality (Petko, 2020), as evidenced by teachers’ lesson plans (Backfisch, Lachner,
Hische, Loose, & Scheiter, 2020) and actual classroom practice (Ocak & Baran, 2019; Yeh, Hsu, Wu, & Chien, 2017). Empirical
research has also shown that teachers’ self-reported TPACK is only weakly related to their objective-TPACK (e.g., measured in terms of
teachers’ ability to discriminate between true and false statements about TPACK) (Drummond & Sweeney, 2017). Although existing
studies of collective TPACK with a component of teacher collaborative discourse have moved beyond self-report and provided initial
evidence for the efficacy of the LBD approach in enhancing teachers’ collective TPACK, it is also critical to gather evidence of indi­
vidual teachers’ personal/enacted TPACK development as a result of collaborative discourse during the LBD process. We argue that
TPACK researchers should focus on the three realms of TPACK in a holistic manner with the aim of connecting both the processes (in the
form of personal and collective TPACK) and the outcomes (in the form of enacted TPACK and/or design artifacts) to provide stronger
evidence for the efficacy of the LBD approach. This is important because a shift in teachers’ knowledge at the group level or changes in
teachers’ individual knowledge may not necessarily translate into decision-making and actions in their own classrooms (Chan & Hume,
2019). Future studies should empirically verify these outcomes and also identify effective strategies to augment the effectiveness of this
approach in developing teachers’ TPACK.
One possible method to analyze teachers’ collaborative discourse beyond the coding-and-counting approach is epistemic network
analysis (ENA) (Shaffer, 2017; Shaffer et al., 2009). ENA is theoretically grounded in epistemic frame theory, which posits that
learning cannot be reduced to isolated components but to the transformation of an individual’s epistemic network, which comprises a
set of relationships connecting the skills, knowledge, and values a learner uses to make sense of and take action in the world. This
network is expressed through discourse and changes over time during the learning process (Shaffer, 2012). ENA allows researchers to
identify and quantify connections among elements in coded data and to represent them in dynamic networks that model learning in
discourse data (Shaffer et al., 2009). Specifically, the use of ENA allows TPACK researchers to model temporal co-occurrences between
teachers’ TPACK (sub)sets and actions during discourse, visualize their co-occurrences, and statistically compare teacher discourse
during different meetings and between different teacher groups with respect to these models. This provides greater insight into how
individual (sub)sets of TPACK/actions occur in relation to one another throughout the whole discourse. It reveals the temporal
development of teachers’ TPACK and the interdependence between different TPACK (sub)sets/actions, and visualizes the dynamics of
learning. Recent TPACK studies have applied this technique to unpack teachers’ TPACK reflected in online collaborative discourse
(Zhang, Liu, Cai, 2019) and lesson plans (Phillips, Kovanović, Mitchell, & Gašević, 2019). We believe this analytical technique can be
similarly applied to understand teachers’ collective TPACK embedded in their collaborative discourse and its development.
In closing, we acknowledge the limitations of our review. First, it was restricted to English-language articles published in high-
quality peer-reviewed journals available from two major databases. Second, only a few articles met our requirement for TPACK
studies that analyzed teachers’ collaborative discourse, and hence our findings were drawn from a limited number of empirical studies.
Third, of the eleven articles we reviewed, eight were written by a select group of four authors and their collaborators, and five were
published in Computers & Education. This suggests that the reviewed studies may have been influenced by the ideas of only a few
researchers and their particular orientation toward collective TPCK/TPACK. While this may be considered a limitation, we hope to
shed light on the lack of attention to teachers’ collaborative discourse and collective TPACK in the context of LBD in the major peer-
reviewed journals, which we see as a potentially promising direction in the TPACK field. From this perspective, our review provides a
starting point for the many TPACK researchers who are interested in using the LBD approach in developing teachers’ TPACK. The
conceptual framework we propose offers an analytical tool for future studies aimed at investigating teachers’ personal and collective
TPACK through LBD, while refocusing researchers’ attention on investigating and connecting teachers’ individual and collective
TPACK and their development during the LBD process. Thus, our review represents a valuable contribution to the TPACK field.

Acknowledgment

This research is supported by the “Institute for Research Excellence in Learning Sciences” and “Higher Education Sprout Project” of
National Taiwan Normal University (NTNU), sponsored by the Ministry of Education, Taiwan, R.O.C.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.compedu.2021.104238.

15
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

Compliance with ethical standards

The manuscript “Toward a framework that connects individual TPACK and collective TPACK: A systematic review of TPACK studies
investigating teacher collaborative discourse in the learning by design process” has not been submitted to other journals for consid­
erations of reviews or publications. To the best of our knowledge, the named authors have no conflict of interest, financial or otherwise.
This is a review study, so all the reviewed studies are published and can be accessed online.

Credit author statement

Yi-Fen Yeh: Conceptualization, Methodology, Literature Search, Formal analysis, Writing, Editing, and Funding acquisition,
Kennedy Kam Ho Chan: Conceptualization, Methodology, Literature Search, Formal analysis, Writing, and Editing, Ying-Shao Hsu:
Conceptualization, Methodology, Literature Search, Formal analysis, Writing, and Editing.

References*

Abbitt, J. T. (2011). Measuring technological pedagogical content knowledge in preservice teacher education: A review of current methods and instruments. Journal of
Research on Technology in Education, 43(4), 281–300. https://doi.org/10.1080/15391523.2011.10782573
Agyei, D. D., & Voogt, J. (2012). Developing technological pedagogical content knowledge in pre-service mathematics teachers through collaborative design.
Australasian Journal of Educational Technology, 28(4), 547–564. https://doi.org/10.14742/ajet.827
Angeli, C., & Valanides, N. (2005). Preservice elementary teachers as information and communication technology designers: An instructional systems design model
based on an expanded view of pedagogical content knowledge. Journal of Computer Assisted Learning, 21(4), 292–302. https://doi.org/10.1111/j.1365-
2729.2005.00135.x
Angeli, C., & Valanides, N. (2009). Epistemological and methodological issues for the conceptualization, development, and assessment of ICT-TPCK: Advances in
technological pedagogical content knowledge (TPCK). Computers in Education, 52(1), 154–168. https://doi.org/10.1016/j.compedu.2008.07.006
Backfisch, I., Lachner, A., Hische, C., Loose, F., & Scheiter, K. (2020). Professional knowledge or motivation? Investigating the role of teachers’ expertise on the quality
of technology-enhanced lesson plans. Learning and Instruction, 66, 101300. https://doi.org/10.1016/j.learninstruc.2019.101300
Baran, E., & Uygun, E. (2016). Putting technological, pedagogical, and content knowledge (TPACK) in action: An integrated TPACK-design-(DBL) approach based
learning. Australasian Journal of Educational Technology, 32(2), 47–63. https://doi.org/10.14742/ajet.2551
Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33(8), 3–15. https://doi.org/10.3102/
0013189x033008003
*Boschman, F., McKenney, S., Pieters, J., & Voogt, J. (2016). Exploring the role of content knowledge in teacher design conversations. Journal of Computer Assisted
Learning, 32(2), 157–169. https://doi.org/10.1111/jcal.12124
*Boschman, F., McKenney, S., & Voogt, J. (2015). Exploring teachers’ use of TPACK in design talk: The collaborative design of technology-rich early literacy activities.
Computers in Education, 82, 250–262. https://doi.org/10.1016/j.compedu.2014.11.010
*Bueno-Alastuey, M. C., & Esteban, S. G. (2016). Telecollaboration to improve CLIL and TPACK knowledge: Aid or hindrance? Estudios Sobre Educacion, 31, 117–138.
https://doi.org/10.15581/004.31.117-138
*Bueno-Alastuey, M. C., Villarreal, I., & García Esteban, S. (2018). Can telecollaboration contribute to the TPACK development of pre-service teachers? Technology.
Pedagogy and Education, 27(3), 367–380.
Carlson, J., Daehler, K., Alonzo, A. C., Barendsen, E., Borowski, A., Berry, A., … Wilson, C. D. (2019). The Refined Consensus Model of pedagogical content knowledge
in science education. In A. Hume, R. Cooper, & A. Borowski (Eds.), Repositioning PCK in teachers professional knowledge (pp. 77–92). Singapore: Springer. https://
doi.org/10.1007/978-981-13-5898-2_15.
Carpenter, J. P., Rosenberg, J. M., Dousay, T. A., Romero-Hall, E., Trust, T., Kessler, A., … Krutka, D. G. (2020). What should teacher educators know about
technology? Perspectives and self-assessments. Teaching and Teacher Education, 95, 13. https://doi.org/10.1016/j.tate.2020.103124
Chai, C. S., Koh, J. H. L., & Tsai, C. C. (2011). Exploring the factor structure of the constructs of technological, pedagogical, content knowledge (TPACK). Asia-Pacific
Education Researcher, 20(3), 595–603.
Chai, C. S., Koh, J. H. L., & Tsai, C. C. (2013). A review of technological pedagogical content knowledge. Journal of Educational Technology & Society, 16(2), 31–51.
Chai, C. S., Koh, J. H. L., & Tsai, C. C. (2016). A review of the quantitative measures of technological pedagogical content knowledge (TPACK). In M. C. Herring,
M. J. Koehler, & P. Mishra (Eds.), Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 87–106). New York: Routledge.
Chan, K. K. H., & Hume, A. (2019). Towards a consenus model: Literature review of how science teachers’ pedagogical content knowledge is investigated. In A. Hume,
R. Cooper, & A. Borowski (Eds.), Repositioning PCK in teachers’ professional knowledge (pp. 3–76). Singapore: Springer.
Cox, S., & Graham, C. R. (2009). Diagramming TPACK in practice: Using an elaborated model of the tpack framework to analyze and depict teacher knowledge.
TechTrends, 53(5), 60–69. https://doi.org/10.1007/s11528-009-0327-1
Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze
verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419–438. https://doi.org/10.1007/s11412-018-9292-z
De Rossi, M., & Trevisan, O. (2018). Technological pedagogical content knowledge in the literature: How TPCK is defined and implemented in initial teacher
education. Italian Journal of Educational Technology, 26(1), 7–23. https://doi.org/10.17471/2499-4324/988
Denzin, N. K. (2017). The research act: A theoretical introduction to sociological methods. Piscataway, NJ: Transaction publishers.
Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38
(3), 181–199. https://doi.org/10.3102/0013189X08331140
Di Blas, N., & Paolini, P. (2016). Distributed and dynamic TPACK: Evidences from a (large) case study. In Paper presented at the society for information technology &
teacher education international conference 2016, savannah, GA, United States. https://www.learntechlib.org/p/172093.
Di Blas, N., Paolini, P., Sawaya, S., & Mishra, P. (2014). Distributed TPACK: Going beyond knowledge in the head. In Paper presented at the society for information
technology & teacher education international conference 2014, jacksonville, Florida, United States.
Doering, A., & Veletsianos, G. (2007). An investigation of the use of real-time, authentic geospatial data in the K-12 classroom. Journal of Geography, 106(6), 217–225.
https://doi.org/10.1080/00221340701845219
Drummond, A., & Sweeney, T. (2017). Can an objective measure of technological pedagogical content knowledge (TPACK) supplement existing TPACK measures?
British Journal of Educational Technology, 48(4), 928–939.

*
The 11 studies that were reviewed are marked with an asterisk.

16
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

van Es, E. A., Tunney, J., Goldsmith, L. T., & Seago, N. (2014). A famework for the facilitation of teachers’ analysis of video. Journal of Teacher Education, 65(4),
340–356. https://doi.org/10.1177/0022487114534266
Fishman, B. J., Marx, R. W., Best, S., & Tal, R. T. (2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and
Teacher Education, 19(6), 643–658. https://doi.org/10.1016/S0742-051X(03)00059-3
González, G., Deal, J. T., & Skultety, L. (2016). Facilitating teacher learning when using different representations of practice. Journal of Teacher Education, 67(5),
447–466. https://doi.org/10.1177/0022487116669573
Graham, C. R. (2011). Theoretical considerations for understanding technological pedagogical content knowledge (TPACK). Computers & Education, 57(3),
1953–1960. https://doi.org/10.1016/j.compedu.2011.04.010
Greeno, J. G. (2003). Situative research relevant to standards for school mathematics. In J. Kilpatrick, W. G. Martin, & D. Schifter (Eds.), A research companion to
principles and standards for school mathematics (pp. 304–332). Reston, VA: National Council of Teachers of Mathematics.
Harris, J. B. (2016). Inservice teachers’ TPACK development: Trends, models, and trajectories. In M. C. Herring, M. J. Koehler, & P. Mishra (Eds.), Handbook of
technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 191–205). New York: Routledge.
Harris, J. B., Grandgenett, N., & Hofer, M. (2010). Testing a TPACK-based technology integration assessment rubric. In Paper presented at the society for information
technology & teacher education international conference 2010, san diego, CA, USA. https://www.learntechlib.org/p/33978.
Harris, J. B., & Hofer, M. J. (2011). Technological pedagogical content knowledge (TPACK) in action: A descriptive study of secondary teachers’ curriculum-based,
technology-related instructional planning. Journal of Research on Technology in Education, 43(3), 211–229. https://doi.org/10.1080/15391523.2011.10782570
Herring, M. C., Koehler, M. J., & Mishra, P. (2016). Introduction to the second edition of the TPACK handbook. In M. C. Herring, M. J. Koehler, & P. Mishra (Eds.),
Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 1–8). New York: Routledge.
Hmelo-Silver, C. E., & Barrows, H. S. (2006). Goals and strategies of a problem-based learning facilitator. Interdisciplinary Journal of Problem-Based Learning, 1(1).
https://doi.org/10.7771/1541-5015.1004
Hmelo-Silver, C. E., & Barrows, H. S. (2008). Facilitating collaborative knowledge building. Cognition and Instruction, 26(1), 48–94. https://doi.org/10.1080/
07370000701798495
Hong, H. Y., Lin, P. Y., Chai, C. S., Hung, G. T., & Zhang, Y. (2019). Fostering design-oriented collective reflection among preservice teachers through principle-based
knowledge building activities. Computers in Education, 130, 105–120. https://doi.org/10.1016/j.compedu.2018.12.001
Howland, J. L., Jonassen, D., & Marra, R. M. (2013). Meaningful learning with technology (4th ed.). Boston: Pearson Higher Education.
Hughes, J. (2005). The role of teacher knowledge and learning experiences in forming technology-integrated pedagogy. Journal of Information Technology for Teacher
Education, 13(2), 277–302.
Jimoyiannis, A. (2010). Designing and implementing an integrated technological pedagogical science knowledge framework for science teachers professional
development. Computers in Education, 55(3), 1259–1269. https://doi.org/10.1016/j.compedu.2010.05.022
Jones, D., Heffernan, A., & Albion, P. (2015). TPACK as shared practice: Toward a research agenda. In Paper presented at the society for information technology & teacher
education international conference 2015, las vegas, NV, United States. https://www.learntechlib.org/p/150454.
Koehler, M. J., & Mishra, P. (2005). What happens when teachers design educational technology? The development of technological pedagogical content knowledge.
Journal of Educational Computing Research, 32(2), 131–152. https://doi.org/10.2190/0EW7-01WB-BKHL-QDYV
Koehler, M. J., & Mishra, P. (2008). Introducing TPCK. In , The handbook of technological pedagogical content knowledge (TPCK) for educatorsAACTE committee on
innovation and technology (pp. 3–29). Mahwah, NJ: Lawrence Erlbaum Associates.
Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1).
Koehler, M. J., Mishra, P., Hershey, K., & Peruski, L. (2004). With a little help from your students: A new model for faculty development and online course design.
Journal of Technology and Teacher Education, 12(1), 25–55.
*Koehler, M. J., Mishra, P., & Yahya, K. (2007). Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy and technology.
Computers in Education, 49(3), 740–762. https://doi.org/10.1016/j.compedu.2005.11.012
Koehler, M. J., Shin, T. S., & Mishra, P. (2012). How do we measure TPACK? Let me count the ways. In R. N. Ronau, C. R. Rakes, & M. L. Niess (Eds.), Educational
technology, teacher knowledge, and classroom impact: A research handbook on frameworks and approaches (pp. 16–31). Hershey, PA, USA: IGI Global.
*Koh, J. H. L. (2019). Articulating teachers’ creation of Technological Pedagogical Mathematical Knowledge (TPMK) for supporting mathematical inquiry with
authentic problems. International Journal of Science and Mathematics Education, 17(6), 1195–1212. https://doi.org/10.1007/s10763-018-9914-y
*Koh, J. H. L., & Chai, C. S. (2016). Seven design frames that teachers use when considering technological pedagogical content knowledge (TPACK). Computers in
Education, 102, 244–257. https://doi.org/10.1016/j.compedu.2016.09.003
*Koh, J. H. L., Chai, C. S., & Tay, L. Y. (2014). TPACK-in-Action: Unpacking the contextual influences of teachers’ construction of technological pedagogical content
knowledge (TPACK). Computers in Education, 78, 20–29. https://doi.org/10.1016/j.compedu.2014.04.022
Kopcha, T. J. (2012). Teachers’ perceptions of the barriers to technology integration and practices with technology under situated professional development.
Computers in Education, 59(4), 1109–1121. https://doi.org/10.1016/j.compedu.2012.05.014
Laurillard, D. (2012). Teaching as a design science: Building pedagogical patterns for learning and technology (1st ed.). New York: Routledge.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, UK: Cambridge University Press.
Lee, C.-J., & Kim, C. M. (2014). An implementation study of a TPACK-based instructional design model in a technology integration course. Educational Technology
Research & Development, 62(4), 437–460. https://doi.org/10.1007/s11423-014-9335-8
Lee, M. H., & Tsai, C. C. (2010). Exploring teachers’ perceived self efficacy and technological pedagogical content knowledge with respect to educational use of the
World wide Web. Instructional Science, 38(1), 1–21. https://doi.org/10.1007/s11251-008-9075-4
Lefstein, A., Louie, N., Segal, A., & Becher, A. (2020). Taking stock of research on teacher collaborative discourse: Theory and method in a nascent field. Teaching and
Teacher Education, 88. https://doi.org/10.1016/j.tate.2019.102954
Macdonald, R. J. (2008). Professional development for information communication technology integration: Identifying and supporting a community of practice
through design-based research. Journal of Research on Technology in Education, 40(4), 429–445. https://doi.org/10.1080/15391523.2008.10782515
McDuffie, A. R., & Mather, M. (2009). Middle school mathematics teachers’ use of curricular reasoning in a collaborative professional development project. In
J. L. Remillard, B. A. Herbel-Eisenmann, & G. M. Lloyd (Eds.), Mathematics teachers at work: Connecting curriculum materials and classroom instruction (pp.
302–320). New York: Routledge.
McKenney, S., & Voogt, J. (2009). Designing technology for emergent literacy: The PictoPal initiative. Computers in Education, 52(4), 719–729. https://doi.org/
10.1016/j.compedu.2008.11.013
Mishra, P., & Koehler, M. J. (2003). Not “what” but “how”: Becoming design-wise about educational technology. In Y. Zhao (Ed.), What teachers should know about
technology: Perspectives and practices (pp. 99–122). Greenwich, CT: Information Age Publishing.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054.
https://doi.org/10.1111/j.1467-9620.2006.00684.x
*Nguyen, G. N. H., & Bower, M. (2018). Novice teacher technology-enhanced learning design practices: The case of the silent pedagogy. British Journal of Educational
Technology, 49(6), 1027–1043.
Niess, M. L. (2005). Preparing teachers to teach science and mathematics with technology: Developing a technology pedagogical content knowledge. Teaching and
Teacher Education, 21(5), 509–523. https://doi.org/10.1016/j.tate.2005.03.006
Njiku, J., Mutarutinya, V., & Maniraho, J. F. (2020). Developing technological pedagogical content knowledge survey items: A review of literature. Journal of Digital
Learning in Teacher Education, 36(3), 150–165. https://doi.org/10.1080/21532974.2020.1724840
Ocak, C., & Baran, E. (2019). Observing the indicators of technologicalpedagogical content knowledge in science classrooms: Video-based research. Journal of
Research on Technology in Education, 51(1), 43–62. https://doi.org/10.1080/15391523.2018.1550627
Papanikolaou, K., Makri, K., & Roussos, P. (2017). Learning design as a vehicle for developing TPACK in blended teacher training on technology enhanced learning.
International Journal of Educational Technology in Higher Education, 14(1). https://doi.org/10.1186/s41239-017-0072-z

17
Y.-F. Yeh et al. Computers & Education 171 (2021) 104238

Petko, D. (2020). Quo vadis TPACK? Scouting the road ahead. In Paper presented at the EdMedia + innovate learning 2020, online, The Netherlands. https://www.
learntechlib.org/p/217445.
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Malden USA: Blackwell Publishing.
Phillips, M., Kovanović, V., Mitchell, I., & Gašević, D. (2019). The influence of discipline on teachers’ knowledge and decision making. Vol. 1112. In B. Eagan,
M. Misfeldt, & A. Siebert-Evenstone (Eds.). Advances in quantitative ethnography. ICQE 2019. Communications in computer and information science (Vol. 1112, pp.
177–188). Cham: Springer ( Cham: Springer).
Pieters, J. M., & Voogt, J. M. (2016). Teacher learning through teacher teams: What makes learning through teacher teams successful? Educational Research and
Evaluation, 22(3–4), 115–120. https://doi.org/10.1080/13803611.2016.1247726
Porras-Hernández, L. H., & Salinas-Amescua, B. (2013). Strengthening TPACK: A broader notion of context and the use of teacher’s narratives to reveal knowledge
construction. Journal of Educational Computing Research, 48(2), 223–244. https://doi.org/10.2190/EC.48.2.f
Putnam, R. T., & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 29(1), 4–15.
https://doi.org/10.3102/0013189x029001004
Rienties, B., Brouwer, N., & Lygo-Baker, S. (2013). The effects of online professional development on higher education teachers’ beliefs and intentions towards
learning facilitation and technology. Teaching and Teacher Education, 29(1), 122–131. https://doi.org/10.1016/j.tate.2012.09.002
Ronfeldt, M., Farmer, S. O., McQueen, K., & Grissom, J. A. (2015). Teacher collaboration in instructional teams and student achievement. American Educational
Research Journal, 52(3), 475–514. https://doi.org/10.3102/0002831215585562
Rosenberg, J. M., & Koehler, M. J. (2015). Context and technological pedagogical content knowledge (TPACK): A systematic review. Journal of Research on Technology
in Education, 47(3), 186–210. https://doi.org/10.1080/15391523.2015.1052663
Schrum, L., & Levin, B. B. (2013). Teachers’ technology professsional development: Lessons learned from exemplary schools. TechTrends, 57(1), 38–42. https://doi.
org/10.1007.
Shaffer, D. W. (2012). Models of situated action: Computer games and the problem of transfer. In C. Steinkuehler, K. Squire, & S. Barab (Eds.), Games, learning, and
society: Learning and meaning in the digital age (pp. 403–431). Cambridge: Cambridge University Press.
Shaffer, D. W. (2017). Quantitative ethnography. Madison, WI: Cathcart.
Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., … Mislevy, R. (2009). Epistemic network analysis: A prototype for 21st-century
assessment of learning. International Journal of Learning and Media, 1(2), 33–53. https://doi.org/10.1162/ijlm.2009.0013
Sherin, M. G., Linsenmeier, K. A., & van Es, E. A. (2009). Selecting video clips to promote mathematics teachers’ discussion of student thinking. Journal of Teacher
Education, 60(3), 213–230. https://doi.org/10.1177/0022487109336967
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189X015002004
Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–22. https://doi.org/10.17763/haer.57.1.
j463w79r56455411
So, H. J., & Kim, B. (2009). Learning about problem based learning: Student teachers integrating technology, pedagogy and content knowledge. Australasian Journal of
Educational Technology, 25(1), 101–116. https://doi.org/10.14742/ajet.1183
Stahl, G. (2011). How I view learning and thinking in CSCL groups. Research and Practice in Technology Enhanced Learning, 6(3), 137–159.
The AACTE Committee on Innovation and Technology. (2008). The handbook of technological pedagogical content knowledge (TPCK) for educators. Mahwah, NJ:
Lawrence Erlbaum Associates.
Thompson, A. D., & Mishra, P. (2007). Breaking news: TPCK becomes TPACK! Journal of Computing in Teacher Education, 24(2), 38–64. https://doi.org/10.1080/
10402454.2007.10784583
Timperley, H., Wilson, A., Barrar, H., & Fung, I. (2007). Teacher professional learning and development: Best evidence synthesis iteration (BES). Retrieved from Wellington,
New Zealand https://www.educationcounts.govt.nz/publications/series/2515/15341.
Tondeur, J., van Braak, J., Sang, G., Voogt, J., Fisser, P., & Ottenbreit-Leftwich, A. (2012). Preparing pre-service teachers to integrate technology in education: A
synthesis of qualitative evidence. Computers & Education, 59(1), 134–144. https://doi.org/10.1016/j.compedu.2011.10.009
*Tseng, J. J., Cheng, Y. S., & Yeh, H. N. (2019). How pre-service English teachers enact TPACK in the context of web-conferencing teaching: A design thinking
approach. Computers in Education, 128, 171–182. https://doi.org/10.1016/j.compedu.2018.09.022
Vangrieken, K., Dochy, F., Raes, E., & Kyndt, E. (2015). Teacher collaboration: A systematic review. Educational Research Review, 15, 17–40. https://doi.org/10.1016/
j.edurev.2015.04.002
Voogt, J., Fisser, P., Pareja Roblin, N., Tondeur, J., & van Braak, J. (2013). Technological pedagogical content knowledge - a review of the literature. Journal of
Computer Assisted Learning, 29(2), 109–121. https://doi.org/10.1111/j.1365-2729.2012.00487.x
Voogt, J., Fisser, P., Tondeur, J., & van Braak, J. (2016). Using theoretical perspectives in developing an understanding of TPACK. In M. C. Herring, M. J. Koehler, &
P. Mishra (Eds.), Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 33–53). London Routledge.
Voogt, J., Laferrière, T., Breuleux, A., Itow, R. C., Hickey, D. T., & McKenney, S. (2015). Collaborative design as a form of professional development. Instructional
Science, 43(2), 259–282. https://doi.org/10.1007/s11251-014-9340-7
Voogt, J. M., Pieters, J. M., & Handelzalts, A. (2016). Teacher collaboration in curriculum design teams: Effects, mechanisms, and conditions. Educational Research and
Evaluation, 22(3–4), 121–140. https://doi.org/10.1080/13803611.2016.1247725
Voogt, J., Westbroek, H., Handelzalts, A., Walraven, A., McKenney, S., Pieters, J., et al. (2011). Teacher learning in collaborative curriculum design. Teaching and
Teacher Education, 27(8), 1235–1244. https://doi.org/10.1016/j.tate.2011.07.003
Vyas, S., & Mishra, P. (2002). The re-design of an after-school reading club. In R. Garner, M. Gillingham, & Y. Zhao (Eds.), Hanging out: After-school community based
programs for children (pp. 75–93). Westport, CT: Greenwood.
Wang, W., Schmidt-Crawford, D., & Jin, Y. (2018). Preservice teachers’ TPACK development: A review of literature. Journal of Digital Learning in Teacher Education, 34
(4), 234–258. https://doi.org/10.1080/21532974.2018.1498039
Willermark, S. (2018). Technological pedagogical and content knowledge: A review of empirical studies published from 2011 to 2016. Journal of Educational
Computing Research, 56(3), 315–343. https://doi.org/10.1177/0735633117713114
Wu, Y. T. (2013). Research trends in technological pedagogical content knowledge (TPACK) research: A review of empirical studies published in selected journals
from 2002 to 2011. British Journal of Educational Technology, 44(3), E73–E76. https://doi.org/10.1111/j.1467-8535.2012.01349.x
*Yan, Z. M., Chai, C. S., & So, H. J. (2018). Creating tools for inquiry-based mathematics learning from technological pedagogical content knowledge perspectives:
Collaborative design approach. Australasian Journal of Educational Technology, 34(4), 57–71. https://doi.org/10.14742/ajet.3755
Yeh, Y.-F., Hsu, Y.-S., Wu, H.-K., & Chien, S.-P. (2017). Exploring the structure of TPACK with video-embedded and discipline-focused assessments. Computers &
Education, 104, 49–62. https://doi.org/10.1016/j.compedu.2016.10.006
Yeh, Y.-F., Hsu, Y.-S., Wu, H.-K., Hwang, F.-K., & Lin, T.-C. (2014). Developing and validating technological pedagogical content knowledge-practical (TPACK-
practical) through the Delphi survey technique. British Journal of Educational Technology, 45(4), 707–722. https://doi.org/10.1111/bjet.12078
Zhang, S., Liu, Q. T., & Cai, Z. Q. (2019). Exploring primary school teachers’ technological pedagogical content knowledge (TPACK) in online collaborative discourse:
An epistemic network analysis. British Journal of Educational Technology, 50(6), 3437–3455. https://doi.org/10.1111/bjet.12751

18