A meta-analysis is a statistical analysis that combines the results of multiple scientific studies.

Meta-
analysis can be performed when there are multiple scientific studies addressing the same question, with
each individual study reporting measurements that are expected to have some degree of error.

This paper serves several purposes. First and foremost, it is devoted to developing a better
understanding of the effectiveness of blended learning (BL) in higher education. This is achieved through
a meta-analysis of a sub-collection of comparative studies of BL and classroom instruction (CI) from a
larger systematic review of technology integration (Schmid et al. in Comput Educ 72:271–291, 2014). In
addition, the methodology of meta-analysis is described and illustrated by examples from the current
study. The paper begins with a summary of the experimental research on distance education (DE) and
online learning (OL), encapsulated in meta-analyses that have been conducted since 1990. Then it
introduces the Bernard et al. (Rev Educ Res 74(3):379–439, 2009) meta-analysis, which attempted to
alter the DE research culture of always comparing DE/OL with CI by examining three forms of interaction
treatments (i.e., student–student, student–teacher, student–content) within DE, using the theoretical
framework of Moore (Am J Distance Educ 3(2):1–6, 1989) and Anderson (Rev Res Open Distance Learn
4(2):9–14, 2003). The rest of the paper revolves around the general steps and procedures (Cooper in
Research synthesis and meta-analysis: a step-by-step approach, 4th edn, SAGE, Los Angeles, CA, 2010)
involved in conducting a meta-analysis. This section is included to provide researchers with an overview
of precisely how meta-analyses can be used to respond to more nuanced questions that speak to
underlying theory and inform practice—in other words, not just answers to the “big questions.” In this
instance, we know that technology has an overall positive impact on learning (g + = +0.35, p < .01,
Tamim et al. in Rev Educ Res 81(3):4–28, 2011), but the sub-questions addressed here concern BL
interacting with technology in higher education. The results indicate that, in terms of achievement
outcomes, BL conditions exceed CI conditions by about one-third of a standard deviation (g + = 0.334, k
= 117, p < .001) and that the kind of computer support used (i.e., cognitive support vs.
content/presentational support) and the presence of one or more interaction treatments (e.g., student–
student/–teacher/–content interaction) serve to enhance student achievement. We examine the
empirical studies that yielded these outcomes, work through the methodology that enables evidence-
based decision-making, and explore how this line of research can improve pedagogy and student
achievement.

Systematic reviews and meta-analyses of higher education research

Higher education research, while a specialist and late developing field, has reached a level of maturity
such that researchers have recently been endeavouring to summarize and synthesize what has been
learnt. This article identifies and analyses many of the systematic reviews and meta-analyses of areas or
aspects of higher education research that have been carried out. It demonstrates the breadth and depth
of higher education research, charting how much we have learnt, what the particular foci of attention
have been, and suggests – by the absence of analysis – which areas may need more attention.
KEYWORDS: Higher education, higher education research, meta-analysis, systematic review

October – 2008

Meta-Analysis: The preferred method of choice for the assessment of distance learning quality factors

Mickey Shachar

Current comparative research literature, although abundant in scope, is inconclusive in its findings, as to
the quality and effectiveness of distance education versus face-to-face methods of delivery. Educational
research produces contradictory results due to differences among studies in treatments, settings,
measurement instruments, and research methods. The purpose of this paper is to advocate the use of a
meta-analytic approach by researchers, in which they synthesize the singular results of these
comparative studies, by introducing the reader to the concept, procedures, and issues underlying this
method. This meta-analytic approach may be the best method appropriate for our ever-expanding and
globalizing educational systems – in general, crossing over geographical boundaries with their multiple
languages, and educational systems in particular. Furthermore, researchers are called to contribute to a
common database of distance learning factors and variables, from which future researchers can share,
glean, and extract data for their respective studies.

Keywords: Distance Learning; Meta-Analysis