FIELD METHODS IN PSYCHOLOGY

Meta-analysis

Topics: Introduction to Meta-Analysis

Discussion of Topics and Sub-topics

DEFINITION

Meta-analysis

- The process of integration of the results of several studies to arrive at evidence synthesis.
Meta-analysis research is secondary analysis of the primary ones.

- It can be performed when there are two or more scientific studies addressing the same
questions.

- It is similar to the systematic review and/or narrative summary; however, meta-analysis

makes use of statistics, numerically gathering the results of the studies and arrives at a summary
estimate. It aims to use statistical approaches to derive a pooled estimate closest to the unknown
common truth.

- Though the essence of meta-analysis is quantitative—that is, it makes use of statistical

approach—qualitative meta-analysis exists as well. It serves as a systematic approach to the
evaluation and synthesis of research studies examining the same phenomena.

- Unless the meta-analysis is properly planned and appropriate both to the review questions
and to the characteristics of the included studies, it may not produce useful or meaningful results;
or worse, it may produce severely misleading results.

PURPOSE

- To provide a concise and comprehensive picture of findings across the chosen studies.

- To examine and evaluate the impact of methodological influences in the original studies on
their findings.

- To discover the essential elements and translating the results into an end product that
transforms the original results into a new conceptualization.

- It also aims to answer the following:

 o The direction of the effect (positive, negative, unclear)

o Size and consistency of the effect across studies

o Strength of evidence (quality/certainty) for the effect.

HISTORY

- The history of meta-analysis can be traced back to 17th century studies of astronomy.

- A paper published by Karl Pearson in 1904 that collected data from several typhoid
vaccination studies is seen as the first instance of which the meta-analytic approach was used to
synthesize the outcomes of multiple clinical studies.

- The term itself “meta-analysis” was coined in 1976 by a statistician named Gene V. Glass,
which led to him being recognized as the modern founder of the method. However, the
methodology itself predates the coinage of the term by several decades (as evident in the
aforementioned study by Pearson)

- Meanwhile, the idea of qualitative meta-analysis was assumed to be proposed by Stern &
Harris, yet the major impact of this particular methodology came from Noblit & Hare’s work on
meta-ethnography in education.

- There exist many types of qualitative meta-analysis, which include meta-ethnography, meta-
study, meta-summary, grounded formal theory, and many other brand name qualitative meta-
analysis methods.

- It is worth noting that there are many more types of brand name qualitative meta-analysis
methods because each of these approaches brings a unique set of skills and perspectives. This
can be attributed to the fact that the approach to conducting qualitative meta-analysis may vary
per researcher and research teams, and that there is no one approach that is consensually agreed
upon.

Research Sample

Research Title: Sleep duration and mortality: a systematic review and meta-analysis

Authors: Lisa Gallicchio and Bindu Kalesan

Abstract: Epidemiologic studies have shown that sleep duration is associated with overall
mortality. We conducted a systematic review of the associations between sleep duration and all-
cause and cause-specific mortality. PubMed was systematically searched up to January, 2008 to
identify studies examining the association between sleep duration and mortality (both all-cause
and cause-specific) among adults. Data were abstracted serially in a standardized manner by two
reviewers and analyzed using random-effects meta-analysis. Twenty-three studies assessing the
associations between sleep duration and mortality were identified. All examined sleep duration
measured using participant self-report. Among the 16 studies which had similar reference
categories and reported sufficient data on short sleep and mortality for meta-analyses, the pooled
relative risk (RR) for all-cause mortality for short sleep duration was 1.10 [95% confidence
interval (CI): 1.06, 1.15]. For cardiovascular-related and cancer-related mortality, the RRs
associated with short sleep were 1.06 (95% CI: 0.94, 1.18) and 0.99 (95% CI: 0.88, 1.13),
respectively. Similarly, among the 17 studies reporting data on long sleep duration and mortality,
the pooled RRs comparing the long sleepers with medium sleepers were 1.23 (95% CI: 1.17,
1.30) for all-cause mortality, 1.38 (95% CI: 1.13, 1.69) for cardiovascular-related mortality, and
1.21 (95% CI: 1.11, 1.32) for cancer-related mortality. Our findings indicate that both short
sleepers and long sleepers are at increased risk of all-cause mortality. Further research using
objective measures of sleep duration is needed to fully characterize these associations.

Purpose of the Study: The purpose of this report was to systematically review the
epidemiologic evidence on the association between sleep duration and the risk of mortality, both
qualitatively and quantitatively.

Methods: Study search; in the study search, the researchers have searched PubMed for articles
on associations between sleep duration and mortality, whether all-cause or cause-specific. Their
search included studies with key terms “sleep”, “sleep initiation and maintenance disorders”,
“insomnia”, “mortality”, “death”, “fatal”, and was limited to humans, adults, and articles written
in English. The search included all studies published in PubMed up to October 31, 2007 and did
not include sudden infant death syndrome. After this they have conducted the study selection and
data abstraction, where they have applied certain criteria to the abstracts for the literature search.
The criteria excluded the following: no original data (reviews, editorials); studies not addressing
the association between sleep duration and mortality; and studies not in human adults. Upon
selection of articles, they are reviewed through these criteria once more. In cases of articles with
similar content, the report with the most updated content is selected, and in cases of duplicate
publication, only one publication was included. Data abstraction was then conducted, which
included year of publication, country of data collection, study design, and sample characteristics.
Lastly, statistical analysis of the gathered data, where relative risks (RR) and 95% confidence
intervals (CI) for all-cause, cardiovascular-related, and cancer-related mortality outcomes were
abstracted from each included study. However, data on other mortality outcomes were also
abstracted but were not considered for meta-analyses because of either too little studies reporting
said outcomes, or the outcome itself was non-specific.
Results:

23 cohort studies were identified that examined the associations between sleep duration
and all-cause and ⁄ or cause-specific mortality. Findings from the quantitative analyses indicate
that among both males and females, short sleepers and long sleepers are at increased risk for all-
cause mortality compared to individuals who report, on average, a medium amount of sleep per
night (generally defined as 7 to 7.9 h). Less evidence exists to draw conclusions regarding the
associations between duration of sleep and specific types of mortality, although the pooled risk
estimates for short sleep among men were in the direction of increased risk compared with
average sleepers for cardiovascular-related mortality, and the pooled risk estimates for long sleep
among men and women were in the direction of increased risk compared with average sleepers
for both cardiovascular related and cancer-related mortality.

Accumulated evidence shows that sleep duration is significantly associated with all-cause
mortality. Despite the consistency of this evidence, there has been some debate in the literature
as to whether these increases in mortality risk among the short sleepers and long sleepers
compared to individuals reporting a medium amount of sleep. Other factors such as pre existing
medical conditions and socio-economic status have been shown to be associated with both
shorter and longer sleep, therefore arriving at the hypothesis that the association between sleep
and mortality rate might be due to either of these two, leaning on the latter (socio-economic
status), though recent studies have shown that these factors and many others do not explain all of
the association between sleep duration and mortality.

In terms of biological plausibility of the association, information remains unclear. Many

studies have shown the potentially adverse effect on short sleep in one’s health, but less research
has focused on potential mechanisms behind the association between long sleep and mortality
even though the body of evidence suggests that long sleep may be more detrimental in terms of
mortality than short sleep. Moreover, studies which discuss these medical conditions hardly give
attention to the association of sleep duration and mortality thus making it hard to tell.

Limitations:

Despite the increasing evidence of association between sleep duration and mortality, it is
difficult to tell the exact number of hours associated with increased mortality risk due to the
limitations of the literature gathered. This can be attributed to the fact that all studies in this
paper measured sleep duration by means of self-report which may not give the accurate amount
of sleep per night. The study would have benefited more from using sleep diaries when
collecting data. Second, the studies included both in the overall systematic review and the meta-
analyses varied in their initial exclusion criteria and their inclusion of adjustment factors in the
analyses. Finally, in general, the studies included in this review used different category cut-
points in their analyses, hence precluding our ability to examine the associations between sleep
duration and mortality at individual hour or smaller sleep duration categories.
Conclusion:

Sleep is a common under-recognized public health issue in industrialized countries across

the globe. This study shows that both short and long sleepers are at increased risk of all-cause
mortality; this is consistent with results from studies examining sleep duration and the
development of health conditions as well as the limited evidence pertaining to the physiological
effects of sleep at various durations. However, the results of this study must be considered with
the limitations it has presented, such as the use of self-report in the studies presented and the lack
of consistency in controlling for comorbid conditions. In terms of strength association, they have
discovered that short sleep has a small increase in all-cause mortality in comparison with other
conditions such as obesity, and the long sleep association was stronger than that reported for
short sleep but this association is more likely to be confounded by unmeasured health problems
or habits.

Personal reflection and recommendation:

Personally, I have just the same knowledge on the harmful effects of less sleep as much
as everyone else, so being able to read up this study is quite an eye-opener for me. I also see that
there are so many studies that have been conducted for this particular topic and so this is one
reason why the use of meta-analysis was best, as there are already existing studies in different
contexts, all that’s left is to weave all these data together in order to come up with newer and
more relevant data that can possibly be applied in most contexts. However, as the limitations
stated, it would be best to aim for better accuracy by reconducting this study a few years from
now, particularly when there are studies of the same topic which makes use of sleep diaries or
anything else that can accurately count the number of hours of sleep that can be associated with
all-cause mortality rates. Further exploration on the association of sleep duration and mortality
on a biological standpoint could also prove to be useful in the future, especially since the
researchers themselves have stated that sleep problems is a public health matter that needs to be
addressed with the help of physicians, and what better way to do so by being able to discover
mechanisms underlying the associations? With that said, I now have a better understanding on
why meta-analysis is considered as having the highest quality of evidence, as I’ve seen in this
study how the researchers have laboriously filtered all the information they have gathered and
formed it into one.

Three epistemological questions:

1. What kind of knowledge does the methodology aim to produce?

The meta-analysis methodology reexamines pre-existing research studies for a

new purpose. This methodology is considered the highest form of evidence
 acquisition, as it synthesizes the most relevant and timely information from
several studies on the same phenomena in order to not only arrive at a certain
conclusion but also explore the limitations of the studies used. It provides a way
to identify what is already known by bringing together different results of studies
and provide a new starting point for research. This particular methodology
employs a critical realist approach. A critical realist position entails the belief that
the data we collect ought to provide us with information about the world, about
how things really are, but that data does not assume a direct reflection of the
world, as evident in the study’s results and conclusion. The researchers are aware
that while they are able to present the data gathered from the 23 studies in
association to sleep duration and mortality, the certain limitations of the studies
used show that there is more to be explored.

2. What kinds of assumptions does the methodology make about the world?

The meta-analytic methodology applies the relativist ontology in terms of the

assumptions it makes about the world. This is defined as maintaining the idea that
the world is not the orderly, law-bound place that realists believe it to be. As
evident to the fact that there are several kinds of qualitative meta-analysis since
there is no consensus on one format/kind alone. It places emphasis on the
diversity of interpretations that can be applied to it. This is also seen in this
particular study as it presents ideas such as the biological plausibility of their
findings, and the establishment of hypotheses even throughout the synthesis of the
results.

3. How does the methodology conceptualize the role of the researcher in the research
process?

Since this methodology employs relativist ontology as its assumptions about the world,
moreover it places emphasis on the fact that it has to be carefully planned and executed for it to
provide useful and meaningful results, it can be safe to say that the researcher is the central
figure for studies that make use of meta-analysis, because it is the researcher who constructs the
findings. As evident in the methods used for this particular study where the researchers have
used key terms in their search for the studies that are to be used for this paper, the careful
selection of only 23 out of the thousand studies they have found, further analysis done by the
researchers, their statistical analysis which have given birth to their discoveries and more queries
for future researchers. In meta-analysis, the authors themselves are the ones who build their
findings, and from these findings others can build their own as well.