Professional Documents
Culture Documents
Systematic Reviews - Karolinska Institutet Universitetsbiblioteket
Systematic Reviews - Karolinska Institutet Universitetsbiblioteket
Search Menu
Karolinska Institutet
University Library
Systematic reviews
"I am preparing a systematic review" – this is something we frequently hear at KIB.
But what exactly is a systematic review? How does it di er from other types of
reviews? And how do you systematically search the literature?
This page is aimed for students and researchers. However, if you are a bachelor's or
master's student and have been asked to conduct a systematic review, you will likely
nd that you cannot ful ll all the criteria stated on this page. As a student you should
rather do a structured literature review, including only certain aspects of the
methodology described on this page.
Introduction
According to the Cochrane Handbook , a "systematic review attempts to collate all
empirical evidence that ts pre-speci ed eligibility criteria in order to answer a speci c
research question". A systematic review also adheres to a speci c methodology and often
includes meta-analyses, wherein the collected data is combined using statistical methods.
Since they summarise the results from all original studies within a given eld, systematic
reviews are commonly regarded as high quality evidence. A pyramid is often used to
visualise this hierarchy of evidence, with systematic reviews placed at the apex of the
pyramid.
https://kib.ki.se/en/search-evaluate/systematic-reviews 1/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
In this pyramid, only study design variations have been included. Other, more complex
pyramids add other types of layers to the systematic reviews, for example synopses of
syntheses (see Alper & Haynes, 2016 ).
Since the total output of medical scienti c literature produced every year is increasing at an
exponential rate, systematic reviews that collate the available evidence have become
increasingly important. Consequently, more and more systematic reviews are published
every year, a trend evident in scienti c databases.
This is one reason why articles summarising the literature are of such importance today.
Accordingly, the number of published systematic reviews has also increased (see Bastian,
Glasziou & Chalmers, 2010 ). Today, there are even reviews of systematic reviews (see for
example Aromataris et al, 2015 ).
As a doctoral student at KI, you are allowed to include a systematic review as part of
your thesis . The Committee of Doctoral Education also emphasises the importance
of a structured review of the literature in the thesis "kappa". A literature review of your
research eld is now mandatory in the half-time summary .
Nowadays, there are also journals – for example, The Lancet and Alzheimer’s & Dementia
– that require primary studies provide a structured, comprehensive and documented
literature search ("Research in context").
https://kib.ki.se/en/search-evaluate/systematic-reviews 2/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
For more about systematic reviews in general, see the further reading section below.
3. At least two people should be involved, particularly for screening articles and
extracting data.
4. Plenty of time resources are needed, but also in terms of availing yourself of others'
expertise – for instance in database searching – and tools and software.
There are a variety of review types, each using a more or less di erent methodology; the
terminology can be confusing. In an article from 2009, Grant & Booth described 14
review types, for example scoping reviews, and their associated methodologies.
For a condensed overview, see the comparison below (from Jesson, Matheson & Lacey,
2011, p. 105 ).
Planning the No de ned path, allows for creativity Transparent process and
review and exploration documented audit trail
https://kib.ki.se/en/search-evaluate/systematic-reviews 3/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
In this section, we cover the speci c techniques used during a systematic search. For a
general introduction, have a look at our searching for information page. There you will
nd basic information about search techniques and search logic, such as controlled
vocabulary, free-text, boolean operators, truncation and parenthesis.
Do you need search help? Read more about our support to students and researchers
under contact us below.
A fundamental part of a systematic review is an exhaustive literature search that nds all
relevant studies on a topic. Hence, it is important that the search strategy is rigorously
developed with a high sensitivity to nd all these potential relevant articles. This means that
you will end up with a large amount of references, often more than thousands of
references. A high percentage will likely be irrelevant.
In addition, grey literature, for example dissertations and clinical trials, should be
considered in your search. These publication types demand other databases. The literature
search is usually also supplemented with a screening of reference lists, article suggestions
from colleagues and sometimes with a citation analysis – i.e. an analysis of which articles
have cited an older, still relevant study.
To construct an exhaustive search strategy, and at the same time avoid too many
references to screen, is a challenging task. In library and information science, the concepts
of sensitivity, precision and speci city are used. According to the Cochrane Handbook ,
sensitivity "is de ned as the number of relevant reports identi ed divided by the total
number of relevant reports in existence." A high sensitivity search strategy should thus
retrieve all relevant studies on a topic. Precision and speci city state the part of non-
relevant literature in the search.
Sensitivity and precision/speci city are almost always irreconcilable: a highly sensitive
search is also often less precise. This is illustrated in the gure below, where the less
theoretical concepts wide (sensitive) versus narrow (speci c) search are used.
For a systematic review, the search strategy should be highly sensitive. A large portion of
the search result will thus not be relevant. In systematic literature searching, a precision of
two-three percent is common, i.e. two-three references out of a hundred will be relevant.
Before you create a search strategy, it is a good idea to look at what has already been done.
Start by collecting published literature reviews on the topic. Even if they are not totally
updated or exactly match your topic, it is often worthwhile to have a look at the method
section of the reviews; sometimes the search strategies are available as supplemental
material.
In addition, there are validated search lters that might be useful. Search lters, or hedges,
are sets of search terms chosen to restrict a search to a selection of references, for example
based on study type (in Cochrane reviews usually randomised controlled trials) or method
https://kib.ki.se/en/search-evaluate/systematic-reviews 5/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
(for example qualitative methods). A search lter is often developed in di erent variants
based on di erent levels of sensitivity, speci city and precision.
Some search lters are integrated into databases like Pubmed/Medline, Psycinfo and
Cinahl. In Pubmed, Clinical Queries allows you to use search lters to restrict the search
to clinical studies, genetic studies or systematic reviews.
McMaster University
A well-de ned and clear research question is an essential starting point for a systematic
search. To create a logical search strategy, always start by identifying the key elements of
the research question – i.e., establish what the main concepts of the topic are. With these
concepts, you can then create the search blocks that form the basis for the search
strategies used in the di erent databases.
Some of the potential concepts that could form the search blocks are marked in bold. The
PICO structure is a common way of formulating clinical research questions: Population,
Intervention, Control and Outcome. In general, you should not include all parts of the PICO
question in the search. Focus is generally on the population and the intervention, in our
example P = patients with myocardial infarction and I = inhaled oxygen.
The search blocks are combined with the boolean operator AND in the search strategy. A
general principle is that a search for a systematic review should consist of few blocks. More
search blocks means a less sensitive search and a higher risk of missing relevant articles.
Two up to maximum four search blocks is a rule of thumb.
The boolean operator NOT should be avoided in systematic search strategies. NOT
decreases the sensitivity of the search and the risk of missing relevant articles increases.
Please note that this applies to the nal search strategy. NOT can be a useful tool when
constructing and comparing searches – see the section Analyse your search below.
https://kib.ki.se/en/search-evaluate/systematic-reviews 6/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
We include two concepts from our search topic: inhaled oxygen and myocardial
infarction. These two concepts will constitute our two search blocks. The concept acute is
thus excluded because a heart attack is always an acute medical condition and therefore
already implicitly included. We also exclude pain and death from the outcome part.
In every search block, you should include all relevant search terms and variants, combined
with the boolean operator OR. In contrast to the number of search blocks, you should also
try to include as many relevant terms as possible, since this makes your search more
sensitive.
Use both subject headings and free-text terms. In this way you will use the potential of the
controlled vocabulary and retrieve articles where the authors used various terminologies.
You will also retrieve articles that lack subject headings, or are indexed with other subject
headings than those you have used in your search.
In many databases, the default search setting is all elds or a combination of elds, which
means searching in both free-text and subject headings. However, in a systematic literature
search we emphasize the importance of specifying the search elds manually. This gives
you more control and facilitates a more logical and cohesive search strategy.
The process of nding relevant search terms is an important part of the systematic search.
Usually you start o with a scoping search, with the subject headings and synonyms you
already know of. A scoping search will give you a sense of how much has been written on
your topic; by screening titles, abstracts and subject headings more relevant search terms
can be found.
You should also identify a set of key articles, i.e. signi cant studies in your eld. These key
articles should correspond exactly to your research question and be retrieved by your
search strategy. In sum, the key articles could be used for both constructing and for
validating your search strategy. If you don't retrieve these studies in your search, modify
your search query.
An additional suggestion is to check the bibliographic information available for the key
articles: title, abstract, author keywords, subject headings etc. These might provide
additional search terms.
There are tools speci c tools for nding search terms which analyses a search query or a
set of references. The tools below are adapted for Pubmed, but the free-text terms are
generic and can thus be used in all databases:
MEDSUM
PubReminer
Subject headings
In many databases, articles are for better retrieval indexed with subject headings or
controlled terms from a thesaurus. In PubMed, the controlled vocabulary is MeSH: Medical
Subject Headings .
You need to nd subject headings to build a good search strategy. Search for your key
concepts in the controlled vocabulary and check what subject headings your key articles are
indexed with to nd relevant terms. Explore the hierarchies and related terms. Please note
that there might be several subject headings for closely related concepts, and that these
headings might be part of di erent hierarchies.
Check if some of the terms are new as subject headings. If so, you might also consider
including the previous subject heading used for indexing the concept.
Avoid limits for subject headings, such as subheadings or main concept, for example MeSH
Major Topic in Pubmed.
The default setting in most databases is the exploding of subject headings, i.e. narrower
terms of the subject headings are included. In general, this functionality is helpful, but you
might in some cases consider if a subject heading should be searched without being
exploded, i.e. not including narrower terms.
Free-text terms
Include all the relevant synonyms and spelling variations in title and abstract from the key
articles and other relevant studies retrieved by the test search. The controlled vocabularies
are useful tools to nd free-text terms. You should, of course, search for the subject
heading terms as free-text too. In addition, you can nd more free-text terms among the
synonyms of a subject heading. These synonyms are named di erently in di erent
databases. In Pubmed, they are called Entry terms .
Narrower terms should also be considered as free-text. These terms are included
automatically in the controlled vocabulary search, thanks to the explode functionality, but
for the free-text search they have to be explicitly stated to be included. One example of this
is di erent forms of myocardial infarction, STEMI and non-STEMI,. Those terms are
automatically included in the exploded MeSH search for Mycardial Infarction , but have to
be added in the free-text part.
https://kib.ki.se/en/search-evaluate/systematic-reviews 8/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
therapy
therapies
therapeutic
Quotation marks are useful for keeping words in phrases together. On the other hand,
you need to be careful with this in systematic searching: quotation marks make the search
more precise and hence can relevant literature be missed. One alternative is to use
proximity operators, which lets you specify the number of words that can appear between
two search terms. Proximity operators thus retrieves di erent variations of phrases, for
instance in the word order, as a contrast to phrases with quotation marks.
Let us use oxygen treatment as an example. If searching for this concept as a phrase, we'll
miss to catch potential relevant references including:
However, these variations are retrieved if we use a proximity operator (oxygen NEAR/3
treatment in Web of science).
Some medical terms, a few of them quite common, are spelled di erently in British and
American English. Consider if your search strategy includes a word with a di erent spelling.
If so, include both of them. Some databases can compensate for this automatically, so
called lemmatization, but you should not rely completely on this functionality.
Tumour / Tumor
Gynaecology / Gynecology
Coeliac / Celiac
Ageing / Aging
Behaviour / Behavior
https://kib.ki.se/en/search-evaluate/systematic-reviews 9/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
Labour / Labor
Read more: Spelling di erences in medical terms in British and American English .
When a tentative search strategy has been constructed, you should analyse if your key
articles have been retrieved by the search.
Do a new search (B) for your key articles only, for instance in Pubmed by using PMID.
Which key articles are not retrieved by your search Search B NOT A. If all key articles
are retrieved, the search result will be zero.
If key articles are not retrieved, you should analyse why. Try to identify which search block
that causes the exclusion of the article. What search terms can be added to include this key
article? Maybe you should consider removing a complete search block? However,
sometimes it's not possible to retrieve all key articles and, at the same time, have a
reasonable precision.
You might also use NOT to nd additional subject headings or free-text terms. Conduct a
search (A) including subject headings only and one search (B) with free-text only. Then
search A NOT B. In this case, you'll retrieve all studies indexed by the relevant subject
headings but without the free-text terms you have included. Check for additional free-text
terms in the title and abstract. After that, do the opposite (B NOT A) to nd more relevant
subject headings.
The next step is to screen the search results. Do the references correspond to your topic?
Because the search strategy should be highly sensitive, the precision will surely be quite low
(hence many irrelevant hits). However, you should of course also nd some eligible
references.
Maybe some of the search terms will generate a lot of false hits. If so, you might consider to
remove them. You can use the NOT operator again in order to analyse the search.
Search A NOT B.
https://kib.ki.se/en/search-evaluate/systematic-reviews 10/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
Analyse the result. If you do not nd any relevant hits, consider excluding the term from the
search strategy.
Always use identical search blocks and free-text terms in all databases. However, the
search strategy has to be translated to the controlled vocabulary in each database and
adapted to the eld tags etc used in the di erent interfaces. In this part, we
demonstrate our search example in four di erent databases: Pubmed, Medline (Ovid),
Embase and Web of Science.
In addition to the search tips below, the database sheet (pdf, 25 kb) made by the search
group at KIB is useful for creating and translating search strategies.
Pubmed
In biomedicine, you'll usually begin the literature search in Pubmed and thus include the
subset Medline – the larger part of indexed articles in Pubmed. However, when
conducting more advanced search strategies, Medline via the Ovid interface is commonly
used (see below).
You are probably quite familiar with Pubmed, and it is an easy database to start with. There
are, however, a few things to consider:
You can use truncation, but the truncation will work completely only if there are
maximum 600 variants – search for test* and have a look!
Not all phrases are searchable, only the phrases included in the phrase index .
Look out for error messages or check if your phrase is searchable by browsing the
phrase index: select Title/Abstract in the Advanced Search Builder , add your
phrase and choose Show index list. If your phrase is not included in the phrase
index it will be separated and searched as independant terms combined with the
boolean operator AND.
Do not trust the automatic term mapping , specify the elds to be searched instead
to gain full control. Moreover, if you are using truncation it will turn o the automatic
term mapping. If you want to see how the automatic term mapping works, search for
a term without specifying which eld to search in and then click on Search Details.
If you are using eld tags (as in the example below) it automatically translates into a
phrase search, similar to using quotation marks.
https://kib.ki.se/en/search-evaluate/systematic-reviews 11/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
You cannot use proximity operators, in contrast to almost all other advanced
databases.
Medline Ovid
In Medline Ovid, the indexed part of Pubmed is included, but also newly added references
("Epub Ahead of Print" and "In-Process") and other non-indexed material. Ovid is a database
platform used for many di erent databases. Through the KI library subscriptions, you also
have access to the databases Psycinfo and Global Health via Ovid.
There are several advantages of searching Medline via Ovid instead of Pubmed. In Ovid it
is possible to use proximity operators, truncate (without any limitations) and search for all
phrases (as opposed to Pubmed, which is limited by the phrase index). It is also easier to
structure comprehensive and advanced search strategies. Information specialists tend to
prefer Medline Ovid and the platform is, in most cases, the standard interface for Cochrane
Reviews.
As in the example below, we deconstruct our search strategy into lines and then combine
them, rst each line within a search block with OR, then the search blocks with AND.
https://kib.ki.se/en/search-evaluate/systematic-reviews 12/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
Embase
There are several di erent options for searching in Embase (via embase.com). In the
example below, the search strategy is divided in lines as in Ovid: one for each Emtree term,
one line for all free-text. These are then combined into one search block. Emtree is the
controlled vocabulary in Embase, similiar but not identical to MeSH in Pubmed. You can
access the Emtree terms in the top menu under "Browse".
Web of Science
Use quotation marks around phrases to keep the words together for greater
precision.
Truncation and proximity operators can be used and are more important because it's
a free-text database.
Consider dividing the search strategy as in the example below, using one search eld
per search block.
The default Basic Search is a good start, but also familiarize yourself with Search
History for a better overview and editing possibilities.
PRISMA
https://kib.ki.se/en/search-evaluate/systematic-reviews 14/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
As a support in the review process there are the PRISMA Guidelines : "an evidence-based
minimum set of items for reporting in systematic reviews and meta-analyses." PRISMA – an
acronym for Preferred Reporting Items for Systematic Reviews and Meta-Analyses –
consists, among other things, of a check list and a ow diagram. Many journals, for example
Lancet , BMJ and Plos One , endorse the guidelines stated by PRISMA and require
a PRISMA ow diagram when publishing systematic reviews.
According to PRISMA, the search strategy should be reported in full for at least one
database; however, we recommend attaching all search strategies as an appendix to the
published article. The majority of journals allow you to upload supplemental material. If the
complete search strategies are attached, you only need to brie y describe the search
process in the methods section and thereafter refer to the supplemental material section.
Item 8: Search – Present full electronic search strategy for at least one database,
including any limits used, such that it could be repeated.
Protocol
As with a clinical trial, a protocol should be established for a systematic review; preferably, it
should also be published. This is actually one of the items for reporting in the PRISMA
Guidelines. Some journals – for instance BMJ, The Lancet and British Journal of Dermatology
– requires a published protocol for systematic reviews. There are several reasons for
this. For instance, it increases the overall transparency of the review, making it more
scienti cally sound. Additionally, the registration of the protocol also adds visibility to your
review. This can help to avoid duplication of the work.
There is also a guideline for developing protocols för systematic reviews: PRISMA-P and a
speci c database aimed for the registration of protocols for systematic reviews: PROSPERO
– International Prospective Register of Systematic Reviews . It's also possible to publish
protocols in the journal Systematic Reviews .
Search in PROSPERO using the phrase "karolinska" and see ongoing review projects
by authors from KI/KS.
Reference management
https://kib.ki.se/en/search-evaluate/systematic-reviews 15/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
Systematic literature searches always produce a large amount of references. Also, when
searching in several di erent databases many duplicate references will appear – i.e.,
identical references that are included in more than one database.
However, Endnote and other types of reference managers have limitations when it comes
to supporting the systematic review process. Other software has therefore been developed
and is dedicated to supporting the systematic review process, for instance in the screening
of abstracts and references according to inclusion and exclusion criteria. For the time
being, KI o ers no institutional licens on this kind of software.
EPPI-Reviewer 4
Covidence
DistillerSR
As the literature search is a fundamental part of a systematic review, it's important that the
search strategy is of high quality. To ensure this, and to avoid errors, there are several
options. One option is to involve an experienced librarian/information specialist in the
review project. In either case, it's good if a second person has a thorough look at the search
strategy. There is also a tool for validating search strategies called Peer Review of Electronic
Search Strategies (PRESS).
PRESS was originally published in 2008–2010, but was revised in 2015. The PRESS checklist
was originally made for expert searchers, such as librarians, but can also be used by
students and researchers when creating more extensive search strategies. The evidence
and conclusions in PRESS are based on a comprehensive research project.
https://kib.ki.se/en/search-evaluate/systematic-reviews 16/20
2019/10/2 Systematic reviews | Karolinska Institutet Universitetsbiblioteket
Further reading
Web
PRESS: Peer Review of Electronic Search Strategies from Canadian Agency for Drugs
and Technologies in Health . Interesting and thorough study that identi es and lists
the errors and omissions that occur in systematic literature searches.
Literature
Egger M, Smith GD, Altman DG, editors. Systematic reviews in health care: meta-
analysis in context . 2. ed. London: BMJ; 2001.
Contact us
If you are a bachelor's or master's student, or a doctoral student writing a literature review
for the half-time summary, please contact the librarians in the library. You can also can
make a booking for a certain time.
https://kib.ki.se/en/search-evaluate/systematic-reviews 17/20