Research Center for Information, Communication,

Culture

ChatGPT as support for systematic scoping reviews: integrating artificial intelligence

with the SALSA framework

Carlos Lopezosa, Lluís Codina and Nú ria Ferran-Ferrer

UNIVERSITY
BARCELONA
Faculty of Information and Mitjans Audiovisuals | University of Barcelona
Mercier Palau, 140
08014 Barcelona

Authors: Carlos Lopezosa, Lluís Codina and Nú ria Ferran-Ferrer

February 2023.
Work distributed under a CC BY-NC-SA 4.0 license

©G8G

Recommended Citation
Lopezosa, Carlos; Codina, Lluís; Ferran-Ferrer, Nú ria (2023) ChatGPT as support for
systematic scoping reviews: integrating artificial intelligence with the SALSA framework .
CRICC Collection . Barcelona: University of Barcelona

Financing
This work is part of the methodological developments of the project "Parameters and
strategies to increase the relevance of media and digital communication in society:
curation, visualization and visibility (CUVICOM)". PID2021-123579OB-I00 (MICINN),
Ministry of Science and Innovation (Spain).
Activity funded by the European Union-NextGenerationEU, Ministry of Universities and
Recovery, Transformation and Resilience Plan, through a call from the Pompeu Fabra
University (Barcelona).

ABOUT THE AUTHORS

2
Carlos Lopezosa has a doctorate in journalism from the Pompeu Fabra University and a
visiting researcher at the University of Barcelona (Margarita Salas postdoctoral
fellowship). His doctoral thesis focused on the study of the positioning factors of
content-intensive sites, and especially online media, as well as the evaluation of SEO
analysis tools. He is a specialist in search engine positioning and monetization systems
based on quality content strategies. He has been an associate professor at the Pompeu
Fabra University, teaching at the Faculty of Communication, in the Degrees in
Journalism, Audiovisual Communication and Advertising and Public Relations.
ORCID: 0000-0001-8619-2194
Contact: lopezosa@ub.edu

Lluís Codina is a professor at the Faculty of Communication, in the degrees of Journalism

and Audiovisual Communication, as well as in the Master's Degree in Communication
Research. He is a researcher in the Communication Department of the Pompeu Fabra
University, coordinator of the Journalism and Digital Documentation Research Unit
(UPF) and co-director of the Cybermedia Observatory. He is also a professor at the
Barcelona School of Management – UPF. He is co-founder and member of the editorial
team of the Hipertext.net Academic Magazine.
ORCID: 0000-0001-7020-1631
Contact: lluis.codina@upf.edu

Nú ria Ferran-Ferrer is a professor at the Faculty of Information and Audiovisual Media

of the University of Barcelona (UB) in the degrees of Digital Information Management
and Audiovisual Communication, as well as in the master's degrees of Digital Content
Management and Digital Humanities ). She is a researcher at the Information,
Communication and Culture Research Center (CRICC). Responsible for the research
project “Women and Wikipedia” and co-director of the academic journal BiD.
ORCID: 0000-0002-9037-8837
Contact: nferranf@ub.edu

Summary:

3
This report proposes practical guidelines to research personnel for the use of artificial
intelligence to support literature reviews with a systematic approach. More specifically,
it presents proposals to combine ChatGPT conversational artificial intelligence (AI) with
the SALSA (Search, AppraisaL, Synthesis and Analysis) framework for systematic reviews
in order to develop reliable literature review studies. For these purposes, first of all, we
present an introduction to both systematic reviews and scoping reviews . Secondly, we
present a description of the components of the SALSA framework , in order to adequately
frame its subsequent use in our methodological proposal. Thirdly, we present a proposal
for the use of AI ChatGPT and its ethical use and critical processing as support for
content analysis and knowledge synthesis methodologies, within which the review
studies can be framed. Finally, our proposal is illustrated with a practical example,
presented step by step how to carry out a scoping review combining ChatGPT with the
SALSA framework.

Keywords:
ChatGPT, systematic reviews , scoping reviews , literature review, content analysis,
artificial intelligence, AI, protocols, review studies, knowledge synthesis, research
methods, research ethics.

4
Abstract:
This report proposes practical guidelines for research personnel on the use of artificial
intelligence as support for systematic literature reviews. More specifically, proposals are
made to combine conversational artificial intelligence (AI) ChatGPT with the SALSA
(Search, Appraisal, Synthesis, and Analysis) framework for systematic reviews in order
to develop literature review studies with guarantees. To these effects, first, we present
an introduction to both systematic and scoping reviews. Second, we present a
description of the components of the SALSA framework, in order to adequately frame its
later use in our methodological proposal. Third, we present a proposal for the use of
ChatGPT AI and its ethical use and critical processing as support for content analysis and
knowledge synthesis methodologies, within which literature review studies can be
framed. Finally, we illustrate our proposal with a practical example, step by step
presented how to carry out a scoping review combining ChatGPT with the SALSA
framework.

Keyword:
ChatGPT, systematic reviews, scoping reviews, literature review, content analysis,
artificial intelligence, AI, protocols, review studies, knowledge synthesis, research
methods, research ethics.

5
 E0 UNIVERSITY
11— BARCELONA

Summary

1. Introduction.........................................................................................................................7
2. An approach to systematic reviews: systematic review and scoping reviews.....................8
2.1 Literature reviews with a traditional approach.............................................................8
2.2 Systematic approaches.................................................................................................9
2.3 Scoping reviews............................................................................................................9
3. SALSA Framework for the development of systematic approaches................................10
4. ChatGPT combined with SALSA for systematic reviews: an approach from research
ethics.........................................................................................................................................13
4.1 Prompt design............................................................................................................13
4.2 SALSA framework.....................................................................................................14
5. Illustrating a scoping review : applying SALSA and ChatGPT step by step....................15
5.1 Search phase...............................................................................................................17
5.2 Evaluation phase (AppraisaL).........................................................................................23
5.3 Analysis phase.................................................................................................................26
5.4 Synthesis phase (Synthesis)............................................................................................32
6. Effects of scale, practicality and connectivity...................................................................36
7. Conclusions.......................................................................................................................37
8. Bibliographic references....................................................................................................37
1.

6
1. Introduction
Systematic reviews (and as part of them, scoping reviews) provide methods to carry out
literature reviews in a rigorous and systematic way (hence their name) so that the results
present scientific validity, both internally (adequate development ) and external
(transferability or generalizability).

The products of a literature review can be part of various types of academic work, among
which the final master's theses and doctoral theses stand out for their importance in the
training of research personnel.

They are also an essential component in the production of reports to request funding for
competitive projects, of decisive importance for consolidating research careers and for
the very existence of research groups.

Additionally, the results of a literature review usually form part of the first pages of
scientific articles, since evaluators (and readers) expect to find the background of the
research that will be reported in the article duly cited.

In this sense, let us say that, in fact, one of the main causes of editorial rejection of
scientific articles is the lack of mention of the theoretical and methodological foundations
in the articles submitted for review, since such lack is understood by the evaluators as
sign of a poor foundation of the work.

Finally, literature reviews can be an academic-scientific genre in themselves and as such

can be presented and published in the form of articles as independent research. These
are so-called review works, or meta-synthesis, of wide and recognized prestige in various
disciplines, among which the biomedical sciences and other health disciplines stand out
for their early use within the so-called evidence-based medicine. . This influence has been
spreading to other disciplines and it is increasingly common to find articles that report
review work in the social sciences and humanities.

The most important issue is that in systematically focused literature reviews there are
critical points that the research staff must carefully address to develop the review with
scientific validity. These points coincide with those presented by the SALSA framework
(Grant and Booth, 2009). They refer specifically to the systematized development of the
search, evaluation, analysis and synthesis phases (Search, AppraisaL, Synthesis and
Analysis, SALSA), which are what make up a review study that aims to have scientific
validity.

Given the new opportunities of artificial intelligence (hereinafter AI), and more
specifically of ChatGPT, a prototype of artificial intelligence conversational bot developed
by OpenAI, focused on dialogue through natural language processing and both
supervised and learning techniques. reinforcement (OpenAI, 2022), this work aims to
propose a method that combines this AI with systematic reviews and more specifically
with the application of the aforementioned SALSA framework (Grant and Booth, 2009).

Likewise, our proposal adheres to the principles proposed by Vand-Dis et al. (2023) that
involve honest, comprehensive and transparent use of AI. In our case, both in the
application of the method and in the collection of data and its subsequent interpretation,
we try to support these principles relying on both ethics and critical thinking, the two
conditions that in previous works we have established as necessary to use the AI (Codina,

7
2022; Lopezosa and Codina, 2023).

In what follows, we will proceed as follows. First of all, we make an approach to

systematic reviews and scoping reviews , then we explain what the SALSA framework
consists of, then we describe what ChatGPT is, in which phases of SALSA this artificial
intelligence can be applied and how to adjust to the principles of Van- Dis et al. (2023),
Codina (2022) and Lopezosa and Codina (2023) and, finally, the SALSA framework is
applied step by step in combination with ChatGPT illustrated with a practical case.

2. An approach to systematic reviews: systematic review and scoping

reviews
Systematic reviews make sense in themselves, obviously, but it can help us to compare
them with traditional reviews to understand their specific contributions in the context of
academic works, so first we will dedicate a few words to the traditional ones.

2.1 Literature reviews with a traditional approach

We start from the simple premise that the traditional approach to literature review does
not have to produce bad products. Some authors are experienced enough to present good
review papers even without a well-determined method. The problem is that it is
impossible to ensure the scientific validity of work done in this way. There are no
guarantees that it is free of bias and it lacks traceability.

Therefore, although a state of the art developed in a traditional way may be brilliant due
to the good work of its author, he himself cannot be sure that he has identified all the
relevant works since he has not followed a protocol to determine the basis of the
evidence. To conclude, we can say that the traditional approach to literature reviews
does not guarantee either rigor or effectiveness because:

• It lacks explicit methods. Instead, it is based on the intuition of its authors, so

it is not possible to know to what extent the selection of the works has been
complete or has been subject to possible author bias, even unintentional.
• It lacks reliability if the author has not considered the need to account
neither the sources nor the methods used to include and exclude the works
nor the way in which they analyzed them to ensure that they were able to
extract the results systematically.
• They lack transparency and traceability, and consequently are impossible to
verify or replicate.

2.2 Systematic approaches

Systematic reviews emerged in a very specific context, namely, research on the

effectiveness of a specific health intervention for specific populations. Later, these
principles of evidence-based science were expanded to other areas, such as the social
sciences and humanities.

Reviews with a systematic approach solve the deficit of rigor and traceability that we
have pointed out in traditional approaches through a four-fold contribution:

8
 • They clearly establish the work phases. In this way, the aspects on which it is
necessary for the author to pay critical attention do not go unnoticed, or it
does not happen that in some cases they have been taken into account and in
others, not.
• For each phase, they foresee (and provide) specific methods for its resolution,
instead of leaving it to intuition or the ability to improvise.

• They ensure systematicity, by having internationally accepted phases and

procedures, which are applied equally to all the documents that form the
basis of the evidence, and it turns out that systematicity is always an essential
component of all scientific work.

• They avoid bias and promote transparency. By having the above elements, the
research staff responsible for a review with a systematic approach can record
the steps followed and offer the corresponding information or make it
available to other research teams.

2.3 Scoping reviews

Scoping reviews , or exploratory reviews, are a specific class of systematic reviews. They
are part of the systematic approaches because they inherit all the principles of rigor,
validity and traceability that we have seen in the previous point.

They differ from other types of reviews because original systematic reviews focus on the
analysis of the effectiveness of interventions in the health field. On the other hand,
scoping reviews focus on the analysis of a field of knowledge. Original systematic reviews
ask whether a health treatment is effective, while scoping reviews ask what the defining
characteristics of a field of knowledge are.

The issue that most affects the social sciences and humanities is that in recent years, the
recognition of scoping reviews as a research method has received recognition from the
most important centers dedicated to the study of systematic approaches.

We can specifically mention the PRISMA Group (Tricco et al. 2018), the Joanna Briggs
Institute -JBI- (Peters, et al. 2015) and the Cochrane Collaboration, although everything
was preceded by a highly cited work by Arksey and O'Malley (2005) who was the first to
propose a methodological framework for this type of reviews.

A scoping review , therefore, is a systematic type of synthesis of evidence whose objective

is to determine, with the greatest possible rigor, the situation of an area, a sector of
science or a field of knowledge.

In this way, one of the characteristic products of a scoping review are the states of the art
or state of the art . Other objectives of exploratory reviews that are usually mentioned in
the examined works are:

• Identify the extent, range and characteristics of an activity

researcher.
• Evaluate the quality and characteristics of the research methods used
in an area of knowledge.
• Identify gaps and research opportunities.
• Clarify key concepts and definitions of a scope of activity of the

9
 science.
• Summarize the main results of research in an area of
knowledge.
• Support decision making through the synthesis of knowledge in a
certain scope.

3. SALSA Framework for the development of systematic approaches

In the SALSA framework , systematic reviews are considered to consist of at least four
phases. It should be said that, in some proposals, these phases can be disaggregated into
subphases, but the core is always made up of the four indicated below (Grant, Booth,
2009):

• Search
• Assessment
• Analysis
• Synthesis

The SALSA framework can be interpreted as a guide that requires answers to four
fundamental questions (figure 1).

Figure 1. The four checking questions and phases of a systematic review. A literature
review can be approached as an investigation that includes data collection because the
phases are well defined and have rigorous and transparent procedures for the selection
and analysis of the evidence base. Source: Codina (2020a) based on the SALSA
Framework (Grant, Booth, 2009).

The curious acronym SALSA comes from the acronym that, in its English expression,
defines each phase, namely, Search , Application , Analysis and Synthesis . Below is an
explanation of the requirements established by each phase, presented in their logical
order (not in that of the acronym):

1
0
 • Search : the search phase must be resolved by identifying and
selection of sources that will provide the basis of evidence. Additionally, by
designing the search equations if we use academic databases, or by designing
other procedures if the sources are of another type, but in any case, the
sources and methods of obtaining documents must be well established.
(Codina, 2020a).
• A ppraisa L : this evaluation phase is required to ensure that the base
of the evidence responds to the requirements of the study. It is carried out
through an application of inclusion and exclusion criteria, which usually
includes the topic, the range of publication dates, the methodology and
ultimately the verification of the quality of each document identified in the
previous phase (Codina, 2020b ).
• Analysis : consists of the design of data extraction schemes with the
which will be systematically analyzed each and every one of the references
that form the basis of the evidence. General tables or matrices can be used to
extract data based on research axes, such as the objectives, methodologies and
main results, or thematic dimensions that are specific to each specific project
(Codina, 2020c).

• Synthesis : in this phase the main results are presented through

tables or matrices, as well as narrative summaries. Diagrams or concept maps
are eventually included. The objective is to present a synthesis that is more
than the sum of the parts, for which a presentation of results is required that is
not merely aggregative, but that includes configuration and interpretation of
results. In particular, the identification of patterns, emerging themes, and
research gaps and opportunities is expected (Codina, 2020c).

In short, thanks to the application of SALSA we can rigorously obtain the evidence base
for a scoping review . This evidence base will be made up of documents (mostly articles
from scientific journals in general) that report research and communicate innovations in
the specific area of knowledge that is intended to be explored.

These same points point out the critical aspects that research personnel must carefully
address to ensure the validity of their review work. These points consist mainly of the
choice of the search equation and the method of carrying out the narrative synthesis. It is
precisely in these two elements where we will apply, as we will see in the following
sections, the use of artificial intelligence.

Likewise, we want to warn that, since the application of artificial intelligence to the
SALSA framework is the central point of this manual, those interested are recommended
to review the Codina chapters (2020a, 2020b and 2020c) in which the different steps are
explained. to carry out a review of the literature with a systematic approach, since in this
report we focus on the use of AI, taking the previous foundations as known.

4. ChatGPT combined with SALSA for systematic reviews: an approach

from research ethics
ChatGPT is a language model developed by the company OpenAI. According to this
research company, it uses natural language processing (NLP) techniques (Guida and
Mauri, 1986) to generate coherent and natural responses in real time. Likewise, its
developers declare that this chatbot is based on a model called GPT-3, which is said to

1
1
have been trained on a wide variety of tasks (OpenAI, 2022).

The point is that this AI is causing a great impact on society since December 2022 and it
seems that it will continue to do so in the short and medium term in different sectors,
from public administration, education to any type of company. Therefore, as researchers,
we must look for ways to integrate the new possibilities that this resource opens up into
our research.

However, it is important to ensure that its use during the development of research has a
positive impact for academia and society. For this reason, for this work we try to take
into account, on the one hand, ethics and critical thinking as a constant in all uses made
of ChatGPT (Codina, 2022; Lopezosa and Codina, 2023). On the other hand, this work is
ascribed to the responsible use of AI referring to the maintenance of human verification,
the development of rules of responsibility for the use of AI and transparency (Van-Dis et
al. 2023) both in the application of the method and in data collection and its subsequent
interpretation.

Finally, it should be noted that currently the ChatGPT chatbot does not declare the
sources of information or textual corpus from which it searches for information and
prepares its responses. This limitation affects the verifiability of the information and the
quality of the sources, essential aspects in the ethics of the research staff; using only the
current version of the ChatGPT tool would not be complying with the necessary
professional rigor of the academy. This aspect is also related to the biases that the texts
used by the chatbot may already have; a tool of this type is not yet trained to be able to
detect defects in this sense, it reproduces them without the essential critical thinking of
the users. humans.

4.1 Prompt design

A critical element in working with artificial intelligence chatbots like ChatGPT is the
appropriate and precise design of the requests or instructions given to the chatbot. This
is what is known as prompt engineering or design . In this sense, for good prompt
development it is important to explicitly declare two principles that we call the
establishment of a context and the questioning operation. We develop them a little
more below:

• Context . It consists of the basic idea that it is advisable to always provide a

description of the circumstances surrounding a given fact, situation or event so
that in the requests made to ChatGPT it is better understood what need for
information is being explained.
• Cross-question. It consists of establishing a dialogue with ChatGPT by asking
again to compare or to complete the first inquiry until obtaining the answer or set
of optimized answers that give us the optimal results expected.

4.2 SALSA framework

Therefore, taking into account the points mentioned above and the demonstrated quality
of some of the functionalities of this AI, our proposal is to use ChatGPT as a tool to
improve the application of SALSA in its different phases. Below, we show an outline of
the SALSA phases and the initial prompts with which we will use the AI (table 1).

1
2
SALSA framework phases and ChatGPT prompts for scoping reviews

Phase Bases for possible prompts

Search
• Ask ChatGPT to provide keyword suggestions to develop the search
equation based on the topics of our research.

1
3
 • Ask ChatGPT to review and eventually improve a search equation.

• Ask ChatGPT to propose a search equation based on the specific topics of

our research.

AppraisaL
• Ask ChatGPT to review the suitability of a candidate document to form
the evidence base, as to whether it focuses on the topic of our research.

Synthesis
• Ask ChatGPT to identify the main themes of each recovered document,
based on the file prepared by the research staff, as well as ask it to group
them by categories.

*Ask ChatGPT to make a summary of a certain number of words for each

of the articles that are part of the selection of articles to review.

Analysis • Ask ChatGPT to identify convergences and divergences of the files

according to the categories: object of study, objectives,
questions/problems/hypotheses, methodology, results and themes to help
us develop the structure of the narrative synthesis of the set of documents
selected for realize the state of the art.

Table 1 . Possible uses of ChatGPT to support literature review with the SALSA
framework . Source: self made.

The following section documents the complete step-by-step process of applying the
SALSA framework with the support of artificial intelligence through the ChatGPT bot.

5. Illustrating a scoping review : applying SALSA and ChatGPT step by step

In order to illustrate the entire process that we propose, let's assume that we want to
carry out an exploratory systematic review (Codina et al., 2021) to carry out a state of the
art on artificial intelligence and new journalistic formats.

To do this, we will analyze the scientific production on these two topics applying the
SALSA framework (Hart, 2008; Grant and Booth, 2009; Booth et al. 2012) with the
support of a chatbot. To better specify the process, all elements of the SALSA phases are
included, even those that are resolved without using ChatGPT, that is, those that are
resolved without AI support. In this way, we show the complete process, both with
contributions from the AI and with the contributions of the research staff.

Below we present a more detailed explanation of the preceding phases to improve your
understanding.

1
4
5.1 Search phase

As we have already advanced in previous points, the first phase of SALSA focuses on the
selection of information sources and the design of the search equation and its application
in the selected databases, generally Scopus and Web of Science to be recovered. the
publications with the most impact in the academic environment, although it can also be
expanded to other sources such as Google Scholar, to give another example used to create
states of the art.

It is important to understand that, in the case of using academic databases (the most
common option), the search equation we use is one of the most sensitive elements when
carrying out a systematic review (Codina, 2020a). The reason is that if we leave out of the
search equation some essential or very representative keyword of the topic we want to
study, it can invalidate the entire study carried out. The same thing happens if we misuse
the Boolean operators with which the keywords are combined.

It is for this reason that we consider ChatGPT to be a good starting point for the first
design of the search equation or for its verification. Specifically we can use ChatGPT to:

• provide keyword ideas to develop the search equation;

• that helps improve a search equation already designed by us;
• that offers a search equation taking into consideration the topics of our research.

The first thing we have to do is access ChatGPT and register. To do this we must access
through the following link: https://openai.com/blog/chatgpt/

@OpenA API RESEARCH BLOG ABOUT

ChatGPT: Optimizing Language Models for Dialogue

We've trained a model called ChatGPT which interacts in a

conversational way. The dialogue format makes it possible for
ChatGPT to answer followup questions, admit its mistakes, challenge
incorrect premises, and reject inappropriate requests. ChatGPT is a
sibling model to InstructGPT, which is trained to follow an instruction
in a prompt and provide a detailed response.

Image 1 . Screenshot of the Open.AI

home page.

Next, once we have entered ChatGPT we will have access to the user interface of this AI.
In the search box, which is located at the bottom (see image 2) will be where we make
requests or prompts to ChatGPT.

1
5
Image 2. Screenshot of the main page of the ChatGPT user interface.

In what follows we will begin with the process of designing the search equation. In this
case, as we have already indicated, we illustrate the three possible proposals in the
following order: (1) ideas to create a search equation with ChatGPT, (2) improvement of
the search equation already created and (3) help for the construction of the search
equation.

Therefore, we can ask ChatGPT to offer us keyword ideas on the topic we want to study
(see image 3).

1
6
 I want to do a Boolean search to find a document on artificial intelligence, journalism and new
journalistic formats, offer me other query words that can help me develop my search equation.
Offer me the result organized by the three proposed topics and also give them to me in English

[ Artificial intelligence:

'Machine learning
'Deep learning
'Natural language processing
'Computer vision
' Robotics

Journalism:

■ News media
'Digital journalism
'Data journalism
'Investigative journalism
'Citizen journalism

New journalistic formats:

'Online journalism
' Multi-media journalism
'Interactive journalism
'Virtual reality journalism
■ Mobile journalism

Image. 3. Screenshot of a question in ChatGPT designing the prompt to identify the

keywords of the specific topics of the scoping review. The ChatGPT responses are also
illustrated, disaggregated by the topics that represent the scoping review that is to be
developed. Finally, answers are requested in English since the search standard in the
databases that we will use (Scopus and Wos) is this language.

As shown in image 3, we can take into consideration the keywords obtained by ChatGPT
and use them to create a search equation.

It is important to keep in mind that all these terms are only a guide, and research
personnel must carefully study this list, and to do so:

• You must have your own knowledge about the subject to be studied;
• If necessary, you must evaluate the relevance of each term through trial and error
by testing searches in databases such as Web Of Science, Scopus, or similar.
Furthermore, it is important to take into account the ethical and transparency principles,
therefore, the result provided by ChatGPT will be the starting point for the development
of the equation.

Therefore, the research staff involved in the scoping review must, on the one hand, verify
ChatGPT's response and apply a critical analysis and, on the other hand, document the

1
7
process in the methodology or methods and instruments section, to apply the principle
of scientific transparency.

The second alternative that the research staff has in this phase of SALSA is to improve a
search equation already created by the research staff (see image 4).

Let's imagine that after our analysis of keyword ideas offered by ChatGPT we have
decided that our search equation is:

("artificial intelligence" OR "neural networks" OR "machine learning" OR "deep learning"

OR algorithms) AND (journalism OR newspapers OR "news outlets" OR "news media")

We can ask ChatGPT to improve it as shown in image 4.

Improve this search equation that is optimized to find documents about artificial intelligence and
journalism: ("artificial intelligence” OR "neural networks" OR "machine learning" OR "deep
learning" OR algorithms) AND (journalism OR newspapers OR "news outlets" OR “news media”

EN The following search formulation may be more effective:

("artificial intelligence" OR "neural networks" OR "machine learning" OR "deep learning" OR
algorithms) AND (journalism OR "news media" OR "news outlets")

Image 4. Capture of the prompt on improvement of a specific search equation developed

by the research staff carrying out the scoping review

Again, the improvement proposed by ChatGPT must be validated by the research staff
through trial and error testing by passing the search equation through the scientific
databases that will be used. In addition, the result of ChatGPT must be collected to
continue working with transparency.

Finally, we can ask ChatGPT to help us build the search equation (see image 5.)
E Offer me a Boolean search equation in English that allows me to find documents on artificial
intelligence, journalism and new journalistic formats.
The search equation must also contain synonyms.

2 Here's a Boolean search equation for finding documents about artificial intelligence, journalism, and
new journalism formats:

("artificial intelligence" OR Al OR machine learning) AND ("journalism" OR "news reporting" OR

"mass media") AND ("new formats" OR "innovative formats" OR "modern journalism")

Image 5. Screenshot of the prompt where ChatGPT is asked to offer a search equation on
artificial intelligence, journalism and new journalistic formats.

It is important to thoroughly review the equation so that it does not contain any errors.
In this case, for example, the search equation it offers us is not entirely correct since the
query word “AI” and “machine learning” should be in quotes.

1
8
In short, the final search equation must be developed and validated by the research staff.
And it must also be documented so that the total transparency of the applied process is
demonstrated, as required by research ethics.

Once we have the search equation created we will go to the selected databases, in our
case Web Of Science and Scopus, we will make the query applying the search equation
and we will download the documents obtained (see image 6 and image 7)

Image 6. Capture of the cover of the results downloaded in Web of Science.

Scopus

Documents

Export Date: 07 Dec 2022

Search: TITLE(("artificial intelligence" OR "neural networks" OR "ma...

1) Tejedor, S.
ARTIFICIAL INTELLIGENCE AND NEWSGAMES IN JOURNALISM Proposals and ideas from the case study of three projects
[Article@INVESTIGAR SOBRE ARTIFICIAL INTELLIGENCE AND NEWSGAMES IN JOURNALISM Case study of three
projects]
(2022) VISUAL Review. International Visual Culture Review / International Culture Magazine, 9. .

Image 7. Screenshot of the cover of the results downloaded in Scopus.

To know the complete process on using Web Of Science and Scopus, it is recommended
to consult the Codina (2020a) manual.

5.2 Evaluation phase (AppraisaL)

As we have already explained previously, the evaluation phase is a stage through which
the research staff examines the documents obtained. This first evaluation is resolved by
reading exclusively the title and summary.

1
9
This evaluation first detects false positives, that is, those documents recovered through
the databases consulted that do not actually correspond to the project topics. This is due
to common problems of false keyword coordinations. The evaluation also consists of the
application of other inclusion/exclusion criteria, which may involve aspects such as the
type of documents (book chapters or communications may be excluded, as reports of
research results are preferred), thematic areas (for the same keywords). we can obtain
articles from very distant disciplines), use of clearly specified methodologies, etc.
(Codina, 2020b).

In this second section of using ChatGPT we focus on the identification of false positives.
To do this, we can ask ChatGPT if each of the recovered documents (see image 8) focuses
on the topic we want to study (see image 9) or not (see image 10).

2
0
Image 8. Capture of the collection of PDF documents recovered after applying the search
equation proposed for our scoping review on artificial intelligence and new journalistic
formats.

Let's imagine that this is our set of documents (see image 8) the result of our search in
Scopus and Web of Science with its corresponding search equation.

We would open each document, copy the abstract of the document, paste it into the
ChatGPT search box and ask it to verify if the document should be part of the corpus we
want to investigate (see image 9) or if it is a false positive. (see image 10) and then we
have to consider if we should discard it.

2
1
 3 You can confirm with a yes or no and justify your answer if this summary has to do with artificial
intelligence and journalism: "Technology and emerging user profiles have opened up interesting
scenarios for the ideation of new journalistic and communicative formats. Artificial Intelligence
(Al) and gamification proposals boost the creation of informative pieces that exploit the
possibilities of cyberspace and, at the same time, adapt to new audiences. Based on three
research projects and using a methodology based on case studies, the study highlights the fact
that these technological developments make it possible to renew the usual formats of journalism
and concludes that they have great informative, educational and awareness-raising potential."
Give me the answer back in Spanish

[ Yes, this summary has to do with artificial intelligence and journalism. The text refers to how
technology and emerging user profiles have created interesting scenarios for the creation of new
journalistic and communication formats. It is specifically mentioned how artificial intelligence
(AI) and gamification proposals drive the creation of informative pieces that exploit the
possibilities of cyberspace and, at the same time, adapt to new audiences. The research is based
on three research projects and uses a case study-based methodology. The study highlights the
fact that these technological developments make it possible to renew the usual formats of
journalism and concludes that they have great informative, educational and awareness-raising
potential.

Image 9. Screenshot of the Prompt where ChatGPT is asked to confirm whether the
attached text addresses the topics of the research project (artificial intelligence and
journalism). The result confirms yes.

2
2
 m You can confirm with a yes or no and justify your answer if this summary has to do with artificial
intelligence and journalism: The main objective of this work is to carry out a comparative analysis
between thirteen digital media websites that have won the Online Journalism Awards (ONA) and
the World Digital Media Awards (WDMA) regarding the dimensions of interactivity, searchability
and web visibility, through the application of an analysis protocol with nine parameters designed
specifically for digital journalism (Santos-Hermosa et al. 2021;2022). In this sense, it is an
innovative methodological proposal applied to a real case study in digital communication. The
results show different levels of achievement of the dimensions of interactivity, navigability and
visibility in the awarded digital media; that allows establishing a ranking from highest to lowest
score. Some outstanding trends are also identified with respect to each dimension of study. This
shows a series of successful strategies and linked weaknesses to interactivity, searchability and
web visibility. The application of the protocol turns out to be useful not only for researchers but
also for digital journalism professionals, providing data for the creation of new services and
products focused on the user experience"

[9 No. This summary has nothing to do with artificial intelligence and journalism. This is a comparative
analysis between award-winning digital media websites and their level of interactivity,
searchability and visibility on the web, through the application of an analysis protocol with nine
parameters designed specifically for digital journalism.

Image 10. Screenshot of the prompt where ChatGPT is asked to confirm whether the
attached text addresses the topics of the research project (artificial intelligence and
journalism). The result this time is negative.

Once the false positives are identified, the research staff is obliged to verify that said
documents are indeed false positives. Once again we want to emphasize that, for
transparency, we should document the entire process followed in the evaluation phase
in which ChatGPT has participated.

The gain in effectiveness is double. Evaluating documents to distribute them into two
sets, that of false positives and that of true positives, consumes more time than verifying
that distribution. Since we entrusted the first task to ChatGPT, then we are making
optimal use of resources. The second effectiveness is that in this first phase the human
researcher only has to verify the false positives.

Once we have the final set of documents, it is time to analyze each of the documents in
the form of a systematic file.

5.3 Analysis phase

Data extraction for the analysis matrices or tables of each document that forms the basis
of the evidence must be carried out by the research staff and, therefore, should not be
replaced by artificial intelligence. This work must be developed by the research staff and
not by the AI, which gives it a scientific rigor, which at least, for the moment, ChatGPT
cannot offer.

Now, there are some aspects in which artificial intelligence can be a good support. More
specifically in the identification of the main themes of each of the sheets and their
thematic organization. Our proposal revolves around this process.

2
3
The first thing we will have to do is offer ChatGPT each of the sheets (see table 2) that we
will have filled out. In the file we will incorporate:

1. An ID: reference number of the recovered document and its name in the form of a
bibliographic reference
2. Object of study: variable focused on identifying the object of study of the
recovered document
3. Objectives: variable focused on describing the research objectives of the
recovered document
4. Questions/problems/hypotheses: variable focused on locating the research
questions, problems or hypotheses of the retrieved document. We may not find
these items. If this circumstance occurs, we will indicate “not applicable.”
5. Methodology: variable focused on identifying the methodology or methodologies
used in the recovered document
6. Results: parameter focused on describing the most significant results of the
recovered document
7. Tags: space designed to incorporate the keywords of the recovered document. If
the keywords are not indicated in the document, we will indicate “not applicable.”
8. Categories (or topics): the categories are the main topics into which we will
group the entire document bank. On the other hand, themes define the content of
each document. We will leave this section empty so that we can process it with
ChatGPT

ID 1: Codina, L., Lopezosa, C., & Rovira, C. (2022). Journalism and algorithms: from the
information age to the algorithm age. Information Sciences Documentation , 45 (1),
1-5.

Parameter Description Categories

Tags
(Themes)
(keywords)

Study object The object of study of this text is the Journalism,

influence of algorithms on society and algorithms,
the journalistic business in the current search engines,
era journalistic SEO,
news
Goals The objectives are to explain why it is recommendation
considered that we are in the era of systems,
algorithms and to highlight how specific
examples such as search engines and
recommendation systems in

2
4
 Social networks show the making of news
decisions that were previously human by aggregation,
algorithms. journalist robot

Questions/ Does not apply

Problems/
Hypotheses

Methodology
The methodology consists of describing
the evolution of Internet search and
Google's role as the most influential
search engine in the West and how its
ranking algorithm uses more than 200
factors to establish the importance of a
page for a given topic.

Results
The most notable results are that
algorithms have a growing influence on
society and the journalistic business and
that search engines and recommendation
systems on social networks are
persuasive examples of the influence of
algorithms on human decision making.

Table 2. Data collection sheet for the synthesis phase for one of the recovered
documents. The category and topics tab remains uncompleted since this element will be
obtained from ChatGPT.

Below is the process to identify topics and categories with ChatGPT. To do this, first of
all, we will select the content of each of the files and we will ask ChatGPT to offer us the
main topic of the file.

2
5
 Offer me the main topic of this text in a maximum of 10 words: "The object of study of this text is
the influence of algorithms on society and the journalistic business in the current era.

The objectives are to explain why it is considered that we are in the era of algorithms and to
highlight how specific examples such as search engines and recommendation systems in social
networks show the making of decisions that were previously human by algorithms.

The methodology consists of describing the evolution of Internet search and Google's role as the
most influential search engine in the West and how its ranking algorithm uses more than 200
factors to establish the importance of a page for a given topic.

The most notable results are that algorithms have a growing influence on society and the
journalistic business and that search engines and recommendation systems on social networks
are persuasive examples of the influence of algorithms on human decision making."

5) Influence of algorithms on society and journalism.

Image 11. Capture of the prompt where ChatGPT is asked to tell us the main topic that
corresponds to the attached file. In this case, it is indicated that the main topic of the ID1
sheet is “Influence of algorithms on society and journalism.

This process must be verified by the research staff, who will be responsible for deciding
the main topic of each document. The process of identifying themes must be repeated for
each of the files we have, that is, one for each recovered document.

Next, we need to group all those topics into categories. Let's imagine that we have ten
documents and that they have been themed as follows, after a process of verification and
agreement by the research staff:

ID 1: Influence of algorithms on society and journalism

ID 2: Study on the evaluation of causes of suicide with AI

ID 3: Perception of AI in journalism

ID 4: Artificial intelligence, television, misinformation

ID 5: Automated journalism and artificial intelligence

ID 6: Automated journalism, algorithms, journalists' opinions

ID 7: Studies on automated journalism

ID 8: Studies on journalism and supervised machine learning (SML)

ID 9: Implementation of AI in journalism

ID 10: Computational news and journalistic routines

In this case we will ask ChatGPT to categorize them by topics. Normally we will find that
some topics are left ungrouped, in that case, it is advisable to ask ChatGPT to group them
into a category called others (see image 12).

2
6
is Group these ideas into 5 categories. If any idea cannot be grouped, include a category called 'other 1
':
"ID 1: Influence of algorithms on society and journalism
ID 2: Study on the evaluation of causes of suicide with AI
ID 3: Perception of AI in journalism
ID 4: Artificial intelligence, television, misinformation
ID 5: Automated journalism and artificial intelligence
ID 6: Automated journalism, algorithms, journalists' opinions
ID 7: Studies on automated journalism
ID 8: Studies on journalism and supervised machine learning (SML)
ID 9: Implementation of AI in journalism
ID 10: Computational news and journalistic routines"

[S Categories:
1. Influence of AI on society and journalism (ID 1, ID 3, ID 5)
2. Automated journalism and algorithms (ID 5, ID 6, ID 7, ID 9)
3. Studies on journalism and AI (ID 2, ID 7, ID 8)
4. Television and misinformation (ID 4)
5. Others (ID 10)

Image 12 . Screenshot of the Prompt where ChatGPT is asked to create 5 categories

based on the topics of our ten documents and group them. The number of categories can
vary according to the needs of the research staff, and ChatGPT can even be asked not to
limit the number of categories.

Next, and after verification by the research staff of the results obtained by ChatGPT, the
result of document ID1 would be as shown in table 3.

This process must be adjusted to each of the tokens in this case from ID2 to ID10.

2
7
ID 1: Codina, L., Lopezosa, C., & Rovira, C. (2022). Journalism and algorithms: from the
information age to the algorithm age. Information Sciences Documentation , 45 (1), 1-5.
Parameter Description Categories
Tags
(Themes)
(keywords)

Study object The object of study of this text is the Journalism, Influence of AI
influence of algorithms on society and algorithms, on society and
the journalistic business in the current search engines, journalism
era journalistic SEO, (Influence of
news algorithms on
Goals
The objectives are to explain why it is recommendation society and
considered that we are in the era of systems, news journalism)
algorithms and to highlight how aggregation
specific examples such as search systems,
engines and recommendation systems journalist robot
in social networks show the making of
decisions that were previously human
by algorithms.

Questions/ Does not apply

Problems/
Hypotheses

Methodology
The methodology consists of a review
of the literature that describes the
evolution of Internet search and the
role of Google as the most influential
search engine in the West and how its
ranking algorithm uses more than 200
factors to establish the importance of
a page. for a given topic.

Results
The most notable results are that
algorithms have a growing influence
on society and the journalistic
business and that search engines and
recommendation systems on social
networks are persuasive examples of
the influence of algorithms on human
decision making.

Table 3 . Data collection form for the synthesis phase completed for one of the
recovered documents. After review by the research staff of the ChatGPT results, the
category tab is assigned the name “Influence of AI on society and journalism” and the
topic “Influence of algorithms on society and journalism.”

2
8
5.4 Synthesis phase (Synthesis)

Finally, synthesis includes presenting the results in a way that is more than the sum of
the parts (Codina, 2020c). These results usually take two formats: matrices or tables for
highly structured or tabular results, and narrative synthesis. The latter can be of three
main types:

• Aggregative: consists of presenting the contributions of each analyzed work, with

no more relationship than, where appropriate, the mere grouping according to
belonging to common themes or subthemes, if applicable.
• Integrative: consists of combining results from two or more different works due
to similarity, opposition or line of continuity by adding comments or
considerations.
• Interpretative: consists of contextualizing, comparing and presenting well-
founded critical evaluations of the analyzed results.

It is important to highlight that ChatGPT should not be asked to perform a narrative

synthesis of the analysis cards, not only because the narrative synthesis it offers may be
erroneous and, in some cases, very artificial (at least for the moment) but also because
the value of incorporating the perspective of the research staff into the synthesis will
make the final result more precise and of greater value and investigative rigor.

However, we can ask ChatGPT to identify convergences and divergences of the files to
help us prepare the narrative synthesis of the set of recovered documents (see image 13
and 14). More specifically taking into account:

Parameters:

1. The object of study

2. The objectives
3. The methodology
4. The questions/problems/hypotheses
5. The results

The categories:

1. Influence of AI on society and journalism

2. Automated journalism and algorithms
3. Studies on journalism and AI
4. Television and misinformation
5. Others

m Identify the convergences and divergences in the following documents and identify which id they
correspond to for both the convergences and the divergences: "ID1: Currently, the process of
evaluating the causes of suicide is highly subjective, which makes efficiency and precision difficult.
of efforts to prevent it. Artificial intelligence (AI) has been presented as a possible solution to
analyze large data sets and detect patterns related to suicide. In this study, AI tools are used to
analyze media texts (press and social networks) to extract information related to suicide, even
though these texts do not contain specific tags on the topic.
ID 2:
This article focuses on examining the research carried out in the field of artificial intelligence and
journalism between 2008 and 2019, with the aim of identifying the topics and contexts in which
these studies have been carried out.

2
9
 ID 3:
The opinion of the labor and academic sector on the use of artificial intelligence in the field of
journalism is studied
ID 4:
Study the project ''A European Perspective", an online news exchange service developed within
the scope of the EBU (European Broadcasting Union). ID 5:
Explore how journalism companies can adopt Artificial Intelligence and algorithms as a tool to
process large volumes of data (big data), as a new source of information for journalistic stories.
ID 6:
The impacts of automated news on media professionals. ID 7:
Studies on automated journalism from academia
ID 8:
journalism studies using supervised machine learning (SML) ID 9:
The use of artificial intelligence in the journalistic field
ID 10:
"Understand the needs of journalists when using artificial intelligence tools in their daily work for
news creation"

Image 13. Prompt where ChatGPT is asked to offer convergences and divergences
taking into consideration the object of study of the ten documents recovered from our
scoping review.
2 Convergences:
‘ ID1 and ID5: both mention the use of artificial intelligence in the journalistic field to process
large volumes of data.
‘ ID2 and ID7: both cover studies on artificial intelligence in journalism.
‘ ID8 and ID9: both cover the use of artificial intelligence in the journalistic field.

Divergences:

‘ ID1 and ID3: the first focuses on the use of artificial intelligence to evaluate causes of suicide
through media texts, while the second focuses on the opinion on the use of AI in the field of
journalism.
‘ ID4 and ID6: the first focuses on the "A European Perspective" project, while the second on
the impacts of automated news on media professionals.
‘ ID5 and ID10: the first focuses on how journalism companies can adopt artificial intelligence,
while the second on the needs of journalists when using AI tools.

Image 14. Results of the Prompt developed on convergences and divergences taking
into consideration the object of study of the ten recovered documents.

As we have indicated in the previous phases, the results obtained through ChatGPT
cannot be treated as final results but as a proposal to be studied by the research staff
involved in the scoping review.

After the explanation of how to integrate ChatGPT to SALSA, we believe it is convenient

to offer a summary table where the central points discussed throughout the process are
found.

3
0
Summary of the SALSA framework combined with ChatGPT

Proposal for prompts with

Phase Task Results
questions

Search Choice of database:

Determine sources of Does not apply. It is resolved by the
Scopus and Web of
inquiry research staff (humans).
Science

Determine search ("artificial intelligence"

equation • “I need a Boolean equation to find OR "neural networks" OR
documents about artificial intelligence "machine learning" OR
and new journalistic formats. Also "deep learning" OR
suggest other keywords that algorithms) AND

3
1
 can help develop the search equation. It (journalism OR
offers the result organized by the three newspapers OR "news
proposed topics with keywords in outlets" OR "news
English.” media")

• “Can you improve this search equation

to find documents about artificial
intelligence and journalism: ("artificial
intelligence" OR "neural networks" OR
"machine learning" OR "deep learning"
OR algorithms) AND (journalism OR
newspapers OR "news outlets" OR
"news media")?”

• “Propose a Boolean search equation

with keywords in English that allows
finding documents on artificial
intelligence and new journalistic
formats. The search equation must also
contain synonyms.”

A ppraisa L Establish Does not apply. It is resolved by the

inclusion/exclusion research staff (humans). Inclusion/exclusion
criteria criteria used: articles with
IMRyD structure (Codina,
2021) or similar (explicit
method and presentation
of results)

Ensure that the “Can you confirm with a yes or no, and
Number initial of
selection responds to justify your answer, if this summary has
documents (N): this result
the requirements of the to do with artificial intelligence and
will depend on each
study journalism?”
scoping review

Final number (N) after

Observation: it is appropriate to copy
identifying false positives
the original summary of the article.
and applying
inclusion/exclusion
criteria: documents (N):
this result will depend on
each scoping review

Synthesis Integration of main Synthesis narrative and

“Indicate what the main topic of this text data tables from the
results
is in a maximum of 10 words” analysis
[At this stage, human researchers attach made for each document
and
text resulting from each summary sheet
ID of the
developed by the researchers so that the
topics and categories
chatbot processes and synthesizes it].
through ChatGPT and

3
2
 *We ask again its verification by the
researchers.
“Group these ideas by topics, include the
IDs and if any idea cannot be grouped in
a topic, assign it to "others"

A analysis Integration of main Texts organized by

results “Identify the convergences and similarities and
divergences in the following documents” divergences, which must
be reviewed and
[Procedure: The human attaches all the verified by the
texts to facilitate the extraction of data human researchers.
by themes or by parameters such as
object of study, objectives,
questions/problems/hypotheses,
methodology or results

Table 4. Adaptation of SALSA framework with a ChatGPT application process. Own

elaboration.

6. Effects of scale, practicality and connectivity

Our proposals, although extremely practical (or at least, this is what we have intended),
do not take into account scale effects. That is, we are moving in a conceptual terrain, and
we are examining and testing the effectiveness of the use of ChatGPT in each and every
one of the SALSA phases from the point of conceptual operation. Our initial tests
demonstrate that the use of this AI in systematic reviews, especially scoping reviews , is
feasible and possibly sustainable.

We leave aside, in this phase of our work, the practicality of ChatGPT in a scale-
independent manner. That is, for scales of a few dozen documents (on the order of 20-
60, to say the least) it seems to be viable.

We do not go into the practicality problems that may appear on scales of hundreds or
thousands of documents. These are unusual scales in scoping reviews , but they may
appear in some systematic reviews.

It is in this area that it is necessary to establish the difference between practical

conceptual viability demonstrated on a specific scale, and its viability at any scale. The
reason is that it is first necessary to demonstrate practical conceptual viability if at some
point we wish to have viability at all scales.

It is in this sense that we refer to connectivity. Certainly in the medium term, and
possibly in the short term, AIs in general, and specifically chatbots, will be connected
through APIs or other forms of integration to other services and, specifically, to office
applications.

In this short/medium term, the practicality of the proposed operations at higher scales
can be resolved by integrating ChatGPT into Microsoft

Office, or from other AI to Google Drive, etc. This is why we must insist on the separation

3
3
between the practical/conceptual viability of our proposals that seems to us to be
reasonably established from their practicality at any scale.

7. Conclusions
In this report we have demonstrated how artificial intelligence, in particular the
ChatGPT language model, can be used as an effective tool in the process of developing
systematic reviews to generate state-of-the-art academic literature.
Likewise, we have emphasized the importance of assuming responsibility in the use of
artificial intelligence, and ethics in research, following the ideas presented by recognized
authors such as Van-Dis et al. (2023) and Codina (2022).
In short, it is essential to highlight that the use of ChatGPT in the preparation of
systematic reviews must always be under a focus on investigative transparency,
integrity, ethics, critical thinking and, ultimately, responsible use. Like any tool and
procedure that is part of the research method.
This responsibility implies using the results generated by ChatGPT as a tool to improve
systematic review processes, but not as definitive results without verification. Therefore,
they should be considered only as a starting point that should be critically questioned by
research staff.

8. Bibliographic references
• Arksey, Hilary; O'Malley, Lisa (2005). «Scoping Studies: Towards a
Methodological Framework». Int. J. Social Research Methodology 8:1, 19-32,
DOI: 10.1080/1364557032000119616 | Repository version

• Booth, Andrew; Papaionnou; Sutton, Anthea . Systematic Approaches to a

Successful Literature Review . London: Sage, 2012.
• Codina, Lluís (2020a). “Systematized bibliographic reviews in Human and Social
Sciences. 1: Fundamentals”. In: Lopezosa C, Díaz-Noci J, Codina L, editors
Methodos Yearbook of Research Methods in Social Communication , 1. Barcelona:
University Pompeu Fabra; 2020. p. 50-60.
DOI: 10.31009/methodos.2020.i01.05
• Codina, Lluís (2020b). “Systematized reviews in Human and Social Sciences. 2:
Search and Evaluation”. In: Lopezosa C, Díaz-Noci J, Codina L, editors Methodos
Yearbook of Research Methods in Social Communication, 1. Barcelona:
University Pompeu Fabra; 2020. p. 61-72.
DOI: 10.31009/methodos.2020.i01.06
• Codina, Lluís (2020c). “Systematized reviews in Human and Social Sciences. 3:
Analysis and Synthesis of qualitative information.” In: Lopezosa C, Díaz-Noci J,
Codina L, editors Methodos Yearbook of Research Methods in Social
Communication , 1. Barcelona: Pompeu Fabra University; 2020. p. 73-87. DOI:
10.31009/methodos.2020.i01.07

• Codina, Lluís (2020d). Databases for research in social communication:

characterization of the optimal group.https://www.lluiscodina.com/bases-de-
datos- comunicacion-social/
• Codina, Lluís (2021). What is a scientific article? IMRyD model: structure, components

3
4
 and meaning https://www.lluiscodina.com/modelo-imryd/
• Codina, Lluís (2022) “How to use ChatGPT in the classroom with an ethical
perspective and critical thinking: a proposal for teachers
and educators,
lluiscodina.com
• Fernández-Sánchez, Higinio; King, Keith; Enríquez-Hernandez, Claudia
Beatriz (2020). «Exploratory Systematic Reviews as a methodology for the
synthesis of scientific knowledge». University Nursing · Vol.17 · No. 1 · January-
March
• Grant, Maria J.; Booth, Andrew . «A typology of reviews: an analysis of 14
review types and associated methodologies». Health Information and Libraries
Journal , 26, pp.91–108, 2009.
DOI: 10.1111/j.1471-1842.2009.00848.x
• Guida, G.; Mauri, G. (1986). "Evaluation of natural language processing systems:
Issues and approaches." Proceedings of the IEEE. 74 (7): 1026–1035.
doi:10.1109/PROC.1986.13580
• Lopezosa Carlos; Díaz-Noci Javier; Codina, Lluís (2020) Methodos Yearbook of
Research Methods in Social Communication , 1. Barcelona: Universitat
Pompeu Fabra; 2020. p. 73-87.
• Lopezosa, Carlos; Codina, Lluís (2023). ChatGPT and CAQDAS software for
qualitative interview analysis: steps to combine OpenAI artificial intelligence with
ATLAS. you, Nvivo and MAXQDA
• Spotted, Dew; Tamames, Sonia; López-González, María; Mohedano, Laura;
Dágostino, Marcelo; See, Jorge. (2009). «Exploratory Systematic Reviews». Med
Segur Trab (Internet) 55 (216): 12-19)
• Mengist, Wondimagegn; Soromessa, Teshome; Legese, Gudina (2020).
«Method for conducting systematic literature review and meta-analysis for
environmental science "research". MethodsX , 7
DOI: 10.1016/j.mex.2019.100777
• Munn, Zachary; Peters, Micah; Stern, Cindy; Tufanaru, Catalin; McArthur,
Alexa; Aromataris, Edoardo (2018). «Systematic review or scoping review?
Guidance for authors when choosing between a systematic or scoping review
approach. BMC Medical Research
Methodology. https://doi.org/10.1186/s12874-018-0611-x
• OpenAI (2022) https://openai.com/blog/chatgpt/
• Peters, Micah; Godfrey Christina; Khalil, Hanan; McInerney, Patricia; Parker,
Deborah; Soares, Cassia (2015). «Guidance for conducting systematic scoping
reviews». International Journal of Evidence-Based Healthcare: September 2015 –
Volume 13 – Issue 3,141-146. doi: 10.1097/XEB.0000000000000050
• Peters, Micah.; Marnie, Casey; Tricco, Andrea; Pollock, Danielle; Munn,
Zacary; Alexander, Lyndsay; McInerney, Patricia; Godfrey, Christina; Khalil,
Hanan (2020). «Updated methodological guidance for the conduct of scoping
reviews». JBI Evidence Synthesis : October 2020, Volume 18, Issue 10, p 2119-
2126 doi: 10.11124/JBIES-20-00167
• Tricco, Andrea.; Zarin, Wasifa; Ghassemi, Marco.; Nincic, Vera; Lillie, Erin;
Page, Matthew; Shamseer, Larissa; Antony, Jesmin; Rios, Patricia; Hwee,
Jeremiah Angeliki, Areti, Moher, David; Hartling, Lisa; Pham; Ba; Straus,
Sharon (2017). «Same family, different species: methodological behavior and
quality varies according to purpose for five types of knowledge synthesis»
Journal of Clinical Epidemiology , Volume 96,

3
5
 133 –
142. https://doi.org/10.1016/j.jclinepi.2017.10.014
• Tricco, Andrea; Lillie, Erin; Zarin, Wasifa; O'Brien, Kelly; Colquhoun,
Heather; Levac, Danielle; Moher, David; Peters, Micah; Horsley, Tanya;
Weeks, Laura; Hempel, Susanne; Akl, Elie; Chang, Christine; McGowan,
Jessie; Stewart, Lesley; Hartling, Lisa; Aldcroft, Adrian; Wilson, Michael;
Garritty, Chantelle; Lewin, Simon; Godfrey, Christina; Macdonald, Marilyn;
Langlois, E tienne ; Soares-Weiser, Karla; Moriarty, Jo; Clifford, Tammy;
Tunçalp, Özge; Straus, S haron (2018). «PRISMA Extension for Scoping Reviews
(PRISMA-ScR): Checklist and Explanation». Annals of Internal Medicine , 04
September 2018. doi:10.7326/M18-0850
• Van-Dis, Eva A. M.; Bollen, Johan; Zuidema, Willem, van Rooij, Robert, &
Bockting, Claudi L . (2023). ChatGPT: five priorities for research. Springer
Nature, vol.614, 224-226 https://www.nature.com/articles/d41586-023-00288-
7

3
6