Accountability in Research

Ethics, Integrity and Policy

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/gacr20

Can ChatGPT be trusted to provide reliable

estimates?

Panagiotis Tsigaris & Jaime A. Teixeira da Silva

To cite this article: Panagiotis Tsigaris & Jaime A. Teixeira da Silva (2023): Can
ChatGPT be trusted to provide reliable estimates?, Accountability in Research, DOI:
10.1080/08989621.2023.2179919

To link to this article: https://doi.org/10.1080/08989621.2023.2179919

Published online: 21 Feb 2023.

Submit your article to this journal

Article views: 905

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=gacr20
ACCOUNTABILITY IN RESEARCH, 2023
https://doi.org/10.1080/08989621.2023.2179919

LETTER TO THE EDITOR

Can ChatGPT be trusted to provide reliable estimates?

a
Panagiotis Tsigaris and Jaime A. Teixeira da Silvab
a
Department of Economics, Thompson Rivers University, Kamloops, BC, Canada; bIndependent
researcher, Ikenobe, Japan
KEYWORDS Artificial intelligence (AI); ethics; knowledge; transparency; trust
Dear Accountability in Research Editors,
We support Hosseini, Rasmussen, and Resnik (2023). regarding the risks
associated with text written by artificial intelligence (AI), such as OpenAI’s
ChatGPT, that are challenging fundamental aspects of academic publishing
like authorship, bias, relevance, accuracy and accountability. Some research­
ers, including those for whom English is a second language, might rely on
translation software or editing firms to assist them with scientific writing,
assistance that is typically acknowledged in a paper. Disclosure is essential so
that the human authors who employ AI can be held accountable when
software fails or generates incorrect or false output. Hosseini, Rasmussen,
and Resnik (2023). allude to the falsification of ideas, i.e., plagiarism, when
ideas sought by AI-driven software like ChatGPT might draw on the ideas of
others, but are then used to claim originality. Typically, scientists support
their work’s claims with relevant references from the scholarly literature.
We address a number of issues in this letter based on an attempt to obtain
information from ChatGPT about the surface area of lakes worldwide.
Interested in the globe’s balance of water supply, as part of our wider interest
in climate change and environmental issues, and the importance of water for
sustainable human development, we wanted to know if ChatGPT could
provide us with an estimate of the area of lakes around the world, in km2
or hectares (ha). We wanted this value to provide an estimate of the value of
ecosystem services, as was performed by Costanza et al. (2014), who esti­
mated the value of rivers and lakes at US$2.5 trillion using 2007 prices based
on 2 million km2 (200 million ha) and a price of 12,502 US$/ha/year
(Costanza et al. 2014). Costanza’s estimated area was based on the
GlobCover data set from the European Space Agency and the UN Food
and Agriculture Organization. However, Messager et al. (2016) reported
values from six studies of lakes with a size≥0.01 km2 (≥1 ha) since 2004
exceeding 200 million ha (Messager et al. 2016). The average value was
3.56 million km2 with a minimum and maximum valuation at [3.1, 4.76
km2]. Verpoorter et al. (2014) provided the highest valuation at 4.76 million
km2 (476 million ha) (Verpoorter et al. 2014). Messager et al. (2016)

CONTACT Panagiotis Tsigaris ptsigaris@tru.ca Department of Economics, Thompson Rivers University, 805
TRU Way, Kamloops, BC V2C 0C8, Canada
© 2023 Informa UK Limited, trading as Taylor & Francis Group
2 P. TSIGARIS AND J. A. TEIXEIRA DA SILVA

Figure 1. A query by the first author to ChatGPT (OpenAI 2023,) requesting information about
the value of the surface area of freshwater lakes and lakes globally. That query required four
consecutive questions in order to arrive at “scientific literature,” according to ChatGPT, that it
claimed supported its data. Ultimately, no scientific literature was provided to support the claims,
while the estimate was inaccurate.
 ACCOUNTABILITY IN RESEARCH 3

estimated the surface area of all lakes at 3.23 million km2 (i.e., 323 million ha)
(Messager et al. 2016).
We wanted to test if ChatGPT could replicate this knowledge and offer
a value within the above range (Figure 1). Unfortunately, it did not. In
response to five queries to ChatGPT (OpenAI 2023) not only was it unable
to provide any of the above valuations, it was seemingly unaware of any of
the three articles we cite, which were published in respectable and ranked
peer-reviewed journals. Furthermore, ChatGPT also made an easy calculation
error when converting km2 to ha. Finally, ChatGPT was unable to, or refused
to, provide literature sources to support its claims, stating that it based the
estimate on “widely accepted scientific estimates and data” (Figure 1).
Apparently, ChatGPT needs to be fed this information, but why should
users be responsible for providing knowledge to AI?
These findings reinforce and amplify the concerns expressed by Hosseini
et al. about AI/ChatGPT (Hosseini, Rasmussen, and Resnik 2023).

Disclosure statement
No potential conflicts of interest was reported by the authors.

Funding
The author(s) reported there is no funding associated with the work featured in this article.

ORCID
Panagiotis Tsigaris http://orcid.org/0000-0002-7965-4554

References
Costanza, R., R. De Groot, P. Sutton, S. Van der Ploeg, S. J. Anderson, I. Kubiszewski,
S. Farber, and R. K. Turner. 2014. “Changes in the Global Value of Ecosystem Services.”
Global Environmental Change 26: 152–158. doi:https://doi.org/10.1016/j.gloenvcha.2014.04.
002.
Hosseini, M., L. M. Rasmussen, and D. B. Resnik. 2023. “Using AI to Write Scholarly
Publications.” Accountability in Research 1–9. doi:https://doi.org/10.1080/08989621.2023.
2168535.
Messager, M. L., B. Lehner, G. Grill, I. Nedeva, and O. Schmitt. 2016. “Estimating the Volume
and Age of Water Stored in Global Lakes Using a Geo-Statistical Approach.” Nature
Communications 7 (1): 13603. doi:10.1038/ncomms13603.
OpenAI. 2023. ChatGpt-3. January 30, 2023 Version. Free Version. Accessed 4 February,
2023. via first author’s account.
Verpoorter, C., T. Kutser, D. A. Seekell, and L. J. Tranvik. 2014. “A Global Inventory of Lakes
Based on High-Resolution Satellite Imagery.” Geophysical Research Letters 41 (18):
6396–6402. doi:10.1002/2014GL060641.