Educational Innovations
ChatGPT for Automated Writing Evaluation in Scholarly
Writing Instruction
Jessica L. Parker, EdD; Kimberly Becker, PhD; and Catherine Carroca, DHS, MSN, RN
ABSTRACT
Background: Effective strategies for developing schol-
arly writing skills in postsecondary nursing students are
needed. Generative artificial intelligence (GAI) tools, such
as ChatGPT, for automated writing evaluation (AWE) hold
promise for mitigating challenges associated with scholarly
writing instruction in nursing education. This article explores
the suitability of ChatGPT for AWE in writing instruction.
Method: ChatGPT feedback on 42 nursing student texts
from the Michigan Corpus of Upper-Level Student Papers
was assessed. Assessment criteria were derived from recent
AWE research. Results: ChatGPT demonstrated utility as an
AWE tool. Its scoring performance demonstrated stricter
grading than human raters, related feedback to macro-level
writing features, and supported multiple submissions and
learner autonomy. Conclusion: Despite concerns surround-
ing GAI in academia, educators can accelerate the feedback
process without increasing their workload, and students
can receive individualized feedback by incorporating AWE
provided by ChatGPT into the writing process. [J Nurs Educ.
2023;62(12):721-727.]
G
enerative artificial intelligence (GAI) in education recent-
ly has sparked widespread interest. In contrast to other
types of artificial intelligence (AI) used in education that
rely on machine learning algorithms or rule-based systems (e.g.,
Grammarly), GAI produces content learned from its training on
a large dataset. ChatGPT, a large language model developed by
OpenAI, is one example of GAI and has been touted to have the
potential to transform health care education, research, and prac-
tice (Sallam, 2023). Specific applications of ChatGPT in health
care education include personalized learning experiences in nurs-
ing education (O’Connor & ChatGPT, 2023), academic writing
Jessica L. Parker, EdD, MS, RDH, is a Lecturer, Massachusetts College of
Pharmacy and Health Sciences, School of Healthcare Business. Kimberly
Becker, PhD, is a Lecturer, English Department, Iowa State University.
Catherine Carroca, DHS, MSN, RN, is an Associate Professor, Department of
Nursing, Massachusetts College of Pharmacy and Health Sciences.
Address correspondence to Jessica L. Parker, EdD, MS, RDH, School of
Healthcare Business, Massachusetts College of Pharmacy and Health Sci-
ences, 179 Longwood Avenue, Boston, MA 02115; email: Jessica.parker@
mcphs.edu.
Disclosure: The authors have disclosed no potential conflicts of inter-
est, financial or otherwise.
Received: May 29, 2023; Accepted: July 3, 2023
doi:10.3928/01484834-20231006-02
Journal of Nursing Education • Vol. 62, No. 12, 2023
development (Kumar, 2023), and immediate feedback (van Dis
et al., 2023).
One promising area in nursing education is using GAI tools
such as ChatGPT for automated writing evaluation (AWE) in
writing instruction. AI applications have been used in higher
education for assessment, evaluation, and automated feedback
(Zawacki-Richter et al., 2019) and to support students’ academ-
ic writing development (Calvo et al., 2011). Student-centered
feedback with AWE has been found to help language learners
improve the accuracy of their writing and save instructors time
(Bailey & Lee, 2020). Further, chatbots have been shown to
grade similarly to human instructors on short writing assign-
ments (Ndukwe et al., 2019). Therefore, this article explores
the suitability of ChatGPT for AWE in writing instruction for
postsecondary nursing students.
Background
Scholarship for nursing practice is a competency domain
recognized by the American Association of Colleges of Nurs-
ing (AACN, 2021) in The Essentials: Core Competencies for
Professional Nursing Education (The Essentials). The Essen-
tials guides curricular development and learner achievement
expectations. Competencies included in the Scholarship for
Nursing Practice domain emphasize students’ ability to de-
scribe, articulate, examine, and integrate. Thus, written as-
signments should be designed to assess these crucial qualifiers
(Lim et al., 2023).
However, nursing students at all postsecondary education
levels struggle in meeting such expectations; moreover, they
face difficulties with organization, grammar, flow, sentence
structure, and critical analysis (DeCoux Hampton & Chafetz,
2021; Cone & Van Dover, 2012; McMillan & Raines, 2011).
Strategies such as peer review, self-assessment, and frequent
feedback have been shown to improve postsecondary nurs-
ing students’ writing proficiency (Bickes & Schim, 2010;
Hampton, 2019; Kilmer et al., 2023; Tornwall & McDaniel,
2022). However, providing thorough and frequent feedback is
time-consuming, posing challenges for instructors and students
(Bjerkvik & Hilli, 2019; Micsinszki & Yeung, 2021; Zhan et
al., 2023).
One possible way to mitigate the challenges associated with
student writing and faculty feedback is using GAI tools, such
as ChatGPT, for AWE. AWE systems originally focused on
analyzing local- or micro-level characteristics, such as gram-
mar, mechanics, lexis, and discourse (Attali & Burstein, 2006;
Deane, 2013; Deane & Quinlan, 2010). However, as AWE
systems have become more sophisticated, their foci increas-
ingly have shifted to global- or macro-level features, such as
organization, communicative goals, and genre characteristics.
721
 Further, AWE systems have
TABLE 1
been shown to be valid and
Input Prompt Sequence and Rubric Criteria
reliable based on studies
Writing
examining the rate of agree-
Construct Criteria Prompt and Description ment between human- and
machine-rated texts (At-
Complexity Thesis/ This input prompt includes a grading rubric with grading criteria and
tali & Burstein, 2006; Lee
purpose a range of scores for a paper written by a nursing student. Review
the paper in the next input prompt and evaluate the paper for the
et al., 2008, 2010; Shermis
thesis/purpose. Provide a score and explain the assigned score, and & Hamner, 2013; Wang &
also include suggestions for improvement with at least one specific Brown, 2007).
example from the paper if a score of 0 to 2 is achieved. Several AWE systems cre-
0 – Thesis is not provided or does not have a discernable central
ated in the past decade have
argument attempted to approach au-
tomation for genre-specific
1 – Thesis identifies a demonstrable central argument, although not
features of academic writing.
clearly stated
The first of these is the Re-
2 – Thesis statement clearly identifies a demonstrable central search Writing Tutor in the
argument United States (Cotos et al.,
3 – Clearly and eloquently identifies a demonstrable and nuanced 2015), which provides feed-
central argument; guides the reader smoothly and logically into the back on the various sections
body of the paper (i.e., introduction, methods,
Evidence Now, evaluate the same paper for evidence according to the results, discussion, and con-
provided criteria. Provide a score and explain the assigned score, and clusion) in academic journal
also include suggestions for improvement with at least one specific article manuscripts. A similar
example from the text if a score of 0 to 2 is achieved. Australian AWE, AcaWriter
0 – Little to no evidence is used throughout the paper (Knight et al., 2020), also in-
1 – Evidence is used to support the central argument, but the cludes genre-based feedback.
evidence is not clearly articulated AWE systems focusing on the
complexities of genre have the
2 – Evidence used to support the central argument is appropriate,
although not particularly rich or detailed; the connection between
potential to provide nursing
argument and evidence is clearly articulated student writers suggestions
without the involvement of
3 – Evidence used to support the central argument is rich, detailed,
human mentors, yet many—
and well chosen
like the two mentioned here—
Fluency Organization Now, evaluate the same paper for organization according to the are not publicly available.
provided criteria. Provide a score and explain the assigned score, and Besides rule- and genre-
also include suggestions for improvement with at least one specific
based tools for AWE, AI re-
example from the text if a score of 0 to 2 is achieved.
cently has become an avail-
0 – Organization of the paper is not logical or discernable able method for feedback. As
1 – Organization of the paper can only be discerned with effort; AI has accelerated dramatical-
only some parts of the paper are effectively integrated; in several ly in recent years, the devel-
paragraphs, there is no distinct or coherent point; topic sentences opment of programs such as
are missing or unclear in some paragraphs; in some paragraphs, the Grammarly, ProWritingAid,
parts do not connect logically and Writefull has optimized
2 – Organization of the paper is logical and apparent, but transitions the identification and correc-
between paragraphs are not consistently smooth; every paragraph tion of micro-level issues.
makes one distinct and coherent point, and for the most part, the Nevertheless, as technology
sentences of each paragraph connect logically and effectively; in all advances, so does its ability
but a few cases, the point of the paragraph is expressed in a clear to improve a variety of writ-
topic sentence ing criteria, such as cohesion,
3 – Organization of the paper is logical and quickly apparent; coherence, and argumenta-
connections among paragraphs are clearly articulated and tive quality. Likewise, AWE
transitions between paragraphs are smooth; every paragraph can improve the process (as
makes one distinct and coherent point, expressed in a clear topic opposed to only the product)
sentence; the parts of each paragraph connect logically, and internal of writing (Adams & Chuah,
2023; Strobl et al., 2019). The
transitions are smooth

availability of free, generative

pretrained transformers, such

722 Copyright © SLACK Incorporated

as ChatGPT, has put these
capabilities at the forefront of
AWE development. Although
ChatGPT’s applications have
been explored in the context
of health care practice (Patel
et al., 2023; Rao et al., 2023),
its potential for facilitating
writing development has yet
to be fully realized.
Method
TABLE 1 (CONTINUED)
Input Prompt Sequence and Rubric Criteria
Writing
Construct Criteria Prompt and Description
Clarity Now, evaluate the same paper for clarity according to the provided
criteria; provide a score and explain the assigned score; and also
include suggestions for improvement with at least one specific
example from the text if a score of 0 to 2 is achieved.
0 – Throughout the paper, the wording is imprecise or ambiguous;
the sentence structure is consistently confusing

To explore the suitability 1 – Wording is often imprecise or ambiguous; the sentence structure
of GAI for AWE, we utilized is often confusing
exemplary nursing student 2 – The paper is, for the most part, precisely worded and
papers from the Michigan unambiguous; the sentence structure is mostly clear
Corpus of Upper-Level Stu- 3 – Throughout the paper, the wording is precise and unambiguous;
dent Papers (MICUSP), a the sentence structure is consistently clear and lucid
broad source of success- Accuracy Mechanics Now, evaluate the same paper for writing mechanics according to
ful, multidisciplinary stu- the provided criteria; provide a score and explain the assigned score;
dent writing (O’Donnell and also include suggestions for improvement with at least one
& Römer, 2012; Römer & specific example from the text if a score of 0 to 2 is achieved.
O’Donnell, 2011). A total of 0 – The paper contains numerous spelling and grammatical errors,
42 texts were downloaded including incomplete and run-on sentences
from MICUSP, of which 18
1 – The paper contains several spelling and grammatical errors; there
were written by undergradu- are some incomplete or run-on sentences
ate nursing students and 24
by first- through third-year 2 – The paper contains minor spelling or grammatical errors; there
graduate nursing students. are minimal incomplete or run-on sentences
Each text was entered sepa- 3 – The paper contains no spelling errors, grammatical errors, or
rately into ChatGPT-3 fol- run-on sentences
lowing an input prompt that
included a rubric with con-
structs of language proficien- assigning a score of 0 to 2 and also provided guidance on
cy intended to represent aspects of complexity, accuracy, and how to improve the writing (Figure 1). ChatGPT responses
fluency (CAF), which have been commonly used as criteria for then were assessed based on its ability to provide feedback
assessing writing development (Ellis & Ellis, 2008; Housen & on macro-level features in writing. In this respect, ChatGPT
Kuiken, 2009). The six criteria and their association with CAF consistently identified and addressed broader aspects of the
are outlined in Table 1, which presents the input prompt and ru- writing rather than examining only surface-level grammar
bric criteria with a 4-point scale, ranging from 0 = unacceptable or sentence structure (Figure 2). ChatGPT also consistently
to 3 = meets or exceeds criteria. A paper with a score of 3 for provided specific suggestions for improvement using ex-
each criterion was considered an “A” paper. The following cerpts from the text. Notably, ChatGPT feedback contained
three criteria from recent AWE research (Zhai & Xiaomei, human-like commentary.
2021) were applied to explore its potential for AWE to ap- Finally, the ability of ChatGPT to support multiple submis-
proximate human raters (Shermis, 2014); relate to macro- sions and learner autonomy was assessed. When a score of 2
level features (e.g., organization and development [Burstein was assigned, users could ask clarifying questions for more
et al., 2013]); and support multiple submissions and learner detailed explanations. Further, users could implement the
autonomy (Wang et al., 2013). feedback and enter revised text or ask ChatGPT to explain
its score and provide additional feedback (Figure 3). In all
Results cases, ChatGPT provided suggestions for improvement and
After collecting ChatGPT feedback for the 42 papers, the answered clarifying questions. Table 2 presents excerpts
criteria mentioned above were applied to assess its forma- from the feedback aligned with best-practice criteria from
tive feedback potential. ChatGPT feedback was compared the literature.
first to the benchmark score of 3 (grade A) assigned by a
human rater. Overall, ChatGPT assigned a score of 3 to one Discussion
graduate nursing paper and a score of 2 (grade B) to all The assessment of ChatGPT’s feedback on undergradu-
other papers, indicating stricter grading. As the prompt in- ate- and graduate-level nursing texts demonstrated its po-
put requested, ChatGPT provided a detailed explanation for tential utility as a tool for AWE in writing instruction. The

Journal of Nursing Education • Vol. 62, No. 12, 2023 723


Figure 1. ChatGPT’s explanation for a score of 0 to 2.
Figure 3. ChatGPT’s response when asked, “Can you explain this score?”
results of this exploratory assessment are not surprising,
as AI writing tools have been used in various educational
contexts for assessment and feedback (Nazari et al., 2021).
However, the capabilities of GAI tools, such as ChatGPT,
extend beyond existing rule-based AI writing tools and
serve as conversational agents that can handle multi-turn
dialogue, maintaining context throughout the conversation.
724
Figure 2. ChatGPT’s feedback on macro-level features in writing.
From the perspective of CAF, complexity should always
be prioritized because finalizing macro-level issues in a
text is the most efficient way to edit and revise. If a writer
only makes changes that contribute to fluency or accura-
cy, as is likely with widely used programs such as Gram-
marly, the improvements may not be substantial enough
to achieve competency. Grammarly and other rule-based
AWEs can provide fluency and accuracy feedback, but only
the most advanced, proprietary AWEs have, up to now, been
able to provide complexity feedback. This is why GAI is
revolutionary.
It would be amiss not to acknowledge the ethical and
privacy concerns surrounding the use of GAI in education.
Concerns range from students using the technology to cheat
on examinations (Thorp, 2023) to plagiarized content in
journals (Zielinski et al., 2023). After a conversation with
ChatGPT, Gunter Eysenbaugh, the founder and editor of the
Journal of Medical Internet Research Medical Education,
expressed significant concern about ChatGPT’s tendency
to hallucinate—or produce a confident response that is not
justified by its training data (Eysenbach, 2023). However,
ChatGPT now includes a new “data control” feature that al-
lows users to exclude sensitive or private information that
should not be processed or shared, reducing the risk of pri-
vacy violations.
To date, GAI tools such as ChatGPT have yet to be ex-
plored for AWE purposes. By incorporating AWE provided
by ChatGPT into writing curricula, instructors can acceler-
ate the feedback process without increasing their workload,
and students can receive immediate feedback at any time
(Liao, 2016). However, the successful implementation of
any technological system depends on an improved under-
standing of the user acceptance processes (Davis, 1989). In
their 2023 op-ed for the Los Angeles Times, Duckworth and
Ungar stressed the necessity for educators to adapt by inte-
grating ChatGPT into their curricula. .
Limitations
The primary limitation of this exploratory test of ChatGPT
for AWE is that its use or acceptance with stakeholders, such
Copyright © SLACK Incorporated
as students or faculty, was
not investigated and the
rate of agreement between
ChatGPT and human rat-
ers also was not calculated.
Although ChatGPT’s scor-
ing performance did not
align with a human rater,
it is important to remem-
ber that a lack of alignment
does not necessarily imply
inadequate feedback. Fac-
ulty, who may have become
expert academic writers
through years of experience,
often have little training in
writing instruction (Mad-
son, 2021) and may struggle
to provide constructive feed-
back (Aguayo-Gonzalez et
al., 2020; Mitchell, 2018);
hence, AI tools potentially
could augment the role
of faculty in the feedback
process.
Learnings
ChatGPT is a new tech-
nology, and there is much to
learn about its applications
in nursing education. When
conducting the exploratory
assessment for this article,
it immediately became ap-
parent that users must know
how to ask the right ques-
tions to obtain the desired
TABLE 2
AWE Assessment Criteria and Feedback Examples: Complexity
Level-Specific Feedback Examplesa
AWE Feedback Criteria 2 3
Approximates human raters “The purpose statement is “The paper is well-formatted with no
(Shermis, 2014) clearly stated in the abstract and significant spelling or grammatical
introduction, and it gives the errors, and quotes are properly
reader a reasonably good sense of attributed and cited.”
the nature of the analysis that will
follow.…I would score the paper
a two for thesis/purpose because
while the central argument is clearly
stated, it could be more nuanced and
specific.”
Relates to macro-level “...provide a more detailed and “The paper has a clearly and
features (e.g., organization, eloquent thesis statement that… eloquently stated thesis that
development) (Burstein et reveals the organizational structure identifies a demonstrable and
al., 2013) of the paper and guides the reader nuanced central argument....The
smoothly and logically into the body thesis is introduced in the opening
of the paper.” paragraph and is reinforced
throughout the paper with well-
supported evidence and analysis. The
author provides a clear sense of the
analysis that will follow and reveals
the organizational structure of the
paper. The thesis guides the reader
smoothly and logically into the body
of the paper.”
Supports multiple “To improve the score to a 3, the “Improvement suggestion: N/A.”
submissions and learner author could…explicitly state the
autonomy (Wang et al., research questions…and provide a Note: When student writers can
2013) brief overview of the methods…” confirm the quality of their content,
then they move on to improving
other areas of their texts.
Note. AWE = Automated Writing Evaluation; N/A = not applicable.
a
Because Michigan Corpus of Upper-Level Student Papers provides examples of A-graded papers, it is unsurprising that the
generative artificial intelligence did not assign any scores of 0 or 1 for the texts in the sample.

feedback. Prompt construc-

tion has received growing
attention within the literature, as generative AIs must be sup-
plied with optimized prompts to produce appropriate content
(Nachson et al., 2023; Sanmarchi et al., 2023). Thus, it will
be important to monitor emerging research on how to prompt
ChatGPT correctly and educate faculty and students on how
to best use ChatGPT for AWE. ChatGPT also can provide
individualized feedback quickly. Emerging research on Chat-
GPT’s capabilities has described its personalized interactions
with users and deemed it to be a “powerful self-learning” tool
(Sallam, 2023). Thus, by using GAI tools such as ChatGPT
for feedback, student learning can be self-driven and indi-
vidualized. Athough GAI is a rapidly advancing technology,
these exploratory applications in providing AWE-style feed-
back are a step forward in improving the efficiency and effec-
tiveness of writing instruction in nursing education.
Aguayo-González, M., Leyva-Moral, J. M., San Rafael, S., Fernandez, M.
Conclusion
This article introduces ChatGPT, a form of GAI, as a tool
for AWE in nursing education. Our experience using Chat-
GPT to provide feedback on nursing student texts demon-
strates its capabilities for individualized writing feedback
and self-driven learning. With proper instruction, ChatGPT
can enhance students’ communication skills (Kumar, 2023).
We plan to formally evaluate ChatGPT’s capabilities for writ-
ing instruction by embedding it within postsecondary nursing
curricula. Like any new technology, it also will be important
to understand ChatGPT’s acceptance among students and fac-
ulty and ensure that faculty and students receive proper guid-
ance and training to facilitate GAI-based learning.
References
Adams, D., & Chuah, K.-M. (2023). Artificial intelligence based tools in
research writing: Current trends and future potentials. In P. P. Churi,
S. Joshi, M. Elhoseny, & A. Omrane, (Eds.). Artificial intelligence in
higher education: A practical approach. CRC Press.
I., & Gómez-Ibáñez, R. (2020). Graduated nurses’ experiences with
baccalaureate thesis writing: A qualitative study. Nursing & Health Sci-
ences, 22(3), 563–569. https://doi.org/10.1111/nhs.12693

Journal of Nursing Education • Vol. 62, No. 12, 2023 725

American Association of Colleges of Nursing. (2021). The essentials:
Core competencies for professional nursing education. https://www.
aacnnursing.org/Portals/42/AcademicNursing/pdf/Essentials-2021.pdf
Attali, Y. & Burstein, J. (2006). Automated essay scoring with e-rater® V.2.
The Journal of Technology, Learning and Assessment, 4(3). https://
ejournals.bc.edu/index.php/jtla/article/view/1650/1492
Bailey, D., & Lee, A. R. (2020). An exploratory study of Grammarly in the
language learning context: An analysis of test-based, textbook-based
and Facebook corpora. TESOL International Journal, 15(2), 4–27.
Bickes, J. T., & Schim, S. M. (2010). Righting writing: Strategies for im-
proving nursing student papers. International Journal of Nursing Edu-
cation Scholarship, 7(1). https://doi.org/10.2202/1548-923X.1964
Bjerkvik, L. K., & Hilli, Y. (2019). Reflective writing in undergraduate clin-
ical nursing education: A literature review. Journal of Nursing Educa-
tion and Practice, 35, 32–41. https://doi.org/10.1016/j.nepr.2018.11.013
PMID:30660960
Burstein, J., Tetreault, J., & Madnani, N. (2013). The e-rater® automated
essay scoring system. In M. D. Shermis & J. Burstein (Eds.), Handbook
of automated essay evaluation: Current applications and new directions
(pp. 55–67). Routledge/Taylor & Francis Group.
Calvo, R. A., O’Rourke, S. T., Jones, J., Yacef, K., & Reimann, P. (2011).
Collaborative writing support tools on the Cloud. IEEE Transac-
tions on Learning Technologies, 4(1), 88–97. https://doi.org/10.1109/
TLT.2010.43
Cone, P. H., & Van Dover, L. (2012). Shaping how graduate nursing stu-
dents write. Nursing Education Perspectives, 33(4), 272–274. https://
doi.org/10.5480/1536-5026-33.4.272 PMID:22916634
Cotos, E., Huffman, S., & Link, S. (2015). Furthering and applying move/
step constructs: Technology-driven marshalling of Swalesian genre the-
ory for EAP pedagogy. Journal of English for Academic Purposes, 19,
52–72. https://doi.org/10.1016/j.jeap.2015.05.004
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user
acceptance of information technology. MIS Quarterly, 13(3), 319–340.
https://doi.org/10.2307/249008
Deane, P. (2013). On the relation between automated essay scoring and
modern views of the writing construct. Assessing Writing, 18(1), 7–24.
https://doi.org/10.1016/j.asw.2012.10.002
Deane, P., & Quinlan, T. (2010). What automated analyses of corpora can
tell us about students’ writing skills. Journal of Writing Research, 2(2),
151–177. https://doi.org/10.17239/jowr-2010.02.02.4
DeCoux Hampton, M., & Chafetz, L. (2021). Evaluating scientific writing
skill in DNP program students. Nurse Educator, 46(3), 164–169. https://
doi.org/10.1097/NNE.0000000000000883 PMID:32658087
Duckworth, A., & Ungar, L. (2023, January 19). Op-ed: Don’t ban chat-
bots in classrooms—Use them to change how we teach. Los Angeles
Times. https://www.latimes.com/opinion/story/2023-01-19/chatgpt-ai-
education-testing-teaching-changes
Ellis, R., & Ellis, S. (2008). The definition and measurement of L2 explicit
knowledge. Language Learning, 58(s1), 1–18.
Eysenbach, G. (2023). The role of ChatGPT, generative language models,
and artificial intelligence in medical education: A conversation with
ChatGPT and a call for papers. JMIR Medical Education, 9, e46885.
https://doi.org/10.2196/46885 PMID:36863937
Hampton, M. D. (2019). A 5-year evaluation of faculty use of practi-
cal within-course writing development strategies in a DNP pro-
gram. Nurse Educator, 44(4), 197?201. https://doi.org/10.1097/
NNE.0000000000000589 PMID:30138150
Housen, A., & Kuiken, F. (2009). Complexity, accuracy, and fluency in sec-
ond language acquisition. Applied Linguistics, 30(4), 461–473. https://
doi.org/10.1093/applin/amp048
Kilmer, M., Bradley, C., Raines, A., & Blair, D. (2023). Integrating writ-
ing throughout the curriculum in Doctor of Nursing Practice pro-
grams: A collaborative model for success. Journal of Nursing Educa-
tion, 62(4), 241–248. https://doi.org/10.3928/01484834-20230208-06
PMID:37021947
Knight, S., Shibani, A., Abel, S., Gibson, A., Ryan, P., Sutton, N., Wight,
R., Lucas, C., Sándor, Á., Kitto, K., Liu, M., Vijay Mogarkar, R., &
Buckingham Shum, S. (2020). AcaWriter: A learning analytics tool for
formative feedback on academic writing. Journal of Writing Research,
12(1), 141–186. https://doi.org/10.17239/jowr-2020.12.01.06
Kumar, H. S. (2023). Analysis of ChatGPT tool to assess the potential of
726
its utility for academic writing in biomedical domain. Biology, En-
gineering, Medicine and Science Reports, 9(1), 24–30. https://doi.
org/10.5530/bems.9.1.5
Lee, Y.-W., Gentile, C., & Kantor, R. (2008). Analytic scoring of TOEFL®
CBT essays: Scores from humans and e-rater®. ETS Research Re-
port Series, 2008(1), i–71. https://doi.org/10.1002/j.2333-8504.2008.
tb02087.x
Liao, H.-C. (2016). Using automated writing evaluation to reduce grammar
errors in writing. ELT Journal, 70 (3), 308–319. https://doi.org/10.1093/
elt/ccv058
Lim, J. Y., Ong, S. Y. K., Ng, C. Y. H., Chan, K. L. E., Wu, S. Y. E. A., So,
W. Z., Tey, G. J. C., Lam, Y. X., Gao, N. L. X., Lim, Y. X., Tay, R. Y.
K., Leong, I. T. Y., Rahman, N. D. A., Chiam, M., Lim, C., Phua, G. L.
G., Murugam, V., Ong, E. K., & Krishna, L. K. R. (2023). A system-
atic scoping review of reflective writing in medical education. BMC
Medical Education, 23(1), 12. Advance online publication. https://doi.
org/10.1186/s12909-022-03924-4 PMID:36624494
Madson, M. J. (Ed.). (2022). Teaching writing in the health professions:
Perspectives, problems, and practices. Routledge.
McMillan, L. R., & Raines, K. (2011). Using the “write” resources: Nursing
student evaluation of an interdisciplinary collaboration using a profes-
sional writing assignment. Journal of Nursing Education, 50(12), 697–
702. https://doi.org/10.3928/01484834-20110930-01 PMID:21956258
Micsinszki, S. K., & Yeung, L. (2021). Adapting «Shut Up & Write!®» to
foster productive scholarly writing in graduate nursing students. The
Journal of Continuing Education in Nursing, 52(7), 313–318. https://
doi.org/10.3928/00220124-20210611-05 PMID:34166156
Mitchell, K. M. (2018). Constructing writing practices in nursing. Journal of
Nursing Education, 57(7), 399–407. https://doi.org/10.3928/01484834-
20180618-04 PMID:29958309
Nachshon, A., Batzofin, B., Beil, M., & van Heerden, P. V. (2023). When
palliative care may be the only option in the management of severe
burns: A case report written with the help of ChatGPT. Cureus, 15(3),
e35649. https://doi.org/10.7759/cureus.35649 PMID:36875254
Nazari, N., Shabbir, M. S., & Setiawan, R. (2021). Application of Artificial
Intelligence powered digital writing assistant in higher education: Ran-
domized controlled trial. Heliyon, 7(5), e07014. Advance online publi-
cation. https://doi.org/10.1016/j.heliyon.2021.e07014 PMID:34027198
Ndukwe, I.G., Daniel, B.K., & Amadi, C.E. (2019). A machine learning
grading system using chatbots. In: Isotani, S., Millán, E., Ogan, A.,
Hastings, P., McLaren, B., & Luckin, R. (eds.), Artificial Intelligence
in Education (Lecture Notes in Computer Science, vol. 11626, pp. 365–
368.). Springer. https://doi.org/10.1007/978-3-030-23207-8_67
O’Connor, S., & ChatGPT. (2023). Open artificial intelligence platforms in
nursing education: Tools for academic progress or abuse? Nurse Educa-
tion in Practice, 66, 103537. https://doi.org/10.1016/j.nepr.2022.103537
PMID:36549229
O’Donnell, M. B., & Römer, U. (2012). From student hard drive to web
corpus (part 2): The annotation and online distribution of the Michigan
Corpus of Upper-level Student Papers (MICUSP). Corpora, 7(1), 1–18.
https://doi.org/10.3366/cor.2012.0015
Patel, S. B., & Lam, K. (2023). ChatGPT: The future of discharge sum-
maries? The Lancet. Digital Health, 5(3), e107–e108. https://doi.
org/10.1016/S2589-7500(23)00021-3 PMID:36754724
Rao, A., Kim, J., Kamineni, M., Pang, M., Lie, W., & Succi, M. D.
(2023). Evaluating ChatGPT as an adjunct for radiologic decision-
making. medRxiv: The Preprint Server for Health Sciences. https://doi.
org/10.1101/2023.02.02.23285399
Römer, U., & O’Donnell, M. B. (2011). From student hard drive to web
corpus (part 1): The design, compilation and genre classification of the
Michigan Corpus of Upper-level Student Papers (MICUSP). Corpora,
6(2), 159–177. https://doi.org/10.3366/cor.2011.0011
Sallam, M. (2023). ChatGPT utility in healthcare education, research,
and practice: systematic review on the promising perspectives and
valid concerns. Healthcare (Basel), 11(6), 887. https://doi.org/10.3390/
healthcare11060887 PMID:36981544
Sanmarchi, F., Bucci, A., & Golinelli, D. (2023). A step-by-step research-
er’s guide to the use of an AI-based transformer in epidemiology: An
exploratory analysis of ChatGPT using the STROBE checklist for ob-
servational studies. medRxiv. Preprint.
Shermis, M. D. (2014). State-of-the-art automated essay scoring: Compe-
Copyright © SLACK Incorporated
 tition, results, and future directions from a United States demonstra-
tion. Assessing Writing, 20(Apr), pp. 53–76. https://doi.org/10.1016/j.
asw.2013.04.001
Shermis, M. D., & Hamner, B. (2013). Contrasting state-of-the-art auto-
mated scoring of essays: Analysis. In M. D. Shermis & J. Burstein
(Eds.), Handbook of automated essay evaluation (pp. 313–346). Rout-
ledge. https://doi.org/10.4324/9780203122761-27
Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O., Proske, A., &
Rapp, C. (2019). Digital support for academic writing: A review of
technologies and pedagogies. Computers & Education, 131, 33–48.
https://doi.org/10.1016/j.compedu.2018.12.005
Thorp H. H. (2023). ChatGPT is fun, but not an author. Science, 379(6630),
313. https://doi.org/10.1126/science.adg7879 PMID:36701446
Tornwall, J., & McDaniel, J. (2022). Key strategies in scholarly writing
instruction for doctor of nursing practice students: A Q-methodology
study. Nurse Education Today, 108, 105192. https://doi.org/10.1016/j.
nedt.2021.105192 PMID:34768153
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L.
(2023). ChatGPT: Five priorities for research. Nature, 614(7947), 224–
226. https://doi.org/10.1038/d41586-023-00288-7 PMID:36737653
Wang, J., & Brown, M. S. (2007). Automated essay scoring versus human
scoring: A comparative study. The Journal of Technology, Learning,
and Assessment, 6(2), 3–28. https://ejournals.bc.edu/index.php/jtla/ar-
ticle/view/1632/1476
Journal of Nursing Education • Vol. 62, No. 12, 2023
Wang, Y. J., Shang, H. F., & Briody, P. (2013). Exploring the impact of
using automated writing evaluation in English as a foreign language
university students’ writing. Computer Assisted Language Learning,
26(3), 234–257. https://doi.org/10.1080/09588221.2012.655300
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019).
Systematic review of research on artificial Intelligence applications in
higher education—Where are the educators? International Journal of
Educational Technology in Higher Education, 16, Article 39. https://
doi.org/10.1186/s41239-019-0171-0
Zhai, N., & Ma, X. (2023). The effectiveness of automated writing
evaluation on writing quality: A meta-analysis. Journal of Educa-
tional Computing Research. Advance online publication. https://doi.
org/10.1177/07356331221127300
Zhan, T., Wang, L., Wang, Y., & Sun, C. J. (2023). Master of nursing spe-
cialist experiences of an internship through the use of written reflec-
tions: A qualitative research study. Heliyon, 9(2), e13299. Advance
online publication. https://doi.org/10.1016/j.heliyon.2023.e13299
PMID:36814604
Zielinski, C., Winker, M., Aggarwal, R., Ferris, L., Heinemann, M., La-
peña, J. F., Jr., Pai, S., Ing, E., & Citrome, L. (2023). Chatbots, Chat-
GPT, and scholarly manuscripts: WAME recommendations on Chat-
GPT and chatbots in relation to scholarly publications. Open Access
Macedonian Journal of Medical Sciences, 11(A), 83–86. https://doi.
org/10.3889/oamjms.2023.11502
727