The Effectiveness of Augmented Reality/Mixed

Reality in Medical Education: A Meta- Analysis

Ruoxuan Zhang
Macao Polytechnic University
Xiaoyan Jin
Peking University Health Science Center - Macao Polytechnic University Nursing Academy, Macao
Polytechnic University
Ming Liu

Peking University Health Science Center - Macao Polytechnic University Nursing Academy, Macao
Polytechnic University
Hoi Yee Tong
Macao Polytechnic University

Research Article

Keywords: Augmented reality, mixed reality, medical education, meta-analysis

Posted Date: July 4th, 2024

DOI: https://doi.org/10.21203/rs.3.rs-4549366/v1

License:   This work is licensed under a Creative Commons Attribution 4.0 International License.
Read Full License

Additional Declarations: No competing interests reported.

 1 The Effectiveness of Augmented Reality/Mixed Reality in Medical Education: A

2 Meta-Analysis

3 Ruoxuan Zhang1,#, Xiaoyan Jin2,#，Ming Liu2,*， Hoi Yee Tong3

1
4 Faculty of Health Sciences and Sports, Macao Polytechnic University, Macao, China
2
5 Peking University Health Science Center - Macao Polytechnic University Nursing

6 Academy, Macao Polytechnic University, Macao, China

3
7 Faculty of Applied Sciences, Macao Polytechnic University, Macao, China

8 #：
These authors make equal contributions.

9 *Corresponding Author:

10 Ming Liu PhD, Professor

11 Peking University Health Science Center - Macao Polytechnic University Nursing

12 Academy, Macao Polytechnic University, Macao, China

13 karryliu@mpu.edu.mo; Tel: +853-88936928

14

15 Abstract:

16 Background: Augmented reality (AR) and mixed reality (MR) are increasingly being applied

17 in the field of medical education. However, the effectiveness and acceptance of AR/MR in different

18 teaching scenarios remain unclear. This meta-analysis aimed to examine the effectiveness of

19 AR/MR in improving medical students' knowledge and skills training.

20 Methods: PubMed, Embase, Cochrane Library, PsycINFO, Education Source, Education

21 Resources Information Center, CINAHL, Web of Science Core Collection, China National

22 Knowledge Infrastructure, WanFang, WeiPu and Chinese BioMedical Literature databases were

23 searched. The search encompassed literature from database establishment to December 2023.

24 Studies examining the use of AR/MR in medical education and reporting outcomes related to

25 knowledge learning or skill training were included. The data were analyzed using Stata version 15.1

26 software.

27 Results: 29 publications were included in this meta-analysis. Compared to traditional teaching

1
28 methods, AR/MR-assisted teaching showed greater effectiveness in medical skills training, as

29 indicated by the higher skill scores (WMD = 12.31, 95% CI: 4.12 to 20.50), reduced failure rate

30 (RD=-0.13, 95% CI: -0.19 to -0.07), and shortened performance time (SMD = -0.19, 95% CI: -0.32

31 to -0.06). However, AR/MR did not significantly improve knowledge acquisition (WMD=2.78, 95%

32 CI: -1.89 to 7.45). The questionnaire survey revealed the advantages of AR/MR in terms of

33 perceived usefulness (PU) (WMD=0.27, 95% CI: 0.00 to 0.53), perceived ease of use (PEOU)

34 (WMD=0.35, 95% CI: 0.13 to 0.57), and enjoyment (WMD=0.67, 95% CI: 0.20 to 1.13).

35 Conclusion: This meta-analysis highlights the effectiveness and user acceptance of AR/MR in

36 medical education, particularly in skill training. However, there was no significant improvement in

37 knowledge learning. The findings provide foundational data for expanding AR/MR applications in

38 medical education.

39 Key words：Augmented reality; mixed reality; medical education; meta-analysis

40 Introduction
41 Augmented reality (AR), along with mixed reality (MR) is increasingly used in medical

42 education and clinical practice. AR and MR technologies have unique abilities to overlay digital

43 information onto real-world environments or blend virtual and real worlds, respectively[1]. AR/MR

44 has three inherent advantages: annotation of the real world, leading to a notable reduction in

45 cognitive load; visualization of process content, resulting in enhanced perception and improved

46 memory retention; and visualization of visual and tactile information, allowing for information

47 acquisition through interaction between humans and the environment[2]. Since AR/MR-based

48 medical devices have already been approved for medical practice[3], this must be included in the

49 curriculum not only for the education purpose but also for practical reasons.

50 In medical education, the application of AR/MR can be broadly categorized into knowledge

51 learning and skill training scenarios. These two types of teaching differ in terms of their learning

52 objectives and assessment methods. Knowledge learning, especially in areas such as anatomy and

53 pathology, requires students to develop spatial thinking skills. The use of AR/MR technologies in

54 education provides medical students with a more intuitive understanding of anatomical structures[4]

55 and pathological changes. Operational training, such as surgical skills[5], physical examinations[6],
2
56 interventional diagnosis and treatment, and emergency skills training[7], requires highly realistic

57 and interactive simulated environments. AR/MR technology can also provide personalized learning

58 experiences and real-time feedback, helping students continuously improve and enhance their

59 learning outcomes[8]. AR/MR, with its significant features of stereoscopic visualization,

60 interactivity, dynamic display, and simulated practice, greatly enriches teaching methods, enhances

61 teaching effectiveness, and promotes the development of clinical thinking and skills in medical

62 students[9]. Compared to traditional teaching methods, which rely on textual instructions, model

63 demonstrations, live observation, animal experiments, or direct patient operations, AR/MR

64 technology offers advantages in terms of safety, autonomy, engagement, repeatability, planning and

65 feedback, and multiuser collaboration[10]. However, the existing published studies have

66 contradictory results, with some suggesting a positive effect on learning outcomes [7] and others

67 indicating no significant impact[11].

68 To our knowledge, apart from a study by Yahia et al. [12], no meta-analysis has been conducted

69 to evaluate AR/MR in medical education. Moreover, Yahia's research included 13 studies as of April

70 2021, mainly analyzing the impact of AR on knowledge learning performance, skill operation

71 completion time, and satisfaction, and lacked a multidimensional assessment of AR effects and user

72 acceptance. The primary goal of this study was to conduct a meta-analysis of the efficacy and user

73 acceptance of AR/MR in two types of application scenarios: theoretical knowledge learning and

74 skill training. The objective indicators used to evaluate the AR/MR effect include knowledge scores,

75 skill scores, failure rates and performance time. The acceptance of AR/MR was evaluated based on

76 the Technology Acceptance Model framework, with subjective indicators including perceived

77 usefulness (PU), perceived ease of use (PEOU), and Enjoyment.

78 The results of this work have many implications for teachers, medical students, education

79 professionals, and the users of AR/MR technology to promote learning and training in the medical

80 field. This meta-analysis aims to address the following research questions (RQs):

81 RQ1: Does the use of AR/MR have an impact on medical education when compared to other

82 instructional methods?

83 RQ2: What medical education application scenarios are AR/MR suitable？

84 RQ3: How is the user experience of AR/MR?

3
 85 Methods
86 Definitions

87 For the purposes of this meta-analysis, we defined AR as a “computer-generated

88 holographic image overlaid into the real clinical environment, permitting the user to interact

89 with the hologram and objects in the real environment in an integrated fashion”[13], and MR

90 as a “blending virtual objects and information onto a physical environment, interacting with it,

91 and responding to it[9]”.

92 Inclusion criteria

93 Based on the PICOS (Population, Intervention, Comparison, Outcomes, Study design)

94 framework, the inclusion criteria were as follows:

95 Population(P): Individuals enrolled in undergraduate or graduate medical programs.

96 Intervention(I): The intervention group received education involving AR or MR (combining

97 AR/MR with traditional teaching, or purely AR/MR-based teaching).

98 Comparison(C): traditional teaching methods, which may include classroom-based learning,

99 no intervention, or other forms of digital and blended education.

100 Outcome(O) (at least one of these): (a) Skill assessment, incorporating: the skill score (SS),

101 evaluated via standardized procedures and converted to a percentage format; failure rate (FR),

102 documenting instances where tasks were not successfully completed; and performance time (PT),

103 measuring the duration taken to finish a single task. (b) Knowledge assessment, incorporating:

104 knowledge score (KS), assessed via a written examination designed to test the comprehension of

105 pertinent theoretical knowledge. (c) Participants' experience was evaluated by a standardized self-

106 report questionnaire survey(QS), that included items on PU, PEOU and enjoyment. Additional

107 Notes: The KS and SS results were presented as scores out of 100 and for QS, the response options

108 ranged from “very dissatisfied” (1 point) to “very satisfied” (5 points) on a five-point Likert

109 scale.

110 Study design(S): Randomized controlled trials (RCTs).

111 Exclusion criteria

112 The exclusion criteria were as follows: (a) studies with unclear outcome measures or

4
113 insufficient data description; (b) duplicate publications; (c) studies without access to the full text;

114 and (d) studies with a high risk of bias.

115 Search strategies

116 We searched seven English-language online databases including MEDLINE, Embase

117 (Elsevier), Cochrane Central Register of Controlled Trials, PsycINFO, Education Resources

118 Information Center (Ovid), CINAHL (EBSCO), Web of Science Core Collection (Thomson

119 Reuters), and four Chinese databases including China National Knowledge Infrastructure,

120 WanFang, WeiPu and Chinese BioMedical Literature. The search encompassed literature from

121 database establishment to December 2023, and the languages were limited to Chinese and English.

122 The keywords used in the search strategy corresponded to PubMed Subject Headings (MeSH) and

123 included, but were not limited to: “Augmented Reality/Mixed Reality/ Education/ Learn*/ Teach*/

124 Train*/ Health/ Medicine/ Randomized Controlled Trial”. Our study protocol was registered with

125 PROSPERO (Code: CRD42024506938). Table S1 shows the literature search strategy for each

126 database.

127 Data Extraction

128 Two researchers conducted literature searches based on the inclusion and exclusion criteria.

129 Noteexpress software was used for literature screening and data extraction. Disputed articles were

130 discussed, and the final decision on inclusion was made by the corresponding author. The included

131 literature was then cross-checked by two researchers to extract the following information: author,

132 publication date, location, specialty, application scenarios, equipment, sample size, outcome

133 measures, and intervention duration.

134 Quality assessment

135 Two researchers independently evaluated the methodological quality of the included studies

136 according to the JBI Critical Appraisal Checklist for randomized controlled trials[14]. The checklist

137 has 13 items to assess the risk of bias, including participants, assignments, measurement, and

138 analysis domains, and 1 overall appraisal item. After each included study was assessed using a 13-

139 item checklist as ‘yes’, ‘no’, or ‘not applicable/unclear’, the final quality judgment was drawn

140 according to the number of “yes”. The risk of bias was evaluated as follows: (1) “yes” ≥10 was

141 considered high quality, (2) “yes” between 7 and 10 was considered medium quality, and (3)
5
142 “yes”<6 was considered poor quality.

143 Statistical analysis

144 Heterogeneity among the included studies was evaluated using the chi-square test, with a

145 significance level of p<0.05 and I2>50% indicating significant heterogeneity among the included

146 studies. A random effects model was employed for the meta-analysis. The pooled effects are

147 presented as weighted or standardized mean difference (WMD/SMD) and 95% confidence intervals

148 (CIs), and the effects on failure rates are presented as risk difference (RD) and 95%CIs. Subgroup

149 analysis was performed based on the teaching scenarios. Sensitivity analysis was conducted by

150 sequentially excluding each study to assess its influence on the pooled effects. The publication bias

151 was evaluated by a funnel plot and Egger’s test. The data were analyzed using Stata version 15.1

152 software. A significance level of p＜0.05 indicated statistical significance.

153 Results
154 Search results

155 A total of 386 potentially relevant articles were identified, of which 128 duplicates were

156 removed. At the screening stage, 202 articles were excluded after reading the titles and abstracts,

157 among which 136 were irrelevant to the topic, and 66 did not match the research type. According to

158 the inclusion and exclusion criteria, 56 full-text articles were assessed for eligibility. Among these,

159 27 studies did not meet the inclusion criteria and had incomplete outcome indicators. Thus, a total

160 of 29 articles were included in this meta-analysis. A summary of the process is outlined in Figure 1.

6
161 Figure 1. Flow chart of literature screening

162 Study characteristics

163 Most of the 29 included studies were published in 2020 or later (n = 23, 79.31%) and the

164 majority of the studies were conducted in Europe and North America (n = 19, 65.51%). The included

165 studies covered 9 specialties, including Anatomy, Surgery, Imaging department, and Emergency

166 Medicine. Among them, studies on theoretical knowledge learning accounted for 53.33% (n=16),

167 and those on skill training accounted for 46.67%(n=14). One of the studies included both knowledge

168 learning and skill training. (Table 1)

169 Table 1 Basic characteristics of the included studies (n=29)

Applica Sample Outcome Intervention
Article Year Location Specialty Equipment(I/C)
tion size(I/C) measures duration
Bogomolova et KS,PU,PEOU,
2023 Netherlands Anatomy KL 3D holography / monoscopic 3D 32/34 45min
al.[15] EN
Cizmic et al.[16] 2023 Germany Laparoscope ST iSurgeon/ verbal guidance 20/20 PT,FR /
Microsoft HoloLens/ regular
D'Aiello et al.[17] 2023 Italy Anatomy KL 19/20 KS 30min
slideware
Farshad et al.[18] 2023 Switzerland Ultrasound ST Microsoft HoloLens/ standard US 22/22 PT /
Felinska et al.[6] 2023 Germany Laparoscope ST iSurgeon/ no AR telestration 20/20 FR，PT,PU /
Guha et al.[19] 2023 UK Surgery ST Microsoft HoloLens/ video 18/18 SS,PEOU,EN 20min
FR,PU,PEOU,
Hayasaka et al.[20] 2023 Japan Anesthesia ST Microsoft HoloLens/ traditional 10/10 10min
EN
Otolaryngolog KS,PU,PEOU,
Malik et al.[21] 2023 UK KL Microsoft HoloLens/ traditional 35/21 45min
y EN
7
 Self-Regulated
An et al.[22] 2022 Korea KL AR app/ textbook 31/31 KS 4weeks
learning
Cardiopulmon
Microsoft HoloLens/ instructor-
Hou et al.[23] 2022 China ary ST 14/13 FR 10min
assisted
Resuscitation
Little et al.[24] 2022 USA Anatomy KL IVALA®/ textbook 38/36 KS,PU,PEOU 60min
Nagayo et al.[25] 2022 Japan Surgery ST Microsoft HoloLens/ video 19/19 SS,PU,PEOU 10min
Pickering et al.[26] 2022 UK,Greece Anatomy KL MR/ screencast 62/84 KS 90min
Veer et al.[11] 2022 Australia Asthma KL Microsoft HoloLens/ textbook 33/34 KS 6min
Yohannan et al.[27] 2022 India Anatomy KL Air Anatomy/ lecture 49/45 KS 3hour
Health
ST,
Chen et al.[28] 2021 Taiwan assessment and 3D holography/ lecture 40/39 SS,KS 18weeks
KL
practice course
Kamonwon et al. [29] 2021 Thailand Ultrasound KL AR/no AR 23/26 KS 4weeks
Pulmonary
Liu[30] 2021 China ST 3D holography/ 2D CT 10/10 SS /
lesions
Weeks et al.[31] 2021 USA Anatomy KL Microsoft HoloLens/ 2D screens 15/15 KS 60min
Bogomolova et stereoscopic 3D AR / monoscopic KS,PU,PEOU,
2020 Netherlands Anatomy KL 20/22 45min
al.[32] 3D EN
59/10 SS,PU,PEOU,
Schoeb et al.[33] 2020 Germany Surgery ST Microsoft HoloLens/ instructor 30min
5 EN
Stojanovska et al.[34] 2020 USA Anatomy KL Microsoft HoloLens/ Dissection 31/33 KS 4hour
New Zealand Microsoft HoloLens/ 3DM laptop KS,PU,PEOU,
Wang et al.[35] 2020 Anatomy KL 19/15 20min
et al. program EN
KS,PU,PEOU,
Noll et al.[36] 2017 Germany Dermatology KL mARble/ traditional 22/22 45min
EN
Ophthalmosco
Leitritz et al.[37] 2014 Germany ST AR ophthalmoscopy/ traditional 19/18 SS，FR,PU 15min
py
Vera et al.[38] 2014 USA Laparoscope ST ART platform/ traditional 9/9 FR，PT 1hour
Forensic KS,PU,PEOU,
Albrecht et al.[39] 2013 Germany KL mARble/ textbook 4/4 30min
medicine EN
Wilson et al.[40] 2013 USA Surgery ST AR goggles/ lecture 13/21 SS，FR /
Perk Station training suite/
Yeo et al.[41] 2011 Canada Surgery ST 20/20 DS 8 time
traditional
170 ST, skill training; KL, knowledge learning; KS, knowledge scores; SS, skill scores; FR, failure rate; PT, performance
171 time; PU, Perceived Usefulness; PEOU, Perceived Ease Of Use; EN, enjoyment.

172 Study quality

173 According to the JBI quality appraisal tool, 8 of the RCTs were of high quality (27.59%) and

174 21 of the RCTs were of medium quality (72.41%) (Table 2). Blinding of participants and interveners

175 was impossible since the intervention was apparent to them. The following three items in most

176 studies were considered“no”: allocation concealment, participant blinding, and intervener

177 blinding.

178 Table 2 Quality appraisal of the included studies (n=29)

Study ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Bogomolova et al.[15] Y N Y Y N N/A Y Y Y Y Y Y Y 10
Cizmic et al.[16] Y N Y N N Y Y Y Y Y Y Y Y 10
D'Aiello et al.[17] U N U N N N/A Y Y Y Y Y Y Y 7
Farshad et al.[18] Y N Y N N N Y Y Y Y Y Y Y 9
Felinska et al.[6] Y U Y N N N Y Y Y Y Y Y Y 9
Guha et al.[19] Y N Y N N Y Y Y Y Y Y Y Y 10
Hayasaka et al.[20] U N Y N N N/A Y Y Y Y Y Y Y 8
Malik et al.[21] U N Y N N N/A Y Y Y Y Y Y Y 8
An et al.[22] Y N Y N N N/A Y Y Y Y Y Y Y 9
Hou et al.[23] Y N Y N N N/A Y Y Y Y Y Y Y 9
Little et al.[24] Y N Y N N N/A Y Y Y Y Y Y Y 9
Nagayo et al.[25] Y N Y N N N/A Y Y Y Y Y Y Y 9
Pickering et al.[26] U N U N N N Y Y Y Y Y Y Y 7
Veer et al.[11] Y N Y N N Y Y Y Y Y Y Y Y 10
Yohannan et al.[27] Y N U N N N/A Y Y Y Y Y Y Y 8
Chen et al.[28] Y N Y N N N/A Y Y Y Y Y Y Y 9
8
 Kamonwon et al. [29] Y N Y N N N/A Y Y Y Y Y Y Y 9
Liu[30] Y N Y N N Y Y Y Y Y Y Y Y 10
Weeks et al.[31] Y N Y N N N/A Y Y Y Y Y Y Y 9
Bogomolova et al.[32] Y N Y N N N/A Y Y Y Y Y Y Y 9
Schoeb et al.[33] Y N Y N N Y Y Y Y Y Y Y Y 10
Stojanovska et al.[34] Y Y Y Y Y N/A Y Y Y Y Y Y Y 12
Wang et al.[35] Y N Y N N N/A Y Y Y Y Y Y Y 9
Noll et al.[36] Y N Y N N N/A Y Y Y Y Y Y Y 9
Leitritz et al.[37] Y N Y N N Y Y Y Y Y Y Y Y 10
Vera et al.[38] Y N Y N N N/A Y Y Y Y Y Y Y 9
Albrecht et al.[39] Y N N N N N/A Y Y Y Y Y Y Y 8
Wilson et al.[40] U N Y N N N/A Y Y Y Y Y Y Y 8
Yeo et al.[41] Y N N N N N/A Y Y Y Y Y Y Y 8
179 Y, yes; N, no; N/A, not applicable; U,unclear. 1.Was true randomization used for assignment of participants to
180 treatment groups? 2.Was allocation to treatment groups concealed? 3. Were treatment groups similar at the
181 baseline? 4.Were participants blind to treatment assignment? 5.Were those delivering the treatment blind to treatment
182 assignment? 6.Were outcome assessors blind to treatment assignment? 7.Were treatment groups treated identically
183 other than the intervention of interest? 8.Was follow-up complete and, if not, were differences between groups in
184 terms of their follow-up adequately described and analyzed? 9.Were participants analyzed in the groups to which
185 they were randomized? 10.Were outcomes measured in the same way for treatment groups? 11.Were outcomes
186 measured in a reliable way? 12.Was appropriate statistical analysis used?13.Was the trial design appropriate and any
187 deviations from the standard RCT design accounted for in the conduct and analysis of the trial? 14.Total number of
188 Y.

189 Sensitivity analysis and publication bias

190 Due to high heterogeneity, a sensitivity analysis was conducted to assess the reliability of the

191 results. In the analysis of performance time, when the study by Felinska et al. [6] was excluded, the

192 overall effect was different between the AR/MR and control groups (SMD=-0.23, 95% CI: -0.45 to

193 0.01, P=0.061), which was different from the primary pooled effect. In addition to the

194 abovementioned studies, no single study was found to significantly influence the overall pooled

195 effects, indicating the stability of our results (Figure S1). Aside from the failure rate analysis (P =

196 0.017), no other analyses revealed significant publication bias based on the the Egger's test (P＜

197 0.05) (Table S2).

198 Effects of AR/MR on skill training

199 A total of 8 publications involving 454 participants (AR/MR group = 205 and control group =

200 249) reported skill scores. Because high heterogeneity was observed across these studies (I2 =

201 95.8%, P=0.000), a random-effects model was used. The pooled effects showed a significant

202 difference in skill scores (WMD = 12.31, 95% CI: 4.12 to 20.50, P =0.003) in favor of the AR/MR

203 group compared to the control group (Figure 2).

9
204

205 Figure 2 Forest plot for the effects of AR/MR on skill scores

206 A total of 9 studies reported failure rates, with 3,690 failures in the AR/MR group and 4,487 in

207 the control group. Due to high heterogeneity in the meta-analysis (I2 = 79.4%, P=0.000), a random-

208 effects model was used. The results showed a significant reduction in the failure rate in the AR/MR

209 group compared to that in the control group (RD=-0.13, 95% CI: -0.19 to -0.07, P=0.000) (Figure

210 3).

211

10
212 Figure 3 Forest plot for the effects of AR/MR on the failure rate

213 A total of 5 publications involving 942 participants were identified (AR/MR group = 471 and

214 control group = 471). Heterogeneity analysis revealed an I2 of 2.3% and a P value of 0.393,

215 indicating no significant heterogeneity. A random-effects model was used for the meta-analysis to

216 ensure the reliability of the results. The results showed a significant reduction in performance time

217 in the AR/MR group compared to the control group (SMD = -0.19, 95% CI: -0.32 to -0.06, P=0.003)

218 (Figure 4).

219

220 Figure 4 Forest plot for the effects of AR/MR on performance time

221 Effects of AR/MR on knowledge learning

222 A total of 14 publications involving 826 participants (AR/MR group = 406 and control group

223 = 420) reported the use of ARs/MRs for knowledge learning. Due to high heterogeneity in the meta-

224 analysis (I2 = 88%, P=0.000), a random-effects model was used. The results showed no significant

225 difference in knowledge test scores between the AR/MR group and the control group (WMD=2.78,

226 95% CI: -1.89 to 7.45, P = 0.243) (Figure 5).

11
227

228 Figure 5 Forest plot for the effects of AR/MR on knowledge scores

229 Effects of AR/MR on student experience indicators

230 A total of 12 publications reported 605 student evaluation questionnaires on the PU (AR/MR

231 group = 281 and control group = 324). Due to high heterogeneity in the meta-analysis (I2 = 77.4%,

232 P =0.000), a random-effects model was used. The pooled effects showed that the score of the

233 AR/MR group was significantly greater than that of the control group (WMD = 0.27, 95% CI: 0.00-

234 0.53, P = 0.048) (Figure S2). To understand the PU of AR/MR in knowledge learning and skill

235 training activities, a subgroup analysis was conducted. In the skills training subgroup, which

236 included 5 studies, the AR/MR scores were significantly greater than those in the control group

237 (WMD = 0.30, 95% CI: 0.11-0.49, P=0.002). Conversely, in the knowledge learning group, which

238 included 7 studies, no significant difference was observed between the AR/MR score and those of

239 the control group (Figure 6).

240 A total of 11 publications reported 568 student evaluation questionnaires on the PU (AR/MR

241 group = 262 and control group = 306).A total of 11 publications reported students' evaluation

242 questionnaires on the PEOU, with 262 in the AR/MR group and 306 in the control group.

243 Heterogeneity analysis revealed high heterogeneity (I2 = 66.5%, P =0.001), and a random-effects

12
244 model was used for the meta-analysis. The pooled effects showed that the AR/MR score was

245 significantlygreater than that of the control group (WMD = 0.35, 95% CI: 0.13-0.57, P=0.002)

246 (Figure S3). Subgroup analysis was conducted for the knowledge learning group (7 studies) and the

247 skill training group(4 studies). The analysis using a random effects model revealed that in both the

248 knowledge learning (WMD=0.40, 95% CI: 0.07-0.73, P=0.018) and skills training (WMD=0.32,

249 95% CI: 0.09-0.55, P=0.007) subgroups, the PEOU scores of the AR/MR group were significantly

250 greater than those of the control group (Figure 6).

251 A total of 9 publications reported 456 student evaluations of enjoyment (AR/MR group = 205

252 and control group = 251). Heterogeneity analysis revealed high heterogeneity (I2 = 90.2%,

253 P=0.000), and a random-effects model was used. The pooled effects showed that the score of the

254 AR/MR group was significantly greater than that of the control group (WMD = 0.67, 95% CI: 0.20-

255 1.13, P=0.005) (Figure S4). The subgroup analysis included 6 studies on knowledge learning and 3

256 on skill training. The results indicated that in both the knowledge learning (WMD = 0.77, 95% CI:

257 0.10-1.44, P=0.024) and skills training (WMD = 0.52, 95% CI: 0.23-0.81, P=0.000) subgroups, the

258 scores for the enjoyment aspect of AR/MR were significantly greater than those of the control group

259 (Figure 6).

260
261 Figure 6 Effects of AR/MR on student experience indicators

13
262 Discussion
263 The current meta-analysis encompasses 29 studies, 23 of which were conducted between 2020

264 and 2023, highlighting the rapid expansion of AR/MR applications in medical education with

265 technological advancements. This meta-analysis indicated that employing AR/MR in skill training

266 significantly reduces both failure rates and performance time, while concurrently yielding higher

267 skill training scores. Notably, no significant difference was detected in the knowledge learning

268 scores. Participants consistently perceive AR/MR as more useful, easier to use, and more enjoyable

269 than conventional instructional methods, particularly within the context of skill training.

270 Information processing theory divides knowledge into declarative and procedural knowledge

271 [42]. AR/MR aids declarative (e.g., anatomy, pathology) via visualizations[43] and enhances

272 procedural skills through immersive simulations, offering real-time operational practice [44].

273 AR/MR enhanced skill training but limited impact on knowledge learning

274 While AR/MR excels in skill training within medical education, as evidenced by reduced

275 failure rates and performance times, its impact on theoretical knowledge learning appears less

276 substantial. This meta-analysis revealed no significant advantage for AR/MR over traditional

277 methods in enhancing knowledge comprehension, which is congruent with the findings of Moro et

278 al.[45] and contrasts with the conclusion of Paloma et al.[8].

279 AR/MR's potential to reduce cognitive load during information processing[46] suggests

280 benefits for learners[27], yet its reliance on visualization and interaction may inadequately address

281 deeper cognitive processes such as understanding and memory, which are crucial for knowledge

282 acquisition. Compared to skill training, the current common evaluation of knowledge learning

283 focuses on recall and memory[47] and may inadequately gauge AR/MR's ability to facilitate 3D

284 comprehension and practical application skills, necessitating more holistic evaluation methods that

285 include performance-based and interactive assessments.

286 In contrast, AR/MR technology significantly improved skill training outcomes and reduced

287 failure rates and performance times, consistent with past meta-analyses[12, 13]. It enhances skill

288 acquisition by offering standardized demos and key explanations, facilitating mental model

289 formation and skill mimicry[6]. The real-time simulation of AR/MR in medical training provides

14
290 authentic intrinsic feedback, which is augmented by extrinsic feedback mechanisms, accelerating

291 skill development and refinement[16, 23, 48]. AR/MR simplifies operational orientation and task

292 execution, reduces cognitive load, and thus accelerates skill formation[30, 49].

293 The impact of AR/MR in medical education in four stages was evaluated based on the

294 Kirkpatrick model[50]: increased student satisfaction (reaction), enhanced skill performance

295 (learning), reduced performance time (behavior), and reduced failure rate (result). In knowledge

296 learning, although students react favorably (reaction), there is no marked improvement in learning

297 outcomes (learning), and behavior and results assessments are lacking.

298 This emphasizes the necessity of refined, multilevel evaluation approaches, including spatial

299 abilities and practical skills. These innovative biometric sensors are expected to achieve accurate

300 quantification[6]. Future research should adopt standardized methodologies to address evaluation

301 gaps and optimiz AR/MR's implementation strategy for both knowledge and skill development,

302 particularly in light of its notable potential for skill training.

303 The acceptance for AR/MR in skill training is better than that in knowledge learning

304 The meta-analysis shows that medical students have varying levels of acceptance of AR/MR

305 in skill training and knowledge learning, which is related to the effectiveness demonstrated by the

306 technology. User acceptance is the key to technological implementation, and Fred D. Davis's (1986)

307 Technology Acceptance Model (TAM) is an important evaluation model. The TAM centers on

308 perceived usefulness (PU) and perceived ease of use (PEOU) as determinants[51]. The PU measures

309 perceived improvement in work performance, while the PEOU reflects ease of use

310 perception[51].These factors are instrumental in forecasting the adoption of emerging technologies.

311 However, to our knowledge, no meta-analysis has yet been conducted to evaluate the user

312 acceptance of AR/MR in medical education, such as PU, PEOU, and enjoyment.

313 In the context of knowledge acquisition, AR/MR technology is highly praised for its user-

314 friendly interface and the enjoyment value it brings to the user. Students reported elevated levels of

315 PEOU and enjoyment, signifying that AR/MR systems are largely seen as intuitive and effortless to

316 navigate. Nevertheless, the PU scores remain unremarkable. This suggests that while AR/MR may

317 excel in delivering an engaging educational experience, students do not believe that it can bring

318 measurable improvements in learning outcomes. The inconsistency between AR/MR-facilitated

15
319 achievements and traditional evaluation frameworks may weaken students' understanding of the

320 effectiveness of AR/MR.

321 In contrast, AR/MR has emerged as a powerhouse in skill training, where it demonstrably

322 heightens PU, PEOU, and enjoyment. Learners acknowledge AR/MR’s capacity to augment the

323 effectiveness of training, aligning with the findings of previous meta-analyses that highlighted AR’s

324 efficacy in enhancing learner satisfaction and proficiency in medical training exercises[12]. The

325 heightened sense of realism and interactivity intrinsic to AR/MR fosters an environment conducive

326 to skill acquisition, thereby reinforcing its PU.

327 To effectively implement AR/MR in medical education, it is important to carefully design the

328 course curriculum. Design courses tailored to the characteristics of AR/MR, fully tapping into the

329 unique value and potential of AR/MR technology in the field of medical education. Implementing

330 assessment models that accurately reflect the unique advantages of AR/MR can facilitate a clearer

331 understanding of learning outcomes among students, thereby increasing their acceptance of

332 AR/MR-integrated curricula. By addressing these key considerations, medical schools can optimize

333 the integration of AR/MR technology and create a more effective and engaging learning

334 environment for their students. By doing so, institutions can leverage the potential of AR/MR to

335 enhance both the theoretical comprehension and practical skill development of future healthcare

336 practitioners, thereby accelerating the development of medical education.

337 Limitations
338 The main limitation of this study is the high heterogeneity among the included studies. There

339 is significant heterogeneity in terms of research methods, outcome measures, data collection, and

340 reporting due to the different research topics covered by the included articles. This is not surprising

341 because there is considerable variation in medical curricula and educational interventions are

342 typically designed based on specific teaching content and targeted populations to meet specific

343 educational needs. High heterogeneity has also been observed in meta-analyses of other medical

344 and health sciences education-related interventions.

16
345 Conclusion
346 This meta-analysis demonstrated that AR/MR has been applied in medical education in Europe,

347 America, and other regions. Its applications include knowledge learning and skill training. When

348 used in skill training, AR/MR teaching methods exhibit significant advantages over traditional

349 methods in terms of mastering operational skills, reducing performance time, and minimizing

350 failure. AR/MR also receives significantly higher ratings in terms of PU, PEOU, and enjoyment

351 than traditional teaching methods. However, in knowledge learning, there is no significant

352 improvement in student performance. Although users appreciate AR/MR teaching and recognize

353 the user-friendliness of AR/MR devices, they do not think it is useful for knowledge learning. This

354 comprehensive analysis of AR/MR teaching provides foundational data for the future expansion of

355 AR/MR applications in education and the improvement of teaching effectiveness. Based on our

356 results, AR/MR can be applied in medical education where spatial thinking and visual perception

357 are required. Furthermore, assessment methods need to be improved to better capture and evaluate

358 students' learning outcomes in AR/MR-based learning environments. However, it is important to

359 consider the heterogeneity of the included studies, and further well-designed research is needed to

360 confirm our findings. Overall, this work provides a solid foundation for further exploration and

361 improvement of AR/MR applications in the context of medical education, to promote progress in

362 educational practice.

363 List of abbreviations

364 AR: Augmented reality; MR: Mixed reality; PEOU: Perceived ease of use; PU: Perceived

365 usefulness; RD: Risk difference; SMD: Standardization mean difference; WMD: Weighted mean

366 difference; TAM: Technology acceptance model.

367 Declarations

368 Funding

369 This study was funded by Macao Polytechnic University-funded research project

370 (Grant number RP/AE-04/2022).

371 Data availability statement

17
372 The datasets are available from the corresponding author upon reasonable request.

373 Authors' contributions

374 Ruoxuan Zhang: Conceptualization, Data curation, Formal analysis, Methodology, Software,

375 Writing – original draft, Writing – review & editing. Xiaoyan Jin: Conceptualization, Data curation,

376 Project administration, Writing – review & editing. Ming Liu: Conceptualization, Methodology,

377 Project administration, Supervision, Writing – review & editing. Hoi Yee Tong: Methodology,

378 Writing – review & editing. All authors have reviewed the manuscript and consented to participate.

379 Ethics approval and consent to participate

380 Not applicable. (This manuscript is a meta-analysis, and does not report on or involve the use

381 of any animal or human data or tissue.)

382 Consent for publication

383 Not applicable.

384 Declaration of competing interest

385 The authors have declared no conflict of interest.

386 Table S1 Databases searched and search terms utilized

Databases search terms
(“Augmented Reality”[Mesh] OR “Augmented Reality”[Title/Abstract] OR “Augmented
Realities”[Title/Abstract] OR “Reality, Augmented”[Title/Abstract] OR “Mixed
Reality”[Title/Abstract] OR “Mixed Realities”[Title/Abstract] OR “Reality,
Mixed”[Title/Abstract]) AND (Education*[Title/Abstract] OR Learn*[Title/Abstract] OR
1 Pubmed Teach*[Title/Abstract] OR Train*[Title/Abstract]) AND (Health[Title/Abstract] OR
Medicine[Title/Abstract] OR Medical[Title/Abstract] OR Healthcare[Title/Abstract] OR
“Health Care”[Title/Abstract]) AND ("Randomized Controlled Trial" [Publication Type] OR
"Randomized Controlled Trials As Topic"[Mesh] OR “Randomized Controlled Trials”
[Title/Abstract] OR “RCT”[Title/Abstract])
('augmented reality':ti,ab,kw OR 'augmented realities':ti,ab,kw OR 'reality, augmented':ti,ab,kw
OR 'mixed reality':ti,ab,kw OR 'mixed realities':ti,ab,kw OR 'reality, mixed':ti,ab,kw) AND
2 Embase (education*:ti,ab,kw OR learn*:ti,ab,kw OR teach*:ti,ab,kw OR train*:ti,ab,kw) AND
(health:ti,ab,kw OR medicine:ti,ab,kw OR medical:ti,ab,kw OR healthcare:ti,ab,kw OR 'health
care':ti,ab,kw OR nursery:ti,ab,kw) AND [randomized controlled trial]/lim
(('augmented reality' OR 'augmented realities' OR 'reality, augmented' OR 'mixed reality' OR
Cochrane Central
'mixed realities' OR 'reality, mixed') AND (education* OR learn* OR teach* OR train*) AND
3 Register of
(health OR medicine OR medical OR healthcare OR 'health care' OR nursery) AND
Controlled Trials
('Randomized Controlled Trials' OR 'RCT')):ti,ab,kw
AB ('augmented reality' OR 'augmented realities' OR 'reality, augmented' OR 'mixed reality' OR
4 PsycINFO
'mixed realities' OR 'reality, mixed') AND (education* OR learn* OR teach* OR train*) AND

18
 (health OR medicine OR medical OR healthcare OR 'health care' OR nursery) AND
('Randomized Controlled Trials' OR 'RCT')
SU ('augmented reality' OR 'augmented realities' OR 'reality, augmented' OR 'mixed reality' OR
Education 'mixed realities' OR 'reality, mixed') AND (education* OR learn* OR teach* OR train*) AND
5
Source(Ebsco) (health OR medicine OR medical OR healthcare OR 'health care' OR nursery) AND
('Randomized Controlled Trials' OR 'RCT')
Education AB ('augmented reality' OR 'augmented realities' OR 'reality, augmented' OR 'mixed reality' OR
Resources 'mixed realities' OR 'reality, mixed') AND (education* OR learn* OR teach* OR train*) AND
6
Information (health OR medicine OR medical OR healthcare OR 'health care' OR nursery) AND
Center(ERIC) ('Randomized Controlled Trials' OR 'RCT')
AB ('augmented reality' OR 'augmented realities' OR 'reality, augmented' OR 'mixed reality' OR
CINAHL 'mixed realities' OR 'reality, mixed') AND (education* OR learn* OR teach* OR train*) AND
7
(EBSCO) (health OR medicine OR medical OR healthcare OR 'health care' OR nursery) AND
('Randomized Controlled Trials' OR 'RCT')
TS=('augmented reality' OR 'augmented realities' OR 'reality, augmented' OR 'mixed reality' OR
Web of Science 'mixed realities' OR 'reality, mixed') AND TS=(education* OR learn* OR teach* OR train*)
8
Core Collection AND TS=(health OR medicine OR medical OR healthcare OR 'health care' OR nursery) AND
TS=('Randomized Controlled Trials' OR 'RCT')
Chinese (增强现实 OR 混合现实 OR AR技术 OR MR技术) AND (医学 OR 医疗 OR 卫生 OR 保健
9 BioMedical
OR 健康) AND (教育 OR 培训 OR 教学) AND (随机对照实验 OR RCT)
Literature (CBM)
China National TKA=(增强现实 OR 混合现实 OR AR技术 OR MR技术) AND TKA=(医学 OR 医疗 OR 卫
Knowledge
10 生 OR 保健 OR 健康) AND TKA=(教育 OR 培训 OR 教学) AND TKA=(随机对照实验 OR
Infrastructure
(CNKI) RCT)

摘要:(增强现实 OR 混合现实 OR AR技术 OR MR技术) AND 摘要:(医学 OR 医疗 OR 卫

11 WanFang 生 OR 保健 OR 健康) AND 摘要:(教育 OR 培训 OR 教学) AND 摘要:(随机对照实验 OR

RCT)

R=(增强现实 OR 混合现实 OR AR技术 OR MR技术) AND R=(医学 OR 医疗 OR 卫生 OR

12 WeiPu
保健 OR 健康) AND R=(教育 OR 培训 OR 教学) AND R=(随机对照实验 OR RCT)

387

388 Table S2 Egger's test assessing the publication bias

Group p value (Egger's test)
Skill scores 0.222
Failure rate 0.017*
Performance time 0.402
Knowledge scores 0.149
Perceived usefulness (PU) 0.539
Perceived ease of use (PEOU) 0.782
Enjoyment 0.152
19
389 *p＜0.05. This analysis revealed significant publication bias.

390
391 Figure S1 Sensitivity analysis assessing the influence of individual studies on the pooled analysis.
392 The underline(—)indicates that after excluding this study, the primary pooled effect has been

20
393 changed.

394
395 Figure S2 Forest plot for the effects of AR/MR on PU compared to control group

396

397 Figure S3 Forest plot for the effects of AR/MR on PEOU compared to control group
21
398

399 Figure S4 Forest plot for the effects of AR/MR on enjoyment compared to control group

400 1. Bruno RR, Wolff G, Wernly B, Masyuk M, Piayda K, Leaver S, Erkens R, Oehler D, Afzal S, Heidari H
401 et al: Virtual and augmented reality in critical care medicine: the patient's, clinician's, and researcher's
402 perspective. Crit Care 2022, 26(1):326. doi.org/10.1186/s13054-022-04202-x.
403 2. Dinh A, Yin AL, Estrin D, Greenwald P, Fortenko A: Augmented Reality in Real-time Telemedicine and
404 Telementoring: Scoping Review. JMIR Mhealth Uhealth 2023, 11:e45464. doi.org/10.2196/45464.
405 3. U.S.Food & Drug administration: Augmented Reality and Virtual Reality in Medical Devices.,
406 https://www.fda.gov/medical-devices/digital-health-center-excellence/augmented-reality-and-virtual-
407 reality-medical-devices.
408 4. Uruthiralingam U, Rea PM: Augmented and Virtual Reality in Anatomical Education - A Systematic
409 Review. Adv Exp Med Biol 2020, 1235:89-101. doi.org/10.1007/978-3-030-37639-0_5.
410 5. Sun P, Zhao Y, Men J, Ma ZR, Jiang HZ, Liu CY, Feng W: Application of Virtual and Augmented Reality
411 Technology in Hip Surgery: Systematic Review. J Med Internet Res 2023, 25:e37599.
412 doi.org/10.2196/37599.
413 6. Felinska EA, Fuchs TE, Kogkas A, Chen ZW, Otto B, Kowalewski KF, Petersen J, Muller-Stich BP,
414 Mylonas G, Nickel F: Telestration with augmented reality improves surgical performance through gaze
415 guidance. Surg Endosc 2023, 37(5):3557-3566. doi.org/10.1007/s00464-022-09859-7.
416 7. Munzer BW, Khan MM, Shipman B, Mahajan P: Augmented Reality in Emergency Medicine: A Scoping
417 Review. J Med Internet Res 2019, 21(4):e12368. doi.org/10.2196/12368.
418 8. Garcia-Robles P, Cortes-Perez I, Nieto-Escamez FA, Garcia-Lopez H, Obrero-Gaitan E, Osuna-Perez MC:
419 Immersive virtual reality and augmented reality in anatomy education: A systematic review and meta-
420 analysis. Anat Sci Educ 2024, 17(3):514-528. doi.org/10.1002/ase.2397.
421 9. Bui T, Ruiz-Cardozo MA, Dave HS, Barot K, Kann MR, Joseph K, Lopez-Alviar S, Trevino G, Brehm S,

22
422 Yahanda AT et al: Virtual, Augmented, and Mixed Reality Applications for Surgical Rehearsal, Operative
423 Execution, and Patient Education in Spine Surgery: A Scoping Review. Medicina (Kaunas) 2024, 60(2).
424 doi.org/10.3390/medicina60020332.
425 10. Yoo S, Son MH: Virtual, augmented, and mixed reality: potential clinical and training applications in
426 pediatrics. Clin Exp Pediatr 2024, 67(2):92-103. doi.org/10.3345/cep.2022.00731.
427 11. Veer V, Phelps C, Moro C: Incorporating Mixed Reality for Knowledge Retention in Physiology, Anatomy,
428 Pathology, and Pharmacology Interdisciplinary Education: A Randomized Controlled Trial. Med Sci Educ
429 2022, 32(6):1579-1586. doi.org/10.1007/s40670-022-01635-5.
430 12. Baashar Y, Alkawsi G, Ahmad WNW, Alhussian H, Alwadain A, Capretz LF, Babiker A, Alghail A:
431 Effectiveness of Using Augmented Reality for Training in the Medical Professions: Meta-analysis. JMIR
432 Serious Games 2022, 10(3):e32715. doi.org/10.2196/32715.
433 13. Cheng A, Fijacko N, Lockey A, Greif R, Abelairas-Gomez C, Gosak L, Lin Y, Education Implementation
434 Team Task Force of the International Liaison Committee on R: Use of augmented and virtual reality in
435 resuscitation training: A systematic review. Resusc Plus 2024, 18:100643.
436 doi.org/10.1016/j.resplu.2024.100643.
437 14. Aromataris E MZ: JBI Manual for Evidence Synthesis. In., 2020 edn: Joanna Briggs Institute.; 2020.
438 15. Bogomolova K, Vorstenbosch M, El Messaoudi I, Holla M, Hovius SER, van der Hage JA, Hierck BP:
439 Effect of binocular disparity on learning anatomy with stereoscopic augmented reality visualization: A
440 double center randomized controlled trial. Anat Sci Educ 2023, 16(1):87-98. doi.org/10.1002/ase.2164.
441 16. Cizmic A, Muller F, Wise PA, Haberle F, Gabel F, Kowalewski KF, Bintintan V, Muller-Stich BP, Nickel
442 F: Telestration with augmented reality improves the performance of the first ten ex vivo porcine
443 laparoscopic cholecystectomies: a randomized controlled study. Surg Endosc 2023, 37(10):7839-7848.
444 doi.org/10.1007/s00464-023-10360-y.
445 17. d'Aiello AF, Cabitza F, Natali C, Vigano S, Ferrero P, Bognoni L, Pasqualin G, Giamberti A, Chessa M:
446 The Effect of Holographic Heart Models and Mixed Reality for Anatomy Learning in Congenital Heart
447 Disease: An Exploratory Study. J Med Syst 2023, 47(1):64. doi.org/10.1007/s10916-023-01959-8.
448 18. Farshad-Amacker NA, Kubik-Huch RA, Kolling C, Leo C, Goldhahn J: Learning how to perform
449 ultrasound-guided interventions with and without augmented reality visualization: a randomized study.
450 Eur Radiol 2023, 33(4):2927-2934. doi.org/10.1007/s00330-022-09220-5.
451 19. Guha P, Lawson J, Minty I, Kinross J, Martin G: Can mixed reality technologies teach surgical skills better
452 than traditional methods? A prospective randomised feasibility study. BMC Med Educ 2023, 23(1):144.
453 doi.org/10.1186/s12909-023-04122-6.
454 20. Hayasaka T, Kawano K, Onodera Y, Suzuki H, Nakane M, Kanoto M, Kawamae K: Comparison of
455 accuracy between augmented reality/mixed reality techniques and conventional techniques for epidural
456 anesthesia using a practice phantom model kit. BMC Anesthesiol 2023, 23(1):171.
457 doi.org/10.1186/s12871-023-02133-w.
458 21. Malik R, Abbas JR, Jayarajah C, Bruce IA, Tolley N: Mixed Reality Enhanced Otolaryngology Case-
459 Based Learning: A Randomized Educational Study. Laryngoscope 2023, 133(7):1606-1613.
460 doi.org/10.1002/lary.30364.
461 22. An J, Oh J, Park K: Self-Regulated Learning Strategies for Nursing Students: A Pilot Randomized
462 Controlled Trial. Int J Environ Res Public Health 2022, 19(15). doi.org/10.3390/ijerph19159058.
463 23. Hou L, Dong X, Li K, Yang C, Yu Y, Jin X, Shang S: Comparison of Augmented Reality-assisted and
464 Instructor-assisted Cardiopulmonary Resuscitation: A Simulated Randomized Controlled Pilot Trial. Clin
465 Simul Nurs 2022, 68:9-18. doi.org/10.1016/j.ecns.2022.04.004.

23
466 24. Little WB, Dezdrobitu C, Conan A, Artemiou E: Is Augmented Reality the New Way for Teaching and
467 Learning Veterinary Cardiac Anatomy? Med Sci Educ 2021, 31(2):723-732. doi.org/10.1007/s40670-021-
468 01260-8.
469 25. Nagayo Y, Saito T, Oyama H: Augmented reality self-training system for suturing in open surgery: A
470 randomized controlled trial. Int J Surg 2022, 102:106650. doi.org/10.1016/j.ijsu.2022.106650.
471 26. Pickering JD, Panagiotis A, Ntakakis G, Athanassiou A, Babatsikos E, Bamidis PD: Assessing the
472 difference in learning gain between a mixed reality application and drawing screencasts in neuroanatomy.
473 Anat Sci Educ 2022, 15(3):628-635. doi.org/10.1002/ase.2113.
474 27. Yohannan DG, Oommen AM, Amogh BJ, Raju NK, Suresh RO, Nair SJ: "Air Anatomy" - Teaching
475 Complex Spatial Anatomy Using Simple Hand Gestures. Anat Sci Educ 2022, 15(3):552-565.
476 doi.org/10.1002/ase.2088.
477 28. Chen CJ, Chen YC, Lee MY, Wang CH, Sung HC: Effects of three-dimensional holograms on the academic
478 performance of nursing students in a health assessment and practice course: A pretest-intervention-posttest
479 study. Nurse Educ Today 2021, 106:105081. doi.org/10.1016/j.nedt.2021.105081.
480 29. Kamonwon Ienghong PK, Thanat Tangpaisarn, Korakot Apiratwarakul: Practical Emergency Ultrasound
481 Flashcards with Augmented Reality in Teaching Point-of-Care Ultrasound in ER. Open Access Maced J
482 Med Sci [Internet] 2021, 9:39-42. doi.org/https://doi.org/10.3889/oamjms.2021.5566.
483 30. Liu S, Xie M, Zhang Z, Wu X, Gao F, Lu L, Zhang J, Xie Y, Yang F, Ye Z: A 3D Hologram With Mixed
484 Reality Techniques to Improve Understanding of Pulmonary Lesions Caused by COVID-19: Randomized
485 Controlled Trial. J Med Internet Res 2021, 23(9):e24081. doi.org/10.2196/24081.
486 31. Weeks JK, Pakpoor J, Park BJ, Robinson NJ, Rubinstein NA, Prouty SM, Nachiappan AC: Harnessing
487 Augmented Reality and CT to Teach First-Year Medical Students Head and Neck Anatomy. Acad Radiol
488 2021, 28(6):871-876. doi.org/10.1016/j.acra.2020.07.008.
489 32. Bogomolova K, van der Ham IJM, Dankbaar MEW, van den Broek WW, Hovius SER, van der Hage JA,
490 Hierck BP: The Effect of Stereoscopic Augmented Reality Visualization on Learning Anatomy and the
491 Modifying Effect of Visual-Spatial Abilities: A Double-Center Randomized Controlled Trial. Anat Sci
492 Educ 2020, 13(5):558-567. doi.org/10.1002/ase.1941.
493 33. Schoeb DS, Schwarz J, Hein S, Schlager D, Pohlmann PF, Frankenschmidt A, Gratzke C, Miernik A:
494 Mixed reality for teaching catheter placement to medical students: a randomized single-blinded,
495 prospective trial. BMC Med Educ 2020, 20(1):510. doi.org/10.1186/s12909-020-02450-5.
496 34. Stojanovska M, Tingle G, Tan L, Ulrey L, Simonson-Shick S, Mlakar J, Eastman H, Gotschall R, Boscia
497 A, Enterline R et al: Mixed Reality Anatomy Using Microsoft HoloLens and Cadaveric Dissection: A
498 Comparative Effectiveness Study. Med Sci Educ 2020, 30(1):173-178. doi.org/10.1007/s40670-019-
499 00834-x.
500 35. Wang C, Daniel BK, Asil M, Khwaounjoo P, Cakmak YO: A Randomised Control Trial and Comparative
501 Analysis of Multi-Dimensional Learning Tools in Anatomy. Sci Rep 2020, 10(1):6120.
502 doi.org/10.1038/s41598-020-62855-6.
503 36. Noll C, von Jan U, Raap U, Albrecht UV: Mobile Augmented Reality as a Feature for Self-Oriented,
504 Blended Learning in Medicine: Randomized Controlled Trial. JMIR Mhealth Uhealth 2017, 5(9):e139.
505 doi.org/10.2196/mhealth.7943.
506 37. Leitritz MA, Ziemssen F, Suesskind D, Partsch M, Voykov B, Bartz-Schmidt KU, Szurman GB: Critical
507 evaluation of the usability of augmented reality ophthalmoscopy for the training of inexperienced
508 examiners. Retina 2014, 34(4):785-791. doi.org/10.1097/IAE.0b013e3182a2e75d.
509 38. Vera AM, Russo M, Mohsin A, Tsuda S: Augmented reality telementoring (ART) platform: a randomized

24
510 controlled trial to assess the efficacy of a new surgical education technology. Surg Endosc 2014,
511 28(12):3467-3472. doi.org/10.1007/s00464-014-3625-4.
512 39. Albrecht UV, Folta-Schoofs K, Behrends M, von Jan U: Effects of mobile augmented reality learning
513 compared to textbook learning on medical students: randomized controlled pilot study. J Med Internet Res
514 2013, 15(8):e182. doi.org/10.2196/jmir.2497.
515 40. Wilson KL, Doswell JT, Fashola OS, Debeatham W, Darko N, Walker TM, Danner OK, Matthews LR,
516 Weaver WL: Using augmented reality as a clinical support tool to assist combat medics in the treatment
517 of tension pneumothoraces. Mil Med 2013, 178(9):981-985. doi.org/10.7205/MILMED-D-13-00074.
518 41. Yeo CT, Ungi T, P UT, Lasso A, McGraw RC, Fichtinger G: The effect of augmented reality training on
519 percutaneous needle placement in spinal facet joint injections. IEEE Trans Biomed Eng 2011, 58(7):2031-
520 2037. doi.org/10.1109/TBME.2011.2132131.
521 42. Ju Y, Lee M, You M: Comparing the Effect of Declarative vs. Procedural Knowledge on Health Behavior
522 Moderated by Outrage Factors. Health Commun 2023, 38(14):3243-3251.
523 doi.org/10.1080/10410236.2022.2145752.
524 43. Barteit S, Lanfermann L, Barnighausen T, Neuhann F, Beiersmann C: Augmented, Mixed, and Virtual
525 Reality-Based Head-Mounted Devices for Medical Education: Systematic Review. JMIR Serious Games
526 2021, 9(3):e29080. doi.org/10.2196/29080.
527 44. Dhar P, Rocks T, Samarasinghe RM, Stephenson G, Smith C: Augmented reality in medical education:
528 students' experiences and learning outcomes. Med Educ Online 2021, 26(1):1953953.
529 doi.org/10.1080/10872981.2021.1953953.
530 45. Moro C, Birt J, Stromberga Z, Phelps C, Clark J, Glasziou P, Scott AM: Virtual and Augmented Reality
531 Enhancements to Medical and Science Student Physiology and Anatomy Test Performance: A Systematic
532 Review and Meta-Analysis. Anat Sci Educ 2021, 14(3):368-376. doi.org/10.1002/ase.2049.
533 46. Kucuk S, Kapakin S, Goktas Y: Learning anatomy via mobile augmented reality: Effects on achievement
534 and cognitive load. Anat Sci Educ 2016, 9(5):411-421. doi.org/10.1002/ase.1603.
535 47. McBain KA, Habib R, Laggis G, Quaiattini A, N MV, Noel G: Scoping review: The use of augmented
536 reality in clinical anatomical education and its assessment tools. Anat Sci Educ 2022, 15(4):765-796.
537 doi.org/10.1002/ase.2155.
538 48. Lamb A, McKinney B, Frousiakis P, Diaz G, Sweet S: A Comparative Study of Traditional Technique
539 Guide versus Virtual Reality in Orthopedic Trauma Training. Adv Med Educ Pract 2023, 14:947-955.
540 doi.org/10.2147/AMEP.S395087.
541 49. Stefanidis D, Korndorffer JR, Jr., Heniford BT, Scott DJ: Limited feedback and video tutorials optimize
542 learning and resource utilization during laparoscopic simulator training. Surgery 2007, 142(2):202-206.
543 doi.org/10.1016/j.surg.2007.03.009.
544 50. El Nsouli D, Nelson D, Nsouli L, Curtis F, Ahmed SI, McGonagle I, Kane R, Ahmadi K: The Application
545 of Kirkpatrick's Evaluation Model in the Assessment of Interprofessional Simulation Activities Involving
546 Pharmacy Students: A Systematic Review. Am J Pharm Educ 2023, 87(8):100003.
547 doi.org/10.1016/j.ajpe.2023.02.003.
548 51. Felber NA, Lipworth W, Tian YJA, Roulet Schwab D, Wangmo T: Informing existing technology
549 acceptance models: a qualitative study with older persons and caregivers. Eur J Ageing 2024, 21(1):12.
550 doi.org/10.1007/s10433-024-00801-5.
551
552

25
Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.

PRISMA2020checklist.docx