Structured training on the da Vinci Skills Simulator leads to improvement
ORIGINAL ARTICLE
in technical performance of robotic novices
€ rtsch, A.,* Teymoortash, A.,* Dworschak, P.,† Werner, J.A.*
Walliczek-Dworschak, U.,* Mandapathil, M.,* Fo
& Gu€ ldner, C.*
*Department of Otorhinolaryngology–Head and Neck Surgery, University Hospital of Marburg, University of Giessen and Marburg,
Marburg, Germany †Center of Orthopedics and Traumatology, University Hospital of Marburg, University of Giessen and Marburg,
Marburg, Germany
Accepted for publication 25 April 2016
Clin. Otolaryngol. 2016, 00, 000–000
Background: The increasing use of minimally invasive
techniques such as robotic-assisted devices raises the question
of how to acquire robotic surgery skills. The da Vinci Skills
Simulator has been demonstrated to be an effective training
tool in previous reports. To date, little data are available on
how to acquire proficiency through simulator training. We
investigated the outcome of a structured training programme
for robotic surgical skills by robotic novices.
Methods: This prospective study was conducted from
January to December 2013 using the da Vinci Skills
Simulator. Twenty participants, all robotic novices, were
enrolled in a 4-week training curriculum. After a brief
introduction to the simulator system, three consecutive
repetitions of five selected exercises (Match Board 1, 2, 3 and
Ring and Rail 1, 2) were performed in a defined order on days
1, 8, 15 and 22. On day 22, one repetition of a previously
unpractised more advanced module (Needle Targeting) was
also performed. After completion of each study day, the
One of the most promising innovations in surgery is the
development of robotic surgical systems such as the da Vinci
surgical master–slave system. This development in medicine
was first promoted by the US Department of Defense in
order to be able to operate on wounded soldiers without the
physical presence of a surgeon.1 The da Vinci surgical system
is named after the famous inventor, anatomist and painter:
da Vinci. His study of human anatomy potentially led to the
first known robot in history, ‘Leonardo’s Robot’, which was
probably constructed around the year 1495.2 The robot was
rediscovered in the 1950s and since then has been developed
further. This was the starting point for robotic development
Correspondence: C. G€uldner, University hospital of Marburg, Department
of ENT, Head and Neck Surgery, Baldingerstrasse, 35043 Marburg,
Germany. Tel.: 004964215869052; Fax: 004964215866367; e-Mail:
gueldner@staff.uni-marburg.de
© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology
overall performance, time to completion, economy in
motion, instrument collisions, excessive instrument force,
instruments out of view, master workspace range and
number of drops were analysed.
Results: Comparing the first and final repetition, overall
score and time needed to complete all exercises, economy of
motion and instrument collisions were significantly
improved in nearly all exercises.Regarding the new exercise, a
positive training effect could be demonstrated. While its
overall entry score was significantly higher, the time to
completion and economy of motion were significantly lower
than the scores on the first repetition of the previous 5
exercises.
Conclusions: It could be shown that training on the da
Vinci Skills Simulator led to an improvement in technical
performance of robotic novices. With regard to a new
exercise, the training had a positive effect on the technical
performance.
in medicine. At the same time minimally invasive surgery has
been developed and introduced in medicine. The break-
through came in 1987 when Philippe Mouret performed the
first laparoscopic cholecystectomy. Technical limitations,
notably the rigid equipment and two-dimensional view of
laparoscopic surgery, supported the development of a system
to overcome those limitations: robotic-assisted minimally
invasive surgery. In 1999, the da Vinci System was launched
on the market by Intuitive Surgical (Sunnyvale, California,
USA). It has become the most common robotic system in
clinical use to the present day.3 The structure consists of the
following 4 components: a surgeon console where the
specialist sits while operating, a patient-side cart where
the patient is positioned during surgery, two to three
interactive robotic arms that carry out the surgeon’s
commands and a high-definition three-dimensional vision
system from inside the patient’s body and EndoWrist
1
2 U. Walliczek-Dworschak et al.
instruments.2 The benefits of robotic-assisted procedures are
especially seen in reduction of blood loss, decreased tissue
damage, reduction of post-operative pain and shorter
recovery times.4 Against this background, otorhinolarnyn-
gology surgeons are more and more interested in using
robotic techniques in the operating theatre. The first pre-
clinical tests with robots in the head and neck area were
performed in 1994 by Kavanagh.5 The first robotic-assisted
intervention in the field of ear–nose–throat (ENT) surgery
was an excision of a vallecular cyst in 2005.6 To the present
day, there are various options for using robot-assisted
surgery in the head and neck area; for example, in the
management of oropharyngeal squamous cell carcinoma,
transoral robotic surgery (TORS) is commonly accepted and
used.7, 8 Furthermore, it can be used in the field of thyroid
and obstructive sleep–apnoea–hypopnoea syndrome sur-
gery.9, 10 Single-case reports can be found about robotic-
assisted procedures in nasopharyngeal and recurrent
nasopharyngeal carcinoma.11, 12 Possible applications of
robot-assisted sinus and skull base surgery have been
reported.13 With the growing integration of robotic-assisted
surgery in the head and neck area, an increasing number of
otorhinolaryngology residents will be confronted with
robotic surgery. Special skills, which differ from techniques
used in laparoscopic or open surgery, are required in robotic
surgery such as camera control, three-dimensional visuali-
sation of the operative field, EndoWrist instrument manip-
ulation and clutching as well as economic use of the hands
and instruments. The integration of robotic surgery in
different medical disciplines raises the question of how to
prepare medical specialists. Various virtual reality simula-
tion systems, for example the da Vinci Skills Simulator
(dVSS), have therefore been introduced onto the market. It
works with proprietary, validated training software (Mimic
Technologies, Seattle, WA) and contains more than 30
training exercises, including tasks that mainly train basic
skills such as camera and clutch control in three-dimensional
visualisation. Furthermore, there are more advanced mod-
ules that concentrate on energy and needle control. A great
advantage of the dVSS is the integration of the original Si
surgeon console with the da Vinci robot, so the trainee
acquires skills using the same EndoWrist instruments and
finger clutch controls that are used in the operating theatre.
The aim is to increase familiarity with the da Vinci surgical
system so as to optimise handling with the robotic system,
especially with the movement of instruments, to make the
residents feel safe, shorten their learning curves for different
interventions and, last but not least, improve surgical
safety.14 Another advantage of the simulator is the possibility
for it to be used in a safe and controlled environment not
connected to the patient. The interface, content and
construct validity of the dVSS have been demonstrated in
various studies.15, 16 Tergas et al. investigated the results of
training with the dvSS regarding time to completion and
economy of motion and reported that both markers
improved significantly.17 By means of the simulation
programme, trainees and expert mentors can see with the
help of a given detailed performance report generated by the
software how they have performed on a given exercise, how
to avoid errors and observe progress over time. To date, there
are no published studies on how useful the offered exercises
are and how often they need to be repeated to prepare the
surgeon sufficiently. Up to the present day, training for
robotic skills remains mainly unstructured.
In this study, basic skills were trained with the help of
Match Board (MB) and Ring and Rail (RR) exercises to
evaluate and describe learning curves referring to time to
complete the exercise, economy in motion, instrument
collisions, excessive instrument force, instruments out of
view, master workspace range, drops, missed targets and
overall performance score. At the end, a new previously
unpractised exercise, Needle Targeting (NT), which is a
more advanced task, was executed. The study objective was
to analyse the learning curve of robotic simulation training
novices and to analyse the skill transfer to a new more
advanced task in order to improve and establish future
robotic surgical curricula.
Methods and materials
Participants
This study was a single-centre, prospective study performed
between January 2013 and December 2013. Twenty test
persons were enrolled in a 4-week training curriculum on the
dVSS in the Department of Otorhinolaryngology–Head and
Neck Surgery, University Hospital of Marburg, Germany
(Table 1). The recruited participants were surgeons within
the first and fifth year of training and had no robotic surgical
experience (performed zero robotic operations and did not
have experience with the dVSS). The study was approved by
the local ethics committee.
Training platform and curriculum
The da Vinci Skills Simulator was chosen for the robotic
training programme. The primary surgeon console of the
da Vinci robot was used without the need for the patient-
side cart or instruments. The dVSS works with Mimic
virtual reality training software (Mimic Technologies,
Seattle, WA), which contains a variety of exercises
specially developed to give trainees the opportunity to
acquire specific robotic skills and improve their handling
of the da Vinci surgeon console.
© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology
 Structured training at the Da Vinci Simulator 3

Table 1. Schedule of the da Vinci Skills Simulator curriculum and the several repetitions presented
Weekday 1 = Weekday 8 = Weekday 15 = Weekday 22 =
First day of Second day Third day of Fourth day of
practice of practice practice practice

Exercise 1 3 times 3 times 3 times 3 times

Match Board 1 Repetition 1–3 Repetition 4–6 Repetition 7–9 Repetition 10–12
Exercise 2 3 times 3 times 3 times 3 times
Match Board 2 Repetition 1–3 Repetition 4–6 Repetition 7–9 Repetition 10–12
Exercise 3 3 times 3 times 3 times 3 times
Match Board 3 Repetition 1–3 Repetition 4–6 Repetition 7–9 Repetition 10–12
Exercise 4 3 times 3 times 3 times 3 times
Ring & Rail 1 Repetition 1–3 Repetition 4–6 Repetition 7–9 Repetition 10–12
Exercise 5 3 times 3 times 3 times 3 times
Ring & Rail 2 Repetition 1–3 Repetition 4–6 Repetition 7–9 Repetition 10–12
New Exercise 1 time
Needle Targeting

The training session was initiated with a brief explanation
of the da Vinci system provided by the study team to
familiarise the participants with the da Vinci console and its
operation. Afterwards, the test subjects were able to proceed
autonomously during the practice. The test subjects per-
formed a total of five pre-defined exercises on 4 week days a
total of three times in a row (Table 1). The exercises that
were performed were chosen from the programme:
‘EndoWrist Manipulation’. The following exercises were
selected in the defined order: Match Board 1, Match Board 2,
Match Board 3, Ring and Rail 1 and Ring and Rail 2. The level
of difficulty increased from Match Board 1 to 3 and from
Ring and Rail 1 to 2. On the last study day, a previously
untrained practice was added: Needle Targeting. This prac-
tice was presented and explained with the help of video
documentation and was completed just one time. All of the
above-mentioned exercises provide diverse abilities to
trainees, which are believed to be necessary for safe
performance of robotic surgery.
Within the ‘MB 1, 2, 3’ exercises, six–three-dimensional
wooden letters and three numbers needed to be positioned in
corresponding panels. The simulation programme marked
the correct position with green colouring (Fig. 1). From MB
1 towards MB 3, the required dexterity increased. In MB 2, in
contrast to MB 1, an extra robotic arm needed to be used, and
in MB 3, the difficulty level was raised again since a third arm
was required to free the pattern cut-outs wherein the objects
needed to be placed.
The following exercises ‘RR 1, 2’ dealt with a ring that was
led along a rod to the correct position. When the ring reached
the correct location, the exercise was estimated as successful
followed by a green colour change (Fig. 1). Within the RR 1
exercise, a ring needed to be led along a twisted rod. In the
more difficult exercise RR 2, three coloured rings were led
along several twisted rods towards the same colour goal.
The final exercise ‘Needle Targeting’ consisted of the
correct positioning of coloured needles in two identical
coloured targets of different sizes. First the larger one and
then the smaller one needed to be pierced.
After having completed successfully the repetition of an
exercise, the simulation terminated automatically and a
detailed performance report generated by the Mimic da
Vinci Trainer software was generated (Fig. 2). The measured
metrics were as follows: time to complete the exercise,
economy in motion, number of instrument collisions,
excessive instrument force, instruments out of view, master
workspace range, number of drops and the overall perfor-
mance score (all metrics combined), and for the exercise
Needle Targeting, the number of missed targets was
indicated additionally (Table 2). The overall score is calcu-
lated from the above measured metrics by the software based
on a defined ratio (Table 2).
Statistics
The statistical data analysis was fulfilled with the program
SPSS 22.0 (SAS Statistics, Cary, NC, USA). The performed
statistical test was the two-sample test for dependent
variables under the assumption of equal variance of the
population. The test was performed with the arithmetic
mean of the overall score and, respectively, the individual
parameters of one session day, in accordance with the
exercises. A P-value ≤0.05 was considered statistically
significant.
Results
A total of 20 participants (10 female, 10 male with a mean age
of 24.2 years  1.4 years) were enrolled in this study. None
had experience either with the da Vinci robot or with the

© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology

4 U. Walliczek-Dworschak et al.

Fig. 1. An overview of the performed training programme is given: (1) Match Board (1; 2) Match Board (2; 3) Match Board (3; 4) Ring & Rail
(1; 5) Ring & Rail (2; 6) Needle Control

Fig. 2. Sample of the detailed performance report generated by the software after one exercise. The colour (red–yellow–green) shows, in
accordance with a traffic light, the level of performance of the subject in each parameter and overall.

© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology

 Structured training at the Da Vinci Simulator 5

Table 2. Description of the single parameters based on which the overall score is calculated (* just for Needle Targeting)
Parameter Description

1. Time to Complete Exercise (in s)
2. Economy of Motion (in cm)
3. Instrument Collisions
4. Excessive Instrument Force (in sek.)
5. Instruments Out of view (in cm)
6. Master Workspace Range (in cm)
7. Drops (in n)
8. (Missed Targets*
Total time the users spends on exercise
Total distance travelled by all instruments
Total number of instruments-on-instruments collisions exceeding
a minimum force threshold
Total time an excessive instrument force is applied above a prescribed threshold force.
Forces from instruments can arise from collisions with each other and from action
such as tissue retraction, driving a needle and pulling on suture
Total distance travelled by instruments outside the user0 s field of view.
Radius of user’s working volume on master grips
Number of times any object is dropped in an inappropriate region of the scene
Number of missed targets)

dVSS. The 80% proficiency benchmark was considered
acceptable, only taking the standard deviation into account.
The exercises were projected on a screen so that the study
crew could take part and give feedback after each exercise.
The fact of the 80% efficacy level results from a recommen-
dation of the company and is therefore mentioned for each
exercise in the following part.
Match board 1 (MB1)
Within exercise 1, the overall performance was over 80% after
the seventh repetition (repetition 8: overall performance
88.15  6.56). The maximal overall performance attained
after the last repetition was 90.0  6.01. The overall perfor-
mance improved significantly between the first and the twelfth
repetitions (P < 0.001) (Fig. 3). The time to completion was
twice as fast after the seventh repetition (138.35  29.81 s)
compared to the first passage (294.75  87.13 s). The time to
completion decreased significantly between the first and last
repetitions (P < 0.001) (Fig. 4).
In repetition 12 compared to the first repetition, the
economy of motion (P < 0.001), the number of instrument
collisions (P = 0.001) and master workspace range
(P = 0.004) improved significantly (Table 3).

Fig. 3. Overall performance measured % (min 0%, max 100%) for exercises 1–5 (see also Figure 2) comparing the several exercises and the
new exercise (red box on the right side)

© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology

6 U. Walliczek-Dworschak et al.
Fig. 4. Time needed for completion for the several exercises 1–5 (see also Figure 2) in comparison with each other and the new exercise (red
box on the right side).
Match board 2 (MB2)
Similar findings were made for exercise 2. The overall
performance was over 80% after the sixth repetition
(repetition 7: overall performance 86  4.91). The maximal
overall performance achieved after the last repetition was
93.0  6.01. The overall performance improved signifi-
cantly between the first and last repetitions (P < 0.001)
(Fig. 3). The time to completion was halved after the eighth
repetition (106.6  16.32 s) compared to the first passage
(217.85  45.25 s). The time to completion decreased
significantly between the first and last repetitions
(P < 0.001) (Fig. 4). Between repetition 1 and repetition
12, the economy of motion (P < 0.001), number of instru-
ment collisions (P = 0.036), total time an excessive instru-
ment force was applied (P < 0.001), master workspace range
(P = 0.004) and number of drops (P = 0.048) improved
significantly (Table 3).
Match board 3 (MB3)
In exercise 3, a mean overall performance >80% was not
reached. The maximal overall performance after the last
repetition was 86.1  7.66. The overall performance
improved significantly between the first and twelfth
repetitions (P < 0.001) (Fig. 3). The time to completion
was halved after the eighth repetition (264.6  42.54 s)
compared to the first passage (561.85  118.53 s). The
time to completion decreased significantly between the
first and last repetitions (P < 0.001) (Fig. 4). Between
repetition 1 and repetition 12, the economy of motion
(P < 0.001), the number of instrument collisions
(P = 0.003), the total time an excessive instrument force
was applied (P < 0.001), the total distance travelled by
instruments outside the user0 s field of view (P < 0.001)
and the number of drops (P = 0.036) improved signifi-
cantly (Table 3).
Ring and rail 1 (RR1)
A mean overall performance >80% was already achieved
after the second repetition (repetition 3: overall performance
89.6  9.18). The maximal overall performance after the last
repetition was 96.35  4.82. The overall performance
improved significantly between the first and twelfth repeti-
tions (P < 0.001) (Fig. 3). The time to completion was
halved even after the sixth repetition (29.5  10.34 s)
compared to the first passage (59.45  20.54 s). The time
© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology
 Structured training at the Da Vinci Simulator 7

to completion decreased significantly between the first and

last repetitions (P < 0.001) (Fig. 4). In repetition 12 com-

0.789

0.055

0.042

0.330

0.789
P
pared to repetition 1, the economy of motion (P < 0.001)
improved significantly (Table 3).

0.15  0.67

0.05  0.22

0.35  0.49
0.1  0.45
0.3  0.66

0.4  0.82

0.3  0.57
0.0  0.0

0.0  0.0

0.0  0.0
MeanSD
Drops
Table 3. Comparison of the first and last repetitions of the several parameters for the five exercises (1 – Match Board 1; 2 – Match Board 2; 3 – Match Board 3;

Ring and rail 2 (RR2)

0.007

0.001

0.049

0.723

0.035
In exercise 5, a mean overall performance >80% was not
P

reached. The maximal overall performance achieved after

12.35  2.23
10.20  2.19

10.35  1.89
12.2  1.79
10.5  1.76

12.0  2.03

8.25  2.49
8.0  1.72
13.2  2.5

9.3  1.3
MeanSD

the last repetition was 85.75  14.7. The overall perfor-

Range

mance nevertheless improved significantly between the

first and the twelfth repetitions (P < 0.001) (Fig. 3). The
0.330

<0.001

0.433

0.055
time to completion was halved only after the eleventh
541
P

repetition (162.95  33.78 s) compared to the first pas-

0.15  0.67

0.75  1.59
0.65  2.03
0.25  0.72
3.55  7.32
0.25  0.72
0.2  0.52
0.1  0.45
8.7  6.33

sage (334.85  86.25 s). The time to completion

0.0  0.0
MeanSD
Out view

decreased significantly between the first and last repeti-

tions (P < 0.001) (Fig. 4). Between repetition 1 and
0.282

<0.001

<0.001

0.117

0.003

repetition 12, the economy of motion (P < 0.001),

P

number of instrument collisions (P = 0.001), total time

96.75  101.88

an excessive instrument force was applied (P = 0.002) and

24.00  15.77

17.1  16.96
1.55  6.26

0.35  1.35

0.85  1.63
0.25  0.55

3.85  3.98
0.7  1.53
Mean SD

0.00  0.0

master workspace range (P = 0.048) improved signifi-

Force

cantly (Table 3).

0.002

0.017

0.002

0.163

0.003
P

Needle targeting (NT)

1.95  2.35
0.25  0.55

4.25  3.75
1.05  1.67
2.3  2.43
0.2  0.52
0.5  0.76
0.1  0.31

0.1  0.31
MeanSD
Collisions

0.0  0.0

The mean overall performance for the before unpractised

exercise was 84.1  15.06 s, and the time to completion was
203.85  48.5 s.
<0.001

<0.001

<0.001

<0.001

<0.001

A significant improvement in the overall performance and

P

the time to completion between the first and last repetition

561.85  118.53
294.75  87.13

217.85  45.25

334.85  86.25
114.5  16.39

220.4  39.97
59.45  20.54

159.2  31.98
86.6  14.57

20.8  7.62

was seen in all exercises (Fig. 3 and 4). A significant

MeanSD
complete
Time to

improvement during training could also be demonstrated

in economy of motion and collisions (except for RR 1). A
significant improvement in avoiding excessive force with the
<0.001

<0.001

<0.001

<0.001

<0.001

instruments could be achieved with training in MB 2, 3 and

P

RR 2. For instruments out of view, a significant decrease

52.25  13.97
63.4  10.44

41.3  13.54
90,0  6.01

83.85  9.33
96.35  4.82

85.75  14.7
86.1  7.66
93  6.01
58  9.7

could be seen just for MB 3. An improvement in the master

MeanSD
Overall

workspace range could be achieved with training in MB 1, 2

and RR 2. A significant training effect in terms of improve-
<0.001

<0.001

<0.001

<0.001

<0.001

ment could be seen in drops in the exercises MB 2 and 3

(Table 3).
P
4 – Ring & Rail 1; 5 – Ring & Rail 2).

Comparing the overall score of the first repetition of each

414.05  113.34

517.10  114.01
750.1  181.21
340.05  88.24
208.85  80.52
357.25  77.31
225.15  27.03

332.10  71.86
39.90  13.41
66.6  22.02
Economy of

exercise with Needle Targeting, it could be demonstrated

MeanSD
Motion

that the overall score for NT was significantly higher (except

for exercise 4). The overall score for NT was approximately at
the same level as the overall score for the fifth repetition of
Repetition

MB 1, the sixth repetition of MB 2, the ninth repetition of

12

12

12

12

12
1

MB 3, the first repetition of RR1 and the eighth repetition of

Day

RR 2 (Fig. 3 and 4). Similar findings were made for the time
1
4
1
4
1
4
1
4
1
4

to completion, which was significantly lower in NT com-

Exercise

pared to the first repetition of each exercise (except for

1

exercise 2). The economy of motion and master workspace

© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology

8 U. Walliczek-Dworschak et al.

range in NT were significantly lower compared to repetition

Table 4. Comparison of the measured metrics between the first repetitions of the different exercises (1 – Match Board 1; 2 – Match Board 2; 3 – Match Board 3; 4 – Ring & Rail 1;

0.272

0.552

0.007

0.247
1 of exercises 1–5 (Table 4).

0.09
P

0.15  0.67
0.55  0.76

0.55  0.76

0.55  0.76
0.05  0.22
0.55  0.76

0.55  0.76
0.3  0.66

0.4  0.82

0.3  0.57
Discussion
MeanSD
Drops

The increasing popularity of minimally invasive techniques

such as robotic-assisted devices makes demands on sur-

<0.001

<0.001

0.021

0.024
geons to ensure proficiency in this special kind of surgery.
0.00
P

Particular skills such as maintaining camera control, three-

12.35  2.23

dimensional visualisation of the operative field, manipula-

9.15  1.27
12.2  1.79
9.15  1.27

9.15  1.27
8.00  1.72
9.15  1.27

9.15  1.27
10.35  1.9
13.2  2.5
MeanSD

tion and clutching of the EndoWrist instruments are

Range

required. The integration of a robotic learning tools into

residency programmes might become even more important
0.531

0.246

<0.001

0.197

0.039
in view of the fact that minimally invasive procedures,
P

especially robotic-assisted interventions, leave little room

0.15  0.67
0.05  0.22

0.05  0.22

0.05  0.22
0.65  2.03
0.05  0.22
3.55  7.32
0.05  0.22
0.2  0.52

8.7  6.33

for error. A curriculum using a proven robotic surgery

MeanSD
Out view

simulator without risk to patients may optimise the learning

curve in supporting efficient preparation for robotic-
assisted surgery and optimal patient outcomes and safety.
0.638

<0.001

<0.001

<0.001
1.00

As significant acquisition of robotic surgery skills also

P

depends on using the Si surgeon console of the da Vinci

0.728 96.75  101.88
24.0  15.77

17.1  16.96

robot, a simulator that incorporates the console in the

1.55  6.26
0.85  2.06

0.85  2.06

0.85  2.06
0.85  1.63
0.85  2.06

0.85  2.06
MeanSD

training might be necessary for optimal training results. So

Force

the da Vinci Skills Simulator, which fulfils these criteria, has

been introduced onto the market. Various studies have
0.414

0.024

0.002

0.012

reported the look, content and construct validity for the da

P

Vinci Skills Simulator.15–17

<0.001 1.95  2.35

<0.001 4.25  3.75

2.3  2.43
1.7  2.15
0.5  0.76
1.7  2.15

1.7  2.15
0.1  0.31
1.7  2.15

1.7  2.15
MeanSD
Collisions

At present, little data are available regarding how

many training hours and which exercises are necessary in
order for a trainee to become secure and efficient with
5 – Ring & Rail 2) in comparison with the results for Needle Targeting (NT)

0.002

<0.001

<0.001

the da Vinci robot.18 This study is a prospective study to

P

evaluate the effect of training with the dVSS on learning

517.10  114.01

curves of defined parameters. In a study from 2013,

750.1  181.21
340.05  88.24
254.2  73.29

254.2  73.29

254.2  73.29

254.2  73.29

254.2  73.29
66.6  22.01
357.25 + 77.31

Lyons et al. performed validation testing for eight chosen

MeanSD
Economy

exercises on the dVSS. 16 They demonstrated furthermore

on 46 surgeons (25 novices, 8 intermediate and 13
experts) that the overall score, economy of motion and
<0.001

0.351

<0.001

<0.001

<0.001

time to complete an exercise were the metrics which

P

could differentiate best between robotic novices and

561.85  118.53

experts.16 This could be interpreted that improvement in

294.75  87.13
203.85  48.50
217.85  45.25
203.85  48.50

203.85  48.50
59.45  20.54
203.85  48.50
334.85  86.25
203.85  48.50

these parameters is crucial in order to evaluate the

MeanSD
complete
Time to

proficiency gained by robotic novices. In the current

study, a significant improvement was seen in the overall
performance and the time to completion between the
<0.001

<0.001

<0.001

<0.001
0.95

first and last/twelfth repetition for all five chosen

P

exercises (MB1–3 and RR 1–2) and in economy of

52.25  13.97
84.10  15.06
63.4  10.44
84.1  15.06

84.1  15.06
41.3  13.54
84.1  15.06

84.1  15.06
83.85  9.33

motion in nearly all exercises (except for RR 1). These

58  9.7
MeanSD
Overall

findings are also consistent with a review from 2012,

which showed that training on VR simulators does have
a significant learning effect on various robotic surgical
Exercise

skills.19 In other studies, the benchmark to achieve

NT

NT

NT

NT

NT
1

proficiency was an overall score of 80%, but no precise

© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology


knowledge about the number of repetitions of the several
exercises required to reach this point exist was reported.20,
21
In the present study, this 80% benchmark, which is
recommended by Intuitive company, was achieved in
three of the five exercises (MB 1, MB 2 and RR 1). The
unanswered question is whether it could be reached in the
most complex and difficult exercises (MB 3 and RR2)
with more than 12 repetitions.
In 2013, Tergas et al. reported that training with the
dVSS led to significant improvements in technical perfor-
mance (time to completion, economy of motion and
instruments out of view) during a 60-min training
session.17 They evaluated 20 participants (10 medical
students and 10 gynaecology trainees) who performed
training on the dVSS versus training on a dry laboratory
platform. The virtual reality platform was preferred by
most of the trainees. Only one exercise was performed:
suture sponge level 1. In contrast, in the current study, 20
trainees were asked to perform 5 different exercises
without a set time limit. We suggest that a variety of
different exercises during training might improve concen-
tration and motivation and reduce sleepiness. A potential
limitation of this study may be that all participants were
young surgical unexperienced persons (residents between
first and fifth year) whose mean age was 24.2 years +
1.4 years. So the age factor dependency of the learning
curves could not be evaluated and needs further investi-
gations. Furthermore, the small number of trainees might
be a limitation and further investigations with higher
collective numbers are needed.
In this study, a significant improvement during training
could also be demonstrated for all exercises (except for RR 1) in
economy of motion and instrument collisions (see Table 3).
These are important parameters that may help surgeons to
optimise surgical handling I order to be more efficient and
minimise tissue damage. It is assumed that trainees may
become better robotic surgeons by learning to be precise and
efficient in their motions before striving to be fast.21
In the present study, training on the dVSS also resulted in a
significant decrease in applying excessive force for all than
MB1 and RR 1(Table 3). It could be that both are just not
difficult enough to get into problems with applying excessive
force with the instruments. Already, in repetition 1, almost no
excessive force could be seen (Tables 3 and 4). For instru-
ments out of view, a significant decrease was achieved just for
MB 3. A significant training effect in terms of improvement
could be seen for drops in exercises MB 2 and 3. However, the
other exercises had such low mean numbers of instruments
out of view and number of drops that improvements in these
two metrics were impossible to interpret statistically. An
improvement in the master workspace range could be reached
with training in MB 1 and 2 and for RR 2.
© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology
Structured training at the Da Vinci Simulator 9
In 2014, Sheth et al. reported a significant improvement
in technical performance in four exercises for 34 gynaecology
trainees with 10 repetitions or less.22 In contrast to our study,
where 12 repetitions split into four training sessions (each
session included 3 consecutive repetitions) within a 4-week
curriculum were performed, all 10 repetitions were per-
formed in a single session. It might be more effective,
particularly with regard to concentration in the long term, to
split the repetitions into more than just one session. It
remained unknown, how many consecutive repetitions are
necessary to reach the manufacturer’s preset proficiency goal
of 80%.20 In the present study, three repetitions of each
exercise were performed. On average, 1.5 h per training day
once a week over a 4-week timeframe was required to obtain
significant improvement in the measured parameters. Fur-
ther investigations with different numbers of repetitions are
needed.
The mean overall performance score for the previously
unpractised advanced new exercise, Needle Targeting, was
84.1 + 15.06, and the time to completion was 203.85
+ 48.5 s. Interestingly, the mean overall performance and
time to completion were significantly better than in the first
repetition of nearly all exercises and was approximately as
high as scores achieved after the last repetition of the most
difficult exercises, MB 3 (overall score 86.1 + 7.66/time to
completion 220.4 + 39.97 s) and RR 2 (overall score
86.25 + 14.7/time to completion 159.2 + 31.98 s) (see
Fig. 3 and 4). This speaks to a positive effect of training on
performance. However, the detected overall score was under
the 80% proficiency benchmark. The transfer of the acquired
skills was not sufficient to reach the level of proficiency level
in the new exercise. The significant improvement in
technical performance demonstrated in all performed exer-
cises can be interpreted as robotic novices learning funda-
mental robotic skills that might be important for robotic
surgery. A transfer of the gained robotic skills to new tasks
could be assumed. The achievement of a defined level of
proficiency in chosen parameters could be used to establish a
threshold that needs to be exceeded before carrying out
robotic surgery on
patients.
Conclusion
This study showed that during training with the da Vinci
Simulator, the technical performance of robotic novices
improved significantly regarding their overall performance
and various parameters. Concerning the precise training
programme itself, it remains unclear which exercises are
necessary and how exactly the time should be structured
so that the training is efficient and time-saving. Further
investigations need to be performed.
10 U. Walliczek-Dworschak et al.
Keypoints
• Structured training with the Da Vinci Skills simulator
results in an improvement of the needed skills.
• Absolute number of repetitions of training seems to be
more important than chosen training program.
• Regarding a new exercise, a positive training effect in
result of the performed training can be seen.
Conflicts of interest
The authors declared that there is no conflict of interest
regarding the publication of this manuscript.
Reference
1 Gourin C.G. & Terris D.J. (2004) Surgical robotics in otolaryngol-
ogy: expanding the technology envelope. Curr Opin Otolaryngol
Head Neck Surg. 12, 204–208
2 Intuitive Surgical (2013) da Vinci skills simulator. Accessed 19 Feb
2013 http://www.intuitivesurgical.com/products/skills_simulator/
3 Bhayani M.K., Holsinger F.C. & Lai S.Y. (2010) A shifting paradigm
for patients with head and neck cancer: transoral robotic surgery
(TORS). Oncology (Williston Park). 24, 1010–1015
4 Orvieto M.A., Marchetti P., Castillo O.A. et al. (2011) Robotic
technologies in surgical oncology training and practice. Surg. Oncol.
20, 203–209
5 Kavanagh K.T. (1994) Applications of image directed robotics in
otolaryngologic surgery. Laryngoscope. 104, 283–293
6 McLeod I.K. & Melder P.C. (2005) Da Vinci robot-assisted
excision of a vallecular cyst: a case report. Ear Nose Throat J.
84, 170–172
7 Moore E.J., Olsen S.M., Laborde R.R. et al. (2012) Long-term
functional and oncologic results of transoral robotic surgery for
oropharyngeal squamous cell carcinoma. Mayo Clin. Proc. 87, 219–
225
8 Genden E.M., Kotz T., Tong C.C. et al. (2011) Transoral robotic
resection and reconstruction for head and neck cancer. Laryngo-
scope. 121, 1668–1674
9 Byeon H.K., Holsinger F.C., Tufano R.P. et al. (2014) Robotic total
thyroidectomy with modified radical neck dissection via unilateral
retroauricular approach. Ann. Surg. Oncol. 21, 3872–3875
10 Vicini C., Montevecchi F., Pang K. et al. (2014) Combined transoral
robotic tongue base surgery and palate surgery in obstructive sleep
apnea-hypopnea syndrome: expansion sphincter pharyngoplasty
versus uvulopalatopharyngoplasty. Head Neck 36, 77–83
11 Wei W.I. & Ho W.K. (2010) Transoral robotic resection of recurrent
nasopharyngeal carcinoma. Laryngoscope. 120, 2011–2014
12 Yin Tsang R.K., Ho W.K. & Wei W.I. (2012) Combined transnasal
endoscopic and transoral robotic resection of recurrent nasopha-
ryngeal carcinoma. Head Neck 34, 1190–1193
13 Eichhorn K.W. & Bootz F. (2011) Clinical requirements and
possible applications of robot assisted endoscopy in skull base and
sinus surgery. Acta Neurochir. 109, 237–240
14 Issenberg S.B., McGaghie W.C., Petrusa E.R. et al. (2005) Features
and uses of high-fidelity medical simulations that lead to effective
learning: a BEME systematic review. Med. Teach. 27, 10–28
15 Hung A.J., Zehnder P., Patil M.B. et al. (2011) Face, content and
construct validity of a novel robotic surgery simulator. J. Urol. 186,
1019–1024
16 Lyons C., Goldfarb D., Jones S.L. et al. (2013) Which skills really
matter? proving face, content, and construct validity for a
commercial robotic simulator Surg. Endosc. 27, 2020–2030
17 Tergas A.I., Sheth S.S., Green I.C. et al. (2013) A pilot study of
surgical training utilizing a virtual robotic surgery simulator. JSLS
17, 219–226
18 Moles J.J., Connelly P.E., Sarti E.E. et al. (2009) Establishing a training
program for residents in robotic surgery. Laryngoscope. 119, 1927–1931
19 Schreuder H.W.R., Wolswjik R., Zweemer R.P. et al. (2012)
Training and learning robotic surgery, time for a more structured
approach: a systematic review. BJOG 119, 137–149
20 Lucas S.M., Gilley D.A., Joshi S.S. et al. (2011) Robotics training
program: evaluation of the satisfaction and the factors that
influence success of skills training in a resident robotics curriculum.
J. Endourol. 25, 1669–1674
21 Patel A., Patel M., Lytle N. et al. (2014) Can we become better robot
surgeons through simulator practice? Surg. Endosc. 28, 847–853
22 Sheth S.S., Fader A.N., Tergas A.I. et al. (2014) Virtual reality
robotic surgical simulation: an analysis of gynecology trainees.
J Surg Educ. 71, 125–132
© 2016 John Wiley & Sons Ltd
Clinical Otolaryngology