Blueprint For A Simulation Framework To Increase D

safety
Article
Blueprint for a Simulation Framework to Increase Driver
Training Safety in North America: Case Study
Kevin F. Hulme 1, * , Rachel Su Ann Lim 1 , Meghan Bauer 2 , Nailah Hatten 3 , Helena Destro 4 , Brenden Switzer 5 ,
Jodie-Ann Dequesnay 6 , Rebecca Cashmore 7 , Ian Duncan, Jr. 8 , Anand Abraham 9 , Jacob Deutsch 10 ,
Nichaela Bald 11 , Gregory A. Fabiano 12 and Kemper E. Lewis 13
1 The Stephen Still Institute for Sustainable Transportation and Logistics, University at Buffalo,
Buffalo, NY 14260, USA; rlim2@buffalo.edu
2 Department of Psychology, College of Arts and Sciences, University at Buffalo, Buffalo, NY 14260, USA;
mebauer@buffalo.edu
3 Gilbane Building Company, Baltimore, MD 21230, USA; nailah.hatten@gmail.com
4 Warsaw Central School District, Warsaw, NY 14569, USA; hdestro9@gmail.com
5 Buffalo Public Schools, Buffalo, NY 14202, USA; BSwitzer@buffaloschools.org
6 The Boeing Company, Los Angeles, CA 90245, USA; Jodie-Ann.V.Duquesnay@boeing.com
7 Williamsville Central School District, Williamsville, NY 14051, USA; rcashmore@williamsvillek12.org
8 Honda R&D America’s Inc., Raymond, OH 43067, USA; duncanijr@gmail.com
9 Adient, Plymouth, MI 48170, USA; anand.faith@gmail.com
10 GE Aviation, Cincinnati, OH 45215, USA; jacob@jacobdeutsch.com
11 Edwards Vacuum, Sanborn, NY 14132, USA; nichaelabald@gmail.com
12 Department of Psychology, College of Arts, Sciences & Education, Florida International University,

Miami, FL 33199, USA; gfabiano@fiu.edu
13 Department of Mechanical and Aerospace Engineering, School of Engineering and Applied Sciences,
Citation: Hulme, K.F.; Lim, R.S.A.;
Bauer, M.; Hatten, N.; Destro, H.;
University at Buffalo, Buffalo, NY 14260, USA; kelewis@buffalo.edu
* Correspondence: hulme@buffalo.edu; Tel.: +1-716-645-5573
Switzer, B.; Dequesnay, J.-A.;
Cashmore, R.; Duncan, I., Jr.;
Abraham, A.; et al. Blueprint for a Abstract: Despite numerous recent advances in the classroom and in-vehicle driver training and
Simulation Framework to Increase education over the last quarter-century, traffic accidents remain a leading cause of mortality for
Driver Training Safety in North young adults—particularly, those between the ages of 16 and 19. Obviously, despite recent advances
America: Case Study. Safety 2021, 7, in conventional driver training (e.g., classroom, in-vehicle, Graduated Driver Licensing programs),
24. https://doi.org/10.3390/ this remains a critical public safety and public health concern. As advanced vehicle technologies
safety7020024 continue to evolve, so too does the unintended potential for mechanical, visual, and/or cognitive
driver distraction and adverse safety events on national highways. For these reasons, a physics-based
Academic Editor: Garrett Mattos
modeling and high-fidelity simulation have great potential to serve as a critical supplementary com-
ponent of a near-future teen-driver training framework. Here, a case study is presented that examines
Received: 29 December 2020
the specification, development, and deployment of a “blueprint” for a simulation framework in-
Accepted: 15 March 2021
Published: 1 April 2021
tended to increase driver training safety in North America. A multi-measure assessment of simulated
driver performance was developed and instituted, including quantitative (e.g., simulator-measured),
Publisher’s Note: MDPI stays neutral
qualitative (e.g., evaluator-observed), and self-report metrics. Preliminary findings are presented,
with regard to jurisdictional claims in along with a summary of novel contributions through the deployment of the training framework, as
published maps and institutional affil- well as planned improvements and suggestions for future directions.
iations.
Keywords: driving simulation; physics-based modeling; human factors; driver training; driver
education; safety; public health; modeling and simulation (M&S)
Copyright: © 2021 by the authors.

Licensee MDPI, Basel, Switzerland.
This article is an open access article 1. Introduction
distributed under the terms and Traffic accidents continue to be a primary safety and public health concern and are
conditions of the Creative Commons a leading cause of death for young, inexperienced, and newly-licensed drivers (aged
Attribution (CC BY) license (https:// 16–19) [1,2]. Per mile driven, young drivers are almost 300% more likely (than drivers in
creativecommons.org/licenses/by/ older age groups) to be involved in a fatal driving accident [2]. Statistics from the same
4.0/).
Safety 2021, 7, 24. https://doi.org/10.3390/safety7020024 https://www.mdpi.com/journal/safety

Safety 2021, 7, 24 2 of 24
source indicate an unfortunate gender effect: the crash fatality rate for young males is
more than 200% higher than that for young females. Furthermore, crash risk has long been
measured to be particularly high just after licensure [3,4]; per mile driven, crash rates are
150% higher for (just-licensed) 16-year-olds than for slightly older drivers who even have a
mere 2–3 years of experience.
Recent scientific evaluations [5] have demonstrated that current driver education
programs (e.g., classroom, in-vehicle, Graduated Driver Licensing programs [6]) do not
necessarily produce safer drivers. While some training programs have been found to be
effective for procedural skill acquisition and others improve driver perception of dangers
and hazards, conventional driver training does not provide teenagers with sufficient hands-
on, seat-time exposure to the varied challenges of driving, which can be visual, manual, and
cognitive [7] in nature. Novice drivers in training need to safely learn essential skills (e.g.,
basic vehicle mechanical operation, steering, turning, and traffic management), without
the complexity of the normal driving environment. Historically, driver licensing in the
United States has served mostly as a formality—if the student does not flagrantly violate
the rules of the road during their brief on-road evaluation, he or she passes the driving
exam [8]. In various countries in Europe (e.g., the Czech Republic, Finland, France, the
Netherlands, and Slovakia), driver training requirements are much more rigorous and
often include a significant component in addition to classroom and in-vehicle training
assessments—through simulator-based training (SBT) [9]. A relevant example from the
literature is the TRAINER project [10], which proposes a framework for SBT (in Europe,
including implementations based in Spain, Belgium, Sweden, and Greece) and offers result
comparisons between training implementation on various classes of simulators and across
simulator fidelities.
In recent times, SBT has been implemented for supplementary teen driver education
and performance assessment [11–15] with numerous key advantages. For example, train-
ing simulators have been implemented to improve driver understanding and detection of
hazards and are useful for enhancing the situational awareness of novice drivers [16,17].
Furthermore, training in the controlled environment of a simulator provides critical ex-
posure to many traffic situations (and some that are rarely encountered while driving a
physical vehicle), trial-and-error (i.e., freedom-to-fail), safety/repeatability, and digital,
real-time metrics of driving performance [18]. Training in a simulator can be extremely
engaging—especially so for young trainees—and provides young learners with critical
additional seat-time to common driving situations in an educational setting that is active
and experiential [19,20]. Likewise, other recent studies [21] have investigated correlations
between simulation performance (e.g., violations, lapses, errors, driving confidence) and
natural tendencies observed when driving a physical vehicle (on-road).
Despite these advantages and advances, SBT also has some noteworthy drawbacks.
A simulator, by definition, is an imitation of reality, and for some trainees, it is difficult to
overcome this perception to provide training that is meaningful. This inherent absence of
both “presence” (i.e., a sense of “being there”, that is natural, immediate, and direct) [22]
and “immersion” (i.e., the objective sensory notion of being in a real-world situation or
setting) [23] in the simulator can result in negative training [24] and acquired skills might
not be appropriately applied when transferred to the “real world”. Accordingly, prior to
implementation, a simulator must be properly verified and validated (i.e., designed and
implemented such that the simulator model serves as an accurate representation of the
“real world”) both (a) to promote positive training for participants and (b) to ensure that
the human behavior data collected are accurate and realistic [25–27].
Another primary concern with many simulator implementations is environment
maladaptation, more commonly referred to as “simulator sickness” [28]. This side effect is
often caused by “postural instability” [29]—a sensory conflict between what is observed and
what is felt when interacting within the artificial environment that commonly manifests
itself through a number of symptoms, including sweating, dizziness, headaches, and
nausea. Fortunately, past studies have demonstrated that simulator sickness is less frequent
Safety 2021, 7, 24 3 of 24
and less severe when experimental duration is modulated, especially so among young
individuals [30]. Finally, simulator environment affordability is a major consideration. The
fidelity of the simulator (e.g., field-of-view, user controls, cabin, motion capabilities, and
various others) should be sufficient to promote positive training while remaining within a
price domain (i.e., acquisition, maintenance, operation, upgrades) that is achievable for a
typical training classroom.
Ultimately, researchers are still trying to determine (and objectively measure) how
simulator exposure compares to actual, real-world driving exposure. Establishing the
efficacy of supplementary simulator exposure in young driver training requires evidence-
based empirical studies; this very notion stands as the primary emphasis of this paper.
Implementing a targeted sequence of SBT exercises and employing a companion holistic
multi-measure assessment, the impact of the training framework was investigated on
various aspects of observed young driver performance. Metrics included: (a) quantitative
measures (e.g., speed and compliance behavior at intersections), as monitored and calcu-
lated by the physics-based vehicle dynamics mathematical model that underlies the driving
simulator engine; (b) evaluator qualitative measures, as a critical “human-in-the-loop”
component to augment numerical assessment; (c) module questionnaires and self-report,
as a means to monitor transfer knowledge (e.g., pre- and post-module and pre- and post-
program); and (d) exit surveys/anecdotal evidence, as a means for teen trainees (and their
parents) to comment on the advantages and disadvantages of the SBT framework.
In Figure 1, a “theory of change” (i.e., a logic model) illustrates both the overall
workflow and the primary novelties of SBT mechanisms that have been developed and
deployed for this work. In Research Methodology, the design and development of the
training environment (including two implemented simulator types) is detailed; the suite
of SBT modules (environments) are described and justified; the experimental cohort is
described, a teenaged cohort who were recruited over a 2+ year timeframe. In Results
and Discussion, preliminary findings are presented by successfully instituting a holistic
performance assessment, the intended outcomes of which could have profound training
benefits. In Broader Impacts of our Methodology and Implementation are discussed the
numerous advantages of using games and modeling and simulation (M&S) in training.
The paper culminates with Conclusions and Future Work. The primary goal is that the
overarching return on investment for this work will be a template for reducing untoward
Safety 2021, 7, x FOR PEER REVIEW 4 of 24
driving outcomes, which could lead to improved national driver safety in the long term.
Figure 1. Teen driving framework—a blueprint theory of change (with simulation).

Figure 1. Teen driving framework—a blueprint theory of change (with simulation).
2. Research Methodology
As a primary component of the training framework, a simulator training environ-
ment was specified and composed that would allow for presentation of training content
at suitable fidelity (i.e., both physical and psychological) while simultaneously enabling
measurement and observation of human performance alongside a holistic assessment of
participant outcomes. The training environment was designed to allow for much-needed
Safety 2021, 7, 24 4 of 24
2. Research Methodology
As a primary component of the training framework, a simulator training environ-
ment was specified and composed that would allow for presentation of training content
at suitable fidelity (i.e., both physical and psychological) while simultaneously enabling
measurement and observation of human performance alongside a holistic assessment of
participant outcomes. The training environment was designed to allow for much-needed
training “seat time” while enabling experiential reinforcement of basic driving skills and
also enabling a trainee’s freedom to fail. In other words, the environment promoted repeti-
tion and was such that the learner could take risks (i.e., where real-world consequences
were greatly reduced). This allowed failure to be reframed as a necessary component
of learning and made students more resilient in the face of obstacles and challenges
(e.g., [31,32]. This study was approved on 21 October 2015 by the ethics committee of 030
University at Buffalo Internal Review Board with the code 030-431144.
2.1. Training Environment Design

The novel implementation offered two simulator types of varying features and physi-
cal fidelity. The fixed-base simulator is comprised of a racing frame/chair, a high-fidelity
spring-resisted steering wheel (with a 240◦ range of motion), game-grade foot pedals
(gas, brake, and clutch), and a PC-grade sound system. The visual system consists of a
four-wall arrangement (each 60 high × 80 wide) that provides drivers with forward, left,
right, and rearward Wide Super eXtended Graphics Array Plus (WSXGA+) -resolution
(1680 × 1050) views arranged at 90◦ (see Figure 2). The motion-based simulator is com-
prised of a 6-Degrees of Freedom (DOF) full-motion electric hexapod platform, a two-seat
Safety 2021, 7, x FOR PEER REVIEWsedan passenger cabin, a moderate-fidelity steering wheel (with force-feedback capability
and a 900◦ range of motion), high-fidelity foot control pedals (with spring-resisted gas and
clutch pedals and pressure modulation on the brake pedal), and a 2.1 sound system. The
visual system consists of a 160 diameter theater arrangement, with a 60 vertical viewing
capability
capability andand
whichwhich implements
implements 6 Full
6 Full High High (FHD)-resolution
definition definition (FHD)-resolution
(1920 × 1080) (192
projectors
projectors to fully envelop
to fully the simulator
envelop participant
the simulator (see Figure 3).
participant (see Figure 3).
Figure
Figure 2. Fixed-base
2. Fixed-base simulator.
simulator.
Safety 2021, 7, 24 5 of 24
Figure 2. Fixed-base simulator.

Figure 2. Fixed-base simulator.
Figure3.3.
Figure Motion-based
Motion-based simulator.
simulator.
The
Theaccompanying
accompanying simulation
simulationsoftware environment
software environmentwas developed in-housein-house
was developed (using (usin
C++)
C++) so
C++)sosothatthat
that its
itsits design
design could
could
design be tailored
be tailored
could to exact
to exact
be tailored training
training
to exact needs.
needs. needs.
training While participating
While participating on on th
on the
While participating
the simulator,
simulator, the the driver
driver adjusts
adjusts the on-board
the on-board Human Human Interface
Interface Devices Devices
(HID) (HID)
controlscontrols
(Stage (Stag
simulator, the driver adjusts the on-board Human Interface Devices (HID) controls
(Stage
1), and 1), andinputs
these theseareinputs
thenare then delivered
delivered to a Windows-based
to a Windows-based workstation workstation where
where a simula-
a1), and theseanalysis
simulation
tion
inputs is
analysis is performed
areperformed
then delivered
(Stage to2).a Physics-based
(Stage 2). Physics-based
Windows-based
model changes
workstation
model changes
manifest
where a simula
manifest
themselves in
tion analysis
themselves in is performed
required updates (Stage
to 2).
three Physics-based
forms of output model
rendering: changes
(i) the manifest
scene themselves i
graphics
required updates to three forms of output rendering: (i) the scene graphics state (Stage
required
state
3a), (ii) the updates
(Stage motion tomotion
threestate
3a), (ii) platform
the forms
platform of state
(Stage output
3b), (Stage
and rendering:
(iii)3b),
theand
audio (i)state
(iii) theaudio
the scenestate
(Stage graphics
3c) of(Stage state
3c) (Stag
the sim-
3a),
of the (ii) the
simulation.motion platform
Notionally, state
this (Stage
signal path 3b),
is and (iii)
illustrated the
in audio
Figure
ulation. Notionally, this signal path is illustrated in Figure 4, whose design and imple- 4, state
whose (Stage
design 3c)
andof the sim
implementation
ulation. Notionally,
mentation has
has previouslypreviously
this been
signal
been described
path elsewhere
described elsewhere
is illustrated [33].in[33]. Each
Figure
Each of these
4, whose
of these primary
primarydesign I/O
I/O statesand imple
states
are are briefly
mentation
briefly has described.
described. previously been described elsewhere [33]. Each of these primary I/O state
are briefly described.
S TA G E 3
S TA G E 1 S TA G E 2
O utput
Input (D river C ontrols) S im ulation A nalysis S TA G E 3
S TA G E 1 S TA G E 2 R endering
O utput
Input (D river C ontrols) S im ulation A nalysis
R endering
a 2 +b 2 =c 2
Steering P edals P addle Shifters C om putation anda 2 +b 2 =c 2V isualC ues M otion C ues A udio C ues
P hysics-based m odeling
Steering P edals P addle Shifters C om putation and V isualC ues M otion C ues A udio C ues
Figure 4. I/O signalP hysics-
pathway based
for m odel
training i
ng
framework.
Figure 4. I/O signal pathway for training framework.
2.1.1. Stage 1: Input—Driver

Figure Controls
4. I/O signal pathway for training framework.
User input commands are obtained with DirectInput (Microsoft), a subcomponent of
DirectX, which allows the programmer to specify a human-interface device (e.g., steering
wheel, gas pedal, brake pedal). The state of the input device (e.g., rotational position of the
steering wheel, translational position of the pedals) can be polled with minimal latency in
near real-time.
2.1.2. Stage 2: Simulation Analysis

The simulation computer uses the control inputs to perform a numerical analysis
whose outputs are used to render the primary output components of the simulation (i.e.,
visual, haptic, aural). Vehicle dynamics are calculated using the classical Bicycle Model of
the automobile [34,35]. This simplified physics-based model has two primary degrees of
freedom (DOF), which are yaw rate (i.e., rotational velocity) and vehicle sideslip angle (i.e.,
the angle between the direction in which the wheels are pointing versus the direction in
which they are actually traveling). Forward velocity is added to the baseline model as a
third primary DOF for the present implementation. Primary model inputs are: steering
Safety 2021, 7, 24 6 of 24
wheel (rotation) angle and the longitudinal forces of the tires, and primary model outputs
include vehicle velocities, accelerations, and tire forces. We then employ basic numerical
integration (i.e., Euler’s method) to solve the equation outputs that define the current state
of the vehicle (e.g., longitudinal/lateral velocity, yaw and rate, and longitudinal/lateral
position). Updates to the state equations have to occur rapidly, so the step size between
the updated and previous state takes place at a rate of 60 Hz. The artificial vehicles
that populate the virtual driving environment alongside the human-driven vehicle are
independently represented by a physics-based traffic model [36] that attempts to emulate
actual human behavior (i.e., including driving errors and lapses).
2.1.3. Stage 3: Outputs—Visual, Haptic and Audio

Once calculated, vehicle output states are subsequently converted into the six motion
DOFs for haptic cueing: (heave, surge, sway, roll, pitch, yaw). Due to the finite stroke length
of the electrical actuators on the motion platform, this conversion requires limiting, scaling,
and tilt coordination (i.e., a procedure commonly employed that helps to sustain longitudinal
and lateral acceleration cues with the gravity vector through pitch and roll, respectively [37])
by way of the washout filtering [38,39] technique. The updated DOFs are sent to the motion
platform computer via datagram packets (UDP) for the entire simulation. Packet delivery
is accomplished using Win64 Posix multithreads; in this manner, the update rate of the
motion process is independent from that of the graphics process to avoid Central Processing
Unit (CPU) / Graphics Processing Unit (GPU) bottlenecks. Simulation graphics have been
developed using OpenGL, an industry-standard 3D graphics Application Programing
Interface (API). The simulation framework employs OpenAL for sound events that are
commonplace in a ground vehicle scenario: ignition/shutdown, engine tone, and cues for
tire-surface behaviors and hazards.
2.2. Training Modules

The training modules that were designed for this framework allow driving participants
to engage with training content in a progressive manner. In this way, training exposure
begins simple and gradually increases in complexity while additional tasks and driving
challenges are introduced. These simulation-based driving modules were specified and
designed to complement existing driver training methodologies (e.g., in-class and in-
vehicle components, primarily) while each targets a specific skillset (i.e., mechanical, visual,
cognitive, or multiple) widely acknowledged for safe driving. The training modules can be
decomposed into three primary classifications: (a) closed course (CC) “proving ground”
tracks (without traffic); (b) standard residential driving (both with/without accompanying
traffic); and (c) challenge scenarios (always with traffic). These classifications are described
and illustrated next.
2.2.1. Closed Course Driving

The closed-course training environments allow us to evaluate human performance
during specialized maneuvers without traffic. The skid pad [40] training course (Figure 5) is
a conventional automotive proving ground environment that is invaluable to help students
better understand the complex relationship between speed and heading (i.e., steering angle).
The tri-radial speedway (Figure 6) integrates straightaways with curves and necessitates
speed adjustments at the segment transitions (i.e., careful interplay between acceleration,
braking, and turning). The figure-8 test track (Figure 7) allows learners to experience both
clockwise and counterclockwise turns in immediate succession. Note that performance
data were not collected during these closed course scenarios. Regardless, they served as
an invaluable mechanism for providing novice drivers with additional and repeatable
hands-on practice within the simulator among varied geometric conditions. This critical
experiential exposure and “seat time” served as an advantageous supplement to the drives
where driver performance was quantified.
acceleration, braking, and turning). The figure-8 test track (Figure 7) allows learners t
experience both clockwise and counterclockwise turns in immediate succession. Note tha
experience both clockwise and counterclockwise turns in immediate succession. Note tha
performance data were not collected during these closed course scenarios. Regardless
performance data were not collected during these closed course scenarios. Regardless
they served as an invaluable mechanism for providing novice drivers with additional an
they served as an invaluable mechanism for providing novice drivers with additional an
repeatable hands-on practice within the simulator among varied geometric conditions
Safety 2021, 7, 24 repeatable hands-on practice within the simulator among varied geometric 7 ofconditions
24
This critical experiential exposure and “seat time” served as an advantageous supplemen
This critical experiential exposure and “seat time” served as an advantageous supplemen
to the drives where driver performance was quantified.
to the drives where driver performance was quantified.
Figure5.5.The
Figure The skid
skid pad
pad scenario
scenario (CC).
(CC).
Figure 5. The skid pad scenario (CC).
Safety 2021, 7, x FOR PEER REVIEW 8 of 2
a notional
Figure 6. Thedepiction of the residential
tri-radial speedway training environment, which can be endeavore
scenario (CC).
Figure6.6.The
Figure The tri-radial
tri-radial speedway
speedway scenario
scenario (CC).
(CC).traffic
either without or with accompanying (as shown).
2.2.2. Residential Driving
The primary SBT environment is based upon a five square mile training area inspire
The primary SBT environment is based upon a five square mile training area inspire
by an actual residential region located in suburban Western New York State (i.e., Buffalo
by an actual residential region located in suburban Western New York State (i.e., Buffalo
Niagara region) and adjacent to the North campus of the University at Buffalo (where thi
Niagara region) and adjacent to the North campus of the University at Buffalo (where thi
research was performed). The residential environment includes actual suburban streets
research was performed). The residential environment includes actual suburban streets
including 2- and 4-lane roads and a 1-mile linear segment of the New York State (290
including 2- and 4-lane roads and a 1-mile linear segment of the New York State (290
Thruway. The residential simulation construct also includes road signs, lane markings
Thruway. The residential simulation construct also includes road signs, lane markings
and traffic control devices and allows the trainee to practice basic safe driving (e.g., spee
and traffic control devices and allows the trainee to practice basic safe driving (e.g., spee
maintenance, traffic sign and signal management) within a controlled, measurable, an
maintenance, traffic sign and signal management) within a controlled, measurable, an
familiar environment inspired by “real world” locations. Refer to Figure 8, which present
familiar environment inspired by “real world” locations. Refer to Figure 8, which present
Figure7.7.The
Figure The figure-8
figure-8 scenario
scenario (CC).
(CC).

The primary SBT environment is based upon a five square mile training area inspired
by an actual residential region located in suburban Western New York State (i.e., Buffalo-
Niagara region) and adjacent to the North campus of the University at Buffalo (where this
research was performed). The residential environment includes actual suburban streets,
including 2- and 4-lane roads and a 1-mile linear segment of the New York State (290)
Thruway. The residential simulation construct also includes road signs, lane markings,
and traffic control devices and allows the trainee to practice basic safe driving (e.g., speed
maintenance, traffic sign and signal management) within a controlled, measurable, and
familiar environment inspired by “real world” locations. Refer to Figure 8, which presents
a notional depiction of the residential training environment, which can be endeavored
Figurewithout
either 8. Residential
or withdriving scenario.traffic (as shown).
accompanying
2.2.3. Challenging Scenarios

Finally, within the residential driving environment, driving challenges were pre
sented that entailed additional vigilance and attentiveness on the part of the traine
through multitasking. To practice these skillsets, which are less common on the roadwa
(and therefore, less frequently encountered during in-vehicle training), a series of specia
Safety 2021, 7, 24 8 of 24
Figure 7. The figure-8 scenario (CC).
Figure8. 8.
Figure Residential
Residential driving
driving scenario.
scenario.

Finally, within the residential driving environment, driving challenges were presented
Finally, within the residential driving environment, driving challenges were pre
that entailed additional vigilance and attentiveness on the part of the trainee through
sented that To
multitasking. entailed
practiceadditional vigilance
these skillsets, whichand attentiveness
are less common onon thethe part of
roadway the traine
(and
Safety 2021, 7, x FOR PEER REVIEWtherefore, less frequently encountered during in-vehicle training), a series of special hazard roadwa
through multitasking. To practice these skillsets, which are less common on the
(and therefore,
(challenge) lesswere
scenarios frequently
devised, encountered during
during which to in-vehicle
observe training),
and measure trainee aresponse
series of specia
Safety 2021, 7, x FOR PEER REVIEW hazard
and (challenge)
driving scenarios
performance. These were devised,
challenge during
scenarios which
included theto observe
following: and measure traine
9 of 2
1.response and driving
A construction zoneperformance.
with roadway These challenge
cones, narrow scenarios
lanes, included
and large vehicles the following:
cognitive/mechanical/visual
2.1. An aggressive driver workload.
(tailgater) segment in A
a specific section
narrowed driving of module
roadway to address the
A construction zone with roadway cones, narrow lanes, and large vehicles
3.impairment) has
Speed modulationbeen designed
segments, and
including aincorporated for
section with speed another
bumps research
(see Figure 9)project [4
2. An aggressive driver (tailgater) segment in a driving
narrowed sectionto of roadway
4.cognitive/mechanical/visual
wasASpeed
complex
not highway
explicitly merge workload.
included with
within
A specific
large vehicles
the current
module
and inclement
work weather
scope.
address
This training
the fifth (i.e
prospe
3.
5.impairment)
modulation
An animalhas
segments,
beenwithin
crossing designed including
andcul-de-sac
a quiet
a
incorporated section with speed
for another
(see Figure 10) research projectFigure
bumps (see [42,43]9)bu
cussed
4.
was not
further
A complex inincluded
explicitly
Futuremerge
highway Work.with
within thelarge vehicles
current workand inclement
scope. weather
This training prospect is di
5. An animal crossing within
cussed further in Future Work. a quiet cul-de-sac (see Figure 10)
According to previously published statistics [41], young and inexperienced adu
drivers are at elevated risk to be involved in a vehicle accident, underestimate hazardou
situations, and drive more frequently in risky situations than more experienced driver
Directly related to these driving deficiencies, the leading causes of fatal crashes amon
teens include: (i) speeding, (ii) driving too fast for road conditions, (iii) inexperience, (iv
distractions, and v) driving while drowsy or impaired. In specification and design effor
for the training modules (and accompanying evaluation protocols), it remained critical fo
us to explicitly address many of these prevailing weaknesses within the supplementa
training framework. Accordingly, thoughtful consideration was given to the layout o
each of the 120 min instructional sessions that comprise the 10 h simulation course. Of th
documented
Figure Speed
9.Speedfivemodulation
Speed leading causes of fatal crashes, the training modules explicitly addres
(challenge).
Figure
Figure9. 9. modulation (challenge).
modulation (challenge).
the first four: (i) influencing the driver to be mindful of posted speed limits, (ii) makin
sure the trainee is monitoring speed in varied driving conditions, (iii) exposing new driv
ers to driving situations they have never seen before—and may only rarely encounter, an
(iv) challenge scenarios that cause external distraction and demand multitasking throug
Figure10.10.
Figure Animal
Animal crossing
crossing (challenge).
(challenge).
Figure 10. Animal crossing (challenge).
2.3. Instructional Sessions and Performance Measures
2.3. Instructional
In advance of Sessions and Performance
each instructional Measures
session, an acclimation drive (of duration fiv
In advance
minutes) is offered of each instructional
primarily session, procedure
to serve as a warm-up an acclimation drive
for drivers (of dura
to become fa
miliar withisthe
minutes) controls
offered and environment
primarily to serve asof athe simulatorprocedure
warm-up [44]. Each for
of the five trainin
drivers to be
Safety 2021, 7, 24 9 of 24
According to previously published statistics [41], young and inexperienced adult

drivers are at elevated risk to be involved in a vehicle accident, underestimate hazardous
situations, and drive more frequently in risky situations than more experienced drivers. Di-
rectly related to these driving deficiencies, the leading causes of fatal crashes among teens
include: (i) speeding, (ii) driving too fast for road conditions, (iii) inexperience, (iv) dis-
tractions, and v) driving while drowsy or impaired. In specification and design efforts
for the training modules (and accompanying evaluation protocols), it remained critical
for us to explicitly address many of these prevailing weaknesses within the supplemental
training framework. Accordingly, thoughtful consideration was given to the layout of
each of the 120 min instructional sessions that comprise the 10 h simulation course. Of the
documented five leading causes of fatal crashes, the training modules explicitly address
the first four: (i) influencing the driver to be mindful of posted speed limits, (ii) making
sure the trainee is monitoring speed in varied driving conditions, (iii) exposing new drivers
to driving situations they have never seen before—and may only rarely encounter, and
(iv) challenge scenarios that cause external distraction and demand multitasking through
cognitive/mechanical/visual workload. A specific driving module to address the fifth
(i.e., impairment) has been designed and incorporated for another research project [42,43]
but was not explicitly included within the current work scope. This training prospect is
discussed further in Future Work.
2.3. Instructional Sessions and Performance Measures

In advance of each instructional session, an acclimation drive (of duration five minutes)
is offered primarily to serve as a warm-up procedure for drivers to become familiar with
the controls and environment of the simulator [44]. Each of the five training sessions
incorporates new elements (e.g., traffic behavior) and training content (e.g., challenge
scenarios) that build upon the previous and scaffold driving knowledge in a progressive
manner. Finally, at the conclusion of the fifth training module, each participant is given
a culminating “final exam” in the form of a virtual road test. This road test serves as
a cumulative mechanism to examine specific training outcomes that were encountered
explicitly within each of the previous five instructional sessions (and the training modules
used for each). Refer to Table 1 for a summary of the instructional sessions, an overview of
the training module(s) that were deployed for that session, and a brief a description of the
primary driving features for each instructional session.
Table 1. Overview of Training Modules for each Instructional Session.
Session Training Module(s) Residential Drive Description/Features

Practice drives for participant acclimation (e.g., the steering
wheel, the paddle shifter signal indicators, and the gas and
Acclimation
Residential brake pedals) and the simulator visual/motion/sound cues.
drive
Note that this drive is encountered at the beginning of each of
the five training sessions.
Course has no traffic and contains three left turns, three right
Residential,
Session 1 turns (including one designated as “no right on red”), five
closed-course
traffic lights, and four stop signs. Course length: 3.2 miles.
Course has no traffic and contains four left turns, two right
turns (including one designated as “no right on red”), six traffic
Residential, closed-course,
Session 2 lights, a roundabout intersection, and three stop signs.
challenge
Challenges (2): construction zone and speed modulation.
Course length: 3.4 miles.
Course introduces traffic and contains four left turns, two right
Residential, closed-course, turns (including one designated as “no right on red”), eight
Session 3
challenge traffic lights, and one stop sign. Challenges (2): construction
zone and speed modulation. Course length: 3.2 miles.
Safety 2021, 7, 24 10 of 24
Table 1. Cont.
Session Training Module(s) Residential Drive Description/Features

Course introduces traffic and contains one left turn, five right
turns (including one designated as “no right on red”), six traffic
Residential, closed-course,
Session 4 lights, and five stop signs. Challenges (3): construction zone,
challenge
speed modulation, and animal crossings. Course length:
4.0 miles.
Course introduces traffic and contains three left turns, four right
Residential, closed-course, turns (including one designated as “no right on red”), six traffic
Session 5
challenge lights, and three stop signs. Challenges (2): tailgater scenario
and speed modulation. Course length: 3.8 miles.
Course introduces traffic and contains two left turns, six right
turns (including one designated as “no right on red”), seven
Virtual
Residential, challenge traffic lights, and five stop signs. Challenges (4): construction
Road Test
zone, speed modulation, animal crossings, and highway merge.
Course length: 4.7 miles.
Table 2 summarizes the experimental protocol for each of the instructional simula-
tion modules. Eligible participants came for their first visit accompanied by their legal
guardian. At the beginning of this first session, parents and teens provided their informed
consent and assent, respectively, to participate in the study. Each session began with a
number of pre-surveys and self-report questionnaires issued in advance of the simulator
exercises. This was followed by a video briefing detailing driving rules and regulations
pertinent to that session’s specific content and a brief acclimation drive. Teens experienced
the actual simulator exercises in pairs (i.e., one driver and one passenger). There were
brief breaks/transitions placed in between driver/passenger load/unload and to assist
with mitigating simulator sickness. The total duration of each of the five sessions was
approximately two hours.
Table 2. Experimental Protocol for Each Instructional Session.
Activity Duration
Informed consent/assent (first visit only) 10–15 min
Pre-surveys and questionnaires 10–15 min
Training module video briefing 10 min
Acclimation drive 5–10 min
Simulator training modules (as driver) 1 30 min
Breaks/transitions 5–10 min
Simulator training modules (as passenger) 1 30 min
Post-surveys and questionnaires 10 min
Discussion/summary/session de-briefing 5–10 min
Total Module Time: 120 min (approx.)
1 Includes seat-time (and repetition) for the closed-course, residential, and challenge scenarios.
To provide a more holistic understanding of simulation-based training on driving

performance, the holistic performance measure assessment consisted of three primary
subcomponents:
2.3.1. Quantitative Measures

A printed simulator score report collected driver performance data for the entire
duration of each training module for each instructional session. The score report rated per-
formance on basic driving skills including: (a) travel speed on every street, (b) travel speed
at every stop sign, and (c) excursion speed and traffic light state (i.e., red/yellow/green)
while traversing both into and out of every stoplight zone. Penalties were assigned when
drivers were traveling above the speed limit or when they failed to fully stop at controlled
intersections. Cumulatively, these penalties resulted in a final percentage score for each
Safety 2021, 7, 24 11 of 24
category (i.e., travel speed, stop signs, stop lights), which were then combined to create
an aggregated total (0–100) score. Note that the simulator also compiled an (X/Y) trace of
lane position in a separate data file. These data were plotted and any obvious/noteworthy
trends were observed by the trainee post-experiment, but trainee compliance was not
(currently) monitored as a component of the score report cumulative score.
2.3.2. Qualitative Measures

The simulator training evaluator completed an evaluation checklist to subjectively rate
the trainee’s performance (in real-time) on various driving skills (e.g., lane maintenance,
use of turn indicators, and speed fluctuations while turning) on a 4-point Likert scale.
Training evaluators also completed handwritten instructor (HI) comments, later coded
to create scores for the number of positive and negative comments (positive HI; negative
HI) issued per instructional session, per driver. These tabulations likewise resulted in a
difference metric (difference HI), which quantified the difference between the positive and
negative comments issued per instructional session, per driver.
2.3.3. Self-Report Measures

A session-based questionnaire (SBQ) measured pre- and post-session knowledge of
basic driving facts, but specifically for driving and roadway details that were pertinent to
each of the five instructional sessions and the corresponding simulator training module.
The simulator exercises were intended to reinforce driving skills introduced during the
pre-SBQ (and an accompanying video briefing), and this information might serve as first-
time exposure for many novice teen drivers. Subsequently, during the simulator exercise,
drivers directly practiced these skills hands-on such that the session-targeted subject matter
should be better understood once the post-SBQ was endeavored at the conclusion of each
training module.
Likewise, a Motion Sickness Assessment Questionnaire (MSAQ) [45] was employed
to query how each driver felt before and after they drove the simulator to assess if, and
to what degree, simulation had a negative impact on physiological response (e.g., nausea,
dizziness, headache). In the full version of the questionnaire, there were 16 questions
total, each rated on a 10-point (0–9) Likert scale, with each question addressing one of
four possible symptom types (i.e., gastrointestinal; central; peripheral; sopite-related). The
MSAQ resulted in sub-scores for each of these four categories, as well as an “overall”
sickness score (i.e., ranging from 0 to 144). The MSAQ was administered post-experiment,
for modules 1, 3, and 5.
Finally, a participant satisfaction form was issued upon program exit to solicit overall
program rating, comments, and suggestions. Separate forms were given both to the drivers
(students) and to their parents. The student form asked a series of questions (10-point
Likert scale) and also contained three open-ended questions asking students for different
types of feedback on the program. The parent form assessed what parents thought of the
program as a whole and was emailed to parents after their child had completed the entire
10 h program. Open-ended feedback comments (from students as well as parents) were
later tabulated coded to provide trend metrics.
2.4. Experimental Cohort

The study cohort size was N = 132 (66 male, 66 female). For deployment, only
teenagers (ages 14–17, permitted in New York State, but unlicensed) were recruited who
had little or no actual driving experience as a requirement of participation—per our
advanced screening procedures. The intention was to recruit only participants who would
be unbiased by pre-existing knowledge of actual on-road driving. This, would enable us
to promote positive training on the simulator with a “clean slate” of truly novice/green
drivers. Other eligibility criteria were strictly health-related: each candidate participant
could not be susceptible to seizures, be claustrophobic, be afraid of dark environments,
or prone to extreme motion sickness (e.g., car, plane, sea). Primary subject recruitment
Safety 2021, 7, 24 12 of 24
sources included digital e-mailings to all regional high schools within a 60 min drive of
campus (where the study was conducted), printed fliers posted about the campus, word-
of-mouth referrals through families who had already completed the program (as well as
related programs on the driving simulator), and printed advertisements within a popular
community newspaper. The Institutional Review Board (IRB) at the University at Buffalo
reviewed and approved our study protocol in advance of its deployment. Subjects were
minimally compensated for their time (a $50 gift card) after completing all five of their
two-hour SBT modules.
3. Results and Discussion

Previously published guidelines [46] have summarized general requirements for the
successful implementation of a motion simulator intended for driver evaluation. These
requirements include: (a) concerns about measurable driver performance criteria, (b) per-
formance with respect to simulator fidelity, and (c) any demerits associated with a simulator
implementation related to maladaptation and simulator sickness. During deployment
and subsequent collection and analyses of data, these guidelines were attempted compre-
hensively through a variety of data forms. Accordingly, our results are decomposed into
three primary categories of performance measurement: (i) quantitative (i.e., as measured
by the simulator), (ii) qualitative (i.e., as observed by the training expert who monitored
each instructional session), and (iii) surveys and questionnaires (i.e., as self-reported by
driving participants and their parents, as appropriate). As a result, the overall analysis is
representative of a truly multi-measure assessment that enabled a thorough and holistic
evaluation of simulator-based driving performance. Results were tabulated, analyzed, and
reported using a combination of both Microsoft Excel and the R statistical software package.
3.1. Quantitative Measures

Frequency histograms were composed to illustrate score report total scores as collected
and calculated by the computational engine of the driving simulator. These data are
representative of core safe driving performance skills as programmed and monitored
in real-time by the simulator (e.g., travel speed and behavior at intersections). Refer to
Figure 11, which plots the average cohort total score on the x axis for each of the five
training sessions (S1–S5), as well as the virtual road test (VRT) on the y axis. The figure
demonstrates a general overall compliance to safe driving habits within the simulator;
however, it indicates that there may be a ceiling effect beyond the first session. For the
remainder of the sessions, ratings from the score report were biased toward the upper
threshold of the rating scale, which is a direct indicator of expected improvement of
cohort core driving skills (with continued practice/exposure). This trend culminated
with the virtual road test, where observed (mean) cohort scores were the highest when
compared against any of the training modules offered within each of the simulator training
sessions. Embedded within each series are the corresponding standard deviations for these
scores report total scores. As expected, there was a wider standard deviation for the first
session; however, these deviations slowly and steadily reduced as training progressed,
culminating with the smallest standard deviation for the virtual road test. This is a likely
indicator that positive training was maintained and actually improved as the program
progressed, as scores tended to increase more uniformly (i.e., with less variation) for the
entire training cohort.
that positive training was maintained and actually improved as the program pro
as scores tended
however, to increaseslowly
these deviations more uniformly (i.e.,
and steadily with less
reduced asvariation) for the entire
training progressed, c
cohort.
ing with the smallest standard deviation for the virtual road test. This is a likely i
that positive training was maintained and actually improved as the program pro
Safety 2021, 7, 24 13 of 24
as scores tended to increase more uniformly (i.e., with less variation) for the entire
cohort.
Figure 11. Score report totals.
In Figure 12, observed results are compared across simulator fidelities amon
bers 11.
Figure
Figure
of11.
the cohort.
Score
Score report
report
Note that all recruited participants were assigned to a cohort (i.
totals.totals.
base or motion-based) for all training sessions. Although it may have been idea
In Figure 12, observed
all participants operate results
both aresimulator
compared acrosstypes,simulator
enablingfidelities among members
asimulator
comparison of fidelitie
In Figure 12, observed results are compared across fidelities amon
of the cohort. Note that all recruited participants were assigned to a cohort (i.e., fixed-base
eachof
bers participant,
the cohort. the
Note overall cohort
that sessions.
all size (N
recruited = 132) provided
participants were a sufficient to abasis for(i.c
or motion-based) for all training Although it may have beenassigned
ideal to have cohort
all
son of
participantshuman
base or motion-based) performance
operate both simulator across
for all types,
trainingsimulator-fidelities.
sessions.
enabling Although
a comparison For the first
it maywithin
of fidelities session,
haveeach coho
been ideal
all participants operate both simulator types, enabling a comparison ofvariance
were higher
participant, the on the
overall fixed-base
cohort size (N simulator
= 132) (but
provided a with much
sufficient basis wider
for result
comparison fidelitie
of human
second performance
session, across
observed simulator-fidelities.
results were For
nearly the
equal
each participant, the overall cohort size (N = 132) provided a sufficient basis forfirst session,
across cohort
simulator results
types, and c
were higher
third, fourth,on the
andfixed-base
fifth simulator
sessions (but
(as with
well much
as the wider
virtual result
road variance);
test), for the
observed resu
son of human performance across simulator-fidelities. For the first session, cohor
second session, observed results were nearly equal across simulator types, and for the third,
slightly
were
fourth, andhigher
higher fifthon for
thethe
sessions (asmotion-based
fixed-base
well as the simulator.
simulator
virtual road(but The
with
test), general
much
observed hypothesis
wider
results were result for
slightly these
variance)
tions for
second
higher is session,
that for the first simulator.
observed
the motion-based session,
resultsthe Thefixed-base
were nearly
general simulator
equal
hypothesis across was
for these easier
simulator totypes,
observations learn and
and
for a for
isthird,
that novice
the trainee,
first session, andthe the second
fixed-base session
simulator served
was easieras
fourth, and fifth sessions (as well as the virtual road test), observed resu toan inflection
learn and point.
operate for Howev
athe
slightlytrainee,
novice
cohort became
higher and the
for the second acclimated
suitably session served
motion-based toasand
simulator.an inflection
comfortable
The generalpoint.with However, once
the companion
hypothesis for theses
the cohort became suitably acclimated to and comfortable with the companion simulator
training
tions environments,
isenvironments,
that for the the firstverythe verythe
session, presence of motion cueswas to accompany theand
sim
training presence offixed-base
motion cuessimulator
to accompany easier
the to learn
simulation
visuals,
for
visuals, as as
a novice wellwell
as theasgreater
trainee, theand greater induced
the second
induced sense
sensesession
of both of both
served
presence aspresence
and an inflection
immersion and immersion
point.
within the Howev w
motion-based
motion-based
the cohort became simulator
simulator display
suitably display
system, system,
acclimated to provided
provided a higher-quality
a higher-quality
and comfortable (i.e., morethe
with (i.e., more au
authentic)
companion s
learning
training environment
learning environments,
environment and and therefore served
thetherefore to better
servedof
very presence promote
tomotion positive
better promote training
cues to positive overall.
accompany training
the simo
visuals, as well as the greater induced sense of both presence and immersion w
motion-based simulator display system, provided a higher-quality (i.e., more au
learning environment and therefore served to better promote positive training ov
Figure
Figure 12.12. Simulator
Simulator fidelity
fidelity comparison.
comparison.
Finally, individual
Finally, components
individual of the score
components report
of the werereport
score investigated
were for noteworthy for not
investigated
trends. Refer to Table 3, which presents a summary decomposed by simulator fidelity. In
trends.12.
Figure
general,
Refer to
Simulator
trainees
Table
showed
3, which
fidelity presents a summary decomposed by simulator fid
comparison.
stronger compliance to posted speed limits on the fixed-base
general, As
simulator. trainees
shown inshowed stronger
the second and thirdcompliance toTable,
columns of the posted speed
trainees limits
drove on the fi
almost
simulator.
twice Finally, individual components of the score report were investigated fordrov
as fast As
over shown
the speed in the
limit second
on the and third
higher-fidelity columns
motion of the
simulator Table,
(2.6 vs. trainees
1.5 mph not
over the
twice asspeed
fastlimit,
overonthe
average),
speedresulting
limit onin slightly lower speed sub-scores.
the higher-fidelity motion This result is (2.6 vs.
simulator
trends. Refer to Table 3, which presents a summary decomposed by simulator fid
general, trainees showed stronger compliance to posted speed limits on the fix
simulator. As shown in the second and third columns of the Table, trainees drov
twice as fast over the speed limit on the higher-fidelity motion simulator (2.6 vs.
over the speed limit, on average), resulting in slightly lower speed sub-scores. Th
is somewhat surprising, as one would think that a higher-fidelity environment
terms of display system and motion capabilities) would result in more authentic
behaviors and stronger vigilance to posted speed limits. These trends were act
Safety 2021, 7, 24 served for the latter two score report sub-scores, where stronger compliance 14 of 24 was o
toward intersection regulation (stop signs and streetlights) on the motion-based s

relative to similar observations with the fixed-base simulator (i.e., reflected by
somewhat surprising,
two columns as one
of the would think that a higher-fidelity environment (both in terms
Table).
of display system and motion capabilities) would result in more authentic driving behaviors
and stronger vigilance to posted speed limits. These trends were actually observed for
Table 3. Score Report Metrics by Simulator Type (Entire Cohort).
the latter two score report sub-scores, where stronger compliance was observed toward
intersection regulation (stop signs and streetlights) on the motion-based simulator relative
Speed Stop Sign Street Lig
to similar observations with the fixed-base simulator (i.e., reflected by the final two columns
Simulator Type Average Speed over Speed Limit (mph)
of the table). Sub-Score Sub-Score Sub-Sco
(0–100) (0–100) (0–100)
Table 3. Score Report Metrics by Simulator Type (Entire Cohort).
Fixed-base 1.5 89.5 92.6 92.8
Motion-based 2.6 Average Speed 87.3
Speed Stop Sign96.1 Street Light 95.1
Simulator Type over Speed Sub-Score Sub-Score Sub-Score
Limit (mph) (0–100) (0–100) (0–100)
3.2.Fixed-base
Qualitative Measures
1.5 89.5 92.6 92.8
Motion-based
Although 2.6
quantitative 87.3
measures served as 96.1
the primary 95.1
mechanism for tra
formance assessment, training evaluators also performed a more subjective quali
3.2. Qualitative Measures
sessment (upon pairs of trainees) during each instructional session. In this way
Although quantitative measures served as the primary mechanism for trainee per-
categories of trainee performance could be monitored and measured that would
formance assessment, training evaluators also performed a more subjective qualitative
challenging
assessment to program
(upon a simulator
pairs of trainees) to quantify
during each withsession.
instructional any meaningful
In this way, accuracy.
cer-
ing,categories
tain it helped of us to maintain
trainee performancean could
ever-critical “human-in-the-loop”
be monitored and measured that would as a critical
be co
of the
more overall human
challenging to programperformance
a simulator assessment. Toany
to quantify with thismeaningful
end, Figure 13 presents
accuracy. In an
so doing, it helped us to maintain an ever-critical “human-in-the-loop” as
depiction of the qualitative assessment for a single participant, in the form of an ina critical con-
stituent of the overall human performance assessment. To this end, Figure 13 presents
evaluation checklist (IEC). The IEC was performed across three qualitative perf
an example depiction of the qualitative assessment for a single participant, in the form of
categories,
an each rated
instructor evaluation on a (0–4)
checklist (IEC). Likert
The IECscale and observed
was performed across over the five primary
three qualitative
tional sessions.
performance The categories
categories, each rated onrepresent an assessment
a (0–4) Likert of (a) overall
scale and observed over thelateral
five lane
primary instructional sessions. The categories represent an assessment of (a)
(b) vigilance for using turn signals to indicate turns and lane changes, and (c) ma overall lateral
lane stability, (b) vigilance for using turn signals to indicate turns and lane changes, and
appropriate speed/distance specifically amidst the “challenge” segments of the
(c) maintaining appropriate speed/distance specifically amidst the “challenge” segments
ofmodules.
the training For this particular
modules. traineetrainee
For this particular (sample), overall,
(sample), qualitative
overall, ratings gene
qualitative ratings
proved improved
generally across the fivethe
across training sessions—particularly,
five training sessions—particularly,compliance
compliance to to“turn
“turn signa
signal indicators”, which demonstrated steady improvement across
tors”, which demonstrated steady improvement across all five sessions. all five sessions.
Figure
Figure 13.13. Sample
Sample instructor
instructor evaluation
evaluation (qualitative).
(qualitative).
As a mechanism by which to culminate these qualitative observations across the

As a mechanism by which to culminate these qualitative observations acros
entire cohort, project evaluators simultaneously completed handwritten instructor (HI)
tire cohort,
comments, project
later coded evaluators simultaneously
to create scores for the numbercompleted handwritten
of positive and instructor (
negative qualita-
ments,
tive later coded
performance to comments
feedback create scores
issuedfor
perthe number
trainee, of positive
per session. and negative qu
These tabulations
performance feedback comments issued per trainee, per session. These tabula
sulted in a difference metric, per training module. To this end, Figure 14 illust
Safety 2021, 7, 24
positive (POS) and negative (NEG) scores, as well as the difference score (D
15 of 24
across the entire training cohort and issued per training module for each
five instructional sessions. The general observed tendency is that the diffe
positively
resulted correlated
in a difference with
metric, additional
per training module.simulator
To this end,exposure, as shown
Figure 14 illustrates thein the p
positive (POS) and negative (NEG) scores, as well as the difference
gram progressed, students continued to progressively demonstrate add score (DIFF), averaged,
across the entire training cohort and issued per training module for each of the primary
driving
five habits
instructional and, through
sessions. experiential
The general exposure
observed tendency is that with the simulation,
the difference metric pr
isperformance improvement
positively correlated amidst
with additional simulatorvaried andasmore
exposure, shown complex
in the plot.driving
As the situa
program
although progressed, students continued toevaluation
the positive/negative progressively demonstrate
differential additional
increased positive
as intend
driving habits and, through experiential exposure with the simulation, practiced gradual
quantity of negative comments issued were observed to steadily increase
performance improvement amidst varied and more complex driving situations. Note that
training
although thesessions, as illustrated.
positive/negative evaluationOne likelyincreased
differential reasonasfor this observation
intended, the overall is th
quantity of negative comments issued were observed to steadily
tion of more challenging driving scenarios as sessions proceeded. increase across the five Facing
training sessions, as illustrated. One likely reason for this observation is the gradual addi-
for the first time creates a new potential for young driver error, and this ma
tion of more challenging driving scenarios as sessions proceeded. Facing a new situation
inthe
for additional negative
first time creates demerits
a new potential for from the human
young driver error, and evaluator,
this may have even as the stu
resulted
into
additional
masternegative demerits from
core driving the human
skills from evaluator,
previous even as the student
exposure. continued to
Likewise, as train
master core driving skills from previous exposure. Likewise, as training progressed across
across training modules and instructional scenarios, the evaluator likel
training modules and instructional scenarios, the evaluator likely exhibited increased ex-
creasedasexpectations
pectations students began to asmaster
students
baselinebegan
drivingto master
skills baseline
and perhaps driving
felt an increasedskills a
responsibility to steadily reinforce more advanced driving behaviors.
an increased responsibility to steadily reinforce more advanced driving be
Figure
Figure 14. Evaluation
14. Evaluation summary.
summary. DIFF,
DIFF, difference difference
score; score;
POS, positive; POS,
NEG, positive;
negative. NEG, negat
3.3. Self-Report Measures
3.3.AsSelf-Report Measures
a tertiary mechanism for the holistic driver performance evaluation, quantitative
and qualitative measures
As a tertiary were supplemented
mechanism for thewith a series
holistic of self-report
driver metrics. An
performance evaluatio
overall data comparison found that post-SBQ scores were improved (with significance) as
and qualitative
compared to the pre-SBQmeasures were
scores, (t(132) supplemented
= +3.64, p = 5.19 × 10−with a series
9 ). Figure 15 plotsof
theself-report
results m
allvs.data
(pre post)comparison
of the session-basedfound that post-SBQ
questionnaire (SBQ) cohort scores
averagewere improved
across each of the five(with s
instructional sessions. When individual sessions were compared,
compared to the pre-SBQ scores, (t(132) = +3.64, p = 5.19e ). Figure the post-SBQ
−9 scores were
15 plots
higher (with significance) than pre-SBQ scores for three of the five instructional sessions:
vs. post)
Session of the
1, t(132) session-based
= +8.89, p =3.18 × 10−11questionnaire
; Session 2, t(132) (SBQ)
= +5.00,cohort
p = 6.02 × average
10−5 ; andacross
instructional
Session sessions.
5, t(132) = +7.92, p = 2.24When
× 10 . individual
− 7 sessions
The lack of observed were compared,
improvement for the other the p
two
were higher (with significance) than pre-SBQ scores for threesufficient
individual sessions (i.e., Sessions 3 and 4) could indicate that there was not of the five in
correlation between the content of the questionnaires and the embedded training content
sions:
within theSession
simulator1, t(132)designed
exercises = +8.89,explicitly
p =3.18e -11; Session 2, t(132) = +5.00, p = 6.02
as the focal point for those sessions.
5, t(132) = +7.92, p = 2.24e−7. The lack of observed improvement for the other
sessions (i.e., Sessions 3 and 4) could indicate that there was not sufficient
tween the content of the questionnaires and the embedded training con
simulator exercises designed explicitly as the focal point for those sessions
Safety 2021, 7, x FOR PEER REVIEW
Safety 2021,
Safety 7, x
2021, 7, FOR
24 PEER REVIEW 16 of 24
Figure 15. SBQ (pre/post).
The Motion Sickness Assessment Questionnaire (MSAQ) was issued post-s

in Sessions 1, 3, and 5. An overall evaluation found that, as expected, simulator
Figure
was
Figure not15.
15. SBQ SBQ
identified(pre/post).
(pre/post). to be a primary concern for these simulator experiments that
signed and deployed for this young driving cohort. Figure 16 displays the coh
The Motion Sickness Assessment Questionnaire (MSAQ) was issued post-simulator
The Motion Sickness Assessment Questionnaire (MSAQ) 144)was issued post-s
inscore (reported
Sessions 1, 3, andas a percentage
5. An of thefound
overall evaluation overall MSAQ
that, score:
as expected, simulator and the number
sickness
wasin Sessions
ipants who 1,
not identified 3,toand
reported be a5. An overall
cumulative
primary concern evaluation
scores foundthan
of greater
for these simulator that, 14as(15)
experimentsexpected,
of
thatthe simulator
total MSA
were
was
designed not andidentified
deployed to be
for a
this primary
young concern
driving for
cohort. these
Figure
(also reported as a percentage of the total N = 132 cohort). Standard deviation simulator
16 displays experiments
the cohort that
signedsessions
mean
three scoreand deployed
(reported as a
(1, 3, and for5)this
percentage
were young
of the driving cohort.
overall
significantly MSAQ score:
large only Figure
144) 16 displays
and
relative the number the coh
to the reported
ofscore
participants
(reported who reported
as a cumulative
percentage ofscores
the of greater
overall MSAQthan 14score:
(15) of144)
the total
and MSAQ
the number
they(also
scale were 2.56%,
reported as 2.55%, and of
a percentage 4.44%,
the totalrespectively,
N = 132 cohort). which is andeviations
Standard indicator forthat the
ipants who reported cumulative scores of greater than 14 (15)
the three sessions (1, 3, and 5) were significantly large only relative to the reported means;16 also
comparatively wide dispersion of results. To this point, note that of the
Figure total MS
a(also
they reported
were
maximum 2.56%,(MAX) as a and
2.55%, percentage
MSAQ 4.44%, ofacross
the total
respectively,
score N =is 132
which
the cohort an for cohort).
indicator ofStandard
each that thethere wasdeviation
three a
sessions
three sessions
comparatively wide (1, 3, and
dispersion 5)
of were
results. significantly
To this point, large
note
this metric was collected and analyzed. While the extremes are noteworthy that only
Figure relative
16 also to the
displays areporte
out
maximum
they (MAX)
were MSAQ
2.56%, score
2.55%, across
and the cohort
4.44%, for each of
respectively, the three
which sessions
is an in which
indicator this that the
overall trends of the data are favorable. The low MSAQ scores across the coho
metric was collected and analyzed. While the extremes are noteworthy outliers, the overall
comparatively
tributed todata wide dispersion
the are
youthful ageThe of results. (14–17),
demographic To this point,who note
are that susceptible
rarely Figure 16 also to
trends of the favorable. low MSAQ scores across the cohort are attributed
athe
maximum
tomore (MAX) MSAQ score across the cohort
than “mild” simulator maladaptation. The experimental design (see Tabl
youthful age demographic (14–17), who are rarely for
susceptibleeach to of the
anything threemoresessions
this“mild”
than metric
cessfully was collected
simulator
imparted and analyzed.
maladaptation.
simulation While
The experimental
best-practices theintentionally
design
and extremes
(see Table 2) are noteworthy
successfully
balanced the 2ouh
imparted
overall simulation
trends ofbest-practices
the data and intentionally
aretransitions
favorable. Thebalanced the 2 hscores
lowdriver/passenger
MSAQ trainingacross
sessions the coho
sessions with breaks, timely between activities, vid
with breaks,
tributed timely
to the transitions
youthful between
age driver/passenger
demographic activities,
(14–17), who video
are training,
rarely survey
susceptible to
ing, survey
activities, activities,
and perhaps mostand perhaps most experiment
critically—simulator critically—simulator
durations thatexperiment
fell below a durat
more
15fell
thana“mild”
minbelow
maximum. 15 min simulator
maximum.maladaptation. The experimental design (see Tab
cessfully imparted simulation best-practices and intentionally balanced the 2 h
sessions with breaks, timely transitions between driver/passenger activities, vid
ing, survey activities, and perhaps most critically—simulator experiment durat
fell below a 15 min maximum.
Figure
Figure 16.16. Motion
Motion Sickness
Sickness Assessment
Assessment Questionnaire
Questionnaire (MSAQ)
(MSAQ) summary. summary.
Finally, exit surveys were issued to both the teen trainees and their parents, and these
Finally, exit surveys were issued to both the teen trainees and their parents, a
data further yielded valuable information about the training program. For the teen trainees,
data
two furtherwere
questions yielded valuable
administered on information
a 1 to 10 Likertabout thefirst
scale. The training
asked,program. For the te
“Overall, rate
Figure
your 16.
ees,experienceMotion
two questions Sickness
in the training Assessment
were program”,
administered Questionnaire
onsecond
and the a 1 toasked,(MSAQ)
10 Likert summary.
scale.
“Overall, Theamount
rate the first asked, “
ofrate
timeyour
that was spent on the driving simulator”. Refer to Figure 17,
experience in the training program”, and the second asked, which co-analyzes both
“Overall
series. Finally,
Overall exit surveys
“experience” were
ratings on issued
the to both
simulator was the teen
observed
amount of time that was spent on the driving simulator”. Refer to Figure trainees
to be and
highly their parents,
favorable, 17, w
data further
analyzes bothyielded
series. valuable information about
Overall “experience” theon
ratings training program.
the simulator wasForobserv
the te
ees, two
highly questionswith
favorable, were administered110
approximately on aof1the
to 10
N =Likert scale. (83%)
132 cohort The first asked,
rating the
rate your experience in the training program”, and the second asked, “Overall
experience as being an 8/10 or better. In contrast, “time on simulator” ratings w
Safety 2021, 7, 24 17 of 24
with approximately 110 of the N = 132 cohort (83%) rating their overall experience as
exercises),
being an 8/10as or opposed to session
better. In contrast, “timetime spent onratings
on simulator” other were
activities (e.g., watching
considerably lower, vid
exercises),
with a weighted
as cohort
opposed to session
average of just
time spent
5.7/10. This
on likely
was
otherbecause
activitiesnot
(e.g.,
enough
watching
time
vid
ing surveys, watching partner drive, experiment debrief). Likewise, teens were
ing surveys,
spent watching
was spent allowing partner
teens to drivedrive, experiment
the simulator debrief).
(performing Likewise, teens
simulator-exercises), as were
the opportunity to answer open-ended questions related to their training exp
the opportunity to answer open-ended questions related to their training exp
opposed to session time spent on other activities (e.g., watching videos, taking surveys,
three questions
watching partner drive, were assessed:
experiment “What
debrief). did you
Likewise, teenslearn about safe
were afforded driving?” (375 t
the opportunity
three questions were assessed: “What did you learn about safe driving?” (375 t
toments
answerreceived),
open-ended “What didrelated
questions you like most
to their about
training your simulator
experiences; training
three questions wereprogra
ments received), “What did you like most about your simulator training progra
assessed: “What did
total comments you learn
received), and about
“What safe did
driving?” (375 totalabout
you dislike commentsthe received),
driving “What
simulation tr
total
did youcommentsmost received),
like comments yourand
aboutreceived). “What
simulator did you
training dislike (330
program?” abouttotalthe driving
comments simulation tr
received),
(112 total The three most common responses (reported as perc
(112
and total did
“What comments received).
you dislike about the Thedriving
three simulation
most common responses
training?” (reported
(112 total comments as perc
for each
received). Thequestion
three most are summarized
common responsesin Figureas18.
(reported Most prominently,
percentages) for each questiontrainees m
for each question are summarized in Figure 18. Most prominently, trainees m
learning
are summarizedabout in hazard
Figure 18. management
Most prominently, and the mentioned
trainees importance of maintaining
learning about hazard an app
learning about hazard management and the importance of maintaining an app
following distance
management and speed
and the importance (relative to
of maintaining anconditions). Similarly,
appropriate following trainees
distance and repor
following
speed (relativedistance
to and speed
conditions). Similarly, (relative
trainees to conditions).
reported most Similarly,
liking the realismtrainees
of the repor
liking the realism of the simulator, the informative nature of the training modu
liking the
simulator, therealism
informative of nature
the simulator, the modules,
of the training informative and thenature
hands-onof the training modu
engagement
the
ofthe
hands-on engagement of driving a simulator for training. By way of criticism
hands-on
driving engagement
a simulator for training. of driving
By way ofa criticism,
simulator for training.
trainees suggestedBy way
that theof criticism
lane
suggested
markers and thatwere
signs the difficult
lane markersto see; and signs
despite were the
acclimation, difficult
steering to see;and
wheel despite
pedals acclima
suggested that the lane markers and signs were difficult to see; despite acclima
steering
were wheel
difficult andtopedals
to adjust were difficult
(i.e., a common complaint to of
adjust to (i.e.,
all driving a common
simulators); andcomplaint
some of
steering wheel and pedals were difficult to adjust to (i.e., a common complaint of
suggested that the graphics were not realistic.
ing simulators); and some suggested that the graphics were not realistic.
ing simulators); and some suggested that the graphics were not realistic.
Figure
Figure 17.17. Teen
Teen Assessment
Assessment Experience/Time
Experience/Time on Drivingon Driving Simulator.
Simulator.
Figure 17. Teen Assessment Experience/Time on Driving Simulator.
Figure 18. Teen Assessment Open-ended Summary.

Figure
Figure 18.18. Teen
Teen Assessment
Assessment Open-ended
Open-ended Summary.Summary.
InIn a similar
a similar manner,
manner, parentsparents werea series
were issued issued of aquestions
series of questions
when when
their teen com- their te
In a similar manner, parents were issued a series of questions when their t
pleted
pleted thethe
training program.
training OverallOverall
program. ratings were extremely
ratings were positive,
extremely as parents nearly-
positive, as parent
pleted the training
unanimously suggested program.
that (a) Overall
simulator ratings
training wereconstitute
should extremely positive, as parent
a mandatory
unanimously suggested that (a) simulator training should constitutefirst
a manda
unanimously
step suggested
in driver training, that (a)
(b) they would simulator
recommend thetraining should
course to other constitute
parents, a manda
and (c) the
step in successfully
driver training, (b) their
theychild
would recommend the course to other parents, an
step in driver training, (b) they would recommend the course to other parents, an
program instructed basic driving skills. Parents were also afforded
program successfully instructed their child basic driving skills. Parents were also
program successfully instructed their child basic driving skills. Parents were also
the opportunity to answer open-ended questions related to their own perception
the opportunity to answer open-ended questions related to their own perception
teen’s training experiences; three questions were assessed (each, on a 3-point Lik
teen’s training experiences; three questions were assessed (each, on a 3-point Lik
“As a result of their training, did you see your child paying more attention to ro
Safety 2021, 7, 24 18 of 24
the opportunity to answer open-ended questions related to their own perceptions of their
teen’s training experiences; three questions were assessed (each, on a 3-point Likert scale):
“As a result of their training, did you see your child paying more attention to road signs,
Safety 2021, 7, x FOR PEER REVIEWroad hazards, and general road compliance?” (i.e., translation of training experiences from
Safety 2021, 7, x FOR PEER REVIEWsimulator to real-world), “How much do you think your child enjoyed being part of the SBT
program?”, and “Overall, how would you rate this program?” The Likert-style results (re-
ported as response percentages) for each question are summarized (favorably) in Figure 19.
As with theirreceived).
comments teens, parents
Forwere
eachafforded
category,the opportunity
the three to mostanswer open-ended
common questions
responses (reporte
comments
(positive andreceived).
negative)For each
related to category,
their child’sthe three experiences
training most common responses
(126 total comments(reporte
centages) Forare summarized in Figure 20. Parents opined that theas primary benefits w
centages)
received). are summarized
each in Figure
category, the three 20. Parents
most common opined
responses that the
(reported primary
percentages)benefits w
teens
are enjoyed in
summarized driving
Figure the
20. simulator
Parents andthat
opined thatthethe instructors
primary benefits were
were of high
that teensquality;
teens enjoyed driving the simulator and that the instructors were of high quality;
highest frequency response (much less significant) was that teens seemed to have
enjoyed driving the simulator and that the instructors were of high quality; the third
highest
highest
frequency response
frequency
(much less significant)
(much less significant)
wasseemed
that teens seemed to have
driving anxietyresponse
(while driving) as a benefit was ofthat teens
their simulatortoexposure.
have reduced However
driving
driving anxiety
anxiety (while(while driving)
driving) as aof
as a benefit benefit of theirexposure.
their simulator simulator exposure.
However, However
parents
also noted that they would have liked to have seen more simulator time and in
alsonoted
also noted thatthat
theythey
would would havetoliked
have liked to have
have seen more seen more
simulator simulator
time time and i
and increased
graphics
graphics realism
realism on onsimulator,
the the simulator,
and many and manyalso
parents parents
noted also
the noted
desire to the desire
attempt the to atte
graphics realism on the simulator, and many parents also noted the desire to att
simulator
simulator themselves.
themselves. Thehighest
The third third highest
frequency frequency response
response (much (much less
less significant) wassignificant)
that
simulator themselves. The third highest frequency response (much less significant)
parents wished for higher visual realism for the simulator.
Figure
Figure 19.19. Parent
Parent (direct)
(direct) Assessment
Assessment Likert-style
Likert-style feedback.
feedback.
Figure 19. Parent (direct) Assessment Likert-style feedback.
Figure 20. Parent Assessment Open-ended Summary.

Figure
Figure 20.20. Parent
Parent Assessment
Assessment Open-ended
Open-ended Summary.Summary.
4.4.Broader
Broader Impacts
Impacts of Methodology
of Our our Methodology and Implementation
and Implementation
4. Broader Impacts of our Methodology and Implementation
The outcomes
The outcomesof theof
current research research
the current aimed to demonstrate the effectivenessthe
aimed to demonstrate of physics-
effectiveness
The outcomes
based M&S of the
and its potential tocurrent
improve research aimed
roadway safety andtopublic
demonstrate the effectiveness
health within North
ics-based M&S and its potential to improve roadway safety and public healt
ics-based
American M&S
young andtraining.
driver its potential to improvecould
Such a demonstration roadway
result insafety
a more and public healt
widespread
North American
deployment young driver
of these advanced training.
technologies Such a demonstration
and augmentation could
of similar training result in a mo
programs
North American young driver training. Such a demonstration could result in a mo
spread
across thedeployment
nation. Agenciesofthat
these
mightadvanced technologies
be interested and augmentation
in such technology of similar
would include high
spread deployment of these advanced technologies and augmentation of similar
programs across the nation. Agencies that might be interested in such technolog
programs across the nation. Agencies that might be interested in such technolog
include high schools, driver training agencies, and law enforcement. The succe
include high schools, driver training agencies, and law enforcement. The succe
plementation of SBT frameworks could, over time, result in improved driving
plementation of SBT frameworks could, over time, result in improved driving
for the teenage demographic, where a disproportionate number of negative driv
Safety 2021, 7, 24 19 of 24
schools, driver training agencies, and law enforcement. The successful implementation
of SBT frameworks could, over time, result in improved driving practices for the teenage
demographic, where a disproportionate number of negative driving outcomes (e.g., crashes,
injuries) continue to occur [1,2]. Ultimately, this could lead to various other benefits to
society, including an increased level of safety on the roadways for all drivers, resulting
in a reduction in accidents, injuries, and loss of life. In turn, this could result in reduced
insurance rates for all drivers, particularly those in the highest-risk pool (i.e., young males).
Likewise, by leveraging the groundwork established in this paper, similar training
programs could be developed for other vehicle types for large equipment civilian vehicle
simulator training, including commercial truck driving. Recent research [47–49] indicates
that skills received from exposure within a large-vehicle simulator (e.g., trucks, aviation)
can directly transfer to skills required to operate an actual truck/aircraft—and are learned
as quickly (and sometimes faster) than is the case with live/physical training. Within
this context, simulators—when employed in addition to (but not as a replacement for)
hands-on training—can be particularly useful for instruction of gear-shifting and linear
and angular maneuvers of increasing complexity. Likewise, in the military, it remains an
ongoing challenge to determine efficient and effective mechanisms for simulator-based
training (SBT). It has been emphasized that what matters most for maximizing SBT ef-
fectiveness is an environment that fosters psychological fidelity (e.g., cognitive skills) by
applying established learning principles (e.g., feedback, measurement, guided practice,
and scenario design) [50,51]. Too often, the evaluation of training outcomes focuses on how
the simulator itself functions, instead of on how humans perform and behave within the
simulator [49,51]. The design and development of a training framework that demonstrates
sufficient physical fidelity, while focusing on the analysis of human factors and objective
metrics for performance within the training environment, remains a point of emphasis for
the current work.
5. Conclusions and Future Work

Traffic accidents remain a primary safety concern and are a leading cause of death for
young, inexperienced, and newly licensed drivers [1–4]. For young driver education, it
therefore remains an utmost public health priority to determine how technology can be
better employed to increase the overall quality and effectiveness of current driver training
methods. Supplemental training afforded by realistic physics-based modeling and hands-
on simulation provides much-needed “seat time”, that is, practice and exposure within an
engaging and authentic setting that promotes experiential learning. Concrete behavioral
and performance feedback can be implemented to enhance and promote positive training,
which could help to reduce negative driving outcomes (e.g., crashes, tickets, property
damage, and fatalities) and thereby improve young driver safety over the long term.
In this paper, a detailed case study was described that examined the specification,
development, and deployment of a “blueprint” for a simulation-centric training framework
intended to increase driver training safety in North America (see Figure 1). Points of focus
in this paper have included an overview of the numerous advantages of expanding the
use of advanced simulation and gaming elements in current and future driver training;
consideration toward simulator fidelity (e.g., including motion capabilities and field-
of-view); and simulator-based driving environments to enhance young driver training
(e.g., including closed-course, residential, and challenge scenarios to accompany baseline
environment acclimation).
A multi-measure (holistic) assessment of simulated driver performance was developed
and demonstrated, including quantitative, qualitative, and self-report rubrics, from which
the following trends were observed:
• Cohort score report total scores exhibited a general upward trend across the entire
program, and these trends were moderately enhanced with the presence of motion
cues (quantitative).
Safety 2021, 7, 24 20 of 24
• Both negative and positive feedback evaluator comments increased as the training
modules proceeded. The overarching hypothesis is that the increase in positive
feedback was counteracted by the driving session evaluator’s tendency for increased
“bias” in the form of growing expectations as the program proceeded (qualitative).
• Post-SBQ scores were higher than pre-SBQ scores (due to module-specific knowledge
gained due to simulator exposure), as expected. Motion sickness (by way of MSAQ)
was found to be insignificant, which can be credited to the experimental design (see
Table 2) and to the young demographics of this specific cohort (self-report).
• Teens rated their experience on the simulator favorably but suggested that more of
the overall session time should have been spent on the simulator itself rather than on
related activities associated with the training program. They recognized the simulator
as an effective way to learn but disliked some of the visual elements and control
aspects of the simulator (self-report).
• Almost all parents agreed that the simulator program should be a first step toward
future driver training. Furthermore, per their own observations of subsequent teen
driving performance, favorable feedback was issued for specific driving skillsets
gained, including improved recognition to road signs and road hazards (self-report).
This paper concludes with numerous suggestions and recommendations that would
enhance future implementations of simulator training to further improve teen driver safety.
5.1. Improved Metrics for Quantifying Simulator Performance

The simulator score report served as a primary measure for quantifying driver perfor-
mance and behavior during the driving simulator exercises. This robustness of this score
report toward comprehensively quantifying simulator performance stands to be improved
from various perspectives. Future implementations would attempt to quantify driver
compliance to lateral lane position, primarily during turn and lane-change maneuvers,
as well as measuring ability to maintain lane-centric behavior during hazard events and
multitasking; would devise enhanced scoring metrics that are specific to each of the three
course environments to monitor overall progress and compliance across the five instruc-
tional sessions; and would incorporate more explicit metrics for each of the specialized
hazard challenges.
5.2. Improvements to Experimental Training Environment

As a primary segment of teen (and parent) surveys, attempts were made to better
understand elements of the current simulator framework that demand improvement in
future implementations. Many teens suggested that signs and lane markings were difficult
to see; although every attempt was made to ensure clarity and visibility of simulation
environment objects, improvements are still required. Many teens suggested that the
simulator (steering/pedals) were difficult to operate; this remains a common complaint
across all driving simulators—particularly so with the feel of the brake pedal [52]. It is
never feasible to make a simulator “feel” like everyone’s own vehicle, both physically
(e.g., the resistive force on the brake pedal) and visually (e.g., the reduced sensation
of longitudinal motion—or vection [53]—while braking within a virtual environment).
Nonetheless, ongoing improvements including additional acclimation and device “tuning”
to each driver should be considered. Finally, various teens (and parents) suggested the
need for improved simulator graphics/realism toward a modern-day “gaming” standard.
5.3. Impairment Training Modules

The current implementation of SBT content was designed explicitly to promote posi-
tive training (i.e., enhancement of knowledge, skills, behaviors [24]) to improve roadway
safety. This was attempted by explicitly addressing the top five (documented) causes for
teen driving accidents and fatalities in New York State [41], with “impairment” being one
of those five. It is well known that teen drivers frequently speak and text-message on
their cell phones while driving [54]. Core simulator exercises are in development that
Safety 2021, 7, 24 21 of 24
compare graphical representations of the teen’s driving to provide concrete performance

and behavioral feedback. Related studies have investigated internal distractions (mind-
wandering) while driving [33,55], which is well-suited to a simulator-based deployment.
Finally, according to data published over the last decade, nearly one-fifth of teenagers
killed in automobile accidents were under the influence (i.e., illicit or prescription drugs,
alcohol), and young drivers are more 17 times more likely to be involved in a fatal accident
when they have a blood alcohol concentration of 0.08% (i.e., the legal limit in New York
State) than when they have not been drinking [56]. Simulator-based exercises have been
designed and installed [42] that enable teen drivers to perform “routine” driving tasks
with/without a simulated state of driver impairment that similarly includes concrete driver
performance feedback.
5.4. Driving Exercises that Promote Reverse Egress

One major deficiency in the present deployment is its current inability to perform a
reverse maneuver within the simulator. Particularly within a driver education context,
this feature could be extremely useful and effective toward improving young driver safety.
For example, reverse maneuvers within a crowded parking area, 2-point turns (e.g., on
a one-way street), 3-point turns (e.g., into a driveway), and the classic parallel-parking
maneuver all require an accurate simulation model for reverse vehicle motion. All of these
maneuvers are encountered during every day driving, and many are requirements on a
standard New York State road test examination [57].
5.5. Driving Exercises for Driver Training on Rural Roads

Previous research [58] has verified that fatality risk is elevated more so on rural roads
than urban roads for all drivers and particularly teen drivers [59]. Per miles traveled,
there have been estimated to be 30–40% more crashes and deaths in rural areas than in
urban areas [59]. Likely causes of these untoward events are attributed to: remote areas
with which young drivers are less familiar; low-lighting conditions during nighttime
driving; increased propensity for elevated speeds on roads that are less traveled [60]; and
as a result of these factors—an elevated likelihood for induced cognitive distraction (e.g.,
mindlessness, task-unrelated thought [33,55]) during the driving task. As such, these
are all critical factors for which to provide exposure—and subsequently examine human
performance behaviors—in future teen driving SBT research.
5.6. Longitudinal Data to Measure Efficacy of SBT

To determine the long-term efficacy of employing simulation for teen drivers on
a broader scale, it would be beneficial to observe the impact of such training exposure
upon lifetime driving statistics. In New York State, the MV-15 report [61] provides vital
information regarding traffic accidents, moving violations, and other negative driving
outcomes of program participants. In future implementations of similar programs, it
would be useful to obtain consent/assent to collect/analyze/compare reports culled from
participants enrolled in simulation-based training programs to similar data culled from
state and national averages. If favorable, such empirical data—presently unavailable in
the literature—could influence future policy for future safety standards related to SBT
courseware.
Author Contributions: Conceptualization, K.E.L. and K.F.H.; methodology, K.F.H.; software, G.A.F.
and K.F.H.; validation, K.F.H.; formal analysis, K.F.H., M.B., H.D. and N.H.; investigation, R.S.A.L.,
M.B., N.H., H.D., B.S., J.-A.D., R.C., I.D.J., A.A., J.D. and N.B.; data curation, M.B. and K.F.H.; writing—
original draft preparation, K.F.H. and R.S.A.L.; writing—review and editing, R.S.A.L. and K.F.H.;
visualization, K.F.H. and R.S.A.L.; supervision, K.F.H.; project administration, N.H., H.D., M.B. and
K.F.H.; funding acquisition, K.E.L. and K.F.H. All authors have read and agreed to the published
version of the manuscript.
Safety 2021, 7, 24 22 of 24
Funding: This research was funded by the National Science Foundation (NSF), grant number 1229861,
the Industry/University Cooperative Research Center (I/UCRC) for e-Design. This research effort
served as a timely contribution to its “Visualization and Virtual Prototyping” track.
Institutional Review Board Statement: The study was conducted according to the guidelines of the
Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of
the University at Buffalo Internal Review Board (protocol code 030-431144 on 21 October 2015).
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: The data presented in this study are available on request from the corre-
sponding author. The data are not publically available due to participant privacy and confidentiality
concerns related to the terms of the study IRB.
Acknowledgments: Toward this extensive, multi-year endeavor, the authors wish to gratefully
acknowledge the financial and technical support provided by Moog, Inc. (East, Aurora, NY) that
further enabled the novel application of M&S to improve STEM, training engagement, and young
driver safety. Lastly, authors Hulme and Lim are grateful for the ongoing support of the University
at Buffalo Stephen Still Institute for Sustainable Transportation and Logistics (SSISTL).
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Centers for Disease Control and Prevention (CDC). WISQARS (Web-Based Injury Statistics Query and Reporting System). US
Department of Health and Human Services. September 2019. Available online: https://www.cdc.gov/injury/wisqars/index.html
(accessed on 12 November 2020).
2. Insurance Institute for Highway Safety (IIHS). Fatality Facts 2017: Teenagers. Highway Loss Data Institute. December 2018.
Available online: https://www.iihs.org/topics/fatality-statistics/detail/teenagers (accessed on 12 November 2020).
3. Mayhew, D.R.; Simpson, H.M.; Pak, A. Changes in collision rates among novice drivers during the first months of driving. Accid.
Anal. Prev. 2003, 35, 683–691. [CrossRef]
4. McCartt, A.T.; Shabanova, V.I.; Leaf, W.A. Driving experiences, crashes, and teenage beginning drivers. Accid. Anal. Prev. 2003,
35, 311–320. [CrossRef]
5. Thomas, F.D., III; Blomberg, R.D.; Fisher, D.L. A Fresh Look at Driver Education in America; Report No. DOT HS 811 543; National
Highway Traffic Safety Administration: Washington, DC, USA, 2012.
6. Vaa, T.; Høye, A.; Almqvist, R. Graduated driver licensing: Searching for the best composition of components. Lat. Am. J. Manag.
Sustain. Dev. 2015, 2, 160. [CrossRef]
7. Strayer, D.L.; Cooper, J.M.; Goethe, R.M.; McCarty, M.M.; Getty, D.J.; Biondi, F. Visual and Cognitive Demands of Using In-Vehicle
Information Systems; AAA Foundation for Traffic Safety: Washington, DC, USA, 2017.
8. Rechtin, M. Dumb and Dumber: America’s Driver Education is Failing Us All—Reference Mark. motortrend.com (Online Article),
Published 20 June 2017. Available online: https://www.motortrend.com/news/dumb-dumber-americas-driver-education-
failing-us-reference-mark/ (accessed on 26 November 2020).
9. Helman, S.V.; Fildes, W.; Oxley, B.; Fernández-Medina, J.; Weekley, K. Study on Driver Training, Testing and Medical Fitness—
Final Report; European Commission, Directorate-General Mobility and Transport (DG MOVE): Brussels, Belgium, 2017; ISBN
978-92-79-66623-0.
10. Nalmpantis, D.; Naniopoulos, A.; Bekiaris, E.; Panou, M.; Gregersen, N.P.; Falkmer, T.; Baten, G.; Dols, J.F. “TRAINER” project:
Pilot applications for the evaluation of new driver training technologies. In Traffic & Transport Psychology: Theory and Application;
Underwood, G., Ed.; Elsevier: Oxford, UK, 2005; pp. 141–156, ISBN 9780080443799.
11. Imtiaz, A.; Mueller, J.; Stanley, L. Driving Behavior Differences among Early Licensed Teen, Novice Teen, and Experienced Drivers
in Simulator and Real World Potential Hazards. In Proceedings of the IIE Annual Conference and Expo 2014, Montreal, QC,
Canada, 31 May–3 June 2014.
12. Mirman, J.H.; Curry, A.E.; Winston, F.K.; Wang, W.; Elliott, M.R.; Schultheis, M.T.; Thiel, M.C.F.; Durbin, D.R. Effect of the teen
driving plan on the driving performance of teenagers before licensure: A randomized clinical trial. JAMA Pediatrics 2014, 168,
764–771. [CrossRef] [PubMed]
13. Milleville-Pennel, I.; Marquez, S. Comparison between elderly and young drivers’ performances on a driving simulator and
self-assessment of their driving attitudes and mastery. Accid. Anal. Prev. 2020, 135, 105317. [CrossRef] [PubMed]
14. Martín-delosReyes, L.M.; Jiménez-Mejías, E.; Martínez-Ruiz, V.; Moreno-Roldán, E.; Molina-Soberanes, D.; Lardelli-Claret, P.
Efficacy of training with driving simulators in improving safety in young novice or learner drivers: A systematic review. Transp.
Res. Part F Traffic Psychol. Behav. 2019, 62, 58–65. [CrossRef]
15. Campbell, B.T.; Borrup, K.; Derbyshire, M.; Rogers, S.; Lapidus, G. Efficacy of Driving Simulator Training for Novice Teen Drivers.
Conn. Med. 2016, 80, 291–296. [PubMed]
Safety 2021, 7, 24 23 of 24
16. De Groot, S.; De Winter, J.C.F.; Mulder, M.; Wieringa, P.A. Didactics in simulator based driver training: Current state of affairs and
future potential. In Proceedings of the Driving Simulation Conference North America, Iowa City, IA, USA, 12–14 September 2007.
17. McDonald, C.C.; Goodwin, A.H.; Pradhan, A.K.; Romoser MR, E.; Williams, A.F. A Review of Hazard Anticipation Training
Programs for Young Drivers. J. Adolesc. Health 2015, 57 (Suppl. 1), S15–S23. [CrossRef]
18. Fuller, R. Driver training and assessment: Implications of the task-difficulty homeostasis model. In Driver Behaviour and Training;
Dorn, L., Ed.; Ashgate: Dublin, Ireland, 2007; Volume 3, pp. 337–348.
19. Hulme, K.F.; Kasprzak, E.; English, K.; Moore-Russo, D.; Lewis, K. Experiential Learning in Vehicle Dynamics Education via
Motion Simulation and Interactive Gaming. Int. J. Comput. Games Technol. 2009, 2009, 952524. [CrossRef]
20. Hulme, K.F.; Lewis, K.E.; Kasprzak, E.M.; Russo, D.-M.; Singla, P.; Fuglewicz, D.P. Game-based Experiential Learning in Dynamics
Education Using Motion Simulation. In Proceedings of the Interservice/Industry Training, Simulation and Education Conference
(I/ITSEC), Orlando, FL, USA, 29 November–2 December 2010.
21. Wickens, C.; Toplak, M.; Wiesenthal, D. Cognitive failures as predictors of driving errors, lapses, and violations. Accid. Anal. Prev.
2008, 40, 1223–1233. [CrossRef] [PubMed]
22. Chiu-Shui, C.; Chien-Hui, W. How Real is the Sense of Presence in a Virtual Environment?: Applying Protocol Analysis for Data
Collection. In Proceedings of the CAADRIA 2005: 10th International Conference on Computer Aided Design Research in Asia,
New Delhi, India, 28–30 April 2005.
23. Berkman, M.I.; Akan, E. Presence and Immersion in Virtual Reality. In Encyclopedia of Computer Graphics and Games; Lee, N., Ed.;
Springer: Berlin/Heidelberg, Germany, 2019. [CrossRef]
24. Groeger, J.A.; Banks, A.P. Anticipating the content and circumstances of skill transfer: Unrealistic expectations of driver training
and graduated licensing? Ergonomics 2007, 50, 1250–1263. [CrossRef] [PubMed]
25. Kaptein, N.; Theeuwes, J.; van der Horst, R. Driving Simulator Validity: Some Considerations. Transp. Res. Rec. 1996, 1550, 30–36.
[CrossRef]
26. Godley, S.T.; Triggs, T.J.; Fildes, B.N. Driving simulator validation for speed research. Accid. Anal. Prev. 2002, 34, 589–600.
[CrossRef]
27. Bella, F. Driving simulator for speed research on two-lane rural roads. Accid. Anal. Prev. 2008, 40, 1078–1087. [CrossRef]
28. Stoner, H.A.; Fisher, D.; Mollenhauer, M., Jr. Simulator and scenario factors influencing simulator sickness. In Handbook of Driving
Simulation for Engineering, Medicine, and Psychology; CRC Press: Boca Raton, FL, USA, 2011.
29. Pettijohn, K.; Pistone, D.; Warner, A.; Roush, G.; Biggs, A. Postural Instability and Seasickness in a Motion-Based Shooting
Simulation. Aerosp. Med. Hum. Perform. 2020, 91, 703–709. [CrossRef]
30. Keshavarz, B.; Ramkhalawansingh, R.; Haycock, B.; Shahab, S.; Campos, J. Comparing simulator sickness in younger and older
adults during simulated driving under different multisensory conditions. Transp. Res. Part F Traffic Psychol. Behav. 2018, 54, 47–62.
[CrossRef]
31. Gee, J.P. What Games Have to Teach Us about Learning and Literacy; Palgrave Macmillan: New York, NY, USA, 2007.
32. Lee, J.; Hammer, J. Gamification in Education: What, How, Why Bother? Acad. Exch. Q. 2011, 15, 146.
33. Hulme, K.F.; Androutselis, T.; Eker, U.; Anastasopoulos, P. A Game-based Modeling and Simulation Environment to Examine the
Dangers of Task-Unrelated Thought While Driving. In Proceedings of the MODSIM World Conference, Virginia Beach, VA, USA,
25–27 April 2016.
34. Milliken, W.F.; Milliken, D.L. Race Car Vehicle Dynamics; Society of Automotive Engineers: Warrendale, PA, USA, 1995.
35. Rajamani, R. Vehicle Dynamics and Control; Springer: New York, NY, USA, 2006; ISBN 0-387-26396-9.
36. Raghuwanshi, V.; Salunke, S.; Hou, Y.; Hulme, K.F. Development of a Microscopic Artificially Intelligent Traffic Model for
Simulation. In Proceedings of the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), Orlando, FL,
USA, 1–4 December 2014.
37. Romano, R. Non-linear Optimal Tilt Coordination for Washout Algorithms. In Proceedings of the AIAA Modeling and Simulation
Technologies Conference and Exhibit, AIAA 2003, Austin, TX, USA, 11–14 August 2003; p. 5681. [CrossRef]
38. Bowles, R.L.; Parrish, R.V.; Dieudonne, J.E. Coordinated Adaptive Washout for Motion Simulators. J. Aircr. 1975, 12, 44–50.
[CrossRef]
39. Sung-Hua, C.; Li-Chen, F. An optimal washout filter design for a motion platform with senseless and angular scaling maneuvers.
In Proceedings of the 2010 American Control Conference, ACC 2010, Baltimore, MD, USA, 30 June–2 July 2010; pp. 4295–4300.
[CrossRef]
40. Akutagawa, K.; Wakao, Y. Stabilization of Vehicle Dynamics by Tire Digital Control—Tire Disturbance Control Algorithm for an
Electric Motor Drive System. World Electr. Veh. J. 2019, 10, 25. [CrossRef]
41. New York State Department of Health (NYSDOH). Car Safety—Teen Driving Safety, (Online Statistics Portal). 2020. Available
online: https://www.health.ny.gov/prevention/injury_prevention/teens.htm (accessed on 16 November 2020).
42. Fabiano, G.A.; Hulme, K.; Linke, S.M.; Nelson-Tuttle, C.; Pariseau, M.E.; Gangloff, B.; Lewis, K.; Pelham, W.E.; Waschbusch, D.A.;
Waxmonsky, J.; et al. The Supporting a Teen’s Effective Entry to the Roadway (STEER) Program: Feasibility and Preliminary
Support for a Psychosocial Intervention for Teenage Drivers with ADHD. Cogn. Behav. Pract. 2011, 18, 267–280. [CrossRef]
43. Fabiano, G.; Schatz, N.K.; Hulme, K.F.; Morris, K.L.; Vujnovic, R.K.; Willoughby, M.T.; Hennessy, D.; Lewis, K.E.; Owens, J.;
Pelham, W.E. Positive Bias in Teenage Drivers with ADHD within a Simulated Driving Task. J. Atten. Disord. 2015, 22, 1150–1157.
[CrossRef]
Safety 2021, 7, 24 24 of 24
44. Overton, R.A. Driving Simulator Transportation Safety: Proper Warm-up Testing Procedures, Distracted Rural Teens, and Gap
Acceptance Intersection Sight Distance Design. Ph.D. Thesis, University of Tennessee, Knoxville, TN, USA, 2012. Available online:
https://trace.tennessee.edu/utk_graddiss/1462 (accessed on 2 December 2020).
45. Gianaros, P.; Muth, E.; Mordkoff, J.; Levine, M.; Stern, R. A Questionnaire for the Assessment of the Multiple Dimensions of
Motion Sickness. Aviat. Space Environ. Med. 2001, 72, 115–119.
46. De Winter, J.C.; de Groot, S.; Mulder, M.; Wieringa, P.A.; Dankelman, J.; Mulder, J.A. Relationships between driving simulator
performance and driving test results. Ergonomics 2009, 52, 137–153. [CrossRef] [PubMed]
47. Hirsch, P.; Bellavance, F.; Choukou, M.-A. Transfer of training in basic control skills from a truck simulator to a real truck. J.
Transp. Res. Board 2017, 2637, 67–73. [CrossRef]
48. Clinewood. Are Driving Simulators Beneficial for Truck Driving Training? 2018. Available online: https://clinewood.com/2018
/02/19/driving-simulators-beneficial-truck-driving-training/ (accessed on 22 November 2020).
49. Salas, E.; Bowers, C.A.; Rhodenizer, L. It is not how much you have but how you use it: Toward a rational use of simulation to
support aviation training. Int. J. Aviat. Psychol. 1998, 8, 197–208. [CrossRef] [PubMed]
50. Straus, S.G.; Matthew, W.L.; Kathryn, C.; Rick, E.; Matthew, E.B.; Timothy, M.; Christopher, M.C.; Geoffrey, E.G.; Heather, S.
Collective Simulation-Based Training in the U.S. Army: User Interface Fidelity, Costs, and Training Effectiveness. Santa Monica,
CA: RAND Corporation. 2019. Available online: https://www.rand.org/pubs/research_reports/RR2250.html (accessed on
18 November 2020).
51. Hays, R.T.; Michael, J.S. Simulation Fidelity in Training System Design: Bridging the Gap between Reality and Training; Springer: New
York, NY, USA, 1989.
52. Wu, Y.; Boyle, L.; McGehee, D.; Roe, C.; Ebe, K.; Foley, J. Modeling Types of Pedal Applications Using a Driving Simulator. Hum.
Factors 2015, 57, 1276–1288. [CrossRef] [PubMed]
53. Kemeny, A.; Panerai, F. Evaluating perception in driving simulation experiments. Trends Cogn. Sci. 2003, 7, 31–37. [CrossRef]
54. Delgado, M.K.; Wanner, K.; McDonald, C. Adolescent Cellphone Use While Driving: An Overview of the Literature and Promising
Future Directions for Prevention. Media Commun. 2016, 4, 79. [CrossRef] [PubMed]
55. Hulme, K.F.; Morris, K.L.; Anastasopoulos, P.; Fabiano, G.A.; Frank, M.; Houston, R. Multi-measure Assessment of Internal
Distractions on Driver/Pilot Performance. In Proceedings of the Interservice/Industry Training, Simulation and Education
Conference (I/ITSEC), Orlando, FL, USA, 30 November–4 December 2015.
56. Centers for Disease Control and Prevention (CDC). Vital Signs, Teen Drinking and Driving. 2020. Available online: https:
//www.cdc.gov/vitalsigns/teendrinkinganddriving/index.html (accessed on 13 October 2020).
57. New York State Department of Motor Vehicles (NYSDMV). Pre-Licensing Course Student’s Manual (Form MV-277.1 PDF
Download). 2020. Available online: https://dmv.ny.gov/forms/mv277.1.pdf (accessed on 13 October 2020).
58. Senserrick, T.; Brown, T.; Quistberg, D.A.; Marshall, D.; Winston, F. Validation of Simulated Assessment of Teen Driver Speed
Management on Rural Roads. Annual proceedings/Association for the Advancement of Automotive Medicine. Assoc. Adv.
Automot. Med. 2007, 51, 525–536.
59. Insurance Institute for Highway Safety (IIHS). Fatality Facts 2018—Urban/Rural Comparison. 2018. Available online: https:
//www.iihs.org/topics/fatality-statistics/detail/urban-rural-comparison (accessed on 2 November 2020).
60. Henning-Smith, C.; Kozhimannil, K.B. Rural-Urban Differences in Risk Factors for Motor Vehicle Fatalities. Health Equity 2018, 2,
260–263. [CrossRef] [PubMed]
61. New York State Department of Motor Vehicles (NYSDMV). REQUEST FOR CERTIFIED DMV RECORDS (Form MV-15 PDF
Download). 2019. Available online: https://dmv.ny.gov/forms/mv15.pdf (accessed on 2 November 2020).

Blueprint For A Simulation Framework To Increase D

Uploaded by

Copyright:

Available Formats

You might also like

Blueprint For A Simulation Framework To Increase D

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Blueprint For A Simulation Framework To Increase D

Uploaded by

Copyright:

Available Formats

safety

Copyright: © 2021 by the authors.

Safety 2021, 7, 24. https://doi.org/10.3390/safety7020024 https://www.mdpi.com/journal/safety

Figure 1. Teen driving framework—a blueprint theory of change (with simulation).

2.1. Training Environment Design

Figure 2. Fixed-base simulator.

2.1.1. Stage 1: Input—Driver

2.1.2. Stage 2: Simulation Analysis

2.1.3. Stage 3: Outputs—Visual, Haptic and Audio

2.2. Training Modules

2.2.1. Closed Course Driving

Safety 2021, 7, x FOR PEER REVIEW 8 of 2

2.2.2. Residential Driving

2.2.3. Challenging Scenarios

Figure 7. The figure-8 scenario (CC).

2.2.3. Challenging Scenarios

According to previously published statistics [41], young and inexperienced adult

2.3. Instructional Sessions and Performance Measures

Table 1. Overview of Training Modules for each Instructional Session.

Session Training Module(s) Residential Drive Description/Features

Session Training Module(s) Residential Drive Description/Features

Table 2. Experimental Protocol for Each Instructional Session.

To provide a more holistic understanding of simulation-based training on driving

2.3.1. Quantitative Measures

2.3.2. Qualitative Measures

2.3.3. Self-Report Measures

2.4. Experimental Cohort

3. Results and Discussion

3.1. Quantitative Measures

Figure 11. Score report totals.

toward intersection regulation (stop signs and streetlights) on the motion-based s

As a mechanism by which to culminate these qualitative observations across the

Figure 15. SBQ (pre/post).

The Motion Sickness Assessment Questionnaire (MSAQ) was issued post-s

Figure 18. Teen Assessment Open-ended Summary.

Figure 20. Parent Assessment Open-ended Summary.

5. Conclusions and Future Work

5.1. Improved Metrics for Quantifying Simulator Performance

5.2. Improvements to Experimental Training Environment

5.3. Impairment Training Modules

compare graphical representations of the teen’s driving to provide concrete performance

5.4. Driving Exercises that Promote Reverse Egress

5.5. Driving Exercises for Driver Training on Rural Roads

5.6. Longitudinal Data to Measure Efficacy of SBT

You might also like