Human Factors Issues in The Collocation of Uret Tma and CPDLC

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 9

HUMAN FACTORS ISSUES IN THE COLLOCATION OF URET, TMA, AND

CPDLC
Ran& L. Sollenberger, Ph.D. and Pamela S.Della Rocco, Ph.D.
Federal Aviation Adminishation, Atlantic City International Airport, New Jersey

Abstract Introduction
The Federal Aviation Administration (FAA) The FAA established the Free Flight Program
Free Flight Program individually deployed the User in collaboration with the aviation community to
Request Evaluation Tool (URET), Traffic increase capacity (airport and airspace) and improve
Management Advisor (TMA), and Controller-Pilot efficiency (flight times and fuel consumption),
Data Link Communications (CPDLC) to a limited while maintaining the current high level of safety.
number of Air Route Traffic Control Centers An important goal of the Free Flight Program was
(ARTCCs). Before deployment expands the delivery of new air traffic control (ATC)
nationwide, it was important to identify any technologies focused on early benefits to users of
potential human factors issues that may arise due to the National Airspace System (NAS). These
the collocation of these tools at the controller’s capabilities included URET, TMA, and CPDLC as
workstation. In this paper, we present the results of en route controller tools. Under the Free Flight
a high fidelity human-in-the-loop simulation we Program, these tools were individually deployed to
conducted to evaluate the impact of URET, TMA, a limited number of ARTCCs nationwide.
and CPDLC collocation on air traffic controllers. The MITRE Corporation developed URET as
We examined collocation issues with a “stovepipe”
a conflict probe tool that automatically predicts and
independent configuration where none of the tools provides early identification of potential aircrafi-to-
were integrated or directly communicated with each aircraft and aircraft-to-special use airspace
other. Twelve certified professional controllers
conflicts. URET has a trial planning feature that
participated in the simulation working in two- allows controllers to determine whether proposed
person teams consisting of a Radar (R-side) and
flight path changes will conflict with traffic or
Data (D-side) controller. restricted airspace. Controllers use trial planning to
The most important collocation issue identified quickly evaluate possible route changes and to
was that controllers had difficulty accessing assign conflict free routing. URET is deployed on a
important information on the D-side display when flat-panel monitor and is operated by the D-side
URET and CPDLC were both operational (i.e., member of controller teams. URET consists of
display clutter). Although neither tool alone caused many different windows for displaying information
display clutter, both tools in combination made it such as aircraft flight data, graphical display of
difficult for D-side controllers to find the conflicts, and trial plans.
information they needed quickly. This was
The National Aeronautical and Space
especially true for accessing CPDLC windows, Administration (NASA) developed TMA as an
which became covered when controllers used
arrival sequencing tool that provides controllers
URET. Good human factors design principles
with information for increasing the efficiency of
prescribe that users must have immediate access to
traffic flow into airports. TMA uses time-based
important information and that critical information
metering to sequence the flow of amval aircraft and
should never be covered. A “stovepipe”
replaces miles-in-trail as a method for aircraft
independent deployment of these tools will result in
spacing. Computer automation calculates the delay
impaired access to timely information. The results
times for amving aircraft based upon aircraft type,
of this study indicated that better efforts should be
flight plan, weather, and winds aloft and displays
made to integrate the information from URET,
the times on a meter list. The TMA meter list is
TMA, and CPDLC on the D-side monitor prior to
displayed on the R-side controller’s Display System
deployment of all three tools at the controller’s
Replacement (DSR) monitor.
workstation.

U.S. Government work not protected by U.S. Copyright


5.B.5-1
CPDLC is a tool that allows controllers and Computer Human Interface (CHI) on the D-side
pilots to communicate with text-based electronic was critical to the successful deployment of URET
messages. CPDLC reduces the number of voice and CPDLC. Kems also noted there would be
communications, which decreases radio congestion changes in the roles and responsibilities of both the
and reduces delays. On the flight deck, CPDLC R-side and D-side controllers.
messages are presented on a small monitor and Della Rocco, Panjwani, Friedman-Berg,
pilots respond using buttons on the sides of the Kopardekar, and Hah [3] conducted a cognitive
display. On the ground D-side, CPDLC Build 1A walkthrough with subject-matter experts (SMEs) to
consists of three different windows for displaying explore the collocation of URET, TMA, and
uplinked messages and their status, a menu of text CPDLC. The study identified a number of CHI
messages, and a history list. This version also has inconsistencies across the tools and raised questions
datablock fly-out windows and symbology for about D-side display clutter with a “stovepipe”
CPDLC messages. deployment of the tools. The authors concurred
Different vendors developed each of the three with Kems [2] in her assessment that the roles and
tools independently in a stand alone “stovepipe” responsibilities of the R-side and D-side positions
design without a plan for integrating their system need clarification when all three tools are deployed
with other support systems. Therefore, we need to together. Furfher, applying human factors
investigate how controllers will interact with the principles to these issues will help controllers use
three tools before they are deployed together at the the tools as intended and attain the expected
controller’s workstation. When evaluating tools benefits.
that will change or add to the number of systems To date, there have been no real-time human-
used by controllers, it is important to identify any in-the-loop simulation studies that have objectively
potential human factors issues that may arise from examined controller performance while employing
the introduction of these new tools. Identifying URET, TMA, and CPDLC. This paper describes
problems and comecting them before they can the first experiment in a series of high-fidelity
negatively impact performance in the field is human-in-the-loop simulations conducted by the
critical in ATC where safety is potentially at stake. present authors to examine the human factors issues
Therefore, the FAA Free Flight Program Office and of collocating these three tools (for more details see
the Human Factors Research and Engineering [4 and 51). In this first experiment, we examined
Division sponsored this study to examine the R-sidem-side controller teams working a high
impact of collocating URET, TMA, and CPDLC at altitude sector and using different combinations of
the controller’s workstation. the three tools at a single sector. The second
experiment examined controller teams interacting
Previous Research with each other while working a high and a low
A few studies have examined issues related to altitude sector and using all of the tools. The third
the collocation of URET, TMA, and CPDLC. experiment examined controllers working a high
Desenti, Gross, and Toma [ 11 examined the use of altitude sector with single controller staffing and
URET and TMA. The authors questioned whether using all of the tools.
there was an emerging concept of use for URET
and TMA in which the trial planning capability of Study Objectives
URET may be compatible to meet the delay times
The specific objectives of the present study are
of TMA. A potential human factors issue identified
was that URET and TMA used different algorithms To assess whether controllers have
to compute an aircraft’s future location. access to infomation when needed,
Kems [2] of the MITRE Center for Advanced To assess controller workload,
situational awareness (SA), and
Aviation System Development examined the human
teamwork, and
factors issues related to collocating URET and
CPDLC. She concluded that D-side controller To identify any other important human
workload would increase, and the design of the factors issues.

5.B.S-2
Method Airspace and Traflc Scenarios
We selected a generic high altitude en route
Particzpanis sector as the airspace for this simulation. SMEs
Ten male and two female Air Traffic Control designed the traffic scenarios with moderate traffic
levels that required traffic flow restrictions, but
Specialists (ATCSs) from Level 11 and Level 12
were not so busy as to overwhelm controllers who
ARTCCs nationwide participated in this study. We
were not experienced with all of the tools. We
recruited six participants who were URET current
prepared one basic test scenario and designed seven
and proficient from ARTCCs where URET is
additional test scenarios based on the basic
operational. We recruited six participants who were
scenario. All scenarios had the same number of
TMA current and proficient from ARTCCs where
arrivals, deparhres, and overflights; however, we
TMA is operational. We trained all twelve ATCSs
changed the aircraft entry times and assigned
in CPDLC after arriving at our research facility.
different callsigns in each of the scenarios. This
Also, all participants received some cross-training
ensured that the test scenarios were similar in
in URET and TMA. Each controller team consisted
traffic, but not recognizable as the same scenario.
of one TMA-qualified ATCS operating the R-side
Each test scenario was 45 minutes in duration.
and one URET-qualified ATCS operating the
D-side position. When CPDLC was in use, 40% of the aircraft
in scenarios were equipped. CPDLC services
Controllers completed a Background
included Transfer of Communication (TOC), Initial
Questionnaire to describe the general demographic
Contact (IC), Altimeter Setting (AS), and Menu
characteristics of participants in the study. The
mean age of participants was 40.52 (range 28-47) Text Messages (MT). In manual TOC mode,
CPDLC will generate a held TOC message that
years old with a mean of 15.10 (range 5-21) years
must be released by either the R-side or D-side
of FAA experience. All participants actively
controller for the system to uplink a new frequency
controlled traffic for the past 12 months.
to aircraft. We allowed the ATCSs to decide for
themselves which team member would release
Simulation Equipment and Setup aircraft with held TOCs.
We conducted the simulation at the FAA
William J. Hughes Technical Center in the Experimental Design and Measures
Research, Development, and Human Factors
The experiment represented a two-factor
Laboratory. Each R-side workstation consisted of a
design with R-side and D-side controller positions
high-resolution Sony 2K monitor, DSR keyboard,
as the first factor and eight different tool
and trackball. We deployed TMA and CPDLC on
the Sony 2K monitor. Each D-side workstation combinations as the second factor. The eight tool
combinations are represented by U, T, C, UT, UC,
consisted of a 21” flat-panel monitor with keyboard
and mouse. We deployed URET and CPDLC on TC, UTC, and No Tools as a baseline where
U=URET, T=TMA, and C=CPDLC. Each team of
the flat-panel monitor. The voice communications
system consisted of individual relay switchboxes, controllers completed eight test scenarios with each
scenario representing a different tool combination.
controller headsets with microphones, and push-to-
talk handsets or foot pedals. Flight strip marking We collected a large set of subjective measures
was optional in our simulation and only one of the that included controller workload and SA, SME
controller teams requested and marked flight ratings and observations, and participant
progress strips. questionnaires. We also collected a large set of
An SME observed each controller team and objective measures for ATC simulation research
provided ratings and comments on controller that included safety, capacity, efficiency, and
communications indicators [6]. For this paper, we
interaction with the tools. A team of simdation
pilots communicated with contrdlers using proper selected a few measures that were directly
ATC phraseology and maneuvered aircraft using influenced by the tools. Table 1 presents these
simple keyboard commands. simulation measures.

5.B.5-3
Table 1. Simulation Measures Results
Controller Ratings of Display Clutter We used univariate inferential statistics to
Controller Ratings of Workload analyze the data in this study. For each simulation
Controller Ratings of Situational Awareness measure, we performed a two-way analysis of
Number of CPDLC TOC Messages Sent variance (ANOVA) with Position (R-side, D-side)
as a between-subjects factor and Tools (None, U, T,
Procedure C, UT, UC, TC, and UTC) as a repeated measures
factor. In each ANOVA, the standard significance
The study consisted of three testing sessions,
level w a s p c .05. When there were significant
each lasting two weeks with different groups of four effects, we performed an analysis ofthe simple
ATCSs. In the first week, we briefed participants effects and Tukey HSD post hoc comparisons to
on the study objectives, airspace operations, and determine which means were different using a
support tools. Controllers completed 18 hours of group comparisons significance level o f p < .05.
training scenarios to become proficient with the
generic airspace and familiar with all three support
tools. In the second week, we began data collection Access to Tool Information
and controllers completed a series of test scenarios. Table 2 shows controller ratings for how often
At the end of each day, we had a group meeting CPDLC caused clutter on the R-side and D-side
with controllers to discuss the issues raised during displays for each of the four tool combinations with
the simulation. CPDLC operational. R-side and D-side ratings
Participants completed several questionnaires were different depending upon the tools in use,
during the course of the study. On the first day, [Position x Tools: F(3,30) = 4 . 3 6 , ~= ,0121. R-side
controllers signed an Informed Consent Form and controller ratings were rather low indicating that
completed a Background Questionnaire to describe they did not think CPDLC caused clutter very often.
the participants in the study. AAer each test D-side controller ratings ranged from low to
scenario, controllers completed a Post-Scenario moderate depending upon the tools in use, [D-side
Questionnaire which consisted of subjective ratings Tools: F(3,15) = 4 . 0 3 , ~= ,0271. When CPDLC
and open-ended comments about the scenario. was displayed alone and when CPDLC and TMA
Also, SMEs completed an Observer Rating Form were displayed together, ratings of display clutter
providing subjective ratings and observations of were very low. Ratings were much higher when
how effectively controllers used the tools in the CPDLC and URET were both displayed and when
scenario. On the last day, controllers completed an all three tools were displayed together (confirmed
Exit Questionnaire consisting of more ratings and by Tukey HSD comparisons).
comments about their overall simulation
experience. Table 2. Mean Ratings for CPDLC Causing
Display Clutter, l=Never to lO=Always
Each team of controllers completed eight test
scenarios. We counterbalanced the presentation
order of the scenarios and tool combinations to
experimentally control for practice effects. At the
start of each test day, controllers performed a
preliminary warm-up scenario followed by four test
scenarios. Before each scenario, the researchers
Table 3 shows controller ratings for how often
informed the participants which tool or tools would
URET caused clutter on the D-side display for each
be operational. Although there were two teams of of the four tool combinations with URET
controllers participating simultaneously, each team
operational. In this case, D-side controller ratings
independently controlled different traffic scenarios
were rather low indicating that they did not think
in the high altitude sector. The sectors adjacent to
URET caused clutter on the D-side display whether
the high altitude sector were automated by “ghost”
used alone or in combination with TMA and
controller functionality.
CPDLC.

5.B.5-4
information can be seen as possible. The CPDLC
Message Out and Menu Text windows are shown
along with the D-side CRD. URET and CPDLC
Tool Combinations windows overlap each other. The URET Aircraft
Position U I UT I uc I UTC List can be resized and made smaller, but
D-side 2.50 I 3.83 I 3.17 I 3.67 controllers do not often do this because important
information would be truncated from the window
In our group discussions, controllers frequently and would not be visible on the display. When a
commented that when URET and CPDLC were controller selects the URET Aircraft List as the
collocated on the D-side, there was display clutter. active window, it becomes front and the CPDLC
Figure 1 illustrates this issue in an example windows move to the back and become covered. In
configuration of URET and CPDLC on the D-side this configuration, updated information in the
display. Controllers expand the URET Aircraft List CPDLC Message Out window is not visible to
to cover most of the display so that as much aircraft controllers.

Figure 1. D-Side Display Showing URET and CPDLC Windows


Some controllers commented that they wanted concerned that simply showing the TMA list on the
TMA delay times available on the D-side display. D-side monitor would add to the display clutter
In the simulation, R-side controllers positioned the problem, but thought there should be a smart way to
TMA list in a location near the edge of the DSR show TMA information on the D-side position.
monitor where D-side controllers could see the Controllers suggested that it would be better if
information. The participants commented that the TMA delay times were available on the URET
R-side sliould be able to position the TMA list Aircraft List where they would not increase display
wherever is most convenient for h i d e r and that clutter.
D-side controllers should have access to this
information on the D-side display. Controllers were

5.B.5-5
Controller Workload and SA from low to moderate and tended to vary for the
Figure 2 shows mean NASA-TLX mental different tool combinations. Mental demand ratings
demand workload ratings provided by the R-side tended to increase slightly in the single tool
and D-side positions for each of the eight tool conditions and further increased in the two and
combinations. For R-side controllers, mental three tool combinations. However, D-side mental
demand ratings were only moderate and there were demand ratings never reached an excessively high
no differences between the tool combinations. For level. Other workload measures showed a similar
D-side controllers, mental demand ratings ranged pattem.

High
9
lo i

R-Side D-Side

Figure 2. Mean Ratings for NASA-TLX Mental Demand


After each scenario, controllers rated their
overall level of SA on a 10-point scale ranging from
l=Low to lO=High. Controller SA ratings were
very high for all conditions of the experiment.
R-side controller ratings ranged from 6.67 to 8.83
and D-side controller ratings ranged from 7.50 to
9.00 in the eight tool combinations.
Tool Combinations
Position C uc TC UTC
Controller Teamwork R-side 19.00 18.50 18.83 19.83
Table 4 shows the mean number of CPDLC D-side 1.oo 1.17 1.50 0.83
TOC messages sent by R-side and D-side
controllers for each of the four tool combinations
with CPDLC operational. The R-side sent from
90% to 95% of the CPDLC TOC messages in each
of the tool combinations [Position: F(1,IO)=
194.96,~ < .001]. Therefore, the controller teams

5.B.5-6
controllers had to visually monitor the CPDLC TMA list in a location near the edge of the DSR
Message Out window to know when their team monitor where D-side controllers could see the
member sent a message or verbally coordinate with information. Both controllers spent a great deal of
each other when messages were sent. This may time viewing the TMA list. Controllers thought it
become a multi-tool issue when one controller was important to have TMA information available
sends a CPDLC TOC message early and, based on on the D-side display where it could be more easily
URET or TMA information, the other team member accessed by D-side controllers. However,
decides to take (or recommends) action on the controllers were concemed that simply showing the
aircraft. TMA list on the D-side monitor might add to the
display clutter.
Discussion Good human factors design principles
prescribe that users must have immediate access to
The most important human factors issue
important information and that critical information
identified in this study was that controllers had
should never be covered. A “stovepipe”
difficulty accessing important information on the
independent deployment of these tools will result in
D-side display when URET and CPDLC were both
impaired access to timely information. The results
operational (i.e., display clutter). Controller ratings
of this study indicated that better human factors
indicated that the CPDLC windows on the D-side
efforts should be made towards integrating the
monitor were the source of this display clutter
information from URET, TMA, and CPDLC. Even
problem. However, all the D-side controllers were
if these systems cannot be entirely integrated, we
accustomed to using URET as a stand alone tool at
should explore integrating the information displays
their own facilities and none had worked with
on the D-side monitor prior to deployment.
CPDLC prior to this study. Therefore, it is not
surprising that these controllers would attribute the Controller NASA-TLX workload ratings
D-side display clutter to CPDLC and not URET. indicated that R-side mental demand was only
Controller ratings indicated that neither URET moderate and did not change with tool use. D-side
alone nor CPDLC alone caused D-side display mental demand ratings tended to increase when two
clutter. Finally, R-side controller ratings indicated and three tools were operational. However, D-side
that CPDLC caused very little display clutter on the workload ratings were only low to moderate and
DSR display. never reached an excessively high level. We
designed the scenarios with moderate traffic levels
This D-side display clutter problem resulted
so that controllers would not be overwhelmed with
from the “stovepipe” independent deployment of
traffic and stop using the tools. We thought
both URET and CPDLC and having to manage the
moderate traffic levels would allow controllers to
multiple windows associated with these tools. It is
use the tools more often and better identify any
important to note that we developed the D-side
collocation issues. Therefore, the workload results
CPDLC CHI for use in this study to be consistent
of this study may have been different with higher
with the “stovepipe” independent deployment of
trafic levels and greater workload demands on
these tools. We added simple CHI features to help
controllers manage the multiple windows associated controllers.
with each tool. SMEs from the CPDLC users group D-side controller workload was rather low in
helped us develop the D-side CHI. At our request, the baseline condition without any tools. Workload
the FAA’s Air Traffic Design Evolution Team for this baseline condition may have been lower
(ATDET) approved the interface for use in this than actual field conditions where flight progress
study. However, this specific D-side CHI was not strips are actively used because five of the six
intended to be the interface that will be deployed to controller teams did not want flight strips posted at
ARTCCs in the future. their workstation. All of our D-side controllers
Another collocation issue identified in this came from URET facilities that do not require flight
study was that D-side controllers had to access strips. The baseline condition was intended to be an
TMA delay time information on the R-side display. experimental comparison for the conditions with
In the simulation, R-side controllers positioned the

5.B.5-7
tools. Without flight strips, controllers used flight together. Another issue was the need for TMA
plan readouts to obtain aircraft routing information. information on the D-side display without adding to
the display clutter. From a human factors
Controllers rated their SA as very high and did
perspective, URET, TMA, and CPDLC should be
not vary with different tool combinations.
integrated on the D-side display. This solution
Although sometimes self-ratings can potentially
avoids the problem of multiple windows or displays
misrepresent true SA, there was no indication that
that were a consequence of the “stovepipe”
this actually occurred during the simulation. With
implementation of these tools in our simulation.
very few exceptions, controllers maintained safety
With URET, TMA, and CPDLC information
throughout the scenarios. For the present study, we
integrated, controllers should have easier access to
used self-ratings of SA as a technique to elicit
information when needed without having to monitor
controller concems and identify collocation issues
and manage multiple displays.
for group discussion.
Controllers also provided some non-integration
Finally, the number of CPDLC TOC messages
ideas including a larger D-side display and an
indicated that R-side controllers sent most of the
improved D-side CHI that would make it easier to
TOCs. There were no mandatory procedures for
manage multiple information displays. We
using CPDLC by the R-side and D-side in our
recommend that additional research be conducted to
simulation. We allowed each controller team to
investigate URET, TMA, and CPDLC collocation
practice together and decide for themselves who
issues and potential solutions. Future simulations
would send the CPDLC TOCs to aircraft. Although
should examine the best presentation of the
the results indicated that D-side controllers did not
information, specific procedures for R-side and
use the TOC service very often, controllers still
D-side tool use, and higher traffic levels for
expressed concems about not knowing what their
scenarios.
team member was doing with CPDLC. Unlike
voice communications, there were no audible cues
with CPDLC to help controllers maintain SA of References
their team member’s actions. Controllers had to
[I] Desenti, C. T., A. E. Gross, and N.E. Toma,
visually monitor the CPDLC Message Out window
2000, Analysis of Atlanta ARTCC operations for
to know when their team member sent a TOC
ATL arrival flows (MTROOW30), McLean, VA,
message. If the CPDLC windows were covered by
The MITRE Corporation.
URET, the D-side controller could miss a sent
message. In addition, CPDLC messages were [2] Kems, K., 2001, User Request Evaluation Tool
visible in the Message Out window for only a few (URET)/Controller-Pilot Data Link
seconds after a pilot “wilco” was received. Communications (CPDLC) collocation:
Therefore, controllers had to be very vigilant. An Development of an operational concept,
even greater concem for safety exists if CPDLC is identification of potential issues, and recommended
used to issue control instructions (e.g., altitude, solutions (MTRO1WOOOOOZ2), McLean, VA, The
heading or airspeed future services). It is possible MITRE Corporation.
that D-side controllers could recommend actions
[3] Della Rocco, P. S., G. Panjwani, F. Friedman-
that create aircraft conflicts when they are not
Berg, P. Kopardekar, and S. Hah, 2004, Collocation
aware of the CPDLC messages sent. More research of User Request Evaluation Tool, Traffic
needs to be conducted examining the roles and
Management Advisor, and Controller-Pilot Data
responsibilities for CPDLC usage, especially when Link Communications: An initial human factors
other tools are being used. evaluation (DOT/FAA/CT-TN04/06), Atlantic City
In conclusion, the purpose of this simulation International Airport, NJ, DOT/FAA William J.
study was to identify human factors issues when Hughes Technical Center.
URET, TMA, and CPDLC were collocated together
[4] Sollenberger, R. L., B.Willems, P. S . Della
at the same sector. In a high-fidelity human-in-the- Rocco, A. Koros, and T. Truitt, 2004, Human-in-
loop simulation, the main issue was D-side display the-Loop simulation evaluating the collocation of
clutter when URET and CPDLC were being used the User Request Evaluation Tool, Traffic

5.B.5-8
Management Advisor, and Controller-Pilot Data real-time air traffic control system simulation
Link Communications: Experiment I - tool (DOTffAA/CT-83/26), Atlantic City Intemational
combinations (DOT/FMCT-TNO4/28), Atlantic Airport, NJ, DOTffAA Technical Center.
City Intemational Airport, NJ, DOT/FAA William
J. Hughes Technical Center.
Disclaimer
[5] Sollenberger, R. L., and P. S. Della Rocco, This document is disseminated under the
2004, Human-in-the-Loop simulation evaluating the sponsorship of the U.S. Department of
collocation of the User Request Evaluation Tool,
Transportation in the interest of information
Traffic Management Advisor, and Controller-Pilot exchange. The United States Government assumes
Data Link Communications: Experiment I11 - single no liability for the contents or use thereof. The
controller staffing (DOTff AA/CT-TN04/29), United States Govemment does not endorse
Atlantic City Intemational Airport, NJ, DOTiFAA products or manufacturers. Trade or manufacturer's
William J. Hughes Technical Center.
names appear herein solely because they are
[6] Buckley, E. P., B. D. DeBaryshe, N. Hitchner, considered essential to the objective of this report.
and P. Kohn, 1983, Methods and measurements in

5.B.S-9

You might also like