Applied Ergonomics 45 (2014) 346e353

Contents lists available at SciVerse ScienceDirect

Applied Ergonomics
journal homepage: www.elsevier.com/locate/apergo

Customization of user interfaces to reduce errors and enhance user

acceptance
Dina Burkolter a, *, Benjamin Weyers b, Annette Kluge c, Wolfram Luther b
a
University of Groningen, Faculty of Behavioural and Social Sciences, Experimental Psychology, Grote Kruisstraat 2/1, NL-9712 TS Groningen,
The Netherlands
b
University of Duisburg-Essen, Department of Computer and Cognitive Science, Lotharstraße 63, D-47058 Duisburg, Germany
c
University of Duisburg-Essen, Department of Computer and Cognitive Science, Business and Organizational Psychology, Lotharstraße 65,
D-47058 Duisburg, Germany

a r t i c l e i n f o a b s t r a c t

Article history: Customization is assumed to reduce error and increase user acceptance in the humanemachine rela-
Received 3 August 2012 tion. Reconfiguration gives the operator the option to customize a user interface according to his or her
Accepted 21 April 2013 own preferences. An experimental study with 72 computer science students using a simulated process
control task was conducted. The reconfiguration group (RG) interactively reconfigured their user in-
Keywords: terfaces and used the reconfigured user interface in the subsequent test whereas the control group (CG)
Reconfiguration
used a default user interface. Results showed significantly lower error rates and higher acceptance of
Customization
the RG compared to the CG while there were no significant differences between the groups regarding
Process control
User acceptance
situation awareness and mental workload. Reconfiguration seems to be promising and therefore
warrants further exploration.
Ó 2013 Elsevier Ltd and The Ergonomics Society. All rights reserved.

1. Customization of user interfaces and expected outcomes
The basic idea of individuation is “to explore ways through
which each and every single individual can customize his or her
own tools to optimize the pleasure and efficiency of his or her own
personal interaction” (Hancock et al., 2005, p. 12). Thus, according
to the idea of individuation, technological tools such as user in-
terfaces should be customizable and adaptable to individuals with
the aim of increasing efficiency and safety (Hancock et al., 2005). In
the context of this study, individuation is perceived as the perfected
or accomplished form of customization. Individuation and cus-
tomization are assumed to increase an individual’s perception of
control over the environment. Perceived control satisfies the need
for autonomy and therefore, has been shown to increase job
satisfaction, work motivation, and positive affect (Fritzsche and
Parrish, 2005, cited in Hancock et al., 2005).
According to Pozzi and Bagnara (2011), research on custom-
ization within the field of humanecomputer interaction is
increasing and the focus in design is changing from designing for
* Corresponding author. Tel.: þ31 50 363 9374; fax: þ31 50 363 63 04.
E-mail addresses: d.burkolter@rug.nl (D. Burkolter), weyers@inf.uni-due.de
(B. Weyers), annette.kluge@uni-due.de (A. Kluge), luther@inf.uni-due.de
(W. Luther).
average users to designing for individuals. Empirical research has
shown that user performance is enhanced when characteristics of
user interfaces are matched to the skill levels of users (Trumbly
et al., 1994). In the present study, we focus on customization of
user interfaces and aim at studying whether performance in a
process control task can be enhanced through customization of
user interfaces.
In our study, participants were provided with a reconfiguration
tool which allowed them to customize their user interface to their
own preferences. The reconfiguration tool allowed different
reconfiguration operations such as the duplication or removal of
interaction elements (e.g., buttons or sliders) or the discretization
of continuous interaction elements, generating various values in a
given interval (e.g., sliders), into one discrete interaction element
(e.g., buttons), generating only one specific value out of the range
the former defined interval (e.g., button; Weyers et al., 2010). Thus,
both, the reconfiguration of the interaction logic (defining the data-
based communication between user and the system to be
controlled) as well as the reconfiguration of the physical repre-
sentation of the user interface were possible. A computer-based
simulation of a feedwater circuit of a nuclear power plant was
employed as a process control task. This process control task
involved controlling and operating the reactor and running pro-
cedures such as start-up and shut-down, and dealing with fault
states (Weyers et al., 2012).

0003-6870/$ e see front matter Ó 2013 Elsevier Ltd and The Ergonomics Society. All rights reserved.
http://dx.doi.org/10.1016/j.apergo.2013.04.017
 D. Burkolter et al. / Applied Ergonomics 45 (2014) 346e353
1.1. Reducing errors through correspondence of mental model and
physical system
We expect the customization of user interfaces, i.e., the possi-
bility to reconfigure one’s own user interface, to enhance
performance in process control. This assumption is based on con-
siderations on the importance of an operator’s mental model of the
system (e.g., the reactor). A mental model is defined as “a mental
structure that reflects the user’s understanding of a system” (Carroll
and Olson, 1987; cited in Wickens and Hollands, 2000, p. 280). The
mental model forms the basis “for understanding the system,
predicting its future behavior, and controlling its actions” (Wickens
and Hollands, 2000, p. 132). An accurate mental model of the sys-
tem is therefore essential for successful performance. However,
mental models can be inaccurate which can decrease performance
and increase the chance of operator errors. Effective user interfaces
require the correspondence of the following three levels of repre-
sentation: (1) the physical system, (2) the operator’s mental model,
and (3) the user interface between the system and the operator
(Wickens and Hollands, 2000, p. 133). We suggest that this corre-
spondence between levels of representation can be improved with
the customization of user interfaces. With the reconfiguration tool,
the operator can reconfigure his or her user interface (level 3)
corresponding to his or her own mental model of the system (level
2) while the physical system (level 1) will be left untouched. This
increased correspondence is expected to reduce errors, because the
operator can ensure an intuitive and effective user interface orga-
nized according to the physical system (cf. Wickens and Hollands,
2000; Weyers et al., 2012). We assume that the process of recon-
figuration can lead to a better mental model of how to control and
handle the system. This mental model of handling and control of a
system may be individual and cannot be predicted for every oper-
ator. Therefore, reconfiguration of user interfaces according to in-
dividual mental models is assumed to enhance performance and
support operators. However, an inaccurate mental model may lead
to a non-optimal user interface which may increase errors and
present a risk to safety. Therefore, it is suggested to incorporate a
safety check of reconfigured user interfaces by experts especially in
safety-relevant fields and industries.
1.2. Enhancing user acceptance
We expect customization to support not only process control
performance but also increase acceptance of the user interface. The
technology acceptance model (e.g., Davis et al., 1989) aims at pre-
dicting the acceptance of a certain technology or computer system
from intentions measured by attitudes, perceived usefulness, and
perceived ease of use (Davis et al., 1989). The model is based on the
theory of reasoned action by Ajzen and Fishbein (1980) which has
been empirically tested in a range of different domains and shown
to be able to predict behavior (Davis et al., 1989). The technology
acceptance model is more specific than the theory of reasoned
action in that it is tailored to computer usage behavior (Davis et al.,
1989). The technology acceptance model suggests that the two
beliefs perceived usefulness and perceived ease of use determine the
behavioral intention to use a certain technology or computer sys-
tem, which is related to subsequent behavior (Davis et al., 1989;
Venkatesh, 2000). Perceived usefulness refers to “the prospective
user’s subjective probability that using a specific application system
will increase his or her job performance” and perceived ease of use
is defined as “the degree to which the prospective user expects the
target system to be free of effort” (Davis et al., 1989, p. 985). The
technology acceptance model is a widely employed model of user
acceptance and has been empirically supported in different studies
(Venkatesh, 2000; Taylor and Todd, 1995). The model’s variables
347
have also been operationalized and the scales and items have been
validated (e.g., Venkatesh, 2000). In the present study, we will
measure user acceptance based on the technology acceptance
model, because we aim at evaluating computer-based user in-
terfaces with a theoretically and empirically supported model.
Research within the context of human-adaptable automation has
shown that flexibility can increase user acceptance (Miller and
Parasuraman, 2007). Therefore, we expect that customization of
user interfaces will increase user acceptance.
1.3. Enhancing situation awareness and reducing mental workload
Situation awareness is referred to as the perception and
comprehension of information and the projection of the informa-
tion status in the near future (Endsley, 1995b). Situation awareness
is important for safety and performance since poor situation
awareness such as incomplete or inaccurate situation awareness
increases the probability of human error. Therefore, situation
awareness has received increased attention in user interface
design. According to Endsley (1995b), user interface design can
have a great impact on situation awareness because the design of a
user interface determines the quantity and accuracy of information
acquired and the degree of compatibleness of the user interface
with the user’s situation awareness needs. Therefore, Endsley
(1995b) suggests to design user interfaces in a way “that will
transmit needed information to the operator without undue
cognitive effort” (p. 50). Thus, both mental workload and situation
awareness should be considered in user interface design (Endsley,
1995b). Based on these considerations regarding situation aware-
ness and mental workload, we assume that situation awareness
will increase with the possibility to reconfigure user interfaces. The
underlying reasoning is that when reconfiguring user interfaces,
the operator can choose a way to present information that suits his
or her preferences. Furthermore, we assume that operators will
come up with a user interface that displays needed information
with little effort. Therefore, we expect that an effectively designed
user interface can help to reduce mental workload (Endsley, 1995b).
2. Method
2.1. Participants and design
In all, 72 participants (12 female, gender unknown: 4) took part
in this between-subjects design involving two groups. The exper-
imental group, i.e. the reconfiguration group (RG; n ¼ 38, 6 female),
trained and worked with the reconfigurable user interface while
the control group (CG; n ¼ 34, 6 female) trained and worked with
the initial user interface. Participants were students of applied
computer science and they participated in the study for course
credit. Their ages ranged from 19 to 41 years with an average age of
22.7 years (SD ¼ 4.3 years).
2.2. Experimental task
We employed a simulation of a feedwater steam circuit of a
nuclear power plant as an experimental task (see Fig. 1). The
simulation was based on an implementation by Eriksson (2012).
More detailed information on the simulation can be found in
another article by Weyers et al. (2012) which is based on the same
study but focuses on formal modeling and reconfiguration of user
interfaces whereas the focus of the present article is on the psy-
chological aspects of reconfiguration. The main tasks of the oper-
ator were (a) to keep the reactor in a safe system state by keeping
the water level in the reactor tank constant (at 2100 mm) and to
prevent accidents from happening, (b) to generate a constant
348 D. Burkolter et al. / Applied Ergonomics 45 (2014) 346e353
Fig. 1. Screenshot of the experimental task (translated from German) with the default user interface.
output of the generator (700 MW) and (c) to react to upcoming
system faults due to malfunctions of certain technical parts such as
water pumps. Two parameters were critical to the safe running of
the reactors: The water level in the reactor tank and the pressure
(Weyers et al., 2012). These two critical parameters could be
controlled directly by the position of the control rods defining the
value of thermal energy generated in the reactor tank and the speed
of the feedwater pumps. Overall, for operating the reactor, various
valves could be opened and closed, water pumps could be regu-
lated, and the position of the control rods could be changed. The
simulation allowed training of different scenarios such as start-up,
shut-down, and fault states.
Fig. 1 depicts the default user interface of the reactor simulation.
This user interface was designed taking into consideration usability
standards such as learnability, consistency, and transparency (cf.,
Dix et al., 1997) as well as Gestalt principles of organization such as
proximity, similarity, and good continuation (cf., Wickens and
Hollands, 2000). The user interface and its usability were tested
in pilot studies prior to this experimental study including in-
terviews with users.
A simulation of a reactor was chosen as an exemplary task to test
the potential of reconfiguration, because it is easy enough to learn
in the available time and complex enough to simulate realistic
scenarios of control and monitoring of two or more variables in a
dynamic system. The simulation has also been tested in a pilot
study with respect to training duration and resulting success in
controlling the feedwater steam circuit. However, reconfiguration
could not be directly applied to this particular field because of
safety regulations and laws (e.g., for Germany, Federal Ministry of
Justice, 2012a,b).
2.3. Measures
2.3.1. Performance: errors
Performance was measured as the number and type of errors
committed according to a method by Hollnagel (1998). Hollnagel’s
cognitive reliability and error analysis method (CREAM) allows the
definition and classification of different error types. Errors were
defined as deviations from the procedures of start-up, shutdown,
and handling of system faults as described in the checklists provided
to the participants. Log files of the participants were compared to
the predefined procedures to identify errors. More detailed infor-
mation on coding and processing of error type data can be found in
Weyers et al. (2012). Due to technical problems, there were some
missing data (leading to a sample of n ¼ 61 and 63, respectively).
Two groups of magnitude and sequence errors were identified.
Magnitude errors were defined as actions taken too far and included
the two types of errors “oversteering” and “understeering”.
Sequence errors refer to the order of actions, i.e., “performing an
action at a wrong place in a sequence or procedure” (Hollnagel,
1998, p. 166). Five different types of sequence errors were identi-
fied in the present study: (1) swapping two neighboring operations
(“swap”), (2) operations performed too early in the sequence
(“premature”), (3) operations performed too late in the sequence
(“belated”), (4) repeated operations (“repetition”), (5) operations
that were not part of the predefined procedure (“false operation”).
2.3.2. User acceptance
User acceptance was assessed employing scales by Venkatesh
(2000) and Taylor and Todd (1995). The scales, sources, Cron-
bach’s alpha, item numbers and item examples are shown in
Table 1. The scales were translated to German and slightly adapted
to fit the purpose if necessary (e.g., “reconfigured interface” instead
of “system”). Furthermore, the items were adapted to each group,
i.e., the items for the RG referred to the reconfigured user interface,
while the items of the CG referred to the default user interface. The
items were assessed on a 7-point scale ranging from 3 (strongly
disagree) to þ3 (strongly agree). In addition to the scales based on
Venkatesh (2000) and Taylor and Todd (1995), two scales for
perceived responsibility and handling of task were developed.
2.3.3. Situation awareness
We measured situation awareness based on the Situation
Awareness Control Room Inventory (Hogg et al., 1995) and the
situation awareness measurement employed by Burkolter et al.
(2010) which are assumed to be objective situation awareness
measurements. The Situation Awareness Control Room Inventory
was derived from the Situation Awareness Global Assessment
 D. Burkolter et al. / Applied Ergonomics 45 (2014) 346e353 349

Table 1
Scales used to assess user acceptance.

Scale Source Cronbach’s alpha No. of items Items

Perceived usefulness
Perceived ease of use
Perceived behavioral control
Behavioral intention to use
Perceived responsibility
Handling of tasks
Venkatesh (2000)
Venkatesh (2000)
Taylor and Todd (1995)
Venkatesh (2000)
Newly developed
Newly developed
0.92 4 Example item: “I find my own reconfigured [the given]
interface to be useful for operating the REACTOR”
0.87 4 Example item: “I found the reconfiguration tool to be
easy to use” (only used for RG)
0.72 3 Example item: “I would be able to control the REACTOR
using my own [the given] reconfigured interface"
0.89 2 Example item: “Assuming I had to control the REACTOR
once again, I intend to use my own reconfigured [the
given] interface for it”
0.63 3 “I was serious about operating the REACTOR because
I did not want to produce a reactor accident”, “I was
afraid to make a mistake that would produce a reactor
accident”, “I know that a reactor accident would have
massive impact on the environment”
0.83 5 “I always knew what to do”, “I understood all tasks”,
“I was able to solve the tasks”, “I was concentrated
while solving the tasks”, “It was easy for me to accept
and solve the tasks”

Technique (Endsley, 1995a) and adapted for process control tasks. (approx. 25 min; see Table 2). Upon arrival, all participants filled in
The situation awareness measurement took place during the last questionnaires on control variables. Participants then received an
testing scenario. The simulation was automatically stopped at three introduction to the reactor simulation, its initial user interface and
times by the system and a message on the screen asked the par- controls, and were given a couple of minutes to explore the simu-
ticipants to answer questions making the whole information lation on their own. After this introductory part, the training ses-
visualization of the process invisible. Following Endsley’s (1995b) sion was held. The training session included three exercises which
concept of situation awareness, the first item referred to the aimed at practicing the start-up of the reactor, the shutdown of the
perception of the situation while the second item concerned reactor, and the handling of a system fault. Participants were pro-
comprehension of the current status and prediction of the future vided with checklists which explained the steps of the procedures
status of the system. The first item was “How is the status of the and were given about 10 min to practice each exercise. Introduction
water level?” which could be answered with the three alternatives to the simulation and the training session were aimed at providing
(a) below normal range (below 2050 mm), (b) within normal range at participants with the most effective training possible within the
2100 mm (þ/ 50 mm), and (c) above normal range (above 2150 mm). given time constraints. Therefore, the training session included
The following item read: “How do you think the course of the water several practice sessions and was held in small groups of five to
level will develop over the next 10 s? Provided that no intervention eight participants to give participants the opportunity to ask
to the system is undertaken, the water level will...” with the answer questions individually. Participants were guided through the task
alternatives (a) decrease, (b) remain stable, and (c) increase. The first procedure by an instructor. First, participants were introduced to
and third measurement of situation awareness was related to the the start-up procedure in a step-by-step manner. In a next step,
water level while the second measurement asked about the output participants conducted the start-up procedure by themselves but
power. These parameters were chosen because they represented with guidance and support by the instructor. Finally, participants
the target parameters to control the simulation. Control of these completed the start-up procedure on their own.
two target parameters was also clearly stated as main objective to
the participants. In total, there were three questions referring to
perception of the situation and three questions referring to
comprehension and projection. To determine whether responses Table 2
made by the participants were correct, they were compared to Study procedure.
logged data (cf. Hogg et al., 1995; Burkolter et al., 2010).
Study parts Reconfiguration Control group (CG)
group (RG)
2.3.4. Mental workload
Introduction Assessment of control variables, introduction to
Mental workload was assessed using a translated and adapted
(approx. 20 min) simulation and exploration of simulation
version of the NASA task load index (Hart and Staveland, 1988) Training session Introduction to Documentary
which was shown to have high validity and user acceptance (approx. 65 min) reconfiguration tool
(Vitense et al., 2003). The scales included items relating to mental, Exercise 1 (start-up of reactor)
Reconfiguration Sketching of improvements
physical, and temporal demand as well as performance, effort, and
to user interface
frustration level. Ratings were made on a 10-point scale ranging Exercise 2 (start-up and shutdown of reactor)
from e.g., low to high or a little to greatly. The questionnaire was Reconfiguration Sketching of improvements
administered to the participants during the testing session before to user interface
assessing situation awareness. Exercise 3 (start-up of reactor and handling of fault state)
Reconfiguration Sketching of improvements
to user interface
2.4. Procedure Testing session Start-up of reactor and practiced fault state
(approx. 25 min) Start-up of reactor and novel fault state
The present study had an overall duration of approximately 2 h Assessment of mental workload
Situation awareness assessment
and consisted of three parts: An introductory part (approx. 20 min), User acceptance questionnaire
a training part (approx. 65 min), and the testing session in the end
350 D. Burkolter et al. / Applied Ergonomics 45 (2014) 346e353

In the subsequent testing session, participants were asked to
start-up the reactor. In addition, participants needed to handle the
practiced fault state (system fault of a feedwater pump) without
knowing in beforehand that the fault would occur. Then, they were
to start-up the reactor and handle a novel fault state which they
had not encountered before (system fault of turbine). Again, par-
ticipants were not aware previously that this fault would occur.
Following these tasks, mental workload and situation awareness
were assessed. Situation awareness was measured as described
above while the participants were performing a combination of the
start-up and shutdown procedures. Finally, the questionnaire
regarding user acceptance was filled in by the participants.
While the exercises and type of tasks in training and testing
session were the same for both groups, the RG used their recon-
figured user interface whereas the CG used the initial one. The RG
was introduced to the reconfiguration tool and its usage before
starting with the exercises. This introduction to the reconfiguration
tool had a duration of approximately 10 min and included expla-
nations of the possibilities and tools as well as time for practicing
with the reconfiguration tool and exploring its options. The par-
ticipants of the RG were then given 10 min after each of the three
exercises to reconfigure their user interface in a way to increase
performance at the task. Reconfiguration was only possible during
the training phase. While the RG group was introduced to the
reconfiguration tool and was given time to reconfigure their own
user interfaces, the CG was given other tasks with the same dura-
tion and similar workload to ensure comparability. During the time
in which the RG was introduced to the reconfiguration tool (about
10 min), the CG was shown a sequence of a documentary about
simulations. The topic of the documentary was supposed to be
unrelated to the reactor simulation and the user interface. While
the RG was reconfiguring their user interfaces, the CG sketched
ideas of possible improvements and adaptations to the user inter-
face using pen and paper.
2.5. Reconfiguration
The reconfiguration tool allowed four different types of recon-
figuration operations applied interactively to the interaction ele-
ments of the user interface (e.g., buttons or sliders): (1) the
duplication of a single interaction element, (2) the removal or
deletion of an interaction element, (3) the sequencing of two or
more operations triggered by discrete interaction elements (but-
tons) into one step that is triggered by one new button, and (4)
discretization of continuous interaction elements (such as sliders)
into one discrete interaction element as has been described above.
Other than the four basic reconfiguration types, it was also possible
(5) to combine discretization and sequencing (cf. Weyers et al.,
2010). These reconfigurations of the user interface mainly
involved adaption of its physical representation such as adding a
new button. Using these reconfiguration operations, it was possible
to sequence various steps of the start-up, shut-down and handling
routines for fault states, such as combining opening and closing of
valves and setting water pumps to previously discretized values.
Some parts of the routines, however, could not be subsumed by
reconfiguration, such as the alternating the increase of water pump
1 and the control rods as part of the start-up routine. A screenshot
of a reconfigured user interface by a participant in our study is
shown in Fig. 2 as an example.
In Table 3, number and percentages of reconfiguration types
used by the study participants are presented. Discretization was the
reconfiguration type that was most frequently used, followed by
the combination of sequencing and discretization and sequencing
alone.

Fig. 2. Screenshot of a reconfigured user interface of one study participant (translated from German). The participant used different reconfiguration operations (e.g., sequencing) to
build up the procedure of start-up (bottom, left), shut-down of the reactor (bottom, right), and the practiced fault state (top, right). In this case, the original user interface still
existed (top, left), slightly modified in position of sliders and color of buttons. (For interpretation of the references to colour in this figure legend, the reader is referred to the web
version of this article.)
 D. Burkolter et al. / Applied Ergonomics 45 (2014) 346e353 351

Table 3 Table 5
Number and percentage of used reconfiguration types (only RG; n ¼ 35). Mean and SD (in parentheses) for error types (both start-up and practiced/novel) as
a function of group and results of t-tests.
Reconfiguration type Number Percentage
Error types RG CG t(df), p (one-tailed),
(1) Duplicate of interaction element 53 13.6%
effect size r
(2) Removal of interaction element 0 0.0%
(3) Sequencing 80 20.6% Start-up and practiced fault state
(4) Discretization 161 41.4% Magnitude error
(5) Discretization and sequencing 95 24.4% Oversteering 0.38 (0.70) 2.38 (1.72) t(45.86) ¼ 6.13, p ¼ .000,
(3 and 4 together) r ¼ .67
Total 389 100.0% Understeering 0.15 (0.37) 0.15 (0.36) t(58) ¼ .07, p ¼ .472, r ¼ .01
Sequence error
Swap 0.04 (0.20) 0.47 (0.86) t(37.40) ¼ 2.83, p ¼ .004,
r ¼ .42
2.6. Assessment of control variables Premature 0.23 (0.43) 0.24 (0.50) t(58) ¼ .04, p ¼ .486, r ¼ .01
Belated 0.42 (0.58) 0.38 (0.70) t(58) ¼ .24, p ¼ .405, r ¼ .03
Repetition 0.38 (0.70) 0.09 (0.29) t(31.54) ¼ 2.04, p ¼ .025, r ¼ .34
We controlled for pre-training knowledge, subjective rating of False operation 0.96 (1.25) 1.82 (1.90) t(56.87) ¼ 2.12, p ¼ .020,
knowledge level, cognitive flexibility, and general mental ability. r ¼ .27
Pre-training knowledge was measured with seven multiple-choice Total 2.48 (1.93) 5.53 (2.71) t(59) ¼ 4.93, p ¼ .000, r ¼ .54
questions about nuclear power plants (e.g., “How is electricity Start-up and novel fault state
Magnitude error
produced in a power plant?”). In addition, participants were asked
Oversteering 0.45 (0.83) 1.50 (1.60) t(51.03) ¼ 3.34,
to give subjective assessment of their previous knowledge about p ¼ .001, r ¼ .42
nuclear power plants on a seven-point scale ranging from (1) very Understeering 0.10 (0.31) 0.18 (0.39) t(61) ¼ 0.82, p ¼ .209, r ¼ .10
poor to (7) very good. General mental ability was assessed with the Sequence error
Wonderlic Personnel Test (Wonderlic Inc., 2002). Due to time Swap 0.21 (0.41) 0.29 (0.52) t(61) ¼ 0.73, p ¼ .236, r ¼ .09
Premature 0.10 (0.31) 0.38 (0.60) t(50.82) ¼ 2.35, p ¼ .011,
constraints, general mental ability was assessed at a later date (on
r ¼ .31
which not all of the participants could take part, n ¼ 43). Cognitive Belated 0.48 (0.69) 0.44 (0.56) t(61) ¼ .26, p ¼ .396, r ¼ .03
flexibility was assessed with the Cognitive Flexibility Inventory Repetition 0.07 (0.26) 0.00 (0.00) t(28.0) ¼ 1.44, p ¼ .081, r ¼ .26
(Spiro et al., 1996). Participants in the two groups did not differ False operation 0.31 (0.71) 0.59 (1.02) t(61) ¼ 1.23, p ¼ .111, r ¼ .16
Total 1.72 (1.62) 3.38 (2.32) t(61) ¼ 3.23, p ¼ .001, r ¼ .38
significantly with regard to control variables which indicates that
there was not an “unhappy randomization” (Mohr, 1995; see
Table 4).
However, the CG committed fewer “repetition” errors during start-
up and practiced fault state. With respect to start-up and novel fault
3. Results
state, the RG committed fewer “oversteering” errors and “prema-
ture” errors.
3.1. Performance: errors

In line with our assumptions, performance of the RG was better 3.2. User acceptance
than performance of the CG which is shown by the total number of
errors committed (see Table 5). Both during start-up of the reactor Descriptive statistics as well as results of t-tests regarding user
and the practiced fault state as well as the novel fault state, the RG acceptance scales can be found in Table 6. As expected, participants
committed significantly fewer errors (practiced: M ¼ 2.48, in the RG showed significantly higher acceptance of the user in-
SD ¼ 1.93; novel: M ¼ 1.72, SD ¼ 1.62) than the CG (practiced: terfaces than the CG, except from ratings regarding perceived re-
M ¼ 5.53, SD ¼ 2.71; novel: M ¼ 3.38, SD ¼ 2.32). Thus, the RG sponsibility and handling of task. On a scale from 3 to 3,
committed about half as many errors than the CG, constituting a participants in the RG rated their behavioral intention to use the
large effect in the case of the practiced fault state (r ¼ .54) and a user interface again almost 2 on average (SD ¼ 1.07) whereas the CG
medium to large effect in the case of the novel fault state (r ¼ .38; rated their behavioral intention to use 0.5 on average (SD ¼ 1.68).
Field, 2005; Cohen, 1992). Regarding the types of errors during This difference between the two groups was significant and
start-up and practiced fault state, the RG committed significantly constituted a large effect (r ¼ .53; Field, 2005; Cohen, 1992).
fewer “oversteering” errors, “swap” errors, and “false operations”.
Table 6
Table 4 Mean and SD (in parentheses) of user acceptance as a function of group and results
Mean and SD (in parentheses) on control variables as a function of group and results of t-tests.
of t-tests.
User acceptance variablesa RG CG t(df), p (one-tailed),
Control variables RG CG t(df), p (two-tailed), effect size r
effect size r
Perceived usefulness 1.95 (1.05) 0.40 (1.47) t(65) ¼ 5.02,
Knowledge testa 12.38 (2.17) 11.74 (2.12) t(69) ¼ 1.26, p ¼ .211, p ¼ .000, r ¼ .53
r ¼ .15 Perceived ease of useb 1.78 (1.31) e e
Subjective assessment 4.13 (1.63) 4.17 (1.78) t(66) ¼ .09, p ¼ .933, Perceived behavioral control 1.65 (0.94) 1.20 (1.06) t(65) ¼ 1.85,
of knowledgeb r ¼ .01 p ¼ .034, r ¼ .22
General mental abilityc 26.28 (5.06) 25.89 (5.19) t(41) ¼ .25, p ¼ .806, Behavioral intention to use 1.99 (1.07) 0.48 (1.68) t(44.79) ¼ 4.22,
r ¼ .04 p ¼ .000, r ¼ .53
d
Cognitive flexibility 0.29 (0.93) 0.49 (1.23) t(61.12) ¼ .77, p ¼ .442, Perceived responsibility 1.71 (1.06) 1.77 (1.10) t(64) ¼ 0.22,
r ¼ .10 p ¼ .414, r ¼ .03
Handling of task 2.16 (0.74) 1.83 (1.09) t(64) ¼ 1.48,
Note:
a p ¼ .072, r ¼ .18
0e14 pts.
b
Scale from 1 (very poor) to 7 (very good). Note:
c a
0e50 pts. Scale from 3 to 3.
d b
Scale from 4 to þ4. Only rated by RG.
352 D. Burkolter et al. / Applied Ergonomics 45 (2014) 346e353

Similarly, perceived usefulness was rated significantly higher by the 2005). We tested whether customization of user interfaces had the
RG (M ¼ 1.95, SD ¼ 1.05) than the CG (M ¼ 0.40, SD ¼ 1.47) and also potential to enhance performance and user acceptance as well as
this effect was large (r ¼ .53). Furthermore, there was a significant situation awareness and reduce mental workload. Participants in
difference between the groups regarding perceived behavioral the experimental group, the RG, were provided with a reconfigu-
control, although the effect was smaller (r ¼ .22). The reconfigured ration tool which allowed different reconfiguration operations such
user interface received thus a higher user acceptance rating than as the discretization of continuous interaction elements or the
the default user interface. However, user acceptance of the default duplication and removal of interaction elements. The RG trained
user interface was still on the positive side on the scale from 3 to and worked with the reconfigurable user interface while the CG
3. Finally, the RG rated perceived ease of use of the reconfiguration trained and worked with a default user interface.
tool on average 1.78 (SD ¼ 1.31) on a scale from 3 to 3. Thus, the First, results showed clearly that the RG committed fewer errors
RG perceived the reconfiguration tool as fairly easy to use. than the CG. Both during start-up and practiced fault state as well
as novel fault state, the RG committed significantly fewer errors
3.3. Situation awareness and mental workload than the CG and this proved to be a medium to large effect. The CG
committed about twice as many errors than the RG. These findings
Results regarding situation awareness and mental workload are show the potential of reconfigurable user interfaces to support
shown in Table 7. Although the possibility to reconfigure one’s own performance.
user interface was expected to enhance situation awareness, the RG Second, not only was performance supported by reconfigurable
did not outperform the CG regarding situation awareness (see user interfaces but the interfaces also enhanced user acceptance.
Table 7). Participants in the RG rated perceived usefulness, perceived behav-
Unlike our assumptions, the groups did not differ significantly ioral control and perceived intention to use significantly higher than
with respect to mental workload. That is, the RG did not show the participants in the CG. However, perceived responsibility and
significantly lower mental workload ratings than the CG. Partici- handling of task were not significantly increased by the use of
pants’ ratings of their mental workload were around four to six on a reconfigurable interfaces. In summary, participants learned to handle
scale from one to ten, which suggests that they perceived their the complex task better with the reconfigurable interface and also
workload as not very high. subjectively liked the reconfigurable interface better than the CG.
Third, results regarding situation awareness and mental work-
3.4. Additional analysis load, however, were not in line with our assumptions. The RG did
not outperform the CG regarding situation awareness nor mental
As an additional analysis, duration of initialization (i.e., duration workload. In general, performance is supposed to be good when
from starting the simulation until power is produced) was situation awareness is good (Endsley, 1995b). Thus, situation
compared between the groups. For the first task (start-up and awareness is expected to be linked to performance. In the present
practiced fault state), the RG needed 20.35 s on average (SD ¼ 17.92) study, however, it seems that performance was good as such (in the
and the CG 45.68 s on average (SD ¼ 18.47). This difference in RG) without the need of support by good situation awareness.
duration of initialization proved to be significant (t(58) ¼ 5.39, Possibly, for the task at hand, the option to reconfigure was more
p < .000) with a large effect size of r ¼ .58 (Field, 2005; Cohen, critical for good performance than having high situation awareness.
1992). For the second task (start-up and novel fault state), The findings may also be explained by usage duration: The study
average duration of initialization of the RG was 18.86 s (SD ¼ 11.72) lasted approximately 2 h in total of which approximately 25 min
and of the CG 38.55 s (SD ¼ 14.80). Also this difference in mean were reserved for the testing session. This usage duration may have
duration of initialization was significant (t(65) ¼ 6.07, p < .000) been too short to build up and maintain high situation awareness,
with a large effect size of r ¼ .60; Field, 2005; Cohen, 1992). Thus, especially when using a novel system. As Endsley (1995b) argues,
reconfiguration reduced the time needed until power was situation awareness “is not necessarily acquired instantaneously
produced. but is built up over time” (p. 38). Further research may thus
consider a longer usage duration. Another possibility is to consider
4. Discussion reconfiguration of the system parameters’ visual representation
(output elements; e.g., single- and multi-value displays and dia-
In the present study, we took up the idea of individuation that grams) in addition to reconfiguration of interaction elements of the
tools such as user interfaces should be customizable (Hancock et al., user interface (i.e., input; cf. Weyers et al., 2012). Reconfiguring
output elements may enhance situation awareness since informa-
Table 7 tion can be displayed as preferred. Regarding mental workload, we
Mean and SD (in parentheses) of situation awareness and mental workload as a
expected the reconfigured interface to decrease mental workload of
function of group and results of t-tests.
the RG. However, mental workload ratings did not differ signifi-
Variables RG CG t(df), p (one-tailed), cantly for the two groups. Maybe this result can be explained by the
effect size r
fact that mental workload ratings were on a rather low level for
Situation awarenessa both groups. Thus, the reconfigurable interface could not decrease
Perception 2.14 (0.92) 1.97 (0.88) t(68) ¼ .77, p ¼ .223, r ¼ .09 an already low perceived workload further. Another explanation
Comprehension 1.54 (0.69) 1.58 (0.97) t(57.15) ¼ .17, p ¼ .432,
and prediction r ¼ .02
could be that reconfiguration led to an increased mental workload
Mental workloadb during the training session and was still on an increased level
Mental demand 5.82 (1.92) 5.85 (1.91) t(70) ¼ .08, p ¼ .468, r ¼ .01 during the testing session.
Physical demand 3.66 (2.35) 4.50 (2.84) t(70) ¼ 1.38, p ¼ .087, r ¼ .16 Although these findings are promising with respect to the use of
Temporal demand 5.76 (1.70) 5.88 (1.80) t(69) ¼ .28, p ¼ .391, r ¼ .03
reconfiguration, some limitations of this study need to be dis-
Performance 4.68 (2.23) 4.29 (2.01) t(70) ¼ .78, p ¼ .220, r ¼ .09
Effort 5.50 (1.75) 5.09 (2.29) t(70) ¼ .86, p ¼ .196, r ¼ .10 cussed. Performance was assessed as the number and type of errors
Frustration level 6.24 (1.94) 5.68 (2.24) t(70) ¼ 1.14, p ¼ .130, r ¼ .14 committed according to Hollnagel’s (1998) classification. The latter
Note:
represents one possible method of assessing performance, how-
a
0e3 pts. ever, future studies should explore other measures such as mea-
b
Scale from 1 (low/little) to 10 (high/greatly). sures for normal operation. Also, future research should assess the
 D. Burkolter et al. / Applied Ergonomics 45 (2014) 346e353
participants’ mental model of the system at the beginning and the
end of the study to compare possible differences and changes in the
mental models of the RG and the CG. Furthermore, participants
wereealthough students of applied computer scienceenovices and
the process control task was unknown to them before the experi-
ment. Also, their mean age was around 23 years. In view of these
facts, the question arises to what extent similar findings could be
expected for more experienced operators or for operators with a
different level of education? With regard to more experienced
operators’ benefit from using the reconfiguration tool, potential
may be seen regarding the decrease of workload. Reconfiguring a
user interface and working with a reconfigured user interface may
enhance performance by reducing workload. Also, further research
may look into how operators work with the reconfiguration tool
when using it for a prolonged duration, e.g., whether they work on
improving their own reconfigured user interface.
Finally, significance of the present findings for practice is dis-
cussed. Reconfiguration presents opportunities to learning and
performing in complex, technical systems. Individuation is
assumed to increase job satisfaction, motivation, and positive affect
through enhanced user control over the technical system (Fritzsche
and Parrish, 2005 cited in Hancock et al., 2005). Furthermore,
correspondence between user interface, mental model of the user,
and physical system (cf. Wickens and Hollands, 2000) is believed to
increase performance. The reconfiguration tool may be used with a
group of operators in the simulator to configure a user interface
according to the needs of these operators. Reconfiguration may also
be employed for learning and training purposes (cf. Weyers et al.,
2011). Reconfiguring a user interface may help to enhance under-
standing of processes and systems.
However, despite possible opportunities, difficulties and risks
with the use of reconfiguration need to be considered as well.
Reconfiguration entails the risk that the reconfigured interface may
not be optimal or even includes erroneous operations. Therefore, a
correction function needs to be implemented to prevent errors as
well as limitations as to what extent reconfiguration is possible.
Furthermore, research is needed to study long-term effects of
reconfiguration. Research needs to rule out the possibility that a
user’s mental model becomes less differentiated over time, because
the user may be prone to think in “macroprocedures” and less in
specific procedures and operations. Thus, further research on
reconfiguration is warranted before possible use in practice.
In conclusion, reconfigurable user interfaces seem to be prom-
ising and the present findings warrant further exploration of
reconfiguration. For instance, the potential of reconfiguration for
learning and training may be further investigated.
References
Ajzen, I., Fishbein, M., 1980. Understanding Attitudes and Predicting Social Behavior.
Prentice-Hall, Englewood Cliffs.
Burkolter, D., Kluge, A., Sauer, J., Ritzmann, S., 2010. Comparative study of three
training methods for enhancing process control performance: emphasis shift
training, situation awareness training and drill and practice. Computers in
Human Behavior 26, 976e986.
Carroll, J.M., Olson, J. (Eds.), 1987. Mental Models in HumaneComputer Interaction:
Research Issues about what the User of Software Knows. National Academy
Press, Washington, D.C.
353
Cohen, J., 1992. A power primer. Psychological Bulletin 112, 155e159.
Davis, F.D., Bagozzi, R.P., Warshaw, P.R., 1989. User acceptance of computer tech-
nology: a comparison of two theoretical models. Management Science 35, 982e
1002.
Dix, A., Finlay, J.E., Abowd, G.D., Beale, R., 1997. HumaneComputer Interaction,
second ed. Prentice Hall Europe, London.
Endsley, M.R., 1995a. Measurement of situation awareness in dynamic systems.
Human Factors 37, 65e84.
Endsley, M.R., 1995b. Toward a theory of situation awareness in dynamic systems.
Human Factors 37, 32e64.
Eriksson, H., 2012. The On-line Nuclear Power Plant. Linköping University, Link-
öping, Sweden. Retrieved August 2nd, 2012 from. http://www.ida.liu.se/wher/
npp.
Federal Ministry of Justice, Germany, 2012a. Gesetz über die friedliche Ver-
wendung der Kernenergie und den Schutz gegen ihre Gefahren (Law on the
peaceful use of nuclear energy and the protection against its risks).
Retrieved August 1st, 2012 from. http://www.gesetze-im-internet.de/atg/
index.html.
Federal Ministry of Justice, Germany, 2012b. Gesetz über die Bereitstellung von
Produkten auf dem Markt (Law on the supply of products on the market).
Retrieved August 1st, 2012 from. http://www.gesetze-im-internet.de/prodsg_
2011/index.html.
Field, A., 2005. Discovering Statistics Using SPSS, second ed. SAGE Publications,
London.
Fritzsche, B.A., Parrish, T.J., 2005. Theories and research on job satisfaction. In:
Brown, S.D., Lent, R.W. (Eds.), Career Development and Counselling: Putting
Theory and Research to Work. Wiley, New York, NY, pp. 180e202.
Hancock, P.A., Pepe, A.A., Murphy, L.L., 2005. Hedonomics: the power of positive and
pleasurable ergonomics. Ergonomics in Design 13, 8e14.
Hart, S.G., Staveland, L.E., 1988. Development of NASA-TLX (Task Load Index):
results of empirical and theoretical research. In: Hancock, P.A.,
Meshkati, N. (Eds.), Human Mental Workload. North Holland, Amsterdam,
pp. 139e183.
Hogg, D.N., Follesø, K., Strand-Volden, F., Torralba, B., 1995. Development of a sit-
uation awareness measure to evaluate advanced alarm systems in nuclear
power plant control rooms. Ergonomics 38, 2394e2413.
Hollnagel, E., 1998. Cognitive Reliability and Error Analysis Method. Elsevier, Oxford.
Miller, C.A., Parasuraman, R., 2007. Designing for flexible interaction between
humans and automation: delegation interfaces for supervisory control. Human
Factors 49, 57e75.
Mohr, L.B., 1995. Impact Analysis for Program Evaluation, second ed. Thousand,
Oaks, SAGE.
Pozzi, S., Bagnara, S., 2011. Individuation and diversity: the need for idiographic HCI.
Theoretical Issues in Ergonomics Science (iFirst article). http://dx.doi.org/
10.1080/1464536X.2011.562564.
Spiro, R.J., Feltovich, P.J., Coulson, R.L., 1996. Two epistemic world-views: prefigu-
rative schemas and learning in complex domains. Applied Cognitive Psychology
10, S51eS61.
Taylor, S., Todd, A., 1995. Understanding information technology usage: a test of
competing models. Information Systems Research 6, 144e176.
Trumbly, J.E., Arnett, K.P., Johnson, P.C., 1994. Productivity gains via an adaptive user
interface: an empirical analysis. International Journal of Human-Computer
Studies 40, 63e81.
Venkatesh, V., 2000. Determinants of perceived ease of use: integrating control,
intrinsic motivation, and emotion into the technology acceptance model. In-
formation Systems Research 11, 342e365.
Vitense, H.S., Jacko, J.A., Emery, V.K., 2003. Multimodal feedback: an assessment of
performance and mental workload. Ergonomics 46, 68e87.
Weyers, B., Burkolter, D., Kluge, A., Luther, W., 2010. User interface reconfigu-
ration for error reduction in human computer interaction. In: Proceedings of
the Third International Conference on Advances in Human-oriented and
Personalized Mechanisms, Technologies, and Services. IEEE Computer Society,
Nice, pp. 52e55.
Weyers, B., Luther, W., Baloian, N., 2011. Interface creation and redesign techniques
in collaborative learning scenarios. Future Generation Computer Systems 27,
127e138.
Weyers, B., Burkolter, D., Luther, W., Kluge, A., 2012. Formal modelling and recon-
figuration of user interfaces for reduction of errors in failure handling of
complex systems. International Journal of HumaneComputer Interaction 28,
646e665.
Wickens, C.D., Hollands, J.G., 2000. Engineering Psychology and Human Perfor-
mance, third ed. Prentice Hall, New Jersey.
Wonderlic, Inc, 2002. Wonderlic Personnel Test. Wonderlic Inc., Libertyville.