Siegel (2017 Dissertation) Team Reflection On Weak Resilience Signals

i
TEAM REFLECTION
ON WEAK RESILIENCE SIGNALS
RESILIENCE ENHANCEMENT
OF A RAIL SOCIOTECHNICAL SYSTEM
Aron Wolf (Willy) Siegel
I
Graduation committee:
Chairman/secretary: Prof. dr. Th. A. J. Toonen

Promotor: Prof. dr. J. M. C. Schraagen
Members: Prof. dr. D. K. J. Heylen

Prof. dr. J. F. M. Koppenjan
Prof. dr. M. A. Neerincx
Prof. dr. N. A. Stanton
Prof. dr. ing. W. B. Verwey
Prof. dr. D. D. Woods
II
TEAM REFLECTION
ON WEAK RESILIENCE SIGNALS
RESILIENCE ENHANCEMENT
OF A RAIL SOCIOTECHNICAL SYSTEM
PROEFSCHRIFT
ter verkrijging van

de graad van doctor aan de Universiteit Twente,
op gezag van de rector magnificus,
prof. dr. T. T. M. Palstra,
volgens besluit van het College voor Promoties
in het openbaar te verdedigen
op donderdag 12 januari 2017 om 16:45 uur
door
Aron Wolf (Willy) Siegel

geboren op 25 november 1956
te Amsterdam
III
Dit proefschrift is goedgekeurd door de promotor:
Prof. dr. J. M. C. Schraagen
CTIT Ph.D. Thesis Series No. 16-416

Centre for Telematics and Information Technology
P.O. Box 217, 7500 AE
Enschede, The Netherlands
ISBN: 978-90-365-4275-3
ISSN: 1381-3617 (CTIT Ph.D. Thesis Series No. 16-416)
DOI: 10.3990/1.9789036542753
https://dx.doi.org/10.3990/1.9789036542753
Cover: Victor Kramnik

Print: Gildeprint, Enschede, The Netherlands
© 2017, Aron Wolf Siegel. All rights reserved.
IV
To my mother and in memory of my father (‫)ז"ל‬,
for Dina, Yaniv, Eyal, and Itai
V
“It's not what you look at that matters, it's what you see.”
Henry David Thoreau
VI
TABLE OF CONTENTS
CHAPTER 1: INTRODUCTION.........................................................................................................................
1. Problem statement.............................................................................................................
2. Research objective............................................................................................................
3. Background.......................................................................................................................
3.1. Resilience engineering.............................................................................................5
3.2. Safety management - from Safety-I to Safety-II.....................................................7
3.3. The rail signaller......................................................................................................8
4. Research approach and overview of the thesis................................................................10
CHAPTER 2: WEAK RESILIENCE SIGNAL - WORKLOAD.......................................................................

I. Measuring workload weak resilience signals at a rail control post 15
Occupational applications.........................................................................................................15
Technical abstract.....................................................................................................................15
1. Introduction.....................................................................................................................16
2. A framework for rail weak resilience signal modelling..................................................18
2.1. Theoretical resilience state model for a railway system.........................................18
2.2. Generic quantifiable weak resilience signal model................................................20
3. A method to measure workload WRS at a rail control post............................................21
4. Observational study during rail operations.....................................................................26
5. Results.............................................................................................................................28
6. Discussion.......................................................................................................................31
Acknowledgement....................................................................................................................33
II. Comparison of real-time relative workload measurements in rail signallers
35
Abstract..................................................................................................................................... 35
1. Introduction.....................................................................................................................36
2. Methods.......................................................................................................................... 39
2.1. Participants & Procedures......................................................................................39
2.2. Measurements........................................................................................................39
2.2.1. Integrated Workload Scale (IWS)..................................................................39
2.2.2. External Cognitive Task Load.......................................................................40
2.2.3. Behavioural cognitive Task Load..................................................................41
2.2.4. Electrodermal activity....................................................................................43
3. Results.............................................................................................................................43
3.1. Data collection and case comparison.....................................................................43
3.2. Short description of case 1 and 2...........................................................................44
3.3. IWS results............................................................................................................44
3.4. EDA results............................................................................................................45
3.5. BTL results............................................................................................................46
VII
3.6. XTL results............................................................................................................48
4. Discussion.......................................................................................................................49
CHAPTER 3: PERFORMANCE WEAK RESILIENCE SIGNAL...................................................................

Towards quantifying metrics for rail-system resilience: Identification and analysis of
performance weak resilience signals
Abstract..................................................................................................................................... 53
1. Introduction.....................................................................................................................54
2. Method............................................................................................................................57
2.1. Research Setting....................................................................................................57
2.2. Resilience observations..........................................................................................58
2.3. Weak resilience signals..........................................................................................60
2.4. Resilience Questionnaire.......................................................................................61
3. Results.............................................................................................................................62
3.1. Real-time dispatching observations.......................................................................62
3.1.1. Example of a high-pressure situation: ‘The hooligan case’...........................63
3.1.2. Resilience behaviour episodes.......................................................................64
3.1.3. Weak resilience signals..................................................................................65
3.1.4. WRS Analysis function.................................................................................67
3.2. Resilience questionnaire........................................................................................69
4. Discussion and conclusion..............................................................................................70
Open Access..............................................................................................................................72
CHAPTER 4: TEAM REFLECTION IN RAIL OPERATIONS.......................................................................

Team reflection makes resilience-related knowledge explicit through collaborative
sensemaking
Abstract..................................................................................................................................... 75
1. Introduction.....................................................................................................................76
2. Method............................................................................................................................78
2.1. Team reflection of rail signal operators at the end of their shift............................79
2.2. Tool for presentation and analysis of relative movements towards the performance
boundary................................................................................................................80
2.3. Reflection global analysis......................................................................................83
2.4. Data-framing..........................................................................................................84
3. Observational study design.............................................................................................86
4. Results.............................................................................................................................88
4.1. Sequenced data-framing case: Suicide attempt (person approaches the rail).........88
4.1.1. Drift towards the boundary and the interdependent relationship...................91
4.2. Global analysis.......................................................................................................92
5. Discussion.......................................................................................................................94
VIII
Appendix – punctuality definition of an area............................................................................97
CHAPTER 5: DESIGN & PROTOTYPE IN RAIL OPERATIONS................................................................

Beyond procedures: Team reflection in a rail control centre to enhance resilience
Abstract...................................................................................................................................101
1. Introduction...................................................................................................................102
2. Methods........................................................................................................................105
2.1. Requirements and design of the reflection tool (Resiliencer)..............................105
2.2. Observational study design..................................................................................108
3. Results...........................................................................................................................109
3.1. Team-reflection observation................................................................................109
3.1.1. Daily observations.......................................................................................109
3.1.2. Week observation overview.........................................................................113
3.2. Near-accident case: Two nose-to-nose stopped trains.........................................115
3.2.1. Background..................................................................................................115
3.2.2. Sequence of events......................................................................................116
3.2.3. Case overview – relevancy for team-reflection...........................................117
4. Discussion.....................................................................................................................120
Acknowledgement..................................................................................................................123
CHAPTER 6: SUMMARY AND CONCLUSIONS...........................................................................................

1. Summary.......................................................................................................................127
2. Conclusions...................................................................................................................131
2.1. Resilience real-time quantification during operations..........................................131
2.2. Operationalizing of Safety-II during operations..................................................132
2.3. Evolution of the rail signaller and others.............................................................132
3. Implications...................................................................................................................133
4. Novelties in operations..................................................................................................135
5. Future research..............................................................................................................135
REFERENCES......................................................................................................................................................
1. Publication list (in chronological research order).........................................................137
2. References.....................................................................................................................139
SAMENVATTING................................................................................................................................................
DANKWOORD.....................................................................................................................................................
CURRICULUM VITAE.......................................................................................................................................
IX
Weak resilience signal - workload
INTRODUCTION
A. W. Siegel 1
Team reflection on weak resilience signals
2 A. W. Siegel
CHAPTER 1
INTRODUCTION
1. Problem statement
Sociotechnical systems, interrelating people and technology, are becoming more complex while
safety expectations grow. Examples of such complex sociotechnical system are civil aviation,
process industry, nuclear industry, and rail systems. The technology is becoming more complex
while the systems themselves are growing in volume. Due to the growing complexity and
volume, the chances that something goes wrong increases, whereas a decrease is expected and
demanded by the general public, customers and politicians.
Existing methods of system development, system maintenance, and system control have
achieved very low failure rates within sociotechnical systems, with a limit of 5x10 -7 risk of a
disastrous accident per safety unit in the system (Amalberti, 2001). These methods get to the
limit of their possibilities and new paradigms are sought to make a next step in achieving safety.
Operators within a sociotechnical system perform more supervisory functions compared to 50

years ago, due to higher levels of automation (Hollnagel, 2014 - chapter 2). They mainly
monitor systems and increasingly lack a deep understanding of the system (Bainbridge, 1983).
Moreover, when they need to act they lack the requisite knowledge and experience. In 2013, the
Federal Aviation Administration (FAA) released a report on the effects of autopilot systems in
commercial aircraft, urging “operators to promote manual flight operations when appropriate”
(Nakamura, 2013). Simulation and training can solve some aspects but are limited in coverage
and time of the operators (Morgan, 2016). Moreover, as Bainbridge (1983) argued, there is a
fundamental irony in automation in that designers leave the most difficult tasks, the ones they
don’t know how to automate, up to humans. The irony is that humans become increasingly
‘deskilled’ in dealing with the difficult cases as automation increases. Methods are needed to
develop the abilities of operators while they are doing their daily work.
The focus in this thesis is on the expanding rail system in the Netherlands. More passengers,
more freight, more capacity utilization, and higher levels of automation characterize this rail
system. In the Netherlands, the passenger kilometres grew from 14.6 billion-km to 16.2 billion-
km between 2010 and 2014 (CBS - Statistics Netherlands, 2016). These large numbers imply
large numbers of passengers, freight, rolling stock, rail personnel, logistic possibilities etc. All of
them interacting with each other exhibits an existing and growing complexity. The complexity is
partly technical, partly social and partly economic in nature. As part of the technical complexity,
the dense rail network is sensitive to disruptions and hard to replan in real-time. As part of the
A. W. Siegel 3
social complexity, there is a consensus culture with multiple players with different
responsibilities and goals. Possible solutions to disruptions need to be discussed and agreed
upon with all parties, where the traveller and national politics need to be taken into account as
well. Changes in the socio-economical context, due to for instance higher customer (traveller)
expectations and increased political scrutiny after major disruptions and in an era of shrinking
budgets, add pressure to those dealing with system disturbances.
Instead of realizing systems that ‘make us smart’ (Norman, 1994), rail operators frequently have
to deal with systems that make them dumb, resulting in workarounds that increase workload. It
may be surprising, therefore, to see that the controlled process as a whole is able to achieve the
strategic rail objectives most of the time. This is undoubtedly due to the so-called ‘human
factor’, which is not a liability but in fact an asset in most complex real-world situations (Woods
& Branlat, 2010). Individuals and organisations must always adjust their performance to the
current conditions; and because resources and time are finite it is inevitable that such
adjustments are approximate. Success in dealing with incidents may be attributed to the
professionalism of local train operators and regional/national traffic operators to anticipate the
changing shape of risk before disruption occurs.
This dissertation is about overcoming the above stated problems by seeking for new ways of
viewing and understanding the rail sociotechnical system dynamics, new ways of analysis and
tooling, enabling new processes for operators to develop their capabilities during their work, and
to improve the whole sociotechnical system dealing with the unexpected and unforeseen.
2. Research objective
The objective of this research is to investigate how to improve the abilities of rail signallers to
enhance the performance of the rail sociotechnical system, when unexpected or unforeseen
events occur. New ways of presenting the status of the system, combined with analysis tools to
understand the underlying reasoning, will improve operators' abilities to see new aspects, learn
and anticipate to enhance system performance. New theories and models are needed to describe
the sociotechnical system dynamics, which can be translated to appropriate human machine
interfaces (HMI) and provide tools for analysis. The models, used in real-time operations and
fed with available on-line information, have to be verified with other non-real-time means. A
process for rail operators, effectuating the HMI and tooling, needs to be designed and theorized
to explain its influence on the operator’s cognition, and the revealing of knowledge to improve
his abilities. Ultimately, the whole set of means and methods needs to be verified and exhibit its
effectiveness in the context of real operations.
3. Background
4 A. W. Siegel
Resilience engineering (section 3.1.) is used as a basis for this research since it is dealing with
sociotechnical systems and their response to the unexpected and unforeseen. Resilience
engineering theory is used and expanded. The evolution of safety management (section 3.2.) is
another foundation related to the problem statement concerning the expectation of low failure
rates of complex systems. The rail signaller (section 3.3.) is the central rail operator role in this
dissertation. The evolution of this function, together with the technological development of the
rail sector, provides relevant background for this thesis.
3.1. Resilience engineering

Resilience is closely related to the problem statement through its definition as the capability of
systems to manage their adaptive capacity and resources to cope with unforeseen and
unexpected events (Branlat & Woods, 2010; Hollnagel, 2011a). Resilience engineering is a
relatively new discipline (Nemeth & Herrera, 2015). The first meeting of the Resilience
Engineering Association was in 2004 resulting in a book based on that meeting (Hollnagel,
Woods, & Leveson, 2006). The first publication mentioning the term resilience engineering can
be traced back to 2003 (Righi, Saurin, & Wachs, 2015; Woods & Wreathall, 2003; Woods,
2003). In the publication of the first meeting, Woods and Hollnagel (Hollnagel, Woods, &
Leveson, 2006; p. 6) defined resilience engineering as: “resilience engineering is a paradigm
that focuses on how to help people cope with complexity under pressure to achieve success”.
Ten years later the Resilience Engineering Association (2016) defines it on its site as:
“Resilience Engineering looks for ways to enhance the ability at all levels of organisations to
create processes that are robust yet flexible, to monitor and revise risk models, and to use
resources proactively in the face of disruptions or ongoing production and economic pressures”.
Main concerns were initially about the understanding what resilience was, and how it occurs in
actual settings. An important theoretical model for understanding resilience is the positioning of
the operating state with reference to system boundaries, where “crossing is irreversible, an error
or accident may occur” (Cook & Rasmussen, 2005; Rasmussen, 1997, p. 189). Rasmussen
(1997) mentioned three boundaries, economic, workload and safety, where each of these puts its
own pressure type on the operating state: efficiency, least effort and safety. It is the behaviour
around the boundaries where the resilience of the system expresses itself. The stress-strain
model (Woods, Chan, & Wreathall, 2014; Woods & Wreathall, 2008), an analogy from
materials sciences mapping external demand onto the material’s stress and system behaviour
onto the material’s strain, identifies different phases around the boundaries. The first phase is
decompensation with a window of opportunity for intervention and the second phase is graceful
degradation.
Righi et al. (2015; p. 144) found in their systematic literature review of resilience engineering
that most publications were about the theory of resilience (52%) and only 16% about safety
management tools, which focusses on the management and measurement of resilience. This last
A. W. Siegel 5
research area, out of six, is of importance for the research objective to develop theory and
models using real-time data to describe the sociotechnical system dynamics. The ability to
measure and quantify the sociotechnical system resilience from operational data would be an
important step. However, the field of Resilience Engineering has not yet developed sufficiently
mature measurement tools and metrics to quantify resilience. When narrowing down to
publications dealing with the measurement of resilience, only a handful were found, with none
of them reporting on real-time measurement of the resilience state during operations. Øien et al.
(Øien, Utne, & Herrera, 2011; Øien, Utne, Tinmannsvik, & Massaiu, 2011) designed a method
for developing resilience-based early warning indicators. It was based on seven organizational
factors, seen as characteristics of a resilient organization: management commitment, awareness,
preparedness, flexibility, just culture, learning culture and opacity. Indirect indicators were
sought to measure these characteristics, but were not based on real-time operational data. Woods
et al. (2013) devised a method for the selection of safety indictors, supporting the identification
of misalignments, overlap and false diversity among metrics. The method uses two dimensions
to plot indicators: reactive-proactive (x-axis) and economy-safety (y-axis). This balancing
economy-safety trade-offs framework is seen as a component of a system resilience measure.
Again, the operational indicators used are not based on real-time operational data. Van der Beek
and Schraagen (2015) developed a questionnaire, an off-line tool, to assess the resilience state of
a team in operation. Being an off-line tool, a questionnaire obviously is not suitable to provide
real-time measurements of the primary process. The lack of real-time quantification of the
concept of resilience proves its difficulty. It is not easy to estimate the activation of hidden
capabilities in the future with unknown situations. This brings us to the first overall research
question: (How) can indicators of resilience be measured during real-time operations? Chapters
2 and 3 will give an answer to this question by describing a resilience model of a rail
sociotechnical system enabling a form of quantification used throughout this dissertation.
6 A. W. Siegel
3.2. Safety management - from Safety-I to Safety-II

Erik Hollnagel (2014) applied the concepts of resilience to safety management to introduce a
new paradigm, called Safety-II. His starting point, Safety-I, is the safety management paradigm
achieving the result of today, but with limitations for the future. The Safety-I philosophy aims to
eliminate risks, by searching and analysing them to take appropriate actions. This approach is so
successful that very low failure rates are achieved. However, the flip side of the coin is that the
effort locating issues becomes larger. Moreover, the complex interrelationships in modern-day
sociotechnical systems add significantly to the reliability sum of all separate components. The
basic approach of making each component reliable is not sufficient for the reliability of the
whole system. New methods like FRAM (Hollnagel, 2012), the functional resonance analysis
method, enable modelling of these complex interrelationships. Safety-II is an answer to these
drawbacks complementing Safety-I. Safety-II aims to focus on what goes right, being the
situation most of the time. Finding out what the adjustments are to cope with the current daily
work “can be more important than finding the cause of infrequent adverse outcomes!”
(Hollnagel, 2014, p. 149). Figure 1 depicts the relationship between resilience, safety in general
and the Safety I and II paradigms. Safety-II is related to resilience having a broader scope of
everyday actions and outcomes, while Safety-I focuses mainly on incidents and accidents.
Safety-II is a very good basis for learning about the real world. Three conditions must be
fulfilled in order for learning to take place. First, there must be an opportunity to learn. Second,
situations or events must be similar in some way. Third, there must be an opportunity for
feedback to verify that what has been learned is relevant and effective. The daily adjustments
fulfil these conditions. They happen continuously, have similarities and provide opportunities to
readjust as a form of feedback. However, Hollnagel admits that these frequent events are
difficult to see. He suggests a few analysis methods to identify and understand these events, but
does not clarify how to operationalize them. The interview and field observations are suggested
as a way to find out how people really do their work. Other methods are mentioned through
workshops, culture change and learning of passive competence, which again are difficult to
operationalize. The second overall research question deals with this gap between theory and
operationalization: How can the Safety-II concept be operationalized during real-time
operations? This question is addressed in chapters 4 and 5 by describing a method to be applied
by control room operators.
A. W. Siegel 7
FIGURE 1 Relationship between Safety-I & Safety-II (Hollnagel, 2014, p. 148, reprinted with
permission of the publisher)
3.3. The rail signaller

The job of the rail signaller has changed drastically in the last century (Kichenside & Williams,
1998), while in essence the function remained the control over train movements to ensure that
trains operate safely, over the correct route and according to the proper timetable. In the
beginning of the last century, the rail signaller, then also called ‘signalman’, operated rail-
switches and signals from a decentralized network of control points. Originally, all signalling
was done by mechanical means. Switches and signals were operated locally from individual
handles, requiring the signalman to walk between the various pieces of equipment to set them in
the required position for each train that passed. The control with the complex interlocking
mechanics, the term for the technical apparatus used to control switches, signals and block
systems, was concentrated in one building (left picture in Figure 2). With the introduction of
electrical relay logic, the mechanical operation was replaced by electronical means, simplifying
the signalling work. Initially, the electronics replaced the mechanical interlocking of a single
switch, keeping the responsibly of setting a whole route with the signaller. This routing function
was eventually executed by the NX (Entrance-Exit) system (middle picture of Figure 2). The
interface of the NX system was a large control panel on the wall with a track diagram. This
diagram displayed all rail-switches, signals, and rail sections (blocks) to be occupied by one
train only. The signaller had to push a button of the entering route and a button of the exit route.
All the rail-switches within the route were automatically set to the required position and,
provided all the switches were detected by the interlocking in the correct position, the related
signals would clear and the section would be available for another train. When needed, each rail-
8 A. W. Siegel
switch could be set manually as well. The signaller managed routes of areas according to a fixed
planning schedule and deviations were communicated by phone. At complex large stations the
operation of the NX was done by two persons. One person would have the overview and
decided on the route, while the second signaller was pushing the appropriate buttons. This
situation lasted approximately until the 1990s.
Automation was the next big step influencing the responsibilities of the rail signaller.
Automation replaced the functionality of the NX, setting complete routes, and did so according
to a given plan, managed by the signaller. The control areas were confined to large rail posts or
control centres. This step motivated much rail human factors research, a fact that has been
overlooked, when compared to the attention devoted to aviation (cockpit and air traffic control)
and road traffic (Wilson & Norris, 2005). When automation was introduced in the Netherlands
in the ‘90s, the rail signaller was heavily occupied solving problems of the system, especially
when trains did not drive according to the pre-planned schedule. Nowadays, the automation is
doing most of the work, where the signaller mainly needs to monitor and act when incidents
occur. Since the automation is performing better, he is most of the time monitoring. This
influences his workmanship, especially when the unexpected occurs. New processes are needed
to develop his skills while monitoring, to be used occasionally. The third and final overall
research question deals with these processes: Which skills of the operator need to be developed
due to increasing automation and the need to act when automation fails? In this dissertation, we
seek for these new processes, as described in chapters 4 and 5, which imply the need for rail
signallers to develop their skills in order to enhance the resilience of the rail sociotechnical
system.
FIGURE 2 Function change of the rail signaller (downloaded from

http://www.nicospilt.com/index_seinhuizen.htm)
A. W. Siegel 9
4. Research approach and overview of the thesis

The approach of this research is to develop a new theory and method to view the behaviour of a
rail sociotechnical system through the lens of resilience engineering. This view will enable
operators to see new aspects of the sociotechnical system they were not able to see before, and
to develop around this vison a new process to enhance resilience of the system. Exploration and
verification in all stages of the research have a naturalistic methodology through observations in
real rail operations (Woods, 2003). This methodology suited the search for the unknown weak
signals in real life and the interaction with them of rail signallers during their daily work. The
naturalistic approach exposed the real world and supported the development of theory and
design to cope with.
In Figure 3 the overall approach and the structure of this dissertation are depicted. Chapter 1,
this chapter, introduces the research by stating the problem domain and defining the research
objective. Three background areas, resilience engineering, safety management and the rail
signaller, are described arriving at gaps to be addressed within the research. Chapter 2 is dealing
with the new view by introducing the construct ‘weak resilience signal’. First, a generic
approach is taken based upon the boundary concept of Rasmussen (1997) identifying three
domains of importance for the resilience of a rail sociotechnical system: workload, performance
and safety. Subsequently this approach is applied to workload. An observational study at a rail
control post is next described using measurement methods, which were logged off-line and
analysed to explore and verify. Chapter 3 uses the theory and methods from chapter 2 and
applies them to the performance domain. Also, an observational study was carried out resulting
in an analysis of logged off-line data. Chapter 4 shift its focus to the operator using the new
view. A process was developed for the operator to use the view and made specific for the
performance domain. An observational study in real operations used real-time tooling for the
operators to employ the new process in their actual work practices. Chapter 5 reports on a study
incorporating the whole concept covering all the domains described in chapter 2. Again an
observational study was performed at a rail control centre with rail signallers employing the new
process with real-time tooling. Chapter 6 summarizes and discusses the results with reference to
the research objective and the gaps mentioned in the introduction
The format of this thesis is a collection of articles. Chapters 2 to 5 are reprints of papers that
were published or accepted for publishing elsewhere. References to the respective published
papers are footnoted on each chapter title page. The papers are unchanged, except for the layout
and some grammar and spelling issues. This implies that each chapter can be read
independently, however each publication refers and builds on the previous publication. The
chapters have a logical development in time, except for a part of chapter 2. This chapter
comprises two articles. The first one describes the foundations of the research and focuses on the
workload domain. The second article deals with workload as well, but was a sub-set of the last
10 A. W. Siegel
field study where the whole concept was tested. It complements the workload results of the first
article.
FIGURE 3 Research structure of the thesis with chapter numbers
A. W. Siegel 11
12 A. W. Siegel
WEAK RESILIENCE SIGNAL -

WORKLOAD
A. W. Siegel 13
14 A. W. Siegel
CHAPTER 2
WEAK RESILIENCE SIGNAL -
WORKLOAD
I. Measuring workload weak resilience signals
at a rail control post1
Occupational applications
This article describes an observational study at a rail control post to measure workload weak
resilience signals. A weak resilience signal indicates a possible degradation of a system’s resilience,
which is defined as the ability of a complex socio-technical system to cope with unexpected and
unforeseen disruptions. A method based upon a weak resilience signal framework introduces a new
metric, stretch, to measure the signals. Stretch is a subjective or an objective reaction of the system to
an external cluster event and is an operationalization of variables in an earlier stress–strain model.
The stretch ratio between the subjective and objective stretch are used to identify workload weak
resilience signals. Weak resilience signals identified during real-time operations revealed obstacles
that influence the resilience state and enabled actions to anticipate and mitigate changes to maintain
the resilience of the system.
Technical abstract
Background: Continuous performance improvement of a complex socio-technical system may result
in a reduced ability to cope with unexpected and unforeseen disruptions. As with technical and
biological systems, these socio-technical systems may become “robust, yet fragile.” Resilience
engineering examines the ability of a socio-technical system to reorganize and adapt to the
unexpected and unforeseen. However, the resilience doctrine is not yet sufficiently well developed for
designing and achieving those goals, and metrics are needed to identify resilience change. Purpose: A
new approach was explored to identify changes in the resilience of a rail system around the workload
boundary to anticipate those changes during normal operations and hence improve the ability to cope
with unexpected and unforeseen disruptions. Methods: A weak resilience signal framework was
developed with a resilience-state model for a railway system, resulting in a generic, quantifiable,
weak resilience signal model. Two workload measurements (i.e., external cognitive task load and
integrated workload scale) were combined into a new metric called stretch. Heart rate variability was
used for correlation and validation. An observational study was used to measure workload weak
resilience signal through workload quantification at an operational rail control post. Results: A
theoretical resilience-state model for a railway system was developed and used to generate a generic
quantifiable weak resilience signal model, forming a weak resilience signal framework that is the
basis for a method to measure workload weak resilience signal through a new metric called stretch
with three variations: objective stretch, subjective stretch, and stretch ratio. A component of the
subjective stretch is the integrated workload scale, for which a real-time tool was developed for
measuring and monitoring. Workload weak resilience signals identified at a rail control post triggered
analysis to reveal anticipated obstacles. Conclusions: A resilience-state model of a rail system can be
used to quantify workload weak resilience signals. Stretch ratio differences represent changes of the
1
Published as Siegel, A. W., & Schraagen, J. M. C. (2014). Measuring workload weak resilience signals at
a rail control post. IIE Transactions on Occupational Ergonomics and Human Factors, 2(3–4), 179–193.
http://doi.org/10.1080/21577323.2014.958632
A. W. Siegel 15
workload state used to measure workload weak resilience signals that aid in revealing obstacles
jeopardizing the resilience state.
16 A. W. Siegel
1. Introduction
The continuous performance improvement of a complex socio-technical system may necessarily
result in a more limited ability to cope with unexpected and unforeseen disruptions. Just as
found with technical and biological systems, these socio-technical systems may become “robust,
yet fragile” (Alderson & Doyle, 2010, p. 839). Resilience engineering investigates, among other
aspects, the ability of a socio-technical system to reorganize and adapt to the unexpected and
unforeseen (Hollnagel et al., 2006). However, the resilience doctrine is not yet sufficiently well
developed for designing and achieving these goals (Madni & Jackson, 2009). An important step
to account for the resilience of a system is information on its resilience state. The resilience state
has been described through theoretical models but so far lacks solid quantification. Woods,
Schenk, & Allen (2009) describe some of these models and compare them with each other. The
Ball and Cup model (Scheffer, Hosper, Meijer, Moss, & Jeppesen, 1993), for example, is aimed
at the system steady state that presents boundaries after which another steady state or system
break-down occurs. However, this model does not have the ability to explain potential
adaptations that may occur around the boundaries.
In another approach, the Stress-Strain (S-S) model (Woods & Wreathall, 2008) takes its analogy
from materials sciences, by mapping the external demand onto the material’s stress and the
system behaviour onto the material’s strain. The S-S model focuses on behaviour near the
boundaries explaining system degradation, system restructuring, and system transitions, which
are potentials that need to be managed during challenging stress events. Woods, Chan, et al.
(2013) extended the Stress-Strain model further to operationalize four cornerstones postulated to
be essential to resilience: anticipating, monitoring, responding, and learning (Hollnagel, 2009),
and introduced regions for base and extra adaptive capacity. The region for base capacity
represents the “normal” functioning of the system to external events. The region for extra
adaptive capacity represents the potential for adaptive shortfalls to arise where responses cannot
match the demands of challenging events that fall near or beyond the boundary area of the base
envelope. These regions explain the behaviour of the system beyond the base envelope, however
they do not provide a means to measure the properties in the extra adaptive region. Furthermore,
the behaviour in the extra adaptive region is a hidden capacity to react to unforeseen
disturbances. An objective of this paper was thus to develop a method to measure properties in
the base capacity region, which signal changes of properties in the extra adaptive region. This
objective makes quantification possible, and provides clues that can be analysed and interpreted
by human operators about aspects of the hidden capacity.
We introduce the concept of weak-resilience-signal (WRS), which we will use to quantify

changes of the resilience state. We define WRS as signals indicating a possible degradation of
the socio-technical system’s resilience that can be traced to its original cause. We contrast a
weak resilience signal with a strong resilience signal, the latter being a clear signal that the
A. W. Siegel 17
resilience of the system has degraded and which should be considered as an alarm triggering a
relevant action. This comparison also emphasizes that a WRS is not an alarm but rather a trigger
of interesting information about the system state. A weak signal in this context can be seen as
analogous to a human feeling some chest pains during daily activities. When investigating this
signal, he may conclude that this is just a spasm or a serious problem with the heart that would
only be evident at the time of a large effort.
A weak signal measuring a minor issue during nominal operations may be a crucial factor of
failure. Dekker (2011) goes even further, theorizing that the accumulation of an unnoticed set of
events is the main cause of the incubation of and surprise at failure. The weak signal can also be
explained through the Stress-Strain model (Woods et al., 2014), in which changes occur in the
base adaptive capacity such as a change in the Young's modulus slope (Woods & Wreathall,
2008), the linear relation between stress and strain. A slope change in the base region indicates a
creeping failure to be exposed at a large stress. Only collecting many detailed weak signals
would not necessarily result in a corrective action in response to a specific signal. It may cause
fatigue or vigilance (Davis & Parasuraman, 1982), and due to many irrelevant weak signals,
which do not need any action, it could cause a "cry wolf" (Breznitz, 1984) effect. Therefore, the
WRS needs an extra set of properties to account for the above. First, it needs to be an
aggregation of a lower/detailed weak signal set, to lower the number of signals, and second, the
aggregation needs to be of interest to the operators to understand the behaviour of the system
beyond resilience. These are "sending" properties of the WRS. Yet, a "receiving" property of the
rail sector is also needed to expand its culture from "working by virtue of many rules and formal
agreements" (Top & Steenhuisen, 2009) to an inquisitive one of understanding, tracking, and
anticipating the relevant weak resilience signals.
In this paper, we focus on a framework for rail weak-resilience-signal (WRS) modelling and we
emphasize one main area - workload - for which we develop a specific method to measure a
workload WRS at a rail control post. We verify and validate this method in real operations
through an observational study during a reorganization of a rail control post. Our research
questions were twofold: 1) How can a weak resilience signal (WRS) be modelled to enable its
quantification and be demonstrated in the area of workload in real operations? 2) How can
workload WRS be measured and utilized at a rail control post? The remainder of this paper is
structured as follows. In section 2, we develop a framework for rail WRS modelling and
describe mathematically its generic quantification. In section 3, we describe a method to
measure workload WRS at a rail control post. Section 4 describes the observational study we
carried out during two separate weeks at the rail control post. We conclude the paper with the
results of the observational study (section 5) and a discussion (section 6).
2. A framework for rail weak resilience signal modelling
18 A. W. Siegel
2.1. Theoretical resilience state model for a railway system

A theoretical model describing the resilience state of a railway system is needed to: (1) better
understand in which areas weak resilience signals (WRS) are to be sought; and (2) provide a
foundation upon which a quantitative model of a WRS can be built. Rasmussen's (1997) safe
operating envelope was used as a starting point since it uses three boundaries – performance,
economy and workload – to describe the envelope of a generic socio-technical system operating
in an economic environment. That model described the various pressures on the Operating State
(OS) that may result in crossing one of the borders or readjusting the border to create a new
steady state. This readjustment is actually resilience, which is defined by the capacity to adapt to
unforeseen events (Hollnagel et al., 2006). In Rasmussen's framework, the performance
boundary is directly linked to safety culture pressure, the economic boundary is linked to
efficiency pressure and the workload boundary is linked to least effort pressure. In our
adaptation of Rasmussen's model, we have introduced some changes to reflect the nature of a
railway system. First, we separated performance from safety to reflect their independent nature,
while their mutual influence on the operating state is made explicit in the new model by
upgrading safety to a boundary entity, which creates safety pressure. Second, we moved the
economic boundary backwards, thereby creating efficiency pressure on the performance
boundary, which in turn creates a performance pressure. This change is justified by the fact that
in rail systems, economic considerations play a more prominent role in the long run than in daily
decisions. However, the performance pressure, created by capacity growth and punctuality to
deliver the planned schedule, plays a major role in daily considerations. The workload boundary
stays intact, reflecting the human importance within a socio-technical rail system, and the result
of these changes is shown in Figure 1(section I).
The above model is considered useful when reasoning about resilience. For example, Cook &
Rasmussen (2005) use different areas in the model to explain the stability of a system: unstable,
low-risk stable, and high-risk stable. The fact that the boundaries put pressure on the Operating
State (OS) is indicated textually with the term "gradient", and grey areas show the OS jump
domain that is due to shallow gradients. These gradients are of interest, since they represent the
internal pressure on the OS, and may be indirectly measured and can help explain the resilience
of the system when the OS is located at any position between the boundaries. When a gradient is
steep, it represents system resilience against external perturbations, while shallowness represents
brittleness. As described by Woods et al. (2009), who related the work of Walker & Holling
(2004) to that of Rasmussen (1997), this gradient can be made explicit by adding a depth
dimension to Rasmussen's model as if it were viewed from above in a landscape of valleys. The
slope (α) of the valley (see Figure 1 section II) describes the internal force gradient (or
Resilience Engineering as in Walking and Holling, 2004) acting on the OS. The vector ⃗ d
describes the external perturbations acting on the OS, while d P=d·CosαP represents the pressure
of boundary BP. This third dimension with the valley slope is important to understand the level
of resilience when moving towards one of the boundaries. A shallow slope is analogous to a
A. W. Siegel 19
small hurdle, representing brittleness, to approach the boundary, while a steep slope represents
resilience. As an example, Figure 1 section III shows an OS that is moving towards the marginal
boundary, a boundary to guard the safety boundary. There are two options to reflect the change
of the internal state. When only the capacity of the system is increased and no safety measures
are taken, this will result in a brittle state, option a, in which the marginal boundary risks being
crossed. However, when measures are taken to also enlarge the safety hurdle, as in option b, it
may result in a deeper valley, thereby maintaining the resilience engineered to cope with a
higher capacity. This theoretical model will be used in the following subsection to model
quantifiable weak resilience signals (WRS) through pressure change acting on the OS near the
boundaries.
FIGURE 1 Resilience state model for a railway system

Section I: Rail-sector boundaries putting pressure on the Operating State (OS)
Section II: Rail-sector boundaries with resilience slope α P, causing pressure dP
Section III: OS move caused by internal change, a or b, influencing system resilience
2.2. Generic quantifiable weak resilience signal model

Assuming an internal pressure αB on boundary B , caused by a certain phenomenon described
through a function fB of n measurable parameters, PiB, can be expressed mathematically as:
α B =f B ( P iB , i=1 , … , n ) (SEQ equation ¿ ¿ MERGEFORMAT 1)
20 A. W. Siegel
When assuming small changes, the pressure change ΔαB can be estimated by the cumulative
weighted changes of the function parameters PiB :
n
∆ α B =∑ ( K iB ∙ ∆ PiB ) ,i=1 , … , n(2)
i=1
Or, as the change of two moments in time, t1 and t2:

n n
∆ α B =∑ K iB P iB ( t 1 )−∑ K iB PiB ( t 2 ) , i=1 , … ,n (3)
i=1 i=1
A weak resilience signal WRSB is created when it is smaller than a Threshold-WRSB , which is a
negative value since by definition a larger αB represents a growing resilience (as in Figure 1):
WRSB: ΔαB < Threshold-WRSB < 0 ( 4)
where the weights KiB ; i=1,...,n and Threshold-WRSB are defined by empirical investigation in
which KiB is used to set the relative proportion of influence among the parameters on the
pressure αB , and may be set initially to 1. Threshold-WRSB is a way to search for a level at
which attention is needed for deeper analysis. A possibility to define Threshold-WRSB is the
added standard deviation of the measurements at t 1 and t2 to make the difference significant, or it
may be set to a value reducing the number of WRS B‘s to the most significant ones. It may be
possible that instead of a hard threshold, a graphical representation, such as a continuous graph,
will be chosen for monitoring by the rail signaller. However, the crux of this model is choosing
the phenomenon that is described by fB. As explained in the Introduction, this phenomenon
needs to cover many possible WRSs and must be chosen in such a way that it is of interest to the
signallers independently of the signals occurring. The following section gives an example of
such a phenomenon worked out with respect to the workload boundary. We assume that passing
the workload boundary with a certain threshold implies a possible degradation of the system
resilience. This is in line with Woods & Patterson (2000), who claimed that unexpected events
produce an escalation of cognitive demands. When cognitive workload change is significant and
identified, it is a signal that the resilience of the system is reduced, due to the reduction of the
spare cognitive capacity, and which may be needed when the unexpected event occurs. There
are two period types of passing the boundary. A short period passage is a real-time signal for
operations to respond to by an intervention. Passages in a long period indicate a possible
structural change to be addressed. With an empirical study we will show the usage of parameter
settings and validate the model with the results through observation.
3. A method to measure workload WRS

at a rail control post
A. W. Siegel 21
Workload measurement methods have been studied extensively (Gao, Wang, Song, Li, & Dong,
2013; Pickup, Wilson, Nichols, & Smith, 2005; Pretorius & Cilliers, 2007; Veltman & Gaillard,
1993). Different factors influence mental workload, such as time, mental tasks, physical tasks,
and stress (Xie & Salvendy, 2000), which makes it clear that one measurement type will not
cover all aspects. Veltman & Gaillard (1996) reason that the measurement of mental workload
needs performance, subjective, and physiological data for a complete understanding of
workload. We suggest using three different measurements: 1) external cognitive task load
(XTL), 2) subjective workload, and 3) heart rate variability to identify arousal created by
workload. To compose the XTL, we built upon Neerincx’ (2003) model of cognitive task load
(CTL) in three dimensions: task complexity, task duration, and task switching. The XTL is
defined specifically to the rail control situation and to parameters that are available in real-time.
The real-time aspect, of all the measurement components, provides possibilities to set up
experiments to close the loop throughout operations. Rail signallers’ task execution can be
divided into four main activities (see Figure 2), which are measurable within the system: 1)
monitoring (Mon), 2) plan mutations (Plan), 3) manual actions (Man), and 4) communication
(Com). Monitoring is keeping track of trains and infrastructure through observation of system
displays. Plan mutations refer to activities concerning the logistic plan, which is the basis of
train movements on the infrastructure as agreed among all parties and used by system
automation. Manual actions are activities performed directly on the infrastructure, like setting a
switch, instead of system automation according to the plan. Telephone calls, with external
parties, are the main communication task. We assumed that monitoring is in proportion with
automated activities executed by the system. This assumption refers to imposed task load, while
in reality the rail signaller can actually ignore the monitoring task. Monitoring can thus be
measured by counting all the automated activities. These activities were counted in 5 minute
base-slots, used throughout all the types of measurement for ease of comparison. We normalized
these counts by dividing them by the maximum count (Mon max) occurring throughout a test
period, causing the measurement to be normalized between 0 and 1. This same idea was applied
to normalizing the plan mutations and the manual actions. Each of these were counted within the
5 minute base-slot and divided by the maximum count, Plan max and Manmax respectively,
throughout a test period. The communication normalization was done differently.
Communication was defined by the percentage of verbal exchanges over the phone, which is
measureable, during the 5 minute base-slot. A rail signaller talking the whole 5 minutes, results
in a 100% communication value.
22 A. W. Siegel
FIGURE 2 Task flow of a rail signaller at his workstation
The combination of these four normalized activities refers to task complexity as stated by
Neerincx (2003). However, Neerincx used the Skill-Rule-Knowledge (SRK) model (Rasmussen,
1997) to express task complexity by rating each task on its SRK cognition load level. Since we
do not know the cognitive relationship among the tasks, we multiplied each with their relative
task complexity constant (Kmon, Kplan, Kman, and Kcom ), and tracked their identity throughout the
whole process. In addition to these activities, task switching and task duration are two extra
dimensions amplifying the workload. To estimate the number of task switches, we examined the
task activations and counted them in each time slot as long as they were activated, to reflect the
task duration. In Figure 2, we list the task activations imposed on a particular workstation. These
activations resulted in the activities discussed above and resulted in workload we measured by
XTL, IWS and HRV.
Since the analysis is based upon log-data, we can search for the maximum number of activations
occurring in the 5 minute base-slots. We divided the number of activations, occurring in the 5
minute base-slot, by the maximum activations occurring throughout the test period to achieve a
normalized switching factor between 0 and 1. Task switching and duration are a cognitive add-
on to the activity load. With the same activity load, 0 to n parallel task switches can occur,
behaving like a cognitive amplifier to the activity load. We added one to the normalized
switching factor to act as a cognitive amplifier by becoming a growth multiplier of the activity
load. Graphically, the multiplication will show jumps attracting the attention needed for
interpretation. Thus, the switching factor becomes:
number of activations∈5 min base slot

K switch = +1( 5)
maximum number of activations∈5 min base slot
A. W. Siegel 23
We calculated the task complexity load with the sum of the four normalized tasks, each
multiplied with their relative task complexity constants: K mon, Kplan, Kman, and Kcom. These
constants are initially set to 1 and may be adjusted proportionally during empirical investigation,
but keeping their sum to the initial value of 4 and only changing their interrelationship. We
multiplied the task switching factor with the task complexity load to achieve a combined XTL
number. This approach creates a number between 0 and 8 to be used as an overall graphical
indication on the XTL magnitude and change. Maximum load due to task execution is 4 X 1 = 4,
multiplied by a maximum switching factor, 2 X 4 = 8. However, it is important to present all the
components and their relationships separately to understand the situation.
The XTL calculations can be performed for workstation WS with its subscripted WS values
using:
(
XTL WS=K switch WS K mon
Mon WS
Monmax
+ K plan
PlanWS
Plan max
+ K man
Man WS
Manmax
+ K com ∙ ComWS (6)
)
Subjective load measurement can be divided into two categories: multidimensional and
unidimensional scales. Multidimensional scales, such as the NASA-TLX (Hart & Staveland,
1988), explicitly represent the dimensions of workload and allow ratings to be obtained from
each dimension. Unidimensional scales (Muckler & Seven, 1992) represent the concept of
workload as one continuum. Hendy, Hamilton, & Landry (1993) claim that a univariate rating is
expected to provide a measure that is at least as sensitive to manipulations of task demand as a
derived estimate from multivariate data. In addition, a unidimensional scale is easier to use and
in our case easier to automate for real-time purposes. Pickup, Wilson, Norris, Mitchell, &
Morrisroe (2005) have developed a unidimensional scale specifically for rail signallers, called
the Integrated Workload Scale (IWS). They have automated the IWS tool for usage of the trial
facilitator for a few-hour period. Our aim was to let the rail signaller assess and enter their own
rating for 24 hours a day. We developed a Java-tool that can run within the operational system
to be seen as part of their routine work. The rail signaller RS i, working at work station WS j, was
alerted every 5 minutes by a peripheral blinking rectangle, to rate their subjective workload.
They were presented with a 9 scale figure containing the following text (from the original
Dutch) (see Figure 3): (1) Not demanding; (2) Minimal effort; (3) Some spare time; (4)
Moderate effort; (5) Moderate pressure; (6) Very busy; (7) Extreme effort; (8) Struggling to
keep up; and (9) Work too demanding. The rail signaller had the option to add a comment to
their rating and received a graphic overview of their scoring.
24 A. W. Siegel
FIGURE 3 IWS application screenshot translated from Dutch (upper-right red rectangle blinked to
draw attention)
The extensively-researched heart rate variability (HRV) was used to identify physiological
arousal due to workload change (Billman & Billman, George, 2011; Goedhart, van der Sluis,
Houtveen, Willemsen, & de Geus, 2007; Hoover, Singh, Fishel-Brown, & Muth, 2012; Jorna,
1992; Malik, 1996; Togo & Takahashi, 2009). The HRV was mainly used to cross-check the
subjective measurement, and will be lower at a higher workload and identify IWS ratings that
are given due to other reasons than a higher workload. HRV was measured with a commercial
device (Zephyr HxM BT) that was positioned on a chest strap and transferred data to a laptop
near each workstation. A signaller wore the device at the start of their work. The device sends
continuous strings with recorded R-R intervals in msec. HRV can be calculated in various ways,
roughly divided into time domain and frequency domain methods (Malik, 1996). We used the
most common occupational health method (Togo & Takahashi, 2009), SDNN, the standard
deviation (SD) of all normal-to-normal (NN) intervals, from the time domain. We calculated the
measures in the same 5-minute base-slot used for the calculations of XTL and IWS.
A. W. Siegel 25
The three measurements described above, XTL, IWS and HRV, are all measured in 5 minute
slots. This timeslot enables comparison of the measurements in a timeline, as Pickup, Wilson,
Norris, et al. (2005) did to validate IWS. We did this for validation of IWS through the HRV,
but it is not sufficient for the analysis of events taking much longer than 5 minutes, which is the
case in the rail environment. Serious events take more than half an hour, as can be seen in the
results section. To compare the XTL and IWS, they should be referenced to a time frame of
events, clustered from and to a steady state. The steady state of a rail control post is the state
when the train activities are occurring as planned, without any intervention. In order to relate the
IWS and XTL measurements, a new metric was introduced – Stretch (see Figure 4).
3 5
Objective Stretch Subjective Stretch

4
2
3
IWS
XTL
2
1
1
IWS
baseline
0 0
Time (Hour:Minute)
XTL IWS
FIGURE 4 Defining Objective and Subjective Stretch from XTL & IWS over time
A Stretch is the cumulative workload effort during a period initially defined by IWS rising from
a baseline until it returns to the baseline. The IWS-baseline is defined as the steady state IWS
rating before and after a disruption. However, the activity in the system may have started earlier
and ended later. Therefore, the starting moment of a Stretch is adjusted to the first XTL-
minimum moment before the IWS rising. Similarly, the ending moment of a Stretch is adjusted
to the first XTL-minimum moment after the IWS return. In other words, a Stretch is the reaction
to an external cluster-event. We use the term cluster-event, since more than one event may occur
during a stretch. An Objective Stretch is the name of the area under XTL, since it is objectively
measured. We name the area under IWS a Subjective Stretch, due to its subjective IWS rating.
The ratio of Subjective Stretch and Objective Stretch is called Stretch-ratio, which is used to
identify a workload WRS. These terms are better related, than the measurements, to the Stress-
Strain (S-S) model (Woods et al., 2014; Woods & Wreathall, 2008) and the resilience state
26 A. W. Siegel
model, developed in the previous section. The objective Stretch is related to the Stress axis of
the S-S model. Stress is the theoretical concept of the demand of the system through Challenge
events. The objective Stretch is the operationalization of the Stress concept through measuring
the factual reaction of the system. The subjective Stretch is the human perception of the system
Strain. The Stretch-ratio relates to αB of the workload boundary (αworkload-boundary), the internal
pressure on the workload boundary, of the resilience state model. When a growing change of a
Stretch-ratio is identified, larger than a threshold, and the Stretch values are larger than a pre-
defined value, a weak resilience signal (WRS) is generated. When comparing two periods, the
accumulated standard deviation (SD) of the Stretch-ratio in each period, can function as the
threshold, indicating a significant change. However, such a principle needs to be validated in
empirical testing. A larger Stretch-ratio during a given period, compared to a baseline period,
indicates more subjective workload in response to similar external events. The Objective Stretch
is used to identify an absolute workload growth, throughout a specific period like a day or a
workweek.
4. Observational study during rail operations

To validate and verify the applicability of the method to measure workload WRS at a rail control
post, we applied it throughout the restructuring try-out of a control post to improve its work
efficiency. In our specific case, the control post was restructuring only one group around a
corridor for a test period of half a year, by: 1) setting focus on a corridor by seating the corridor
team together; 2) splitting-up the responsibility of a rail signaller’s tasks to planning and safety
related activities by adding a planner to the team; 3) enforcing standardization through position
rotation; and 4) growing their expertise level through training as part of the position rotation.
This efficiency step can, however, affect the post’s spare, and sometimes hidden, adaptive
capacity needed when an unexpected disruption occurs. In addition, this efficiency step can also
affect the organization’s ability to manage this capacity. As improved work efficiency, may
conflict with an organization’s resilience due to common resource demands, methods are needed
to identify this potential conflict, which can be shown by a WRS. A rail control post is
responsible for a large area containing railway stations, controlled by rail signallers managing
the traffic on the rail infrastructure. The post we studied is active 24/7 with 10 to 20 rail
professionals. A rail control post is an example of a socio-technical system due to the critical
human-system interaction.
The generic setting is a rail control post with m Post workstations and nPost rail signallers
evaluating a new organizational form to increase their performance. Each workstation, WSj, is
allocated to a set of railway stations and operated by one rail signaller, RSi, who is responsible
for all the workstations’ aspects. These aspects are roughly divided into logistics and safety, and
the workstations are split into two groups. The first group, G T, is the target group that will
reorganize, as described above, to improve its performance. The second group, G R, is the
A. W. Siegel 27
reference group that will not reorganize throughout the testing period. All the n Post rail signallers
of the control post may be allocated to each of the groups and to each of its workstations. In
group GT there are mT workstations, and in group G R there are mR workstations. In addition,
there is a calamity workstation, WScal, which is added to give support to the workstation being at
the core of a calamity. The calamity workstation, which is not related to the reorganization, can
be added to each group, GT or GR. The setting is depicted in Figure 5:
FIGURE 5 Rail control-post setting with observer O
In our case, we carried out structured observations at a Dutch rail post with 44 participating rail
signallers (nPost=44), during two periods of one working week (Monday until Friday). The age of
the participants ranged between 23 and 64 years, with a mean of 43.6 years, and the population
contained 79.5% males. All of them rated their subjective workload with the IWS tool, though
39% consented to wearing a heart rate sensor during their work. The work experience varied
between 0 and 37 years, with mean 17.6. The first measurement period was immediately before
the reorganization of the target group, and the second measurement period was two months
afterward. In the first period, measurements were recorded in two shifts from 7AM until 9PM
with the IWS tool on a separate laptop near each workstation. During the second period, the
measurements were recorded continuously, 24 hrs a day, with the IWS tool integrated within the
operational system (see Figure 6). Initially, there were three workstations at the target and
reference group (mT = mR = 3). After the reorganization, one workstation was added to the target
group (mT=4), for planning activities of the corridor. The protocol guiding the observations was
28 A. W. Siegel
approved by the ethical committee of the University of Twente, except for its request to obtain
written consent by participants, which was replaced by oral consent by each participant at the
request of Post management.
FIGURE 6 Integration of the IWS tool within operations
5. Results
The quantitative results of the Stretch measurements before and during the reorganization are
summarized in Table 1. Before the reorganization, the mean Stretch-ratio of the target group was
5.30 [IWS/XTL] with a standard deviation (SD) of 2.61. The mean Stretch-ratio of the reference
group was 5.82 [IWS/XTL] with and SD of 2.55. Since the standard deviations were large, and
the means were similar, we may conclude that the Stretch-ratio of both groups were in the same
order of magnitude, indicating the similarity of work in both groups. The duration of the Stretch
varied substantially. This can be seen clearly by comparing the Stretch with the Stretch divided
by its duration (Table 1: SS/Dt and OS/Dt), the latter representing the mean workload
throughout the Stretch. For example, the subjective Stretch of both groups before the
reorganization was 21.13 [IWS x min] with a SD of 15.60, whereas subjective Stretch divided
by its duration was 3.09 [IWS] with a SD of 0.80.
During the reorganization, a planner was added to the target group. The mean Stretch-ratio of
the planner was 11.83 [IWS/XTL] with a SD of 5.54. The reason the planner had a much larger
Stretch-ratio than the normal rail signaller is because their XTL was much lower since that
individual does less work. The planner had no monitoring task, no manual action task, and fewer
phone calls since they do not communicate with the train drivers. In contrast, the planner rated
IWS similar to colleagues, causing the Stretch-ratio to become larger. This could be solved by
adjusting the relative task complexity constants, which were initially set to 1, and give more
A. W. Siegel 29
relative weight to plan activities. However, more empirical research is needed in this area,
causing the existing Stretch-ratio to be valuable for comparison of similar tasks, but not yet
suitable to compare between different tasks. For that reason, we have added to the summary
table entries where the planner is excluded (Table 1: Target-planner and All-planner). The mean
Stretch-ratio of the Target group during the reorganization without the planner was 6.17
[IWS/XTL] with a SD of 2.81. The mean Stretch-ratio of the Reference group during the
reorganization was 6.36 [IWS/XTL] with a SD of 1.80. The Stretch-ratio for both groups
remained similar, but increased in the measurement week during the reorganization. The reason
for the increase can be found in the figures of the objective Stretch, which are lower during the
reorganization than before. Deeper investigation shows that fewer phone calls are the cause for
the objective Stretch reduction. In summary, in the measurement week during the
reorganization, no evidence was found that the reorganization significantly influenced the
workload adaptive capacity needed for system resilience.
TABLE 1 Stretch measurements over one week, both before and during reorganization (cells that
are not relevant for the line of argumentation are not filled in)
Group ## Stretch Stretch-ratio Subjective Stretch (SS) Objective Stretch (OS)
Mean SD Mean [IWS SD Mean(SS/Dt) SD(SS/Dt) Mean SD Mean(OS/Dt) SD(OS/Dt)
[IWS/XTL] x min] [IWS] [XTL x min] [XTL]
Before Target 35 5.30 2.61

reorganization
Reference 107 5.82 2.55
All (T & R) 142 5.69 2.57 21.13 15.60 3.09 0.80 4.28 3.58 0.62 0.26
During Target 170 7.37 4.24
reorganization
Target - planner 134 6.17 2.81
Reference 134 6.36 1.80
All (T & R) 304 6.92 3.42 21.17 24.30 2.75 0.59 3.49 3.82 0.47 0.21
All - planner 268 6.26 2.36 21.18 25.59 2.75 0.59 3.70 4.00 0.50 0.20
Another representation of the measurement results is a plot of the objective Stretch versus the
subjective Stretch before and during the reorganization (Figure 7). The two Stretch types are
highly correlated, with r (Pearson) = 0.90 before reorganization and 0.88 during reorganization.
Most Stretches in both weeks are small. We have drawn a threshold line with a Stretch-ratio of 9
[IWS/XTL], since the mean Stretch-ratio in the first week was 5.69 [IWS/XTL] with a SD of
2.57 (Table 1). A first threshold line would be the rounded sum of the means with one standard
deviation above (i.e., 6+3). It is the threshold, as explained in the previous section, which needs
to be set empirically to optimize the number of WRSs to handle. With this threshold, two weak
resilience signals during the reorganization need further investigation (WRS-1 and WRS-2,
labelled “1” and “2” in Figure 7).
30 A. W. Siegel
FIGURE 7 Objective versus Subjective Stretch in one week, both before (left) and during (right)
reorganization
WRS-1 has a Stretch-ratio of 14.11 [IWS/XTL], with a subjective Stretch of 163 [IWS x min]
and an objective Stretch of 11.55 [XTL x min], which are numbers for comparison of Stretches
in the given setting The WRS occurred on the first measurement day at Workstation-3, at 7:10
AM with a duration of 195 minutes, while performing shunting of rail material as the main
activity. The rail-signaller subjectively rated their mean workload during this Stretch as
“moderate effort” (4.17), which is higher than the mean IWS-rating (“some spare time” = 2.75)
of the whole group during the test week. The higher IWS-rating, combined with the long
duration of shunting activities, triggers further investigation or at least causes the tracking of the
shunting for a longer period to understand the phenomena and take appropriate actions. This is
an example of a WRS causing the identification of an obstacle, which could become a main
cause of incubation and surprise at failure, as stated by Dekker (2011).
WRS-2 has a Stretch-ratio of 9.16 [IWS/XTL] with a subjective Stretch of 211 [IWS x min] and
an objective Stretch of 23.03 [XTL x min]. The WRS occurred on the second measurement day,
at Workstation-3 at 8:40 AM, with a duration of 350 minutes, which again performing mainly
shunting of rail material. The rail-signaller subjectively rated their mean workload during this
Stretch as “some spare time” (3.01). Although the mean IWS-rating was lower than that of
WRS-1, the duration was much longer. This recurring shunting activity emphasizes the
importance of investigating the reasoning for the long periods. Such an investigation is an
example of actions taken as a result of a WRS.
The above results and reasoning give some confidence in the validity of the data, since they
correlate with the observations in both weeks. In both weeks, no special events occurred, and
both groups were able to cope with daily disturbances. The shunting issues of the WRSs were
recorded as well, and were caused by the three train companies, who had had extensive
unplanned rail materiel to be treated manually by the rail signallers. The reorganization did not
have a visible effect on the average disturbances. To further validate the data, we analysed the
work distribution, based upon the XTL components, and verified it as well with the
observations. In Figure 8, we have shown the work distribution of the Target group before and
during the reorganization. It is clear from the graphs that the extra workstation (WS 4) does
most of the planning, communicates less than the other workstations, and does not perform
manual or monitoring activities. These figures are consistent with the observations, where all
planning activities that were more than 10 minutes ahead were allocated to WS 4.
A. W. Siegel 31
FIGURE 8 Work distribution of target group before (left) and during (right) reorganization (mon =
monitoring ; plan = plan mutation ; man = manual action ; com = communication ; act =
activations)
In addition, HRV was been correlated to the objective Stretch. The following algorithm has been
applied to identify a lowering HRV during a Stretch. First, the highest value of the HRV on the
boundaries of its Stretch was marked. Then, this value was multiplied by the Stretch-duration
and the integral under the HRV throughout the stretch was subtracted. A negative value was
assumed to confirm the subjective Stretch by the physiological response. This algorithm was
applied to the data available in the week before the reorganization. A lower HRV was recorded
during 83% of the subjective Stretches, which is in line with the literature (Togo & Takahashi,
2009). This finding provides an additional means to evaluate Stretches passing the Threshold
boundaries.
6. Discussion
There is a need during real-time operations to quantify the system resilience state. Quantification
is challenging because, on the one hand, socio-technical systems are complex and non-linear
(Doyle & Csete, 2011), while on the other hand resilience is about hidden capacity that is
measured only during the response to such disruptions (Woods et al., 2013). Woods et al. (2013)
have made some progress in the quantification of resilience parameters by looking at the system
boundaries. This paper focused on the area of daily operations, seeking quantifiable weak
resilience signals (WRS) around the workload. The aim of this research was to show how a
WRS can be modelled, to enable its quantification and to demonstrate this in the area of
workload in real-time train operations. In addition, we wanted to determine whether, and how,
we can measure workload WRS at a rail control post and demonstrate how it can be utilized.
A WRS framework was developed and used to concretize a workload WRS at a rail control post,
specifically for the work of a rail signaller. The modelling was built up from specific types of
workload measurements adjusted to the rail context, resulting in three measurements: 1)
eXternal Task Load (XTL), 2) Integrated Workload Scale (IWS), and 3) Heart Rate Variability
32 A. W. Siegel
(HRV). The first two measurement results were merged into a new metric, Stretch, describing
the efforts during clusters of events occurring at the control Post. HRV measurement was used
for validation. The two variations, objective and subjective Stretch, are an operationalization of
Woods’ Stress-Strain (S-S) model variables (Woods, et al., 2013; Woods & Wreathall, 2006).
An objective Stretch is related to the stress on the system and the subjective Stretch is the human
response perception related to strain. Stretch-ratio is the relation between both Stretches and
relates to the slope of the S-S line. Stretch seemed to describe well the variations of the same
task set. However, more research is needed to tune the multiplying constants of the sub-tasks,
initially being set to 1 here, to compare with other task sets. For comparison of the groups here,
we have excluded the planner, who had a consistently larger Stretch-ratio than the others.
Overall, the Stretch gave a clear picture of the events occurring at the control Post and created
two Workload WRSs. These were analysed and triggered further analysis of the shunting
activities engaged in at workstation 3, and which is a concrete example of anticipation driven by
a WRS. Beyond this finding, there was no indication of a resilience reduction caused by the
reorganization. A longer period, with significant disruptions, is needed to understand the impact
of the reorganization on the workload resilience border and resilience as a whole. This longer
testing period can also contribute to validation of the workload WRS, since more WRSs will
occur that can be analysed and reveal other obstacles influencing the resilience state. In the
current testing, we have validated components of the Stretch against observations.
In summary, the Stretch, which is based upon the WRS theoretical and quantification model,
offers the ability to quantify a workload WRS. Such WRSs provide new means to measure the,
sometimes creeping, resilience changes. When analysed during operations, it creates awareness
of obstacles that can become a (main) cause of incubation and surprise at failure. This awareness
stimulates the anticipation to take actions in the period before the unexpected and unforeseen
external event occurs. In such a way, the hidden extra adaptive capacity is maintained and can
be utilized through the ability of managing this capacity. This will improve the performance of
the signallers. A future research step is to measure for longer periods and extend the specific
WRS modelling to the other two boundaries, Safety and Capacity. WRS coverage, the identified
percentage of obstacles compromising the resilience state, will be investigated as well. Our aim
is eventually to test and validate the contribution of the total WRS concept to managing the
resilience of the socio-technical rail system.
Acknowledgement
We are grateful for the hospitality of the ProRail control post at Zwolle, who gave us all the
freedom for this research and were willing to use our experimental tooling. We thank Jaldert van
der Werf for his development of the IWS and analysis software tooling, and his contribution to
the observational study. We appreciate the guidance by Alfons Schaafsma. This research was
A. W. Siegel 33
conducted within the RAILROAD project and was supported by ProRail and the Netherlands
organization for scientific research (NWO) (under grant 438-12-306).
34 A. W. Siegel
II. Comparison of real-time relative workload

measurements in rail signallers2
Abstract
This exploratory field study investigated the weak resilience signals of workload in a rail traffic
control room. The goals of this research are to see whether real-time system information of a rail
control post can be used to predict workload of a rail signaller in real-time (Siegel & Schraagen,
2014d), and to further improve this method. In order to investigate this question, three workload
measures were used. The first was the subjective Integrated Workload Scale, the second was a
physiological measurement of electrodermal activity and the third was behavioural observation.
For two cases the subjective workload was compared to the system information algorithm and
the two other workload measures. The results show that the system information for
communication, manual actions and switch cost are discriminating for workload.
2
Published as Van Broekhoven, R., Siegel, A. W., Schraagen, J. M. C., & Noordzij, M. L. (2016).
Comparison of real-time relative workload measurements in rail signallers. In B. Milius & A. Naumann
(Eds.), Rail Human Factors Proceedings of the 2nd Germany Workshop March, 8th and 9th, 2016,
Stadthalle Braunschweig (pp. 30–40). Braunschweig: ITS automotive nord. Retrieved from
http://doc.utwente.nl/99353/
A. W. Siegel 35
1. Introduction
It is Wednesday 14:00 h. at a rail control room in Alkmaar, The Netherlands. A rail signaller
throws a glance at his screen and makes some adjustments in the rail traffic planning. It is a
calm shift and the train traffic runs as normal. At 16:23, a train driver calls in to report that the
train has hit an object. The driver has stopped the train to check out what happened, as the
procedure prescribes. As a response to the incoming call, the rail signaller notifies the
decentralized traffic manager and rail signaller of the adjacent area of the stop location of the
train. Next, he proceeds to inform the rail signaller of another adjacent post about the situation.
Because the situation is rather unclear, the rail signaller calls all the approaching trains and
orders them to stop. Each call includes exact prescribed actions and mileage to avoid
miscommunication. After seven minutes (16:30) the inspecting rail driver reports that he did not
find anything that could explain the sound he heard and that there is no sign that the train has hit
a person. Therefore, the rail signaller gives permission to drive again. The local co-workers, the
decentralised traffic manager and the rail signaller from the other post are informed and the
restrictions are cancelled. The rail signaller calls all related trains to abrogate the restrictions and
informs them that they may start driving again. He requests to remain vigilant around the
reported area.
This case presents the effects of one train stopping for 7 minutes with the consequences of a
workload increase for more than half an hour. Around 17:00, the last telephone call was
conducted. While the events unfolded, the rail signaller had to monitor and act on different
trains and events. The rail signaller was constantly switching between incoming calls from train
drivers, informing co-workers, being updated by co-workers, anticipating on all new incoming
trains in the area, manually rerouting these trains and informing all involved train drivers by
telephone. This case describes a possible urgent and alarming situation where lots of different
actions are necessary and a lot of different people need to communicate. Does the workload
increase? Are rail signallers aware of their own (perceived) workload?
Resilience engineering studies how socio-technical systems deal with unexpected and
unforeseen circumstances, such as described in the case above (Hollnagel et al., 2006). Siegel
and Schraagen (Siegel & Schraagen, 2014d) proposed that dealing with such circumstances in a
resilient fashion requires sociotechnical systems to focus on so-called ‘weak resilience signals’.
Weak resilience signals are signals that indicate a possible degradation of the sociotechnical
system without immediately triggering a predefined alarm. An example of a weak resilience
signal could be a change in experienced workload that is not noticed or is not recognized as an
alarming signal. For an organisation, both Madni and Jackson (2009) and Hollnagel (2009) state
that the level of resilience is not merely a given factor, but an ability that can be developed to
make the organization more flexible and proactive. Important factors in developing resilience
36 A. W. Siegel
are the ability and opportunity to anticipate, monitor, respond and learn from situations
(Hollnagel, 2009).
To improve resilience in a railroad setting, Siegel and Schraagen (Siegel & Schraagen, 2014d)
developed a real time support system presenting weak resilience signals to increase the ability to
respond to events and learn from them. Weak resilience signals provide this ability without the
need for escalation or accident. One of the weak resilience signals described by Siegel and
Schraagen (2014d) is the relative increase or decrease of subjective and objective workload.
Presenting workload weak resilience signals was done by presenting rail signallers with changes
of their subjective workload and objective workload configured from their information system.
Subjective workload was operationalised by a one-dimensional workload scale designed for rail
signallers. This scale is called the Integrated Workload Scale (IWS; (Pickup, Wilson, Norris, et
al., 2005)). Objective workload was operationalised by means of an algorithm based on the
model of cognitive task load (CTL(Neerincx, 2003)). The cognitive task load for a certain task is
based on three dimensions: task complexity, task duration and task switching. The more
complex, the longer the duration or the more switching between different tasks, the higher the
cognitive task load. Siegel and Schraagen (Siegel & Schraagen, 2014d) also developed an
algorithm, derived from log data of the traffic system, resulting in a measure called the external
cognitive task load (XTL; Siegel and Schraagen (Siegel & Schraagen, 2014d)). The XTL is
based upon four main measurable tasks of the rail signaller: monitoring, plan mutations, manual
actions and communication by telephone. Because these measures are taken from system
information, there could be a discrepancy between the behaviours that the system data would
predict and the actual behaviour of the rail signaller. For example, an automatic mutation in the
planning can change to something else, or can change back to the original planning without the
rail signaller’s awareness. This will have no effect on the executed behaviours of the rail
signaller, but the system will register activity. Therefore, this study will compare workload as
measured by means of the XTL with other workload measures. In this way, this study
investigates whether the results of the XTL correspond to other workload measures.
However, the literature is not consistent about the exact definition of workload (e.g., (Young,
Brookhuis, Wickens, & Hancock, 2015)). The problem is that there is no exact empirical
definition and no physical unit to measure workload. Still, there is a whole range of methods,
attempting to measure workload. Those methods generally focus on different facets of workload,
such as self-report questionnaires (NASA-TLX; (Hart & Staveland, 1988), heart rate variability
(Jorna, 1992) and EEG (Brookhuis & de Waard, 2010)). There is consensus, however, that at
least three components are important for measuring workload. These components are subjective,
physiological and performance measures (Young et al., 2015). Therefore, the current study will
use three different ways of measuring workload, corresponding to these three components.
First, for the subjective measure, we used the Integrated Workload Scale (IWS; (Pickup, Wilson,
Norris, et al., 2005). The IWS consists of a 9-point one-dimensional scale on which the rail
signallers could indicate their perceived workload for a certain period of time. The IWS is
A. W. Siegel 37
specifically designed to measure rail signallers’ subjective workload and gives an insight in their
perceived cognitive workload.
Second, for the physiological measure, we used electrodermal activity (EDA). Electrodermal
activity is an online physiological measure of workload and there is consensus that it at least
reflects a general measure of arousal or stress (Healey & Picard, 2005). The EDA is expressed in
skin conductance (SC) units (Boucsein, 2012). In the EDA measurement, there are several
parameters that can be extracted. Some parameters are related to the phasic, short lived Skin
Conductance Responses (SCR), others are related to tonic, slow changes in the average level of
the skin conductance level (SCL). The EDA measurement directly reflects activity of the
sympathetic nervous system without being affected by parasympathetic activity (Boucsein,
2012) and it is a non-intrusive measurement that minimizes motion artefacts (Poh, Swenson, &
Picard, 2010). As the EDA measurement is not intrusive, the rail signallers will not be disrupted
in their work.
Third, for the performance measure we used behavioural observation, enabling to make a good
comparison with the XTL (Siegel & Schraagen, 2014d). The behavioural observation will focus
on the executed behaviours, forming the basis for a behavioural performance measure. We
expect that certain behaviours correlate with the XTL measure.
The current research builds upon a previous study by Siegel and Schraagen (Siegel &
Schraagen, 2014d) by adding behavioural observations and EDA. Those measurements will be
compared with the algorithm used. Furthermore, these measurements can be used to further
calibrate this algorithm. This exploratory field study attempted to answer two research
questions. The first is whether the four workload measurements described (IWS, XTL, EDA and
behavioural observations), support each other in the identification of changes in observed
workload. The second question is whether the objective workload measure employed by Siegel
and Schraagen (Siegel & Schraagen, 2014d) can be compared and complemented with the
measurements used.
38 A. W. Siegel
2. Methods
2.1. Participants & Procedures
The observations took place at a rail control post responsible for the area to the north of
Amsterdam. The post was located in the city of Alkmaar. The rail control post consisted of four
workstations (WS) with four rail signallers on active duty, one backup rail signaller for
calamities, one decentralized train traffic manager and one team supervisor. This observational
study focused on one of the four work stations of the railroad control post. The 10 (9 male and 1
female) rail signallers that participated were between 22 and 52 years old (M = 37.6; SD =
11.12). The participating rail signallers had experience from half a year up to 34 years (M =
11.8; SD = 11.01). The protocol guiding the observations included an oral recorder consent, due
to cultural constraints and at the specific request of the post management, and was approved by
the ethics committee of the University of Twente. Before the observations started, the
instructions and goals of the study were explained. When the participants were ready and
everything was clear, they were asked to wear the EDA-sensor and were informed that the
behavioural observations would start in a few minutes. The IWS measurement was running
during the whole day and evening. The EDA measurement, as well as camera monitoring, was
conducted during the day provided participants were willing to wear the EDA sensor and agreed
to be recorded. In total, 34 hours of EDA measurement and 26 hours of behavioural
observations were recorded. The camera monitoring was done to capture possible unique events
and to look back for specific behaviours. Coding of observed behaviour was restricted to half an
hour before the subjective IWS measure indicated “Some spare time” or higher. This was done
for practical reasons in analysing all recorded material. During and between shifts, it was
possible for the rail signallers to rotate positions. If this happened, the EDA-sensor was retrieved
and data were extracted and logged before the EDA-sensor was passed on to the next rail
signaller. Camera and behavioural observations continued, but a change of shift was marked in
the video file. When the shift was coming to an end, the participants were asked for any remarks
about the shift and were thanked for their participation.
2.2. Measurements
2.2.1. Integrated Workload Scale (IWS)

Siegel and Schraagen (2014d) (Figure 1) developed a tool based on the Dutch validated IWS
(Wilms & Zeilstra, 2013). The IWS would pop up for 30 seconds on the rail signaller’s work
station. This was repeated every five minutes. In this way, we received an automatic and
continuous rating of the IWS and a longer duration of a higher IWS will be referred to as a
“stretch” of increased subjective workload as defined in Siegel and Schraagen (2014d). To
A. W. Siegel 39
maintain a high response rate, it was possible for the rail signallers to open and fill in the IWS
during the whole five minutes. It was also possible to adjust the last response they had given.
This gave the rail signaller the opportunity to primarily focus on handling the situation, while
still having the ability to fill out the IWS. If no response was given, the last value was copied
under the assumption that there was no change of experienced workload.
FIGURE 1 Integrated Workload Scale tool (Siegel & Schraagen, 2014d),

translated from Dutch to English
2.2.2. External Cognitive Task Load

The External Cognitive Task Load (XTL) was calculated from real-time data retrieved from the
operational control system (Siegel & Schraagen, 2014d). The XTL was adjusted by adding a one
to the formula used by Siegel and Schraagen (2014d). This was done to achieve the same range
as the IWS (from 1 to 9). The algorithm was based on the number of automatically executed
plan rules in 5 minutes per workstation (monitoring, mon), the number of mutated plan rules in 5
40 A. W. Siegel
minutes per workstation (plan mutations), the number of non-executed plan rules in 5 minutes
per workstation (manual actions, man), and the percentage of seconds spoken through the
telephone of 5 minutes per workstation (communications, com). The constant k’s were
Initialized with 1 and adjusted during post-processing, optimizing the relation with IWS:
K mon = 0.4, K plan =0.9 , K man= 1.2 and K com =1.5 .
(
XTL WS=K s w itchWS K mon
Mon WS
Mon max
+ K plan
PlanWS
Plan max
+ K man
Man WS
Manmax
+ K com ∙ ComWS +1 )
;
1 ≤ XTLWS ≤ 9
The XTL formula of Siegel and Schraagen (2014d) has been altered by adding a 1, causing the
XTL values to be between 1 and 9, just like the values retrieved from the IWS. The switch cost
was taken into account by the number of activations that is composed from: 1) the number of
delayed trains, 2) the number of telephone calls, and 3) the number of incidents reported in 5
minutes per workstation divided by the maximum number of activations in a 5-minute time slot.
The XTL gives a general relative cognitive task load configured from system output each five
minutes. It will also provide a relative load of each of the four categories (monitoring, planning,
manual and communication) which can be used to look at specific components in the XTL
formula.
2.2.3. Behavioural cognitive Task Load

The Behavioural cognitive Task Load (BTL) was calculated in a similar way as the XTL. The
BTL is based on the model of Neerincx (Neerincx, 2003) to be able to compare the variables of
both measures with each other. The difference between the XTL and the BTL is that the
information for the XTL comes from the ProRail information system, whereas the information
from the BTL comes from executed behaviours of the rail signaller. Behaviours were selected
based on observations, interviews with rail signallers and the four different categories of the
XTL. Behaviours were observed using the Observer XT (version 11) and were rated based on
how long (s) a behaviour was executed and how many switches occurred between different
behaviours in five-minute time frames. This was done by observing for how long rail signallers
showed behaviours that were linked to observation, manual actions, planning behaviour,
communication with team members and making telephone calls with others outside the rail
traffic control post. These categories were further specified, taking into account different
behaviours and implementation locations (Table 1; Figure 2). A differentiation was made, for
example, between telephone calls originating from different parts of the socio-technical system.
More specifically, a call from a bridge operator is likely to cause a low increased workload
A. W. Siegel 41
because the waterway bridge is manually controlled with one button. On the other hand, an
incoming alarm call is more likely to increase workload because it needs immediate action.
TABLE 1 Overview categorized behaviours BTL
Monitor Planning Manual Communication

Fast and global Manual Railway Local Communication
glance on the plan screen occupation communication through
screens screen telephone
Railway Plan screen Overview Decentralized traffic Bridge
occupation monitor screen regulator
screen
Overview Writing Co-RS, Co-RS-in other
screen report specific case post
Other Co-RS or other co- Train driver
worker, general but
work related.
Alarm
FIGURE 2 Screen one is the occupation screen, screen two is the planning screen, the three screens
under number three are the overview screens
42 A. W. Siegel
The formula of the BTL is based on the time(s) that behaviours in the categories (mon, plan,
man, com) were observed. The constant k’s were initialized with 1.
(
BTLws =BswitchWS ∙ k mon ∙ mon ( s ) + k plan∙ plan ( s ) +k man∙ m an ( s )
+ k com∙ com ( s ) )
Again, the factor ‘switch cost’ from Neerincx (2003) was integrated. The switch cost for BTL
was based on the number of switches in 5 minute intervals divided by the maximum observed
number of changes in behaviour. The maximum behavioural switches observed during the study
were 60 switches in 5 minutes.
2.2.4. Electrodermal activity

Rail signallers were asked to wear the Affectiva QTM sensor. This is a wrist worn, watch-like
sensor that measures EDA with 1 cm diameter Ag-AgCl dry electrodes at the ventral side of the
wrist. EDA data were pre-processed with a Continuous Decomposition Analysis (CDA) as
implemented in Ledalab (Benedek & Kaernbach, 2010), which requires MATLAB (Mathworks,
Natick, MA, USA). From the EDA, an estimate of the skin conductance level (SCL) as well as
the overlaying phasic activity (occurrence and amplitude of SCR’s) can be acquired. The phasic
activity, coming from classical Trough-to-Peak analysis, was reported (threshold for an SCR
amplitude was set at .03 µS; Boucsein (2012)). As recommended by Boucsein (2012), visual
checks were performed on plots of skin conductance data to identify failed measurements, “non-
responding” (indicated by an absence of SCRs in a given measurement) and incorrect
classification of SCR’s. Data from these problematic measurements were removed from further
analysis. The SCL and SCR parameters were expressed in 5 minute intervals to allow for
comparisons to the XTL and IWS the values.
3. Results
3.1. Data collection and case comparison
In order to compare the methods with each other, we took IWS as a baseline to make a
distinction between low (IWS, 1-2) and high (IWS, 3-9) workload. We chose for IWS as a
baseline because it has an uni-dimensional scale and because the IWS is used and validated for
rail signallers (Pickup, Wilson, Norris, et al., 2005; Wilms & Zeilstra, 2013). However,
occurrences of incidents or high workload were rare during the study. Therefore, behavioural
observation was only further analysed around IWS elevations. During the observations, the IWS
rose 14 times above “minimal effort (2)” and there was only one period of “very
busy”(6)/”extremely busy”(7). In three of the IWS elevations the pattern showed a clear stretch
A. W. Siegel 43
in the IWS and the data collected from the other measurements were usable. Two of these cases
(briefly describe below) contained sufficient data points for further statistical analyses.
3.2. Short description of case 1 and 2

Case 1 Case 2
A train driver is calling in thinking he hit a The rail signaller is informed by mail that
person and is going to check it. The train trains need to reduce velocity between a
signaller informs co-workers and starts certain trajectory to a maximum of 40 km/h.
informing train drivers to stop the train or Colleagues are informed and all approaching
slow velocity as prescribed. After a few trains for this trajectory are called and
minutes the train driver reports he could not informed according to procedures.
find anything and that it must have been
something else. The train signaller informs
co-workers and train drivers that they can
start driving again. The short time it took to
stop and get going again had over half an
hour delay involving all trains on the
trajectory
3.3. IWS results

The average IWS during the day of the main event was “minimal effort” with a small deviation
(M = 2,06; SD = 1,13). The IWS pattern for the cases further analysed have a stretch lasting 10
or more minutes. The IWS pattern for the two cases are presented in Figure 3. In the other cases,
the IWS scores did not become this high or the duration was too short. For further analyses, the
IWS pattern will be used as a reference for the other methods and a distinction will be made
between low IWS (0-2) and high IWS (3-9).
44 A. W. Siegel
9 Case 1 9 Case 2
8 8
7 7
IWS
6 6
5 5
IWS
4
4
3
3
2
2
1
1 5 0 5 0 5 0 5
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 :5 4:1 4:2 4 :4 4 :5 5 :1 5 :2
13 1 1 1 1 1 1
FIGURE 3 IWS scale 1 to 9 for case one and two
3.4. EDA results

The EDA data were visually inspected for any non-responders. All participants that seemed to
provide usable EDA data were further analysed. Statistical analyses were conducted for the two
cases using a MANOVA, comparing different EDA measurements (SCR, Amplitude, SCL)
during the period of high IWS with the corresponding measurements during a period of low
IWS. We found significant differences between periods of high and low IWS for all three
measures in Case 1 (Figure 4, case 1). First of all, we found the SCR to be significantly different
for periods of high IWS and for periods of low IWS (F(1,18) = 8.58, p < .009). The SCR signals
occurred significantly more for periods with a high IWS (M = 87.4; SD = 18.96) than for periods
with a low IWS (M = 63.60; SD = 17.68). The amplitude was significantly (F(1,18) = 8.59, p
< .009) higher for periods with a high IWS (M = 27.73 µSiemens (µS); SD = 7.90) than for
periods with a low IWS (M = 17.08 µS; SD = 8.33). Also the SCL was significantly different for
periods of high compared to periods of low IWS (F(1,18) = 11.18, p < .004). Again, the SCL
was significantly higher for periods of high IWS (M = 25.94 µS; SD = 5.37) than for periods
with low IWS (M = 17.27 µS; SD = 6.19). These results show that the three EDA measures can
discriminate between high and low IWS in Case 1.
For case 2, only the SCL was significantly different (Figure 4, case 2; F(1,19) = 1.66, p < .02)
for periods of high IWS (M = 0.05 µS; SD = 0.08) compared to periods of low IWS (M = 0.65
µS; SD = 0.62). The results for the SCR and Amplitude were not significant for case 2.
Moreover, the effect for SCL in case 2 is incongruent with the results of case 1. In case 2, the
SCL is significantly higher for periods with low subjective workload.
A. W. Siegel 45
FIGURE 4 Average number of SCR, Amplitude (average µS) and SCL(average µS) for case one and
two for high and low IWS. Significant differences are indicated with (*)
3.5. BTL results

To corroborate the scoring system used, two of the researchers scored a sample of half an hour
of observations. The inter-rater reliability between the two observers had 85% agreement
(Cohen’s kappa = 0.82) in number of seconds per behaviour.
For the behavioural observation results, we performed a similar MANOVA comparing the four
BTL categories, number of switches between behaviours and observed behaviours during
periods of high IWS with the corresponding measurements during periods of low IWS. For case
1 the factor communication differs significantly between periods of high and low IWS (F(1,18)
= 4.74, p < .04). When looking at the subcategories of communication (Figure 6), we see that
there is a significant difference for communication through telephone with a train driver
(F(1,18) = 10.70, p = .004). This means that this behaviour occurs more during periods of high
IWS (M = 87.73s (out of 300); SD = 75.38) than during periods of low IWS (M = 8.86s (out of
300); SD = 11.48). Also the local communication with the decentralized traffic manager was
significantly different (F(1,18) = 4.54, p = .05) during periods of high IWS (M = 9.53s (out of
300); SD = 13.52) compared to periods of low IWS (M = 0.38s (out of 300); SD = 1.20). This
means that communication through the telephone with a train driver and local communication
with a decentralized traffic manager are significantly higher in a high IWS situation than in a
low IWS situation.
46 A. W. Siegel
FIGURE 5 BTL observed behaviours for case one for high and low IWS. Significant differences are
indicated with (*)
For case 2 the four BTL categories communication (F(1,19) = 17.85, p < .001), manual (F(1,19)
= 11.23, p < .003), planning (F(1,19) = 5.85, p < .05) and monitoring (F(1,19) = 21.70, p
< .001) were significantly different between high and low IWS. Also the number of switches
between behaviours was significant (F(1,19) = 36.73, p < .001) with more switches in 5 minutes
for high IWS (M = 35.43; SD = 11.33) than for low IWS (M = 12.36; SD = 6.30). On
behavioural level (figure 6), communication through telephone with a train driver was
significantly different (F(1,19) = 10.36, p < .005) for high IWS periods (M = 63.7s (out of 300) ;
SD = 76.10) compared to low IWS periods (M = 0.00s (out of 300); SD = 0.00). Also local (case
specific) communication with colleagues was significantly different (F(1,19) = 8.08, p < .01)
with more communication in high IWS periods (M = 16.22 ; SD = 21.94) than in low IWS
periods (M = 0.00s (out of 300) ; SD = 0.00). Also manually writing (F(1,19) = 10.68, p < .004)
was significantly different for periods of high IWS (M = 16.96 ; SD = 19.95) compared to
periods of low IWS (M = 0.00s (out of 300) ; SD = 0.00). Monitoring planning screen was also
significant (F(1,19) = 10.95, p < .004), with more monitoring during high IWS periods (M =
17.82s (out of 300) ; SD = 7.33) compared to low IWS periods(M = 7.27s (out of 300) ; SD
= .6.68). Also monitoring the overview screen was significantly higher (F(1,19) = 19.59 p
< .001) during high IWS periods (M = 38.89s (out of 300) ; SD = 18.05) compared to low IWS
periods (M = 8.73s (out of 300); SD = 6.68). In conclusion, these results show that some specific
behaviours were able to discriminate between periods of high and low IWS.
A. W. Siegel 47
FIGURE 6 BTL observed behaviours for case two for high and low IWS. Significant differences are
indicated with (*)
3.6. XTL results

For the XTL data, we performed a MANOVA for low versus high IWS with five factors ( Mon,
Plan, Man, Com and Switchcost). For Case 1, the factor communication differed significantly
(F(1,18) = 11.20, p < .004) between periods with high IWS (M = 0.25 ; SD = 0.20) and periods
with low IWS (M = 0.03 ; SD = 0.06). This means that for periods with high IWS.
For Case 2, the factor communication was significantly different (F(1,19) = 8.58, p < .009),
being higher during periods of high IWS (M = 0.25; SD = 0.33), compared to periods of low
IWS (M = 0.00 ; SD = 0.00). Manual was also significantly different (F(1,19) = 9.00, p < .006),
with more manual data in high IWS (M = 0.10 ; SD = .13) compared to low IWS (M = 0.00; SD
= 0.00). Switch was also significantly different (F(1,19) = 5.09, p < .04) with more switches
during periods of high IWS (M = 0.07; SD = 0.06) than during periods of low IWS (M = 0.01;
SD = 0.04). In conclusion, these results show that communication, manual and switch cost
discriminated between high and low IWS.
48 A. W. Siegel
FIGURE 7 XTL parameters and switches for case one and two for high and low IWS. Significant
differences are indicated with (*)
4. Discussion
This study investigated whether the four workload measurements (IWS, XTL, EDA and BTL)
supported each other in the identification of changes in observed workload, and whether the
XTL algorithm can be confirmed and complemented.
The results show that EDA is a good discriminator between high and low IWS values in case 1,
which is in line with the consensus that electrodermal activity is an online physiological measure
of workload that at least reflects a general measure of arousal or stress (Healey & Picard, 2005).
This effect was not found in case 2 except for the SCL which was opposite but small. This
discrepancy in case 2 could be explained by the smaller change of IWS, which did not pass the
physiological arousal or stress threshold. The EDA measurement results show that the EDA
seems to be a promising method to use in measuring workload in rail signallers. The method is
not intrusive with their work and it is theoretically possible to process the data in real time
(although this was not the case in this research). However, the current experiment was relatively
short and the observed periods were relatively calm, so further research is necessary.
BTL shows that different behaviours occur with high versus low IWS. Mainly the category
“communication” seems to be important. Looking at different behaviours, “telephone
communication with train driver” and “contact with the decentralized” came back in both cases.
When compared with the XTL, these effects reoccurred partially. The effects on the XTL,
however, are less pronounced. The reason for this might be that, in the XTL, no distinction was
made between with whom the telephone communication took place. This information seems to
be important for interpreting these results, considering the BTL data. Also the factors “manual”
and “switch cost” seem to differentiate between high and low workload. The BTL observations
show that there is a high correlation between subjective and behavioural patterns, but that this
A. W. Siegel 49
highly depends on the behaviour in combination with other factors. For example, the same
behaviours (communication/calling on the telephone) can have a different impact on
experienced workload if the context or communicating partner is different. For the XTL, it
would be desirable to make a differentiation for different categories or interactions in the socio-
technical system. For example, calling with a train driver has a greater correlation on
experienced workload that calling with a bridge operator.
Finally, The XTL formula in the investigated cases shows a differentiating ability in both
communication and manual actions. This shows that the XTL and in particular the parameters
“communication” and “manual” could differentiate between high and low workload. However,
for manual action the effects are not congruent and should be examined in perspective of more
cases. Also Switch cost seems to show a trend (although not significant in case 1). For case 1
this could be explained by lag of the IWS/XTL. If the last high IWS for case 1 is removed the
XLT switch cost is also significant. The XTL could be further improved in further research by
differentiating the input data for XTL for the different categories. In this way, for example, a
distinction could be made through the other party in a telephone call. These steps will make the
XTL more sensitive and will create a better match between performance and experienced
workload.
Overall, the current research shows that real-time observation of subjective measures using IWS
and XTL can be done and are corroborated by EDA and behavioural observation. IWS, XTL,
EDA and BTL are capable of making distinctions between high and low experienced workload.
Further research and specifications are necessary to determine and validate which of the
system’s data have a high predictive validity and which do not. The current research contributes
to a better understanding of measuring workload of rail signallers by showing that system
information can be used to give a relative indication of the workload of the rail signaller.
50 A. W. Siegel
PERFORMANCE WEAK RESILIENCE

SIGNAL
A. W. Siegel 51
52 A. W. Siegel
CHAPTER 3
PERFORMANCE WEAK RESILIENCE
SIGNAL
Towards quantifying metrics for rail-system
resilience: Identification and analysis of
performance weak resilience signals3
Abstract
This paper aims to enhance tangibility of the resilience engineering concept by facilitating
understanding and operationalization of weak resilience signals (WRSs) in the rail sector.
Within complex socio-technical systems, accidents can be seen as unwanted outcomes emerging
from uncontrolled sources of entropy (functional resonance). Various theoretical models exist to
determine the variability of system interactions, the resilience state and the organization’s
intrinsic abilities to reorganize and manage their functioning and adaptive capacity to cope with
unexpected and unforeseen disruptions. However, operationalizing and measuring concrete and
reliable manifestations of resilience and assessing their impact at a system level have proved to
be a challenge. A multimethod, ethnographic observation and resilience questionnaire, were
used to determine resilience baseline conditions at an operational rail traffic control post. This
paper describes the development, implementation and initial validation of WRSs identified and
modelled around a ‘performance system boundary.’ In addition, a WRS analysis function is
introduced to interpret underlying factors of the performance WRSs and serves as a method to
reveal potential sources of future resonance that could comprise system resilience. Results
indicate that performance WRSs can successfully be implemented to accentuate relative
deviations from resilience baseline conditions. A WRS analysis function can help to interpret
these divergences and could be used to reveal (creeping) change processes and unnoticed
initiating events that facilitate emergence that degrades rail-system resilience. Establishing
relevant change signals in advance can contribute to anticipation and awareness, enhance
organizational learning and stimulate resilient courses of action and adaptive behaviour that
ensures rail operations reliability.
3
Published as De Regt, A., Siegel, A. W., & Schraagen, J. M. C. (2016). Towards quantifying metrics for
rail-system resilience: Identification and analysis of performance weak resilience signals. Cognition,
Technology & Work, 18(2), 319–331. http://doi.org/10.1007/s10111-015-0356-9
A. W. Siegel 53
1. Introduction
We currently live in an increasingly tightly coupled and interactively complex world in which
unpredictable events are omnipresent and the velocity with which unanticipated events can
amplify into unwanted outcomes is continually increasing (Weick & Sutcliffe, 2001). Within
this setting, the railway industry is broadly recognized as an example of a safety critical and
complex socio-technical system (e.g. Wilson, Farrington-Darby, Cox, Bye, & Hockey, 2007;
Belmonte et al., 2011). To maintain control, enhance efficiency and improve safe operations in
the rail industry, a rise in automation (Wilson & Norris, 2005), standardization and strict
adherence to protocols and predefined timetables (off-line timetabling; Goverde & Odijk, 2002;
Hansen, 2010) has been notable over the years. This optimization rationale enabled the
European railway to become an ultra-safe system (one accident per one million events;
Amalberti, 2001; European Railway Agency, 2014). This, combined with the search for
sustainable transport solutions, induced political focus on rail transportation throughout Europe
(Ferreira, Wilson, Ryan, & Sharples, 2011). However, to meet and maintain the high levels of
performance (e.g., punctuality, capacity and safety) that are required to realize this potential,
linear and additive optimization solutions (e.g., more rules and regulations) may prove to be
insufficient (e.g., Bieder & Bourrier, 2013). Unwanted outcomes in ultra-safe complex socio-
technical systems ‘emerge’ from a combination of unanticipated, non-linear relationships
between constituent parts of the system that can arise under dynamic operating conditions
(Dekker, Hollnagel, Woods, & Cook, 2008; Leveson, 2004). This causes the system to contain
hidden fragilities with respect to rare and relatively unpredictable perturbations, making the
system robust yet fragile (RYF; Doyle et al., 2005). The railway system thus faces the challenge
of finding alternative methods to enhance performance and outmanoeuvre (confusing) system
complexity (De Carvalho, 2011).
The resilience (systems) approach is considered to be the next step (e.g., Qureshi, 2008) and has
become, arguably, the dominant paradigm in the study of complex socio-technical systems
(Underwood & Waterson, 2013). Resilience engineering can be defined as a proactive approach
concerned with enhancing organizations’ intrinsic abilities to reorganize and manage their
functioning and adaptive capacity prior to, during, or following events, so that the system can
sustain the required level of operations under both expected and unexpected conditions
(Hollnagel, 2014; Woods & Branlat, 2010). Resilience should thus be seen as an emergent
property originating from what an organization does, rather than what an organization has,
emphasizing function over structure and ability over capacity (Hollnagel, 2004). Different
theoretical models are available and have been used over the years to describe the resilient state
of a system (e.g., The Ball and Cup model: Scheffer, Hosper, Meijer, Moss, & Jeppesen, 1993;
Safe operating envelope: Rasmussen, 1997; Stress-Strain (S-S) model: Woods & Wreathall,
2008; Resilience Analysis Grid (RAG): Hollnagel, 2011; Functional Resonance Analysis
Method (FRAM): Hollnagel, 2012). Among resilience researchers there is general consensus
54 A. W. Siegel
that people are the primary source of resilience (e.g., Woods, Patterson, & Cook, 2007). In
accordance, providing techniques and system designs that help people and organizations cope
with complexity might thus be one method to enhance system resilience. However, without a
clear understanding of what manifestations of resilience look like (Back, Furniss, Hildebrandt,
& Blandford, 2008), it will be difficult to identify such manifestations in practice and quantify
the theoretical models developed, creating a research–practice gap (Underwood & Waterson,
2013). This is especially true when focusing on quantifying resilience for infrastructural
systems, in which the current quantification methods used (e.g., graph theory: Berche, von
Ferber, Holovatch, & Holovatch, 2009; fuzzy interference: Heaslip, Louisell, Collura, & Urena
Serulle, 2010) emanate from other well established and well-elaborated methodological
frameworks, but as such, are not fully capable of capturing the underlying interrelations of
system modules (Tamvakis & Xenidis, 2013). Research aimed at operationalizing theoretical
resilience models and prospective analysis frameworks for quantifying resilience of
infrastructure systems is required (e.g., Madni & Jackson, 2009).
Siegel and Schraagen (2014d) contributed to diminishing the research-practice gap by

developing a so-called ‘resilience state model’ for railway systems. This model is based on
Rasmussen's (1997) safe operating envelope, the Stress-Strain model described by Woods and
Wreathall (2008), and adheres to the notion that knowledge and error flow from the same mental
sources (Hollnagel, 2012; Mach, 1905). Siegel and Schraagen (2014d) adapted the three
(relative) system boundaries defined by Rasmussen (performance, economy and workload;
1997) to describe and explain the various (external) pressures -safety, performance (capacity &
punctuality) and workload- that affect the operating state of a railway system. In addition, a
depth dimension was added to the model that enables differentiation between internal changes
that keep the system in a resilient state or have it move towards brittleness. The Stress-Strain
model (Woods & Wreathall, 2008) characterizes the properties of an organization as an adaptive
system by using an analogy from materials engineering which focuses on the relationship
between the external and varying demand on a mechanical structure (stress), and how the
structure stretches in response (strain). Siegel and Schraagen (2014d) use changes in the linear
relation between stress and strain (i.e., Young's modulus slope) to model quantifiable rail weak
resilience signals (WRS). A WRS indicates a change in the system's operating state and is
defined by measuring properties in the base capacity region of the system that signals changes of
properties in the extra adaptive region of the system. In this model, the base capacity reflects the
‘normal’ functioning response of the system to external events. The extra adaptive capacity
reflects the potential discrepancy between adaptive system responses and external demands that
challenge or fall outside the boundaries of the base operating capacity (Woods et al., 2014). In
other words, WRSs represent uncertain snippets of information, hidden within the ‘normal’
system variability, which could be used as situated indicators to signal potential change
processes in the organizations’ resilience level. The ‘resilience state model’ for railway systems
is thus described as a framework for generic quantifiable modelling of rail WRSs around three
(relative) system boundaries (workload, performance and safety; Siegel & Schraagen 2014b).
Previous research indicated that changes around the workload boundary could be successfully
A. W. Siegel 55
measured, identified and used to quantify workload WRSs (Siegel & Schraagen, 2014d). In this
paper we extend implementation of the WRS framework to the performance boundary. Since
within rail systems the quality of performance is, to a large extent, based on time related key
performance indicators, specific methods to measure and quantify changes in punctuality and
rail capacity need to be defined.
To develop operational parameters that identify changes around the performance boundary,
baseline conditions (acceptable levels of performance) of the current operating state should be
established where deviations can be measured against. Examining the current operating state
will enhance insight and understanding in how WRSs arise and support the interpretation of the
WRSs indicated through the WRS framework. In addition, focusing on different elements that
comprise, or serve as alternative and additional performance indicators (i.e., analysis functions)
might help to further enhance interpretation, understanding and analysis of rail WRSs indicated
through the framework. To investigate the factors influencing the operating state this paper
adheres to Hollnagel's (2009) notion that management of uncertainty and system variability in
the (real-time) operations is built around four main system capabilities defining resilience:
responding to the actual (knowing what to do), monitoring the critical (knowing what to look
for), anticipating the potential (knowing what to expect) and learning from the factual (knowing
what has happened).
The overall aim of this research is to enhance tangibility of resilience by facilitating

understanding and operationalization of rail weak resilience signals. The research question is
threefold: 1) How can the baseline conditions that comprise the current resilience operating state
of the socio-technical rail system be determined? 2) How can the resilience state model for
railway systems be further developed to enable quantification and operationalization of
performance WRSs? 3) Can a WRS analysis function enhance interpreting and understanding of
(creeping) factors that underlie performance WRSs?
The rest of this paper is organized as follows: the multi-method research approach will be
described and operational WRS parameters will be introduced. Resilience baseline conditions
will be identified and quantification of WRSs on the punctuality boundary will be explicated
upon. In conclusion, the main points and results will be discussed and an outlook on future
research will be provided.
56 A. W. Siegel
2. Method
A multi-method approach was used to acquire data and knowledge about the current operating
state of the system (i.e., identify performance indicators). Influenced by the interdisciplinary
research fields of human-computer-interaction and computer-supported cooperative work (e.g.
Herrmann, Hoffmann, Kunau, & Loser, 2004; Millen, 2000), an ethnographically informed
method was adopted to construct adequate understanding of the working environment, discover
exceptional and beneficial user-behaviour and provide additional insights into social and
organizational phenomena.
2.1. Research Setting

The research took place at a rail control post in Zwolle. This rail control post, one out of
thirteen, handles rail stations and train traffic in the north-eastern part of the Netherlands. To
bolster rail operation reliability and efficiency, the setup of the control room (Figure 1) consists
of eight workstations divided into two main rail corridors: south-north, called corridor North
(four workstations), and west-east, called corridor East (three work stations). A corridor is
defined as a large area of rail infrastructure that consists of adjacent rail sections from various
(interdependent) work stations. Within this setup train dispatchers work in small ‘corridor’
teams consisting of approximately 3-4 people, 1 planner (executive logistics focus) and 2-3 train
dispatchers (safety focus), who rotate in positions. The one remaining workstation is exclusively
utilized during calamities and is readily available to either corridor as a means to provide added
resources and capacity.
FIGURE 1 Corridor North (1, 2, 3, 4) and corridor East (5, 6, 7) & corresponding rail trajectories
The population consisted of a convenience sample of 25 rail dispatchers (5 female, 20 male)

working at least 24 hours a week. Because both senior as well as junior dispatchers were
included in the sample, years of experience in train dispatching ranged from 2 to 37 years (M=
18.5, SD= 10.55). In addition, age varied between 29 and 65 years (M=47, SD=8.72).
A. W. Siegel 57
2.2. Resilience observations

The observations took place over a period of 2 non-contiguous weeks. During the first week,
three researchers pulled rotating shifts of 8 hours to fully cover the rail dispatcher’s proceedings
during day and night shifts (24/7). In the second week, two researchers observed the domain
practitioners between 7:00 AM and 9:00 PM, logging a total of 266 hours of observation. Since
the presence of people other than the rail dispatchers is not uncommon in the control room,
disruption of the natural setting was minimized. This non-obtrusiveness allowed the researchers
to roam around freely to observe and interview the rail dispatchers as opportunities presented
themselves. In order to guide this process and enhance the efficiency, outcomes of real-time
workload measurements were used to focus on situations in which train dispatchers experienced
increased workload. Workload measurements were conducted using an automated version of the
Integrated Workload Scale (IWS; Pickup, Wilson, Norris, Mitchell, & Morrisroe, 2005), a uni-
dimensional self-report tool consisting of a 9-point scale in which 1 = not demanding and 9 =
work too demanding, that enabled numerical entering of rail dispatchers’ subjective workload at
5-minute intervals. An interface was developed to display the 5-minute interval entering of the
subjective workload for all individual rail dispatchers and all individual work-stations in real-
time (figure 2). To distinguish between different rail-dispatchers who worked at the same
working-station, a unique but anonymous numerical code had to be entered at the beginning of a
rail dispatcher’s shift. The purpose of the interface was to enhance rapid identification of
(relatively) busy work situations. To enable this, the height of the bar-charts was adjusted
according to the 5-minute interval scores and different (rainbow) colours were chosen to
visualize the IWS-scores (with 1= purple and 9= dark red). The general configuration of the
interface was set to display the current hour, however time slots of two, three, six and twelve
hours were also available.
FIGURE 2 Monitoring interface for the observer. (8 workstations with resolution time setting)
58 A. W. Siegel
Although the observations as well as the conversations were mainly centred around the
dispatchers’ current actions, the work in general and prior situations (that either turned out either
unexpectedly good or resulted in unwanted outcomes and safety issues) were also discussed
with dispatchers retrospectively. The purpose of this was twofold: 1) enhance understanding and
2) fill the (potential) observational research gap which occurs due to the fact that the European
railway has become an ultra-safe system (Amalberti, 2001). Since in an ultra-safe system on
average only one accident occurs per a million events, the chances of us observing such an event
during the observation period are close to zero. It is however of utmost importance to know how
the dispatchers would handle, or have handled, such situations. Therefore, within the context of
the current as well as the prior situations, particular attention was placed on capturing concrete
examples in which the complexity of the environment, in either routine or disturbed situations,
required resilient behaviour. The Resilience Markers Framework by Furniss et al. (2011), which
is used to systematically observe concrete manifestations of resilience at different levels of
granularity within and across domains, was used to document the cognitive, collaborative
strategies and facilitating activities used to handle situation demands. This framework
distinguishes three levels: 1) resilience markers, 2) resilience strategies and 3) resilience
observations. Attributes and an example of the different levels are shown in Table 1.
A. W. Siegel 59
TABLE 1 Levels, attributes and an example of the hierarchy (Furniss et al., 2011).
1. Level 2. Generalizability 3. Quantity 4. Example

8. Maximizing information
5. Markers 6. High 7. Low
extraction
9. Strategy 10. Moderate 11. Moderate 12. Creating an external cue
16. A paperclip to bookmark a
13. Observation 14. Low 15. High page in the procedure
someone is following.
2.3. Weak resilience signals

To resolve potential resilience gaps that surface from the base activity (i.e., through
observations) and reveal other issues and sources of future resonance that could comprise
system resilience in the long-term, operationalization and implementation of WRSs was
conducted. To measure workload WRSs, Siegel and Schraagen (2014d) introduced a new
metric, Stretch, which can be defined as an objective or a subjective system reaction to an
external (cluster) event. An objective stretch is used to identify an absolute workload growth and
is comprised of two main components: task complexity and events. Task complexity is
composed of real-time technical system measurement and readily available (log) data of rail
dispatchers’ main job requirements: monitoring of rail movement, performance of plan
mutations, execution of manual actions, and communication activities. Events are defined as
external events, i.e., not controlled by the operators themselves, that can influence the operator’s
task load (e.g., section and switch disruptions, rail track maintenance data, number of delayed
trains and the number of phone calls). The total number of events is calculated and measured in
five-minute intervals. This value is normalized between 1-2 and multiplied with the task
complexity resulting in the eXternal Task Load (XTL). The subjective stretch is the human
perception of the system’s strain and embodies the (cumulative) workload effort during a period
of time in which IWS scores deviate from an IWS-baseline. In this context an IWS-baseline is
defined as the steady state IWS rating before and after a disruption occurs (i.e., the IWS rating
during scheduled rail movements). Since we were looking for relative changes in workload
experienced by individual rail dispatchers it was decided not to weigh the IWS scores to account
for subjective variability due to competence. The timespan of the objective and subjective
stretch can differ if the activity in the system started earlier or ended later than the workload
shift perceived by the rail dispatcher. Therefore, the start and end time of a stretch are adjusted
to the first XTL-minimum moment before the IWS rising from, and after the IWS returning to,
the IWS-baseline. The ratio between the subjective and objective stretch is used to identify
workload WRSs.
When a growing change of a stretch-ratio is identified and the stretch values are larger than a
predefined (threshold) value, a weak resilience signal (WRS) is generated. To indicate
60 A. W. Siegel
significant and relative changes when comparing two periods, the accumulated standard
deviation (SD) of the stretch-ratio in each period was used (for more information see; Siegel &
Schraagen, 2014d). The workload WRSs were measured and used as a starting point and
reference frame, to extend operationalization of the rail WRS framework to the performance
boundary. To operationalize and utilize performance WRSs, identification and implementation
requirements had to be established.
The operational parameters for the performance boundary are entirely based on technical system
measurement and readily available (log) data. Since the rail capacity is generally stable over the
year (e.g., due to pre-defined and optimized offline time-tabling; Goverde & Odijk, 2002;
Hansen, 2010) differences between the pre-defined timetable (scheduled planning) and the
working timetable (real-time measurements of rail movements) were used to measure
performance WRSs. The main focus was placed on delay development, and propagation, within
the rail system and how this impacts the punctuality boundary (i.e., buffering capacity,
flexibility, margin and tolerance; Woods & Cook, 2006). Adhering to rail-dispatching
guidelines concerning time-lag and rail movement, a train was considered delayed if the
deviation from the pre-defined time-table exceeded a three-minute threshold.
By providing actionable attention cues, the WRSs will contribute to revealing eroding levels of
(operational) system resilience. The attention cues are visualized by generating graphical
representations (WRS graphs created in Excel). In addition, the WRSs serve as an objective
method (i.e., based on technical system data measurements) to approach the resilience base
capability levels observed.
To guide the process of selecting WRSs that need to be dealt with, analysis functions will be
constructed. An analysis function serves as an alternated frame of reference that is based on
other or additional performance indicators (i.e., besides the punctuality data). The aim of
implementing an analysis function is to exclude the ‘evident, known and obvious’ causes of
resonance, and shift attention to reveal ‘hidden, unmarked or ignored’ processes and incident
precursors that could affect rail-system resilience.
2.4. Resilience Questionnaire

Concurrent with the second week of observations, a resilience questionnaire was distributed by
e-mail amongst all rail dispatchers and the management (N=67) as a cross-referential method to
measure the (operational) resilience level within the rail control post. An online rather than on-
paper survey method was chosen due to the shift roster. To boost the response rate, a reminder
e-mail was sent the following week and a reminder message was placed in the organization’s
weekly newsletter.
The ADAPTER questionnaire (Analysing & Developing Adaptability & Performance in Teams
to Enhance Resilience; van der Beek & Schraagen, 2015) was selected because the questionnaire
A. W. Siegel 61
is suited to diagnose team resilience requirements of safety critical jobs and can be administered
within a relatively short time period. In ADAPTER, the four essential abilities of resilience
(Hollnagel, 2009) are supplemented with relation-oriented abilities such as leadership and
(cross-boundary) cooperation, to operationalize the concept of team resilience. Although the
questionnaire was already available in Dutch, it was decided to slightly change the wording of
some questions to better fit the terminology used within the railway organization. In addition, an
N/A category option was added, where appropriate, to ensure valid answers and avoid positive
skewness of answer categories which were not applicable for our specific situation (e.g., the
N/A category was added to questions relating to ‘cooperation with other teams’ since in rail
control this often involves cooperating across organizational boundaries in which not all
information about the other teams is known or available). To evaluate the ADAPTER results we
used the method of van der Beek and Schraagen (2015) to compute descriptive statistics and
reliability estimates for the whole sample.
3. Results
3.1. Real-time dispatching observations
The transportation planning within the Dutch railway system is a highly dynamic multifaceted
process. Within this context, train dispatchers coordinate and manage the (conflicting) demands
placed on track use and integrate multiple sources of information to conduct trade-off decisions
and actions necessary (e.g., re-routing, re-ordering and re-timing of trains, tracks and signals) to
maintain performance, regain control and mitigate potential threats. Especially in uncertain,
time-pressured and variable traffic situations, in which train dispatchers are pushed towards the
limits of their regular operating (base) capacity and the adaptive capacity of the system is
challenged (e.g. Woods et al., 2014), handling the situational demands proves to be a cognitively
complex task. It is in those instances that resilient strategies and behaviours are required and
boundary conditions of adaptive capacity, as well as localization of those boundaries, might be
exposed (Dekker, 2011; Woods & Cook, 2006). For this reason, observations and description of
resilient behaviour were focused around high-pressure situations.
In the next section one of the observed high-pressure situations will be delineated. This
illustrative case provides insight into the concrete manifestations of resilient dispatcher
performance, as well as subsequent vulnerabilities, and serves as a baseline measure for the
(operational) resilience conditions currently present within the organization.
3.1.1. Example of a high-pressure situation:

‘The hooligan case’
62 A. W. Siegel
On 2-4-2014 a major disruptive event unfolded when soccer hooligans ignited fireworks and
smoke bombs on a rail station platform and the mobile police unit was forced to intervene. As is
standard procedure in such high-pressure situations, the emergency workplace was put into
operation (as a means of reinforcement to handle performance variability, minimize timetable
disruptions and mitigate the rapidly increasing delays). In these situations, the dispatcher
responsible for the rail trajectory in which the disruption occurs focuses on the direct (short-
term) actions involved with the disruption handling (i.e., quick responsive action to train and
time table delays directly resulting from the disturbance) while the ‘emergency’ dispatcher
assists by taking over verbal communication with other actors (i.e., telephone calls from train
drivers) and (long-term) planning activities. As is the case in almost all high-pressure situations,
the corridor team was unable to integrate and develop implications for this specific situation
based on the full set of information held by all actors involved (Woolley, Gerbasi, Chabris,
Kosslyn, & Hackman, 2008). This is due to the fact that no direct line of communication could
be established between the rail control centre and the commander, nor other members, of the
mobile police unit. Therefore, the corridor team initially chose to arrange disruption handling of
delayed trains based on the incoming order of notifications in the system. This method of
prioritizing (short-term), proved to be inadequate and even counterproductive in the long-term
due to escalating knock-on delays for connecting trains (i.e., working at cross-purposes). This
process was noticed by the post manager, approximately 15 minutes after the incident occurred,
who directly stressed the importance of developing and implementing an action plan to properly
deal with this situation. To fill the information gap, rail dispatchers used the live camera feed
from the station platforms (Figure 3). By monitoring the police actions on scene, rail dispatchers
were able to enhance their overall situation awareness. Concurrently, the internal
communication channels/ structure was optimized. Two corridor team members gathered behind
the rail dispatcher’s workstation (responsible for the rail section where the disturbance took
place) in order to listen in on communication and look at the monitor displays to gain insight
into train movement and the overall rail situation on the surrounding tracks. Subsequently, this
information was shared with train dispatchers manning the neighbouring rail sections.
Implementation of the action plan resulted in highly selective rail movement in the disruptive
rail section (e.g. prioritizing international trains), and gradual redirection of stationary and
delayed trains occupying adjacent rail tracks to the nearest available railway station.
A. W. Siegel 63
FIGURE 3 Live camera feed from the station platforms.
3.1.2. Resilience behaviour episodes

Administering the resilience markers framework by Furniss et al. (2011), two resilience
behaviour episodes were distinguished for this specific situation. 1) Recognition of inappropriate
situation handling and avoiding escalation of commitment. 2) Tailoring of existing artefacts to
maximize information extraction.
Recognition of inappropriate situation handling and avoiding escalation of commitment (i.e., the
tendency to continue a chosen course of action even when changing to a new course would be
preferable; (Staw, 1981)) were related to the strategy 'provision of feedback to enable error
correction' (Blandford & Furniss, 2006) and the broader marker of ‘recognizing and responding
to failure’. Although the recognition and notification of malfunctioning initiated the corrective
actions necessary to manage the performance variability in this situation, the insight came rather
late and was only noted by one actor (the post manager) within the corridor team. Although it
could be argued that the post manager has a high level of experience, and as such might
outperform the operational competence skill level of the other corridor team members, the tasks
of a post manager and a rail dispatcher are of a different nature. As such, the post manager’s
skills and experience do not translate one-to-one to the abilities and experience of the corridor
team members. An alternative explanation could be that the post manager provided a fresh
perspective which led to a broader set of actions. This situation exposed potential vulnerabilities
(e.g., maintaining adequate situational overview and awareness in high-pressure demands,
acknowledgement of inappropriate actions and or routines) which could influence learning and
anticipation of future resonance and disruption handling. This notion was strengthened by
irregularities observed in the levels of operational performance within and between dispatchers
and situations. Similar prioritizing decisions could be observed with other dispatchers over
64 A. W. Siegel
different shifts (e.g., answering incoming phone calls rather than prioritizing time-table changes,
which would have been more efficient).
Tailoring of existing artefacts to maximize information extraction can be related to the strategies
'prepare for future work' (Blandford & Furniss, 2006) and 'cue creation in action' (Perin, 2005),
with the broader markers of 'preparation' and 'strategies that maximize information extraction'
(Blandford & Furniss, 2006). The awareness of (incoming) data limitations and the proactive
steps taken at present (i.e., enhanced monitoring), increased the readiness to adequately respond
to ongoing developments (efficient management of the performance variability) and provided
the opportunity to anticipate and prepare for future situational demands.
3.1.3. Weak resilience signals

The operational parameters for the workload WRS set by Siegel and Schraagen (2014d), were
used to determine if the hooligan case could indeed be labelled as a high-pressure situation and
if the system detected it as such. A WRS can be defined looking at three features: a (relatively)
long stretch duration, high (average) IWS or XTL scores and discrepancies in the stretch-ratio.
A graphical representation of the workload stretch measures was generated by plotting all the
objective vs. the subjective stretches of that day relative to an empirically drawn threshold line
(i.e., the rounded sum of the means with one standard deviation above). Since the stretches in
figure 4 are significantly correlated (r = .94, p < .05), the threshold line serves as a visual guide
to optimize the selection of stretches that deserve attention, by serving as a WRS. In figure 4 the
x-axis represents the stretch duration x IWS scores. The stretch duration is derived by the sum of
total minutes a stretch occurred representing the five-minute time-slots. In addition, the mean
IWS score can be calculated for each stretch. The y-axis indicates the sum of technical system
activity measured in a specific stretch, also taking into account the 5-minute time slots.
A. W. Siegel 65
FIGURE 4 Workload stretch graph 2-4-2014 with hooligan case stretches
From the graph it becomes clear that most stretches that occurred on 2-4-2014 were small and
do not exceed the boundaries of the safe operating envelope (Rasmussen, 1997). Ad hoc analysis
revealed that five workload stretches in figure 4 are caused by the same underlying
(decompensation) event, the ignition of fireworks and smoke bombs on the rail tracks and
station platform by soccer hooligans (‘Hooligan Case’ in figure 4). Looking at these five
stretches in relation to the three WRS features it is evident that all stretches have a (rather) long
stretch duration with increased mean IWS scores (circa 5 - 6, indicating moderate pressure –
very busy). In addition to an increased IWS average, all stretches also contained 5-minute
periods rated with the three highest IWS scores (7 = Extreme effort, 8 = Struggling to keep up,
and 9 = work too demanding). The stretches also have increased levels of technical system
activity (i.e., due to telephony and manual re-routing quantities) and enlarged deviations in the
stretch-ratio (see stretch number 2). All in all, the hooligan case can indeed be classified as a
high-pressure situation.
To validate and verify the performance WRS constructs, (log) data were examined to determine
whether the decompensation event that unfolded during that day could also have been identified
using performance WRS data methods. A spike in delay development was identified for trains in
the 1700 series, indicating a segment of 36 trains traveling the same rail trajectory (Figure 5).
The upward slope could be explained by three ‘hooligan trains’ (all part of the 1700 series). Two
trains suffered imminent, rapidly increasing time table delays due to the fact that they could not
leave the station as a direct result of hooligans and fireworks on the tracks. The third train was
used by the riot police to forcefully transport soccer hooligans out of the station. In addition,
knock-on delays occurred due to the fact that trains retained from departure occupied the rail
66 A. W. Siegel
platforms. This induced red rail signals (an increase of 9.2 % above average) for connecting
trains, forcing trains to wait on rail tracks surrounding the station.
Average delay per train in the 1700 serie

Time delay in minutes
31-3-2014 to 4-4-2014
6.0
5.0
4.0
3.0
2.0
1.0
0.0
-1.0 31-Mar 1-Apr 2-Apr 3-Apr 4-Apr
workstation 1 workstation 2 workstation 3
total SD
FIGURE 5 Performance WRS graph 31-3-2014 to 4-4-2014
3.1.4. WRS Analysis function

The successful identification of the known hooligan event (i.e., it was observed during the
ethnographic study) contributes to verification of the WRS method. Deviations from normal
operational baseline periods, defined as the steady state of a rail control post in which rail
movements occur as planned without any intervention, could be established on the workload as
well as the punctuality boundary. However, signalling of the hooligan event does not
immediately create insight and understanding into the unknown variables in the normal
performance variability that could indicate potential creeping sources of future resonance that
may underlie the incident. Relatively long-lasting disruptive events with a big impact factor (i.e.,
affecting multiple trains and dispatchers) are likely to gain attention amongst actors in the
system even without WRS indications. However, when such an event is already known,
attention is needlessly diverted which may result in obscuring potential other unidentified
factors that influence the resilience state. To enhance the organization’s feedback control loop
(Doyle, Francis, & Tannenbaum, 2013) and increase the understanding, tracking, and
anticipation of potential sources of future resonance and or the impact factors of the different
WRSs indicated by the framework, implementation of WRS analysis functions is proposed. An
analysis function is described as an alternated frame of reference, based on other or additional
performance indicators, which guides the process of selecting WRSs that need to be dealt with.
The aim of this analysis function is to exclude the ‘evident, known and obvious’ causes of
resonance, and attempt to shift attention and reveal ‘hidden, unknown or ignored’ processes that
could affect rail-system resilience. To demonstrate the concept and implementation of this
A. W. Siegel 67
principle, a punctuality WRS analysis function was established for the hooligan scenario which
will be described in more detail in the next section. It is important to note however, that the use
of analysis functions is not limited to high-pressure situations. Analysis functions are equally
applicable to, and well-suited to uncover (creeping) incident precursors in, routine situations.
Implementation of the WRS analysis function allows the frame of reference for the punctuality
boundary to be manually altered by excluding trains with exorbitant delays due to well-known
escalation events (i.e., the hooligan trains) and by comparing the real-time delay measurements
to specified base line conditions (performance indicators; train-series and specified dates). To
test the applicability of the WRS analysis function method, the delay data from the second week
of observations were re-examined (figure 6).
Average delay per train in the 1700 serie with exclusion of

time delay in minutes
the 'hooligan trains' on 2-4-2014

3.0
2.5
2.0
1.5
1.0
0.5
0.0
-0.5 31-Mar 1-Apr 2-Apr 3-Apr 4-Apr
workstation 1 workstation 2 workstation 3 total
Linear (total) SD
FIGURE 6 WRS analysis function applied on delay data from the 1700 series
The three hooligan trains, which caused the exorbitant delays, were excluded from the analysis.
Ad hoc analysis revealed an upward trend in delay development for the 1700 series. It could be
argued that an average delay development increase of 1.7 minutes (102 sec.) per train does not
exceed the predefined organizational threshold of ≥ 3 min delay and, as such, does not require
further investigation. However, it could be beneficial to examine whether specific trains in this
series contribute invariably to this delay development and whether this upward trend continues
over time (e.g., the consecutive days or weeks). In addition, the time delays may impact the time
buffers built-in on the pre-defined timetable and as such influence the rail dispatcher’s
workload. Such information could aid in forestalling and anticipating future resonance emerging
from ‘seemingly insignificant’ (creeping) change patterns and might even identify
commonalities in the operating state preceding well known events.
WRSs and WRS analysis functions should be created to (visually) support the train dispatcher’s
comprehension of the current operating state, resilience status and to enhance prediction of
possible incidents and accidents in the future by guiding attention to aspects that deserve further
68 A. W. Siegel
analysis. They provide a means to an end and will not in itself present an integrated approach to
improve the resilience or related aspects of the system. In other words, rather than directing the
domain practitioners along a defined path, exploratory content that allows for comparison
between data is provided.
3.2. Resilience questionnaire

The response rate to the questionnaire was calculated according to the American Association for
Public Opinion Research (2015) RR1 definition. Of the 67 employees contacted, one person no
longer worked for the company and a second person abstained due to prolonged absence. This
resulted in a RR1: 22/65 = 34%, which is acceptable for online surveys (Nulty, 2008). The
sample demographics were as follows: 16 rail dispatchers, 3 managers and 3 front office
employees. In total 2 female and 20 males answered the resilience questionnaire. Results from
the ADAPTER questionnaire (Table 2) are consistent with the resilience baseline conditions and
current operating state of the system ascertained during the observations in that the domain
practitioners rated the resilience constructs monitoring and responding higher than the resilience
constructs anticipating and learning. It should be noted, however, that the scores for anticipating
(Mdn=3.25) and learning (Mdn=3.17) fall within the average range of the five point Likert scale,
being at variance with the observational results which indicated underperformance for these
constructs. This could indicate miscalibration of resilience levels (Woods & Wreathall, 2008;
i.e., learning construct α=.70 and SD=.43) within the organization. However, the scores might
also be explained by fluctuation in resilient behaviour that was observed within and between
(senior and junior) rail dispatchers when they were coping with the (dynamic propagation of)
random disturbances during real-time operation. A Mann-Whitney U test was conducted to
evaluate whether anticipating resilience scores differed between senior and junior rail
dispatchers. Although the results indicated that anticipating resilience scores between senior rail
dispatchers (Mdn = 11,83) and junior rail dispatchers (Mdn = 10,00) were not significantly
different, U = 30.0, p = .608, r = .11, differences in resilient behaviour cannot be ruled out
completely since common and socially desirable answers could have been given by the rail
dispatchers answering the questionnaire. During real-time operation, specific situational
demands could elicit differences between junior and senior rail dispatchers based on experience.
Situational demands in itself might provide an indication as to why the acceptable degree of
internal consistency (α=.70; Tabachnick & Fidell, 2001) was not met since it could also explain
the operational variability observed within individual dispatchers. This essentially reflects the
rail dispatchers’ notions that no situation is alike, even though situations might appear similar to
outsiders since they for example both entail disruption handling due to a broken rail switch. In
addition to the resilience constructs, domain practitioners assessed the relation-oriented abilities
(i.e., shared transformational leadership and cooperation with other teams), which are
incorporated into the ADAPTER questionnaire to operationalize the concept of team resilience,
as the least well-represented within the organization. These results are in line with the resilience
observations. Transformational leadership has proven to be a leadership style that effectively
A. W. Siegel 69
stimulates knowledge creation and knowledge sharing at the individual and group levels
(Bryant, 2003). The fact that that this ability is under-represented (Mdn=3.14) could affect the
learning capabilities of the organization (Zagoršek, Dimovski, & Škerlavaj, 2009), and as such
explain the performance variability observed. The low rating for cooperation with other teams
(Mdn=2.54) indicates improvement opportunities for handlings across organizational and sub-
system boundaries, such as the communication breakdown that occurred in the hooligan
example.
TABLE 2 ADAPTER questionnaire; Descriptive Statistics and Reliability Coefficients
ADAPTER Scales n items Mdn SD Cronbach’s Alpha (α)

Monitoring 13 3.69 .75 .95
Responding 12 3.46 .75 .90
Anticipating 8 3.25 .77 .68
Learning 9 3.17 .43 .70
Shared transformational leadership 14 3.14 .60 .91
Cooperation with other teams 12 2.54 .65 .71
Note. N= 22. Total scale team resilience: α= .91
4. Discussion and conclusion

Practical implementations and concrete measurement of resilience is a challenging issue that is
inadequately addressed in current research practices. The purpose of this research was therefore
to take initial steps towards enhancing operationalization and understanding of resilience metrics
in the railway sector and quantification of the rail-system resilience state. Overall results
indicate that the multi-method approach adopted to establish operational baseline conditions,
based on the four system capabilities that comprise resilience, is a reliable method to determine
the overall level of rail-system resilience. In addition, WRS’s prove to successfully measure
deviations from predetermined resilience baseline conditions. Although WRS analysis functions
show the potential to enhance understanding of the underlying and complex system dynamics
that comprise future resonance, more research is required to determine their full potential.
More specifically, it can be stated that triangulation of the quantitative and qualitative research
methods proves to be a useful means to capturing more detail, minimize the effects of research
biases and limitation boundaries of the individual research techniques, and understand causal
mechanisms. The observation (behavioural) and the resilience questionnaire outcomes
(attitudinal) both indicated that the system capability ‘monitoring’ was best represented within
the organization and that the relation-oriented abilities that represent team resilience were the
least developed, reinforcing the outcomes. Grounding these results within Hollnagel's (2009)
framework, a (recurring) pattern emerges that generates insights into the current practices and
how this influences the resilience level and operating state of the socio-technical system. The
70 A. W. Siegel
commonality across all real-time dispatching processes is that by means of continuously

monitoring and quick responding, dispatchers try to control the situation and mitigate potential
threats. In essence this reflects exactly the current practice of dispatch activities, monitor the
traffic flow and act accordingly. This generally yields positive and acceptable levels of
performance. However, performance variability increases when the “normal” system
functioning (i.e., the corridor team serves as base adaptive capacity; Woods et al., 2014) is
challenged and disturbances (external events) cascade across sub-system and organizational
boundaries, enhancing the chances for a system decompensation collapse (Branlat & Woods,
2010). To prevent such system breakdowns from happening, timely notification and anticipation
to incident precursors is crucial. To accomplish this, the theoretical resilience state model for
railway systems (Siegel & Schraagen, 2014d) was implemented to measure deviations from the
resilience baseline conditions on the performance system boundary. The measurements in
performance variability were translated into WRSs that act as prompts for variables that should
be considered. In this process, the observations and resilience questionnaire provide the
necessary contextualization that the technical system metrics alone are unable to fully capture.
The quantification of WRSs and visualization of cues in a constructive manner helps to close the
feedback loop and enhance situation awareness. Boosting the relation-oriented abilities within
the organization can strengthen these processes and as such reinforce the overall level of system
resilience (Hollnagel, 2009). Enhancing cooperation and knowledge sharing with other teams
could for example aid in minimizing information-processing failures (Woolley et al., 2008)
where transformational leadership (Bass, 1990) could contribute to sense-making, interpretation
and understanding of a situation amongst all members of the (corridor) team (Bartone, 2006). In
this context a WRS analysis function should be seen as a means to an end. By providing a means
to uncover potential factors that comprise a WRS, a WRS analysis function can be implemented
to guide the WRS selection process and enhance corridor team reflectivity. In this context,
reflectivity is defined as the deliberate process of discussing and evaluating team goals,
processes and outcomes, learning from failure and successes and craft action intentions for
improved future functioning (Ellis, Carette, Anseel, & Lievens, 2014; Schippers, Edmondson, &
West, 2014).
Although results are promising and preliminary feedback from domain practitioners indicates a
positive attitude towards implementation of this method, research limitations should be
considered. Even though operational parameters were chosen that allow for real-time
measurement in the future, the current implementations are based on technological measurement
of readily available log data and were created in retrospect. Further empirical research is needed
in order to validate and verify these prospects and results during real-time operations. In
addition, the fact that we operated in a real-life setting poses a limitation with respect to the
replication of the study. Since every rail control post (e.g. the practitioners, the vibe) is different,
and the rail control post themselves play a crucial role in the outcome, an exact replication of
this study would not necessarily yield similar results. Although it could be argued that this
would be possible in a simulated environment, replication in such a literal way was never the
main priority for this study. We would rather invite and encourage other researchers to use this
A. W. Siegel 71
study as a base and build upon this work. A potential implication for future work in the line of
real-time WRS research would be to create a fully operational advanced graphical user interface
design, which can be used to test and capture the complex interactions generated by interrelated
components at system level (e.g., usability enhancement based on ecological resilience design
principles). Another option is to mature the implementation of WRSs and analysis functions by
adding specification criteria and including other resilience boundaries (i.e., safety boundary).
Furthermore, it could be examined whether the punctuality WRS and analysis function, which
were created to enhance the system capabilities learning and anticipating in the railway system,
can be used to enhance these system capabilities in other control room operations of complex
socio-technical systems. In addition, it could be interesting to explore these metrics in the
broader context of other scientific fields like data science and predictive statistics.
Acknowledgement
The authors wish to thank the ProRail control post at Zwolle for their hospitality and Dolf van
der Beek for sharing the ADAPTER resilience questionnaire. This research was conducted
within the RAILROAD project and was supported by ProRail and the Netherlands organization
for scientific research (NWO) (under grant 438-12-306).
Open Access
This article is distributed under the terms of the Creative Commons Attribution 4.0 International
License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use,
distribution, and reproduction in any medium, provided you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons license, and indicate
if changes were made.
72 A. W. Siegel
TEAM REFLECTION IN RAIL

OPERATIONS
A. W. Siegel 73
74 A. W. Siegel
CHAPTER 4
TEAM REFLECTION IN RAIL
OPERATIONS
Team reflection makes resilience-related
knowledge explicit through collaborative
sensemaking4
Abstract
Resilience is defined as the ability to adaptively deal with system boundaries in the face of the
unexpected and unforeseen (Branlat & Woods, 2010). We hypothesize that drawing upon
resilience-related knowledge is a prerequisite for such adaptivity. This paper proposes team
reflection (Ellis et al., 2014) as a macrocognitive function to make the resilience-related
knowledge explicit. This knowledge is implicitly available with individual team members active
at the sharp end but is never explicitly shared due to invisibility of goal-relevant constraints. To
overcome this invisibility, we suggest an application that makes changes in the current rail
socio-technical system visible in terms of the three system boundaries, a variation of the
originally proposed by Rasmussen (1997): safety, performance and workload. This allows a
team of rail signallers to analyse movements toward system boundaries and share knowledge on
these movements. An observational study at a rail control post was conducted to assess the value
of team reflection in making resilience-related knowledge explicit. For this purpose, we
developed a first prototype of the application concerning the performance boundary only. Using
naturalistic observations of a team during a week, we observed how they reflected at the end of
their shift on salient system changes. A global content analysis was used to show the relevance
of the content to resilience and to test the increase of the resilience-related knowledge
throughout the observation period. A specific case of a human approaching the rail tracks, as a
potential suicide, was analysed in detail. The results show the value of team reflection on system
movements towards their boundaries, thus making goal-relevant constrained knowledge explicit
within the operational rail environment.
4
Manuscript submitted for publication after peer review in Cognition, Technology & Work as Siegel, A.
W., & Schraagen, J. M. C. (2016). Team reflection makes resilience-related knowledge explicit through
collaborative sensemaking.
A. W. Siegel 75
1. Introduction
Resilience engineering studies, among other aspects, the ability of a socio-technical system
(STS) to reorganize and adapt to the unexpected and unforeseen (Hollnagel et al., 2006).
Hollnagel (2009) theorizes that a resilient STS needs four essential capabilities: responding,
monitoring, learning and anticipating. These capabilities differ in moment and scope – actual,
critical, factual and potential – but have in common the need for explicit relevant knowledge and
the ability to apply this knowledge. This knowledge is partly available at the sharp end of the
system, for instance with control operators, who are a component of the STS interacting with the
system environment. However, this knowledge is frequently implicit due to the fact that goal-
relevant constraints, required to deal with unforeseen disturbances, are not visible for operators,
thus hampering efficient knowledge-based behaviour (Burns & Hajdukiewicz, 2004;
Rasmussen, 1983, 1985; Vicente & Rasmussen, 1992). The question arises which method
should be used to make the available knowledge explicit. We have addressed this question in the
context of a naturalistic observation study conducted at a rail control post. A rail STS is an
example of a system that constantly needs to adapt to disturbances, and rail signallers working at
different control posts are responsible for making the rail infrastructure available in a safe and
timely manner, in the face of daily disturbances.
We propose team reflection (Ellis et al., 2014; Reymen, 2003; Schippers, Den Hartog, &
Koopman, 2007; Schippers et al., 2014; West, 2000; Wiedow & Konradt, 2010) as a mechanism
for the team at the sharp end to make the resilience-related knowledge explicit. The resilience-
related knowledge is defined as the knowledge required to adapt to goal-relevant constraints
(Rasmussen, 1985) imposed by the goals of the STS as they occur due to unexpected and
unforeseen events (Branlat & Woods, 2010). Team reflection includes behaviours such as
questioning, analysis, making use of knowledge explicitly, reviewing past events with self-
awareness, and coming to terms over time with a new awareness (West, 2000). Team reflection,
in a loop with planning and action, is commonly used in a broader reflexive process (Schippers
et al., 2014; West, 2000) where team members collectively reflect upon the team’s objectives,
strategies (e.g., decision making) and processes (e.g., communication). The results of such a
reflection can be fed back into the planning and action/adaptation loop to improve team
performance (Schippers et al., 2014). However, in our case the objectives of reflection are to
transform implicit to explicit knowledge, at the sharp end, relevant to the resilience of a socio-
technical system as a whole rather than the team itself as a focal point for reflection. This
knowledge goes beyond the direct responsibility of the team. Implicit knowledge is tacit
knowledge, a form of private knowledge that is treated as “informal,” and even, in a sense,
“unconscious” knowledge (Day, 2005; Polanyi, 1969), that can be transformed to explicit
knowledge (Frappaolo, 2008). We are interested in knowledge, relevant to system resilience,
acquired throughout the regular work of the rail signallers. Resilience is about the behaviour of
the socio-technical system (STS) when it approaches and possibly crosses its boundaries (Siegel
76 A. W. Siegel
& Schraagen, 2014a; Woods, 2006b). The behaviour of an STS is an interrelated process of all
the different participants and technology (Simon, 1996; Waterson et al., 2015), which goes
beyond the direct responsibility and the team’s span of control at the sharp end. However, the
team at the sharp end is exposed and aware outside its span of control, and is able to reveal
knowledge related to resilience of the whole STS. Therefore, reflection should be applied to
system goals rather than team goals.
Rail STSs have three main system boundaries: safety, performance and workload (Siegel &
Schraagen, 2014d). Since resilience of an STS manifests itself through its ability to adapt and
reorganize (Woods, 2006b) around its boundaries, it is the case here as well for the rail STS. The
movements of the operating system towards these boundaries during a shift of the control room
operators, relative to the movements in a previous period, may include hints of system behaviour
around the boundaries. As system behaviour around the boundaries is the essence of resilience,
we expect that reflection on these movements will cause resilience-related topics to arise. We
assume that the broad nature of system boundaries will cause discussion beyond the scope of the
team. For example, the team is responsible for setting timely train paths, but through
communication it has knowledge about the personnel on the train, which is beyond its
responsibility but can be discussed. These system movements, relative to a previous period,
represent weak signals that could possibly signal a ‘drift into failure’ (Dekker, 2011). These
changes can function as cases to learn from, when the system succeeds in dealing with the
situation and things go right as opposed to wrong (Hollnagel, 2011a). The challenge is to make
weak signals explicit and institute a process within the rail traffic control organization to
explicitly reflect upon these weak signals to be able to learn from and therefore possibly
anticipate subsequent disturbances.
The proposed team-reflection process is applicable, among others, to control rooms of STSs and
in particular a rail control post. Our research question, in this context and based upon the
introduction above is: Does team-reflection, on STS movements towards its boundaries, make
resilience-related knowledge explicit? This research question can be divided into three sub-
questions: 1) What is required, in terms of information presentation to operators, to make
knowledge explicit? 2) Is the explicit made knowledge resilience-related? 3) Does ongoing
practice of team reflection increase the use of resilience-related knowledge? We attempted to
answer these questions by conducting an empirical observational study at a rail control post
following our proposed process of reflecting with the whole team at the end of their shift. To
facilitate team reflection, we developed a prototype, named Resiliencer-performance, which
presents movements towards the performance boundary and provides simple analysis functions
to retrieve the data behind the daily movements. In order to motivate team reflection on topics
related to resilience, we captured movements of the Operating System (OS) towards and from
the boundaries as described in our previous research (De Regt, Siegel, & Schraagen, 2016;
Siegel & Schraagen, 2014a, 2014d). In this paper, we focus only on the performance boundary
and particularly on punctuality. We assume that one boundary is sufficient to study the proposed
team-reflection process and trigger discussions that will make resilience-related knowledge
A. W. Siegel 77
explicit. Resilience is a result of interrelated forces and trade-offs caused by the three boundaries
(Amalberti, 2001; Cook & Rasmussen, 2005; Hoffman & Woods, 2011). However, a weak
resilience signal may be triggered by identified movements towards one boundary only (Siegel
& Schraagen, 2014d). We have focused on the interaction with all three boundaries in a separate
publication (Siegel & Schraagen, 2017), which enlarges the discussion topics with interrelated
cases. In the next section (2) we describe the methods used for the after-shift team reflection, for
the Resiliencer-performance implementation, and for the analysis. In section 3, we describe the
observational study design and in section 4 we present the results followed by a discussion
section (5).
2. Method
We first describe the setting to understand the context of the methods. The sociotechnical
system we have studied is a Dutch rail-post responsible for the area North and West of
Amsterdam with about fifty rail stations and a thousand daily train trajectories (see figure 1).
The work, performed 24/7, is assigned to rail signallers during the day across four workstations
and to one regional dispatcher, who is out of scope of this study. The rail signallers must
monitor the system planning and execution. During disruptions, they adjust the planning,
manually direct the system and follow safety procedures and protocols including communication
with train drivers and other personnel. They enter information about every train delay of more
than three minutes through a dedicated application, noting the cause of the delay. This is the
only place where they systematically capture their knowledge about the system. The rail
signallers perform their tasks, transfer the status to the next signaller at the end of their shift and
currently go home without any organized discussion about their work. In case of large
disruptions, they may be approached for questioning by staff members or their managers, in
most cases a few days or more afterwards, but never immediately after an incident or calamity
occurred. The team reflection by the rail signallers that we introduced at the end of their shift is
a new activity described in the next subsection. The following subsection describes the
requirements and prototype of a tool to support this reflection process. The last two subsections
describe the reflection analysis method used to identify the global content and the data-framing
method to analyse the expression of explicit knowledge.
78 A. W. Siegel
FIGURE 1 The rail map north of Amsterdam with stations mentioned in the text
2.1. Team reflection of rail signal operators

at the end of their shift
The team mentioned in this paper is a group of rail signal operators working together during a
shift at a rail control post. We depict the effects of after-shift reflection on the team’s knowledge
in figure 2. The signal operators interact individually with the rail STS while being part of it as
well. Throughout the interaction they gain individual knowledge on the rail STS, which is
partially made explicit through data entry by the signallers themselves into the system. Changes
of the rail socio-technical system are expressed by “to/from boundary movements”. Through the
proposed reflection process with analysis of the “to/from boundary movements”, they will
exchange some of their knowledge causing it to become explicit.
A. W. Siegel 79
FIGURE 2 After-shift reflection of the operator team
2.2. Tool for presentation and analysis of relative movements

towards the performance boundary
In the introduction, we have explained that operators lack visibility of goal-relevant constraints.
This visibility, to stimulate and support discussion, can be obtained by the reflecting team
through a tool presenting a view with analysis functions on its shift. The view should present
movements towards the system boundaries relative to a previous period (De Regt et al., 2016;
Siegel & Schraagen, 2014d). In this research, we focus only on the performance boundary.
Performance in the rail sector is a combination of punctuality and capacity. In the short term,
mainly punctuality plays a role since capacity is nearly constant through its year planning and
the effect of disturbances on the capacity is presented through punctuality as well. Punctuality of
rail operations is well-defined as the difference between planned (i.e. according to the latest
published timetable) and actual moments of arrival or departure from a specific station (R. M. P.
Goverde, 2005; Hansen, 2010). However, in our case we deal with many stations in a large area,
many trains, different routes and shift periods, which need an extended punctuality definition.
We have taken all the delayed trains (equal or more than 3 minutes) of a shift within the
controlled area and calculated their average delay increase between entering and leaving the area
(see detailed description in the appendix).
The presentation should be of a real-time nature to capture data until the start of the reflection
and it should have an ecological interface (Siegel & Schraagen, 2017), “to reflect the constraints
of the work environment in a way that it is perceptually available to the people who use it”
(Burns & Hajdukiewicz, 2004, p. 1). The changes in punctuality of an area with respect to a
previous period is a value well understood by the rail signaller, but it is difficult to translate a
system punctuality number of an area to a specific identifiable component. For that reason, the
application needs to provide a simple analysis function, which helps to make the link between
the high-level punctuality change of the area to the identifiable component. We used the
adjective “simple” to emphasize that the analysis function needs to fit an operator, as opposed to
an analyst. We built a prototype, referred to as Resiliencer-performance, fulfilling these
requirements (fig. 3). The application used real-time data and presented in live mode the relative
punctuality of the whole area. The area was split into 4 main trajectories to have an initial clue
which of the trajectories contributes most to the overall result (left side of figure 3). The
trajectories were divided in passenger and freight trains, since both have a different
characteristic concerning time delays. Passenger trains are tightly coupled to the on-line
published time-table, while freight deviates frequently and has a lower punctuality priority. For
the analysis mode (right side of figure 3), we implemented a search function to locate the
contributing train with only 3 button pushes (see an example in the results section 4.1 figure 7).
80 A. W. Siegel
On the right-hand side in the analysis mode, the rail signallers can zoom into one of the
trajectories. The top window displays average train delays of train series. Choosing one of the
train series will result in the middle display with all the specific trains. Choosing one specific
train will result in a delay diagram across its trajectory of this train in the actual shift and the
average delay of the specific train in the reference period. The graphs expose the specific train
behaviour within the shift and expose patterns or deviations of the same train in the reference
period. This information with its reasoning beyond the technical data, represents the constraints
of the performance boundary in terms identifiable by rail signallers. It is the basis for discussion
on information beyond the hard technical figures, enabling the related knowledge to be made
explicit.
A. W. Siegel 81
FIGURE 3 The Resiliencer-performance in live mode (top) and analysis mode (bottom)
82 A. W. Siegel
2.3. Reflection global analysis

We recorded the team-reflection sessions and analysed them. With this approach, we created a
global overview of the content throughout the reflection, which is dependent on the events
occurring during the shifts. We segmented the discussion topics in the global overview to areas
that are essential to resilience behaviour, to illustrate that the content is resilience-related. The
aim of the global content analysis is to show the relation of the discussion topics to resilience. In
the introduction, we explained our assumption that reflection on system movements to and from
system boundaries, with their margins (Rankin, Lundberg, Woltjer, Rollenhagen, & Hollnagel,
2013), will evoke resilience-related topics. This follows from the definition of resilience as the
ability to adapt and anticipate to changes around system boundaries (Woods, 2011). In the
following, we describe a coding scheme enabling us to decide whether discussion topics are
related to resilience or not. The first category in the coding scheme is “adequately dealing with
procedures”. Dekker (2003) claims that organisations should invest in their understanding of the
gap between procedures and practice, and help develop operators’ skill at adapting. He sums up
four reasons why procedures do not work, although they are thought to represent the best
thought-out, and thus the safest way to carry out a job. The first reason is that there exists a
mismatch between procedures and practice. Second, the real world has limited resources to
follow the procedures. Third, procedures cannot describe complex situations well. Fourth,
procedure-following can be antithetical to safety. Discussion about the flexibility of procedures
(Bourrier & Bieder, 2013) in the real world will improve the performance when applying,
adjusting or even neglecting them in real situations. For this reason, our first category of interest
to look for in the discussions is “adequately dealing with procedures”. The second category is
“communication with counterparties” (Baysari, McIntosh, & Wilson, 2008; Murphy, 2001;
Shanahan, Gregory, Shannon, & Gibson, 2007). Baysari et al. (2009) reviewed nineteen rail
safety investigation reports in Australia and found, among others, that the communication
between driver and signaller was the only recurring error to contribute to 12 analysed railway
incidents. In the UK, research has identified types of communication errors involved in railway
incident occurrence (Murphy, 2001; Shanahan et al., 2007). These findings emphasize the
importance to discuss communication as contributing to resilient behaviour. Moreover, in
discussions about communicative actions that need to take place to deal with disturbances the
adaptive capacity (Branlat & Woods, 2010; Lundberg & Johansson, 2015) is reflected and cross-
scale interactions (Woods, 2006a) are reflected in discussions about the communication that
took place across levels in the rail system (e.g., between posts). The adaptive capacity is also
reflected in discussions about similar cases, which is the third category - “reference to similar
cases”. Discussing similar cases in the same context shares understandings, decisions made and
reasoning, which can be adopted by other team members in future situations (Haunschild &
Sullivan, 2002; Hovden, Størseth, & Tinmannsvik, 2011).
We categorized each discussion topic as follows:

 Procedure mentioned
A. W. Siegel 83
 Communication – this is split-up to the main parties the rail signaller communicates
with.
o Train driver
o Police
o “Knoco” – name abbreviation of the station-node coordinator
o Regional dispatcher or another rail signaller
o Another person
 Reference to similar cases
A combination of the three categories, when a train with a deviation from the plan is discussed,
indicates a relation to resilience as discussed above.
2.4. Data-framing
We view team reflection on relative system movements as a macrocognitive process of
collaborative sensemaking (Fiore et al., 2010) to explain how knowledge is made explicit. The
data-frame theory of sensemaking (Klein, Moon, & Hoffman, 2006) postulates that elements are
explained when they are fitted into a structure that links them to other elements. The term frame
is used to denote an explanatory structure that defines entities by describing their relationship to
other entities (Klein, Phillips, Rall, & Peluso, 2007, p.118). The initial trigger for the
sensemaking process in our domain of interest is information (data) on relative operating state
movements towards the boundary. Each of the reflecting team members is exposed to this data
and frames it based upon his or her own implicit knowledge. In figure 4 we have depicted the
interaction between the individual data-framing and the shared data-framing of a team reflecting
together. As seen in the data-frame diagram in the left side of figure 4, the individual is
questioning his data-frame model by tracking anomalies, detecting inconsistencies and judging
plausibility. The questioning may cause the need to reframe or to elaborate/preserve the data-
frame. When some equilibrium is achieved one of the team members may express himself
explicitly and share his data-frame with the team. In the social domain, the team will verbally go
through a similar process of discussing the shared data-frame (the right side of figure 4) by
questioning and when needed by reframing or elaborating. During the discussion, the knowledge
is shared and becomes explicit and thus available for all team members. The data-frame cycle in
the team domain will influence the individual domain, through personal questioning and new
frames to compare with. This interaction between the team and individual is continuous, until a
satisfactory equilibrium is found, through which related explicit knowledge is made available.
This generic process of an individual making his implicit knowledge explicit through a social
team process is not new. However, we are trying to explain how resilience-related knowledge is
made explicit. We use the behaviour of the system towards its boundaries as stimulator, being a
derivative of resilience by its definition (Woods, 2006b), as mentioned above. The relative
system movements also present drifts toward the boundary, which may not be noticed on a daily
basis, but may be amplified by presentation of work-shift data in comparison to previous longer
84 A. W. Siegel
periods. Our research focuses on making drifts toward the boundary more notable for team
members, such that they can explicitly reflect upon this drift during team reflection and
collaborative sensemaking. This noticed drift is seen by Dekker (2011) as a resilience
component to act upon. The data-frame theory depicts the process from relative system
movement data, through relevant framing, towards resilience-related knowledge.
FIGURE 4 Individual Data-Frame interacting with shared Data-Frame
Within the data-frame theory of sensemaking (Klein et al., 2006), a process of questioning,
elaboration or reframing is described. In our method of data-framing we are mimicking this
process to show how knowledge is made explicit. We record the verbal expressions during the
discussion and transcribe them in line with verbal analysis (Chi, 1997). This time we take a few
sentences expressed on a topic and refer to it as data, which needs a frame to fit in. The frame, a
construct to that of cognitive schema (DiMaggio, 1977), is an organized pattern of thought or
behaviour that organizes categories of information and the relationships among them. Since the
reflection starts with the Resiliencer-performance, the first frame discussed is triggered from
that domain. The main frame triggered will be the Delay-frame, triggering thought and
information on the occurrence of delays, but other frames can be triggered as well. For example,
a Graph-frame concerning information of graphs, or a Reference-frame about thoughts and
experiences of the reference period. This method is used to describe and cluster the knowledge
made explicit, where the mapping within a frame group is only a means to structure the
knowledge. In the results section, we will present a case which on the one hand clarifies the
method and on the other hand shows, with help of sequenced data-framing, how knowledge is
made explicit and how the Resiliencer-performance has triggered the discussion, through the
initial frame.
3. Observational study design

A. W. Siegel 85
The study design at the Dutch rail-post described above introduced team reflection with consent
of the management and the rail signallers at the rail control post (figure 6 in the top-centre). At
the end of the rail signal operators’ duty (figure 6 in the bottom-centre) the team discussed
delays within its controlled area. For that, they used the Resiliencer-performance (figure 6 left
side). The application was configured for the specific rail-post under investigation. It presented
in live mode the punctuality status and provided in analysis mode the ability to search for
logistic details (i.e., the delay progress of a specific train). The post-area was split up into four
main trajectories covering all stations and each trajectory was controlled by two out of the four
workstations. This caused at least two rail signallers to relate to the results of a main trajectory.
The results of the four trajectories were joined into a result of the whole post during a shift.
The observational study took first place on one try-out day with the new developed Resiliencer-
performance prototype, which was successful. A full working week followed from Monday to
Friday all with two shifts, except for Friday when only the morning shift was taken into account.
The early shift lasted from 6:30 AM until 2:30 PM and the late shift from 2:30 PM until 10:30
PM. The reflections took place at 2 PM for the early shift and at 9 PM for the late shift for about
thirty minutes (see figure 5 for an impression).
FIGURE 5 Team reflection with the Resiliencer-performance
The four rail signallers on duty voluntarily finished their work between half an hour and an hour
before their scheduled ending time, during the observational study period, for a reflection
86 A. W. Siegel
session together with their team leader (the next team voluntarily started earlier to fill in this
gap). They asked themselves the following generic questions:
 Did our shift today proceed better than the average of last period? Why?
o What were the circumstances for the difference?
 Which of the identified circumstances could occur again in the future?
o What can we learn from that?
 How can we deal with these circumstances and what can we do differently?
For answering these questions, they used the Resiliencer-performance and analysed the
numerical punctuality progress in their area. However, reasoning beyond the numerical data
could only be accomplished with help of their personal knowledge and notes made during their
shift (figure 6 in the centre). We recorded the discussion and analysed it (figure 6 top right side)
as described in the previous section. The protocol guiding the observations included an oral
recorder consent, due to cultural constraints and specific request of the post management, and
was approved by the ethics committee of the University of Twente (No. BCE15199 dated 17-4-
2015).
FIGURE 6 Illustration of study design
4. Results
4.1. Sequenced data-framing case: Suicide attempt (person
approaches the rail)
We describe here a case, which we use to explain how knowledge was made explicit, throughout
a reflection session, with help of the Data-framing model. The case is a suicide attempt, which
A. W. Siegel 87
occurs almost daily within the Netherlands. In 2011, 205 attempts and in 2012, 188 attempts
were successful (CBS - Statistics Netherlands, 2015). This frequency causes adaptations in
behaviour of train drivers when dealing with human approaches to the rail tracks. The standard
procedure of a rail signaller in this case is as follows. When a train-driver reports the rail
signaller that he has seen a person approach the tracks, the rail signaller alerts other train-drivers
on the trajectory to slow down and watch for that person. Only when a second time the person
has been spotted the rail signaller will call the police. The train-drivers are expected not to stop
unless the person is on the tracks and should let the police approach the person in question.
However, reality does not follow these stated procedures. During our observation period, the
situation of a person approaching the rail occurred twice, without deadly ending. The first time
train-drivers stopped near the person until the police came. The second time the train-driver
decided on his own to take the person into his cabin up to the end-station where he delivered the
person to the police. These cases arose during the reflections. We have used the data-framing
theory to show the process by which information is made explicit and how it is related to the
initial trigger of the Resiliencer-performance. In the left side of figure 7 we marked on the
Resiliencer-performance in analyse mode the steps followed. The signallers identified that the
maximum delay on the trajectory Den-Helder – Zaandam was 13 minutes (marking 1a) and
searched to find the train with that delay on the trajectory (marking 1b). Marking 2 is the
identification of the 3000 series and marking 3 is the identification of the specific 3023 train.
The delay-trajectory is shown in the results window (marked R) and enlarged at the right side of
figure 7.
FIGURE 7 The suicide case (Resiliencer-performance in analyse mode)
88 A. W. Siegel
In Table 1 we have detailed the verbalization of the reflection, the related frame, the frame
group it belongs to and the explicit knowledge within the team-reflection domain. The first step
is associated with the Resiliencer frame group, making sense of what is presented on the screen.
The next step is the identification of the actor and further framing is used to get to the relevant
details of the case discussed. Table 1 includes abbreviations of the following persons: Team
leader (TL) and Rail signaller (RS).
A. W. Siegel 89
TABLE 1 Data-framing making available explicit knowledge
To explain how knowledge was being made explicit, team-reflection can be seen as a frame
sequence process, after which each frame has been elaborated or reframed. The first frame is
from the Resiliencer frame group concerning the relative system movement – the performance
of the shift relative to a previous period, the performance of the main-trajectories and extreme
performance. Reframing causes focus on the actors, train 3023 with its train driver, and is
followed by focus on the case and details within. The following frame sequence occurred in our
case:
 Delay frame was triggered by the Resiliencers relative system movement
o an extreme delay (13 minutes) on trajectory Den-Helder - Zaandam
 Train frame, as an Actor, was triggered by the Delay frame
o Series 3000 with train 3023
o Train 3023 – Driving from Den-Helder to Amsterdam Sloterdijk. 0 minutes
delay at Alkmaar (06:42) and 11 minutes delay at Heiloo (06:46 + 00:11).
 Suicide frame was the case frame triggered by the Delay and Train frame
o Reporting of a person approaching the rail
 Procedure (formal versus reality) frame was a detail frame triggered by the suicide
frame
o Reporting of and to the train-drivers
o The stop of the train-driver not according to the procedure
o Point of interest
90 A. W. Siegel
 The train-driver. His behaviour when locating the person near the
rail.
o Higher abstraction
 Train-drivers and suicides
Through the sequence of data-framing, rail signallers revealed to each other their work “as
done” (Cowley & Borys, 2014, p. 21; Lundberg, Rollenhagen, & Hollnagel, 2009, p. 1298) as
opposed to the work “as designed” or “as described” in the procedures they need to follow. In
our case, they learned from each other how train drivers behave in reality when a person is
approaching the rail, being triggered by information of the Resiliencer-performance. The
standard procedure of ordering the train driver to continue his journey, to wait for a second
spotting and then to call the police, did not work. Understanding the real world, as it unfolds,
includes knowledge, which may be important in next occurrences of similar cases and can also
help in new cases. When the unforeseen and unexpected occurs, it may be crucial to use
knowledge and experience on accepted deviations from standard procedures.
4.1.1. Drift towards the boundary and

the interdependent relationship
Another aspect seen in this case is the slow drift towards the boundary. The average delay of
delayed trains during this shift was 1.8 minutes while the delay in the previous month was 1.3
minutes. This means that the movement towards the performance boundary was only 30
seconds, where only trains with 3 minutes or more are counted officially. If the prototype would
not have presented data for discussion on this seemingly small movement, then in-depth
discussion, making related knowledge explicit, may not have taken place on this suicide case.
Moreover, when searching for the reason of this small daily drift it appeared that the same train
(no 3023) had 6 delay occurrences in the last month (see the brown reference line in figure 9).
All delays started between Alkmaar and Heiloo, indicating that this area causes more delays due
to suicidal behaviour. Curiously, this particular fact has been discussed in the 2 nd chamber of the
Dutch parliament as well (Minister van Verkeer & Waterstaat, 2010), an example of the
interrelatedness of different areas influencing the events and behaviour of the rail STS. These
facts can be used for learning and anticipating. But beyond these interesting understandings it
becomes clear that the relationships of all entities are very interdependent (Collis Lynne,
Schmid, 2012). It is not a matter of a direct link between a signal and its cause. The signal hides
many interrelated entities, which can be revealed through the knowledge exposed by the sharp
end. Like in our case, a short and incomplete list: train-delay, suicide, deviation from
procedures, 2nd chamber, many occurrences between Alkmaar and Heiloo, etc. The reflection on
this weak signal, a delay drift, caused discussion and awareness of complex interrelationships
and enhanced the explicit knowledge as an important resilience behaviour component (Dekker,
2011).
A. W. Siegel 91
The description above is a typical example of a weak resilience signal (WRS) triggering
discussion beyond the movement towards the boundary. It supports our assumption that even a
WRS on one boundary can reveal knowledge with a variety of dimensions and
interrelationships. This case with only thirty seconds drift, got attention because on that day train
3023 had the largest delay, which was the reason for the team to choose that train in the
sequence of section 4.1. The team could choose any deviation, which draw their attentions. The
triggers, weak resilience signals, make knowledge explicit which include resilience-related
components, analysed for the observation week in the next section (4.2).
4.2. Global analysis

The global results of the reflections are presented in Table 2. The first column contains the date,
the day of the month (April 2015, last day was on the first of May), and the shift (E=early and
L=late). The second column contains the number of trains discussed throughout the reflection.
Some trains were discussed, without having any issue, having a role in the logistic stories of the
rail signallers. The next column contains the number of train cases having an issue. The column
after that contains the number of procedures discussed. In hindsight, it became apparent that any
train having an issue was reasoned about with the help of at least one procedure to explain the
actions. These procedures were central in the team’s reflection and signallers discussed whether
they were or should be obeyed as well as the need to adjust them. Some procedures concerned
logistics decisions while others consisted of sequences to act. An example of a logistics decision
is the priority of an express-train over a stop-train. An express-train has priority over a stop-train
until it is delayed more than “x” minutes, minimizing extra delays within the system. An
example of a sequence to act procedure is the moment to call the police when the train driver has
spotted a person near the rail tracks. The standard procedure is to call all train drivers on the
trajectory to spot the person for a second time to make sure the police will not come
unnecessarily. However, this second spotting may cause a delay being too late to prevent an
accident. The next column contains the number of communicators the rail signaller has contact
with, as mentioned in the reflection. The first sub-column lists the total of train-driver, police,
colleague who may be another signaller or a regional dispatcher, or any other person. In most
cases the train-driver is contacted while the others are strongly dependent on the situation. The
last column contains the number of reference cases cited during the reflection. Most cases where
procedures were mentioned were tied to another reference case.
The test week was very quiet, yet cases occurred and were discussed. On average 7.2 trains were
discussed during each shift. On average, 3.1 trains out of the 7.2 were considered to have an
issue.
TABLE 2 Global analysis results
92 A. W. Siegel
Date Num. of Num. of Num. of Num. of communicators discussed Num. of

& trains trains with procedures ref. cases
shift discussed deviation discussed Ttl T-Driver Police Colleague Other discussed
9E 7 4 4 5 4 1 0 0 4
9L 13 3 3 3 1 0 2 0 3
27 E 8 8 11 11 6 3 0 2 8
27 L 9 7 7 11 7 0 1 3 7
28 E 8 3 3 5 3 1 1 0 3
28 L 5 1 3 1 1 0 0 0 1
29 E 8 3 4 3 2 0 1 0 3
29 L 4 0 1 0 0 0 0 0 0
30 E 4 3 4 3 3 0 0 0 3
30 L 4 0 1 0 0 0 0 0 0
1E 9 2 2 4 2 1 0 1 2
From the information in Table 2 we can test whether there was an increase in the use of
resilience-related knowledge categories throughout the observation week. Assuming that the
number of procedures and communicators discussed is an indicator of the quantity, we can
divide it by the number of delayed trains to test an increase per delayed train. The results are
plotted in figure 8, showing a shallow increasing trend line for both ratios during the week. Days
29L and 30L were not plotted since no deviated trains occurred on that day. Days 9E and 9L
were test days of the Resiliencer-performance with full compliance of the reflection protocol,
justifying to add the data to the observation week. Those days were actually the first time the
reflection was performed in operations. The increase during the observation was not substantial,
although a jump on the procedure rate occurred on day 28L. This result indicates a possible
increase in the resilience-related knowledge categories discussed over the course of the
observation week A longer period of observation is needed to strengthen our hypothesis that this
type of knowledge increases when team reflection is facilitated by presenting goal-relevant
constraints.
A. W. Siegel 93
FIGURE 8 Change of procedure and communication ratio with respect to number of deviancy trains
5. Discussion
The purpose of this paper was to test the proposition that team reflection on system movements
towards it boundaries will make resilience-related knowledge explicit. The high level
requirements of a tool used by the operators during their team reflection are: 1) the interface
should be ecologically designed (Burns & Hajdukiewicz, 2004) combining all boundaries; 2) the
data must be of a real-time nature; 3) the interface includes simple analysis functions to relate
system boundary values to specific identifiable components. These requirements give an answer
on the first question posed in the introduction on the needs of the operators to make knowledge
explicit. The tool was prototyped and used during our observation at a rail control post. Through
the analysis of their discussions during a specific case we showed how knowledge was made
explicit, using the data-frame theory of sensemaking (Klein et al., 2006). In this case were able
to show as well the interrelationships between entities during a drift towards the boundary. With
help of a global analysis we showed that the explicit knowledge is related to resilience and that
its use indicates a possible increase throughout the observation. These findings give an answer
on the second and third question in the introduction on the relation to resilience and the increase
of knowledge.
However, proving that a specific knowledge detail will play a role in resilient behaviour when
the unexpected or unforeseen occurs is extremely difficult. Other researchers (Heese, Kallus, &
Kolodej, 2014; Herrera et al., 2015; Van der Beek & Schraagen, 2015; Woods, Chan, &
Wreathall, 2014) have looked at other properties than knowledge to assess the resilience of an
STS. They strongly based their research on two fundamental theories: the four cornerstones of
Hollnagel (2009) and the stress-strain theory of Woods & Wreathall (2008). Schraagen (2015)
94 A. W. Siegel
has introduced the relation between networks, knowledge and resilience. We have made a first
attempt to show the relation between knowledge and resilience in an empirical setting through
reflection (Ellis et al., 2014) and the data-frame theory of sensemaking (Klein et al., 2007). We
showed the relation indirectly by splitting the analysis of the reflection discussion into two –
global and specific. In the global analysis, we showed the relation of the topics to resilience
through three categories derived from previous research - “adequately dealing with procedures”,
“communication with counterparties” and “reference to similar cases”. In the specific analysis,
we showed, with help of sequenced data-framing, how knowledge was made explicit in those
resilience-related topics. In addition, we demonstrated the relation of the frames to the
Resiliencer-performance, a support tool that presents relative system movements towards the
performance boundary, where resilience behaviour is needed. The results support the
assumption that knowledge is related to resilience, but a further quasi-experiment would
definitely strengthen our argument. In addition, we tested the increase in the use of resilience-
related knowledge throughout the observation week. To this end, we controlled for the number
of delayed trains, as any delayed train would surely lead to more knowledge being made
explicit. By the end of the week, an increase was observed in resilience-related knowledge being
discussed, even when controlling for the number of delayed trains, indicating that the
introduction of team reflection was successful. Of course, this result needs to be interpreted
with caution as we did not observe over a longer period of time and cannot state whether this
effect would hold up.
The observation was done in real operations without a control group. We did not organize
reflection sessions without the Resiliencer-performance, since in real operations the cases vary
largely from day to day and would be difficult to compare. However, rail signallers stated that,
in the past, previous debriefing attempts had been less successful than their current team
reflection sessions, because:
 The discussions were not interesting and focused mainly on major events during the
shift, which all of them were already aware of;
 No on-line system exists today, which gives a good picture of the shift. The operational
systems are real-time and are not designed for debriefing.
The reflection-tool we introduced addressed the shortcomings of previous debriefing attempts

by being an on-line tool dedicated to the after-shift team-reflection as it presents the information
from a different perspective than the major events during the shift. Although the field
observations took place during a quiet period, the tool was able to trigger topics rail signallers
were not directly aware of. An example is the identification of delay trends compared to
previous periods. Moreover, the tool caused discussion on things that go right and is therefore in
line with resilience thinking (Hollnagel, 2011). Rail signallers became explicitly aware of
procedures not being followed and shared this knowledge. We assume that through this
awareness learning took place and that in subsequent scenarios anticipation and responding
might be improved. As these activities are thought to underlie resilience performance
A. W. Siegel 95
(Hollnagel, 2009), we may assume resilience of the team as a whole improved. However, due to
limited observation possibilities, we were in the current study unable to look at the team’s
behaviour in the long run. Therefore, any conclusions on the causal relation between knowledge
and resilience are premature at this stage.
We have shown in this paper how the resilience engineering approach of sensemaking on post-
event reconstruction adds value beyond the simple traditional system monitoring. However, the
approach contains an inherent complexity. The success of the reflection making resilience
relevant knowledge explicit depends on the individuals, group dynamics and culture of the
environment (Gabelica et al., 2014; Schippers et al., 2014). Moreover, it also depends on the
information provided to the team to reflect on. It is in the end the group’s responsibility to
identify the right event and reveal the relevant information to each other. More research is
needed to understand ways to overcome these limitations and provide methodologies that result
in a consistent set of information under similar conditions.
The team-reflection in the experiment focused only on the performance boundary, while in
theory (Siegel & Schraagen, 2014b) there are two more boundaries: the safety boundary and the
workload boundary. The concept of gaining knowledge of all the complex interdependencies
beyond this boundary was discussed above in the context of the suicide case study. A more
complete picture may arise through the usage of the three boundaries. An experiment with all
three boundaries (Siegel & Schraagen, 2017) will give more empirical insight on the
contribution of team-reflection in a broader sense and will deepen the understanding of
collaborative sensemaking on related subjects from different boundaries of a system.
Acknowledgement
We would like to thank the post Alkmaar for their openness and cooperation. The good
atmosphere and their enthusiasm has contributed a lot to the success of the research. We greatly
appreciate the guidance and review comments by Alfons Schaafsma. Last but not least we
would like to thank Bert Bierman and Victor Kramnik for the software development and
graphical design of the Resiliencer-performance. This research was conducted within the
RAILROAD project and was supported by ProRail and the Netherlands organization for
scientific research (NWO) (under grant 438-12-306).
Appendix – punctuality definition of an area

An extended punctuality definition is needed to deal with many stations in a large area, many
trains, different routes and shift periods. This definition is the basis for the presentation and
analysis during reflection (see variable definitions used in Table 3). The context is a control area
A and m stations Sj , j=1, …, m. In this area are n A trains, Ti , i=1,…, nA, driving during de shift
96 A. W. Siegel
start end
period between t shift and t shift . Train Ti has at station Sj a punctuality of
act , de p /arr plan ,dep /arr act , dep /arr
Pi , j=t i , j −t i , j being positive when the train is delayed. Where ti,j is
plan,dep / arr
the actual moment of arrival (arr) or departure (dep) of train T i at station Sj and t i,j is
the planned moment. The train Ti has a route starting at station S Bj and ending at station SEj
where S Bj , S Ej ∈ { S j , j=1 , … , m} ∈ A . The punctuality of train Ti at the start of its route in
act , dep plan ,dep
area A (station SBj) is: Pi , Bj=t i, Bj −t i ,Bj and at the end of his route (station S Ej)
act , arr plan,arr
Pi , Ej=t i , Ej −t i , Ej . A train, in this context, is defined as delayed when
( Pi , Bj∨Pi , Ej ) ≥ t d, where td is a time duration set by the rail sector. In our case t d = 3 min. This
definition causes delays of train Ti within its trajectory at area A not be counted as a delay.
TABLE 3 variable definition
Variable notation Definition

A Control area with m rail stations and nA trains
Sj , j=1, …, m Rail station within control area A
Ti , i=1,…, nA Train within control area A
Pi,j Punctuality of train Ti at station Sj
plan ,arr
ti,j Planned arrival moment of train Ti at station Sj
∆ A Pi Increased punctuality of train Ti in area A
∆ A Pgroup Average increased punctuality of group trains in area A
Team reflection needs an indication on the performance of the trains within area A. We have
chosen to calculate the punctuality increase of delayed trains during the shift. We present its
relation to the same parameter during a reference period, which is the last week, month or year.
The increased punctuality of train T i in area A is ∆ A Pi=Pi , Ej−Pi , Bj . The average increased
start end
punctuality of delayed trains Ti in area A during shift period between t shift and t shift is
n
1
∆ A Pshift = ∑∆ P
n i=1 A i ,shift
where n is the number of delayed trains driving in area A within
start act ,dep act , arr end

the shift intervalt shift ≤ t i , Bj ∨t i , Ej ≤ t shift causing trains, crossing the shift boundary,
counted in both shifts. The average increased-punctuality of delayed trains in area A during a
reference period of shifts is ∆ A Pref . Movements towards the performance boundary are
identified through the relation between ∆ A Pshift and ∆ A Pref . When the first is larger than we
talk about, a movement occurs towards the boundary, otherwise the movement is away from the
boundary. We have implemented the above in an application called the Resiliencer-performance
(Figure 3).
A. W. Siegel 97
98 A. W. Siegel
DESIGN & PROTOTYPE IN RAIL

OPERATIONS
A. W. Siegel 99
100 A. W. Siegel
CHAPTER 5
DESIGN & PROTOTYPE IN RAIL
OPERATIONS
Beyond procedures: Team reflection in a rail
control centre to enhance resilience5
Abstract
Resilience engineering concepts can complement proceduralization of complex sociotechnical
systems (STS). Proceduralization aims at defining precise and quantified system objectives, and
at defining a process that describes and prescribes how to achieve those objectives. Although
proceduralization has been successfully implemented to capture knowledge and experience, it is
limited when the unexpected and unforeseen occurs. Resilience engineering focuses on this
drawback and seeks for concepts to enable adaptive responses in these situations. We propose a
team reflection process to enhance resilience of a rail STS, complementing its proceduralization.
In the present study, we describe how rail signallers used team reflection, supported by a tool
that allowed in-depth post-shift inspection of train movements. A near accident, occurring
during a one-week observation, is described and used for two purposes. First, it was used as an
example to explain the usage of the support tool. Second, it was used as a reference case of
topics playing a role in evolving accidents. The analysis showed that the topic categories
discussed during the team reflections were similar to the incident categories. This means that
relevant topics are available, when things go right, to learn from and anticipate on. In addition,
we showed that rail signallers, over the course of the observations, increasingly analysed and
reasoned about their work. This enriched knowledge beyond procedures, enhancing the ability to
cope with the unexpected and unforeseen.
5
Published as Siegel, A. W., & Schraagen, J. M. C. (2017). Beyond procedures: Team reflection in a rail
control centre to enhance resilience. Safety Science, 91, 181–191. http://doi.org/10.1016/j.ssci.2016.08.013
A. W. Siegel 101
1. Introduction
The approach of resilience engineering (RE) seems contrary to the proceduralization approach
enabling sociotechnical systems (STSs) to cope with variability of external events. Resilience
engineering deals with the ability of STSs to manage their spare capacity to cope with the
unexpected and unforeseen (Leveson et al., 2006; Madni & Jackson, 2009). Margins are needed
to manage the adaptation to these situations (Branlat & Woods, 2010; Cook & Rasmussen,
2005) when procedures do not exist for the unforeseen or are inapplicable during the
unexpected. The emphasis in these situations is on the management of available abilities, for
which RE seeks methods and tooling with relevant data to manage. On the other hand, the
proceduralization approach focuses on procedures capturing knowledge on “how to do”, job
rules, ingenuity and know-how (Fucks and Dien, 2013, p. 27). Rules and procedures are key
features for a modern organisation to function (Bourrier & Bieder, 2013) and can lead to
confidence in task performance, but also allow a retreat from initiative and responsibility
(Fowler, 2013; Schulman, 2013). Proceduralization aims at defining precise and quantified
system objectives, and at defining a process that describes and prescribes how to achieve those
objectives (Bieder & Bourrier, 2013). This contrast can also be seen as the search for balance
between stability and flexibility in operations (Grote, 2014). The procedures have a stable
character while the resilience approach has a more flexible one.
Combining both resilience and proceduralization may be beneficial despite seemingly divergent
starting points: the rigidity of procedures capturing past experience may be joined with the
flexibility to manage available nontangible capacities. Procedures embody the knowledge base
of an organization with respect to the operation of its technical system but rigidify behaviour
and may result in mindless routine (Langer, 1989; Schulman, 2013; Taylor, 1911). Resilience
promotes mindfulness but is as yet less tangible due to the complexity it is dealing with (Madni
& Jackson, 2009; Woods et al., 2007). We propose to combine these approaches for a team in a
rail control centre.
Rail signallers continuously fit unplanned train movements in the real-time flow of trains. They
are responsible for their own part of the system, a particular geographical area in which they
monitor the traffic situation or cope with a disturbed situation (Farrington-Darby, Wilson,
Norris, & Clarke, 2006; Heath & Luff, 2000; Steenhuisen, 2012). Within their decision space,
they are expected to work according to prescribed procedures. When they have a break or finish
their work, they transfer the status to the next signaller. Procedures with their deviations and
other irregularities, at the moment of transfer, are communicated as facts and are seldom
discussed. This type of information during their shift is only discussed when things go wrong
and need justification and explanation. The results of these discussions, in some occasions, are
fed back into procedure updates. The tools rail signallers work with support real-time
operations, but offer few opportunities to look back to the past and discuss details. The log
102 A. W. Siegel
information is only available to analysts in the back-office, who analyse requested situations.
Neither the tooling nor the regular process provide opportunities to the rail signaller to step out
of the procedure space to learn to cope with its limitations. Their professionalism is mainly
focused on following procedures. During training and inquiries, they can use their
professionalism and think beyond procedures. This happens occasionally and even then the
organizational directive to follow procedures remains. It would be desirable to be able to
regularly distance oneself from procedural thinking (Norros, Liinasuo, & Savioja, 2014), to see
the continuous minor deviations of procedures, to be critical and open minded, and to share
knowledge beyond procedures, which may be used when the unforeseen and unexpected occurs.
We propose that before going home the whole signaller team will reflect (Reymen, 2003;
Schippers et al., 2007, 2014; West, 2000; Wiedow & Konradt, 2010) on their shift with help of
weak resilience signals (Siegel & Schraagen, 2014d). Team reflection includes behaviours such
as questioning, analysis, making use of knowledge explicitly, reviewing past events with self-
awareness, and coming to terms over time with a new awareness (West, 2000). Team reflection,
in a loop with planning and action, is used in a broader reflexive process (West, 2000) where
team members collectively reflect upon the team’s objectives, strategies, and processes. This
concept aims to improve the effectiveness of the team itself and stimulates organizational
innovation in the context of the team’s work. We extend this well-established reflection process
in two directions. The first one is the scope of reflection and the second one is the subject to
reflect on. The scope of reflection has so far mainly been limited to the reflecting team itself.
We expand this to the whole STS, where the team is part of. Rail signallers operate at the sharp
end of the system and are aware of the operating system beyond their scope of control (Flin,
O’Connor, & Crichton, 2008). For example, they are aware of missing personnel on the trains,
which, although not their responsibility, may cause a delay they do have to deal with. Second,
the subject to reflect on are weak resilience signals (WRSs) (Siegel & Schraagen, 2014d), which
contrasts to previously studied strong and explicit objects of reflection such as plans and
performance failures (Schippers et al., 2014; Wilson & Norris, 2005). The knowledge made
explicit through the reflection process may also relate to objectives, strategies, and processes but
is not limited to these elements it and should go beyond them (Siegel & Schraagen, 2016). This
resilience related knowledge, beyond the knowledge embedded in procedures, will enrich
professionalism as well as knowledge for learning, acting and anticipation purposes. These
abilities are three of the four resilience building blocks (Hollnagel, 2009), learn, act, anticipate
and monitor, and as such are expected to enhance resilience.
A WRS is a resilience related signal which needs further investigation, as opposed to a strong
signal which demands immediate action. We have developed a model measuring weak resilience
signals of a rail STS (Siegel & Schraagen, 2014d). The WRSs are derived from movements of
the operating system towards its boundaries. We adjusted the boundary categories initially
proposed by Rasmussen (1997) for a rail system: performance, workload and safety. Each
category was modelled to enable quantification and identification of relative movements –
changes in value during the working shift compared to a previous period like a week, month or
year. These changes can visualize unnoticed drifts, which may contribute to a failure (Dekker,
A. W. Siegel 103
2011). These relative movements do not need absolute values of the boundaries, which exists in
theory but are not known in the real world. The workload measurement model was split into
two. The first component was an objective model measuring data from the operational system
and was based upon the cognitive task load model (Neerincx, 2003). The second workload
component was a subjective unidimensional model (IWS - integrated workload scale; Pickup,
Wilson, Norris, Mitchell, & Morrisroe, 2005) measured through a real-time App for each rail
signaller. The workload models were tested during an observational study with off-line data. A
discrepancy between both models stimulated additional inquiry by rail signallers in that study,
which revealed an underlying operational obstacle concerning shunting (Siegel & Schraagen,
2014d). The performance measurement model, related to train punctuality, was tested through a
second observational study with off-line data. An identified movement towards the performance
boundary triggered further investigation and revealed an operational obstacle, which in that
study concerned the communication between the police and rail signaller during a hooligan case
(De Regt et al., 2016). The performance model has subsequently been extended to measure the
punctuality of a controlled area and translated into a real-time application, which was used by
rail signallers to reflect at the end of each shift during a third observational study (Siegel &
Schraagen, 2016). We showed how reflection made resilience related knowledge explicit and
how the reflection progressed throughout the observation week.
The aim of this paper is to investigate the influence of team reflection, at the end of each shift,
on relative system movements with respect to all three boundaries. We are interested in
capturing team knowledge used during the shift that goes beyond procedures. In addition, we are
interested in comparing team reflection on three boundaries, one boundary and without any
tooling. Our research question is how team reflection complements procedures and how that
possibly influences resilience of the STS. How does the reflection progress in time? Do the three
boundaries make a difference in the type of topics discussed? In order to answer this research
question, we conducted a fourth observational study at a rail control post with a real-time
prototype presenting system movements towards the three boundaries with analysis functions to
support the reflection. As it happened, at the start of the observation a near-accident occurred,
which we analysed for types of topics discussed as they naturally occurred and used this as a
reference to the reflection processes occurring later on that week. We analysed whether there is
a relationship between the reflection and the near-accident to answer the first question above. In
the next section (2) we describe the methods used, which include the design and requirements of
the reflection tool, and the observational setup and analysis. In section 3, we present the results
including the reporting and analysis of the near-accident case. In the last section (4) we discuss
the results to address our research questions and their theoretical implications.
2. Methods
2.1. Requirements and design of the reflection tool
(Resiliencer)
104 A. W. Siegel
Team reflection needs a tool to support the process of identifying weak resilience signals (WRS;
Siegel & Schraagen, 2014d) and making resilience related knowledge explicit (Siegel &
Schraagen, 2015). The team needs this tool because without it, we hypothesize there would be
insufficient and also invalid information to draw upon while reflecting (this hypothesis will be
empirically tested by comparing a reflection process without a tool to a reflection process with a
tool). For designing such a tool we propose the following method in accordance with ecological
interface design (EID) principles, as it was defined as an interface to expose “the constraints of
the work environment in a way that is perceptually available to the people who use it” (Burns &
Hajdukiewicz, 2004; p. 1). The EID approach is based upon the work domain analysis (Naikar,
2013; Vicente, 1999) using the abstraction hierarchy (AH; Leveson, 2016; Rasmussen, 1985) as
a model to understand the work domain constraints. The AH has a similar theoretical
background as the WRS, dealing with constraints through system boundaries. The AH uses five
levels connecting the overall functional purpose to the physical object of the system. Each level
contains a set of what-objects, which have a why-what-how relationship (Lintern, 2009) with
objects in the level above and below. The objects in the level above explain why it is needed and
the objects below explain how this is done. Figure 1 shows the AH of the reflection tool using
the CWA application V1.0.2.1 of BAE systems (Jenkins et al., 2007). The functional purpose is
to enable team reflection on Operating State (OS) relative movements to make resilience related
knowledge explicit. The relative OS movements are changes of boundary field values in the
work shift of the team with reference to those values in a reference period, like the previous
week, month, or year. The functional purpose is achieved by presenting relative changes on the
three boundaries – performance, workload and safety. Each of the boundaries needs a purpose
related function to express its value. From this point, downwards in the AH, the details are more
and more depending on the specific nature of the system. Below, we will work out in detail the
design for our case – a rail system for reflection of rail signallers. Generically, these functions
represent the relative movement of the overall rail system towards the three boundaries.
However, this overview screen provides information on a system level, which is not sufficient
enough to reflect on. Specific details are needed for the team members to recognize and identify
with. The operators are not educated analysts and need a simple search mechanism to relate the
system level information to object level details (Siegel & Schraagen, 2014b). The design of the
tool needs to provide a simple search mechanism to link changes of each boundary domain to
specific details. In the description below we explain, in our case, how this is done.
To make the AH specific for a rail system, we mention in the top row that the functional purpose
is aimed at rail signallers, being part of a rail system. The values and priority measures are
generic, since they refer to the system boundaries. We have edited the performance boundary
description to emphasize that we focus only on punctuality, as an performance indicator. The
punctuality is defined as the cumulative delay of delayed trains within a control area (Siegel &
Schraagen, 2016). The workload value is split into an objective measure from the operational
systems and a subjective workload. The objective measure uses information available in the
system: planning mutations, monitoring driven by the number of trains, command lines entered,
and number and duration of phone calls (Van Broekhoven, Siegel, Schraagen, & Noordzij,
A. W. Siegel 105
2016). The subjective measure is collected via a rating scale application requesting a 1-9 rating
every five minutes (Siegel & Schraagen, 2014a). The safety value is a derivative of possible
SPADs (signals passed at danger), which are mandatory reportable events and are used as a
safety performance measure (Nikandros & Tombs, 2007). The number of SPADs that occurred
in the Netherlands in 2014 was 112 (Dutch Ministry of Infrastructure and Environment, 2015).
This is on average 10 times a year for each of the 12 posts or less than once a month. This
number of occurrences is not enough to reflect on each shift. Instead of the SPADs, we have
therefore taken the number of red signal approaches as safety indicator. The more red signal
approaches, the higher the risk of a SPAD (Siegel & Schraagen, 2014c). The other purpose
related function next to the movements towards the boundary is the search function for
reasoning and discussion, which allows the team to search for relevant technical data. All of the
purpose related functions use object related processes, which are related to rail physical objects,
like a train, signal and plan (see lowest level in Figure 1).
FIGURE 1 Abstraction hierarchy of the reflection tool (Resiliencer)
This description, through the AH analysis with high-level requirements, has been transformed
into a real-time application, we named the Resiliencer, with a main screen, visualising the
relative operational state vis-à-vis its system boundaries (see figure 2). The green triangle
represents the results of the reference period with values in the green circles. The values in the
106 A. W. Siegel
black circles represent the actual shift and have three position possibilities with respect to the
reference – towards the boundary, from the boundary and similar distance to the boundary. The
combination of all boundaries enables the understanding of their interrelationships and trade-
offs (Cooke, Stout, & Salas, 2001; Qureshi, 2008; Tamvakis & Xenidis, 2013). For reaching the
specific details from system level values, we designed for each boundary a three-click search
(see an example with explanation in the results section 3.2.3.). The performance boundary
analysis links the punctuality change of the controlled area to the contribution of a specific train,
searched through train series and time. The workload boundary analysis finds the relation
between a subjective/objective workload change and a specific workstation in time through
workload stretches (Siegel & Schraagen, 2014d) occurring during a shift. The safety boundary
analysis locates the relation of a relative safety change and the hourly occurrences of red-signal
approaches, or safety messages, in the area of a specific workstation. The specific details should
stimulate discussions, to make related knowledge explicit (Siegel & Schraagen, 2015) beyond
the procedures followed or adjusted – one of the goals to be observed during the study.
FIGURE 2 The Resiliencer prototype used in the current study (translated from Dutch)
2.2. Observational study design

The naturalistic observational study took place in the Dutch rail control centre Alkmaar, which
is responsible for all train movements to the North and West of Amsterdam. The area contains
approximately fifty rail stations and controls about thousand train routes a day. The 24/7 control
A. W. Siegel 107
centre contains workstations for different roles (see figure 3): 1) four rail signallers (RS), 2)
regional dispatcher (RD), 3) bridge operator, 4) three public transport announcers, 5) calamity
support, and 6) team leader (TL). On the wall near the team leader was a large presentation
screen, which was used to display the Resiliencer. In front of the screen was the reflection area,
where all RSs and TL gathered. The 24 hours a day are divided into three shifts – the early shift
(6:30 AM – 2:30PM), the late shift (2:30 PM – 10:30 PM) and the night shift. In this study, we
focussed on the rail signallers only. However, the team leader led the reflection and others in the
control room were mentioned in the discussions (section 3.1.) and the safety case described
(section 3.2.). During the observation period the late shift arrived at 2 PM, making it possible for
the early shift to reflect with the team leader from 2:00 to 2:30 PM. The late shift reflection took
place at about 9:30 PM, during which the backup rail signaller and the regional dispatcher were
monitoring the four workstations. The observation took place in a single week from Monday
6:30 AM until Friday 2:30 PM, as well as the Thursday before. This Thursday was used as a
baseline measurement when reflection was carried out without the Resiliencer. During the
observation week, the Resiliencer was used as the central reflection tool. The team reflection
sessions were recorded and transcribed for analysis, while during the shift the rail signallers
were observed and interviewed on their findings.
FIGURE 3 Rail control centre layout with reflection area and Resiliencer on the wall
3. Results
3.1. Team-reflection observation
108 A. W. Siegel
3.1.1. Daily observations

In the following text, we number in brackets in-depth discussion topics, which we summarize
and analyse in the section 3.1.2..
The first team reflection observation took place on Thursday at the late shift before the
observation week with the Resiliencer. The team reflection was done without any support and
was driven only by the memory of the participants for what had occurred during their shift.
During the whole discussion, no specific train number was mentioned. When the RS discussed
an event, he only mentioned the train series, and while the other participant knew immediately
its trajectory, he did not remember the exact train number. The whole discussion was only about
logistics and of a generic nature. For example, a train from the 3300 series coming from Hoorn,
a city north of Amsterdam, had been cancelled because of cumulative delays. The discussion
topic moved to the role of the regional dispatcher (RD) (1), who was absent at the control centre
and whose role was taken over by the RD positioned in Amsterdam. The majority thought that a
RD can communicate electronically from another physical location and function well, as was the
case in their shift. This topic was about a procedure change. Another topic discussed was the
decision of a Dutch railway company (the NS), that decided to add stops to intercities (2) since
they had problems with their stop-trains. Their decision influenced freight trains, owned and
managed by other rail companies, whose delays grew due to the intercities stopping in front of
them on small stations, so the freight trains could not pass. This topic was about the reasoning
behind the procedure. The last topic discussed was the inconsistency of the NS with cancellation
procedure of trains (3), which happened at that day with the 3400 series. Workload and safety
matters were not mentioned at all during the discussion. This strengthens one of the Resiliencer
goals: to display information on workload and safety as well, stimulating attention, discussion
and interrelationship of system boundaries.
The first reflection with the Resiliencer (see figure 4) took place on Monday afternoon of the
early shift. Similar to the late shift, there was scepticism about the value of reflection for the rail
signallers. Some of them laughed saying “we have entered a lot of ones” (referring to the
subjective workload scoring scale of 1 to 9), which indeed was the case. On the first day, 80% of
the IWS scores were one, while the rest of the week on average 59% IWS scores were one with
a standard deviation of 8%. One of the TC’s wondered “Do we gain anything personal from this
process?”. A large percentage of the time was spent explaining the functionality of the
Resiliencer. The serious case of a near accident, described in section 3.2., was barely discussed.
The high workload stretch associated with the near accident was identified, but the RS in
question was sent home and nobody else knew the details. A few specific trains (1514, 14525,
3328, etc.) were mentioned but only one of the RS’s explained what happened to trains without
response and discussion with the other team members. No in-depth discussion took place, which
seems a start-up phenomenon of introducing the new process.
A. W. Siegel 109
FIGURE 4 Rail signallers and their team leader reflect on their shift with help of the Resiliencer
The Tuesday late shift was the next recorded observation, since there had been a technical
problem with the voice recording during the early shift. The reflection topics shifted from train
logistics to workload and annoying personnel. Many trains seemed to have problems with the
lead guard (LG) starting from Den Helder, which is the start station in the north. The LG’s are
sometime missing or too late on the trains (4). This is context information influencing
procedures. The team started focusing on pattern repetitions of train numbers with similar delay
occurrences in the previous period. They understood that the 5400 series was mainly delayed
due to delays of the 2100 series in the rush hour – a planning issue to be solved. They identified
eight workload stretches and discussed each of them: “I had to do three things at the same time”,
“I could not get in contact with the node coordinator”, etc. There was a long discussion on an
instruction-form filled out incorrectly by the NS. They identified an unknowledgeable NS-
worker and discussed how they should deal with that. At the end of the discussion they analysed
the correlation between safety data and the workload stretches, finding the information to be
consistent.
On Wednesday, the teams started to grasp the concepts of the reflections by analysing the
reasoning behind their workload stretches. They recalled a case with someone who walked along
the rail tracks who happened to be a refugee (5). It seemed that the train driver had taken the
person into his cabin until the next station, which is not according to the procedures. A train
driver should not take any person into his cabin. A discussion developed on refugees in the
Netherlands, which is beyond the procedure concerning a rail-walker but may reveal relevant
110 A. W. Siegel
knowledge to understand the behaviour of the rail walker and the train driver in this context. The
3008 train had a delay caused by a defect train compartment, and they analysed the
consequences on the 800 train series. They also identified the delay of train 5156, which was
discussed on Thursday as well. This is a good example of returning topics, where the team may
identify patterns, which are normally not seen. The next topic were the differences between train
drivers, some of whom drive faster to diminish the delay while others drive normally enlarging
the delay (6). The difference between rail signallers was discussed as well (7). The workload
was analysed and they identified six workload stretches, where one workstation had no stretch at
all. All stretches were treated and discussed systematically.
On Thursday, the early team identified topics they analysed in depth and enjoyed their
understanding. The late team started experimenting with the reflection sequence. The early team
had a case with a troublesome traveller (8) in train 3047 arriving at the small station of
Castricum, with no possibilities for other trains passing by. The train caused an extra delay for
train 3441 that had to wait before the station until the police arrived. The team analysed the
difficulty of forecasting the expected delay and the new role of the back-office to be the contact
entity with the police. Before, they had the contact themselves and could better estimate the
delay. In the new situation, they communicate only with the train driver, who in some cases is
not on the train or inaccessible. This influences their replanning task. They concluded that train
3047 should have continued until the next station, Uitgeest, where there are more shunting and
switch possibilities. This topic clarified reasoning behind events and needed further discussion
with the NS. Another topic analysed was “the rush hours with thousands of passengers and
missing rolling stock” (9) and different ways to deal with that. During the discussion on
workload they explained to each other how they rated the subjective workload. One RS said “via
IWS we tell our story even if we do not feel extra work stress”. “We use the low rating until
IWS=5 for small events to tell the story”. The early team finished their discussion on red
planning lines (10) and their use for controlling the system. A red plan-line cannot be executed
by the automated system for various reasons and needs manual handling of the RS. The late
team experimented with the discussion sequence starting from the safety boundary to the
performance boundary, through the workload boundary, which was the opposite direction until
then. This approach was the own initiative of the team leader, which showed trust in the process.
They identified many red planning lines (10) around the rail station Hoorn, revealing delays of
trains coming from Amsterdam crossing each other at Hoorn. In addition, they identified a
delayed train, the 5156, which one of the RSs discussed the day before - “we discussed that one
yesterday and now it is part of the reference period”. The repetitiveness was an issue to be
reported to the planning department. From this point on they put more emphasis in their analysis
on the delayed trains appearing in the previous period. At the end, they discussed the large
number of stretches (11), 15, mainly caused by the many delays that day. During one stretch the
RS was 9 minutes on the telephone, a parameter they were not aware of.
The early shift on Friday was the last reflection. The team seemed to enjoy the analysis and the
word “interesting” was mentioned eight times. The special procedure of “high green” (12) was
A. W. Siegel 111
discussed. Through manual settings of the RS they were able to set a rail path with only green
signals without yellow ones, even when entering a station. In a normal situation, the exit signal
of a station is red causing the previous signal to be yellow, forcing the train driver to reduce its
speed on entrance. By setting the exit signal to green together with the previous signal, the train
driver brakes at a later moment and leaves the station earlier. This way rail signallers can reduce
the delay. Some of the RS’s use this tactic, while others do not. The pros and cons of this tactic
were discussed, which could become a formal procedure. One RS expressed how he was rating
his subjective workload; “IWS=1 – I have nothing to do, 2 – I have one thing to do, 3 – I do a
few things parallel, 4 – it is getting complex, … 7 – an alarm is additionally coming in”. They
also discussed three tactics (13) to deal with delays. One RS was updating his delay continuously
in the plan. The second RS judged each train individually within its context whether the plan
should be updated. The third RS left the plan as much as possible untouched. The pros and cons
of these strategies were discussed, enriching the formal procedures. The team wrapped up the
reflection week expressing their understanding of the contribution and belief in the concept,
emphasizing the need for organisational change to feed the results back into the organisation -
“yes, it is very useful, but only if something is done with the things we say”. In addition, they
suggested to incorporate other parties in the reflection process like the passenger, freight and
maintenance companies.
3.1.2. Week observation overview

From the daily description in section 3.1.1., we discerned progress throughout the week. The
first day started with scepticism about the value of team reflection. The second day the topic of
discussion expanded from train logistics, which was their main concern, to topics related to
workload. The third day, RSs started to understand the new concepts. The fourth day, they
started to analyse the topics in more depth. The last day, RSs were really searching for topics
they framed as “interesting.”
To quantify their growing trust in and ease of use with the support system, we counted their
subjective load stretches with IWSs smaller or equal than 4. We assume that RSs rated IWS =<4
when something happened they wanted to explain, while stretches with IWS>=5 were used
when a serious external event occurred. This was also expressed by one of the RSs on Thursday.
When RSs rate more stretches with IWS=<4, they are more involved, as this indicates a more
urgent internal need to explain, rather than an externally imposed need caused by external
events. We saw that during the week the number of small stretches increased (see Figure 5).
This finding supports our observation of a growing involvement during the week.
112 A. W. Siegel
FIGURE 5 Growth of small (=<4) subjective stretches throughout the observation week
We summarized the description in section 3.1.1. into a list of topics, which were discussed in
depth, throughout the week. In Table 1, we added a column of topic categories, to enable
comparison with the topics contributed to the near accident described in the following section
(3.2.). Most topics are related to procedures and add knowledge beyond them in three ways.
First, the knowledge leads to procedure update and improvement. Second, the topic discussion
enriches details of the procedure description, which is always limited (Fucks & Dien, 2013).
Third, the topic reveals knowledge about the real complex world, contrasting the world assumed
when constructing the procedure.
A. W. Siegel 113
TABLE 1 Discussion topics during reflection (numbered according to appearance sequence marked
in the text of section 3.1.1.) with related underlying procedure.
Discussion topic
No during reflection Topic category Related procedure
1 Role of RD with cancelling trains Work overview Cancel train
2 Adding extra stops for intercities Procedures Order acceptance
3 Inconsistent train cancelling Procedures Cancel train
4 Absence of train staff Staff functioning Update plan,
Recall train path
5 Rail-walker Procedures Lower speed,
Inform police
6 Dependency on train driver to Staff functioning Delay handling
decrease delay
7 Culture differences between posts Work conditions -
8 Troublesome traveller External conditions Call police,
Update plan
9 Rush hour with missing rolling Infrastructure -
stock
10 Reasoning about red plan-lines Planning Manual
commands
11 Explaining workload Work conditions -
12 The “high green” procedure Alternatives Manual
commands
13 Tactics coping with delays Alternatives Update plan
3.2. Near-accident case: Two nose-to-nose stopped trains

In the case description, we entered numbers in brackets near the main contributions of the
accident evolvement, which we summarize and analyse in the end of section 3.2.3..
3.2.1. Background
During the day shifts of post Alkmaar four workstation are ordinarily manned: 1) Haarlem;
2) Uitgeest; 3) Alkmaar and 4) Hoorn. During the night shift workstation Uitgeest is unmanned
and divided up across the other workstations. The area of station Zaandam is moved from
workstation Uitgeest to Hoorn and projected on a separate screen (1) (see Figure 6). Between
rail-station Zaandam and the next rail-station Zaandam-Kogerveld is Zaanbridge BR4151
crossing the Zaan river. Signal 194 is positioned before the Zaanbridge and the previous signal
on the track, located at the exit of station Zaandam, is number 278. The bridge operator can open
114 A. W. Siegel
the bridge twice an hour: 1) between 10 to 20 minutes over the hour and 2) between 40 to 50
minutes over the hour. Train 1514 is daily planned to leave Zaandam at 6:42 AM and arrive at
Zaandam-Kogerveld at 6:47 AM being in conflict with a potential opening of the bridge (2)
when a boat is requesting to pass. This conflict in the plan has been discussed among all related
parties and accepted. A boat passing through between 6:40 AM and 6:50 AM will cause train
1514 to wait before red signal 194 and to be delayed for several minutes. This situation does
occur occasionally.
3.2.2. Sequence of events

Rail signaller X (RSX) worked in the evening before the accident in the late shift (3). Normally
RSs do not work in the early shift the following day, but in this case RS X finished the late shift at
7PM instead of 11PM having more time to rest before getting up early next morning. The
responsible team leader believed that the gap between 7PM and 6AM was sufficient to start the
next day in the early shift. At 5 AM one of the rail signallers called in sick (4) and the team
leader on duty decided to continue with the night shift formation into the morning shift (5) until
a replacement had arrived. RSX, a new employee working for half a year independently on
workstation Hoorn but never in the night formation during the day shift (6), arrived at 6 AM
working at the extended workstation Hoorn including rail-station Zaandam. This morning
showed dense fog (visibility less than 100 m) (7) and train 1514 was driven by a train driver in
training (8). The Zaanbridge was open causing signal 278, at the exit of rail-station Zaandam, to
be yellow, indicating signal 194 to be red and demanding velocity reduction. Unfortunately, the
red lamp of signal 194 was faulty and did not show (9). The rail-driver in training, detected the
defect signal too late for braking on time, and stopped the train after the ”red” unlit signal – a
SPAD (signal passed at danger).
The alarm went off at the control centre, where RS X contacted the train driver of train 1514 and
started the prescribed procedure. A facet of the procedure is to take the tracks, PP and PC, on the
bridge out of order and to take measures instructing trains approaching the bridge to turn. Train
driver 3325, being at track PB, called RS X to turn in Zaandam-Kogerveld according to the
measure. RSX discussed two options to turn (10) (see Figure 6). Option 1 was to enter the station
on track PC, which is the normal platform to continue towards Zaandam. Option 2 was to enter
the station on track PP, which is the platform used to drive in the other direction, making clear to
the passengers that the train is turning. RS X decided for option 2 for passenger clarity and gave
train 3325 permission to approach track PP. At the end of the procedure with train driver 1514,
he asked for permission from RS X to progress to rail-station Zaandam-Kogerveld on track PP,
which had been taken out of order. The request was granted with the instruction to drive slowly.
Train 1514 approached the station carefully and spotted train 3325, through the fog. He stopped
the train, avoiding a collision and standing nose-to-nose.
A. W. Siegel 115
FIGURE 6 Situation of the nose-to-nose near accident
3.2.3. Case overview – relevancy for team-reflection

This near-accident case, occurring during the observation period, is in two ways relevant for
team reflection. First, the Resiliencer presented technical information of the incident enabling
discussion. Second, the analysis of the incident revealed a list of facts playing a role in the
outcome, which could have been different when treated through previous reflections. The
Resiliencer supported the reflection in the afternoon of the first day. The workload analysis
revealed the incident. The subjective workload (IWS) was the largest at workstation Hoorn.
During the shift the average IWS at Hoorn was 3.7, while the average of the whole post was 2.8,
as seen in Figure 7 marking 1. This caused the team to push on the “Hoorn” button (marking 2).
The workload analysis showed two stretches, where the largest was 4.68 at 6:40 AM (marking
3). When choosing this stretch, the Resiliencer presented its workload over time (the window of
marking 4) and detailed alphanumeric information of the stretch (the window of marking 5). The
subjective workload (IWS) was initially a 4 for 5 minutes, then a 5 for 20 minutes, followed by a
7 starting at 7:20 AM for almost an hour. At 8:15 the IWS dropped to 4 until 11:30 AM with a
jump to 5 at 9:45 for 25 minutes. Alphanumeric data of the stretch showed a duration of 315
minutes, an average subjective workload score of 4.68 an objective workload score of 2.38, list
of involved trains from the 1500, 3300, 3400 series, 54 minutes on the phone, and 23 red plan-
lines, which were not executed by the automated system. The information was noticed by the
team during the reflection but no relevant knowledge was made explicit because RS X, handling
the near-accident had been sent home. The other RSs did not know the details, which were
sensitive at the time they discussed this.
116 A. W. Siegel
1
5
FIGURE 7 Resiliencer - Workload analysis of the near-accident case. 1 - Subjective workload at

workstation Hoorn; 2 - Choosing Hoorn analysis; 3 - Choosing stretch at 6:40 AM;
4 - Objective & subjective workload during the stretch; 5 - Alphanumeric data of the
stretch (duration, average workload, involved trains, minutes on phone, etc.)
We analysed the incident described in sections 3.2.1. and 3.2.2. and used as reference an
unreleased internal report of ProRail, the company incorporating all rail posts and responsible
for the Dutch railway network infrastructure, on the incident. We listed the topics contributing to
the incident and added a topic category (see Table 2) to enable comparison with the topics
discussed during the reflection (see Table 1).
A. W. Siegel 117
TABLE 2 Topics contributing to the incident (numbered according to appearance sequence marked
in the text of sections 3.2.1. and 3.2.2.)
No Topic contributing to the incident Topic category Related procedure

1 Information split between two Work overview Night formation
screens
2 Bridge opening in conflict with train Planning Bridge opening
route
3 Early shift after late shift (until Work conditions Working hours
7PM)
4 Call in sick at 5AM Procedures Sick reporting
5 Night formation during early shift Work overview Night formation
rush hour
6 RS without experience in night Work conditions Authorization
formation
7 Fog with low visibility External Extreme weather
conditions
8 Train driver trainee drives Staff functioning Authorization
9 Broken signal Infrastructure Disruption
handling
10 Two options turning train Alternatives Rerouting
The topic categories listed above are similar to the topic categories during the team reflections
(Table 1) and imply that the discussions of the reflections can influence the interrelated process
of incidents. A concrete example, mentioned in a talk the researchers had with an RS and team
leader, concerned the schedule conflict of the train-bridge interference. Due to performance
pressure planners made a conscious decision that a train may be delayed by a boat, which does
not occur daily. RSs on the post warned about the consequences of possible delays, but were
overruled. If the conflict would have been resolved in the planning, the bridge would not have
been open and the interrelated sequence would not have occurred. Another example, is an action
taken after the incident, which could have been discussed and activated before the incident and
influence the outcome. The post decided to have an extra RS on partial duty at home, ready to be
called at 5 AM joining the early shift at 6 AM. A sick call in at 5 AM occurred before, but was
not discussed in depth. Such a discussion and action could have been triggered through team
reflection. Previous research shows that signalling itself is not enough to cause related actions
(Vaughan, 2002). The results above indicate growth of relevant knowledge beyond procedures
influencing in three ways. First, some knowledge items are related to deficiencies and may lead
to anticipation by updating the procedures. An example is the procedure of open bridge
planning. Second, some knowledge includes items related to unwritten details enabling to learn
and act better. An example is the combination of information on split screens. Third, some
knowledge refers to the understanding of the real world, as it unfolds. An example is the rail-
118 A. W. Siegel
walker with another cultural background, behaving differently than most Dutch people would
typically behave in similar circumstances.
4. Discussion
We wanted to know how team reflection, at the end of each shift, enriches the knowledge of
procedures followed, and how it influences the resilience of the STS. To answer this question,
we conducted a naturalistic observational study. A team of rail signallers reflected on their shift
with help of a real-time tool, presenting relative OS movements towards system boundaries and
providing analysis functions to relate the OS movement to identifiable details. The team
discussed topics similar to topics contributing to a near-accident that occurred during the
observation. Most topics were related to underlying procedures and revealed a context which is
not described in the procedures and went beyond them. The similarity of these topics support
our assumption that relevant topics contributing to accident evolvement are topics occurring
daily, when thing seemingly go right, and contain valuable knowledge. This adheres to the
notion that only the outcome of a process can distinguish between knowledge and error, given
that both stem from the same cognitive sources (Mach, 1905). The team reflection executed is a
type of after action review (AAR), which is a de-briefing process for analysing what happened,
why it happened, and how it can be done better (Morrison & Meliza, 1999). The AAR is
frequently used after a training or exercise to unfold the scenario step by step and discuss the
response of each team member. The main novelty of our approach is that we enable this process
in actual and continuous operations by providing real-time access to operational data in a
dedicated lay-out for this purpose. To design this lay-out, we used ecological interface design
principles (Burns & Hajdukiewicz, 2004) such as the abstraction hierarchy (Rasmussen, 1985),
which is corresponding to system boundary constraints. Since the method is used during normal
daily operations, the emphasis is more on learning from what goes right as opposed to what goes
wrong (Hollnagel, 2011a, 2014). It goes beyond the usage of team reflection to discuss the
mission, strategy and processes at ad hoc moments (West, 2000). It expands the scope of
reflection beyond the scope of the team’s span of control to the whole rail-STS and influences it
as such. The team at the sharp end sees how all parties of the STS are acting, for which they are
not responsible, and can give them feedback, anticipate by adapting the written and unwritten
procedures, or use the knowledge during future events. For example, RSs can adapt their
procedures on the handling of rail-walkers by train drivers, give feedback to the train company
or understand and react on a rail driver in a future case. Knowledge, beyond procedures, on the
whole STS is an essential component for managing the unforeseen and unexpected (Schraagen,
2015). This implies that the proposed method contributes to resilience enhancement of the STS.
Additional novelties of our research are 1) the simultaneous presentation on three boundaries, 2)
the use of weak signals, and 3) the ability to translate, out of big data, abstract system values to
identifiable details. Quite often research is focussed on single domains such as performance
A. W. Siegel 119
(Marsden & Bonsall, 2006), workload (Lowe & Pickup, 2008) or safety (Jeffcott, Pidgeon,
Weyman, & Walls, 2006). We have shown to rail signallers the dynamic interrelationships
among these domains to trigger interrelated discussion. Through comparison with a previous
observation study with reflection on only the performance boundary (Siegel & Schraagen, 2016)
and one reflection session without tooling (current study), we could clearly see a difference with
the reflection on multiple domains. The reflection without the tool focussed solely on the
logistics of train series of main events during the shift. The rail signallers did not remember
specific trains nor minor events, which is to be expected when only memory is consulted. They
referred to obvious patterns in the previous period, but creeping changes (Dekker, 2011) were
not identified. The lack of specific information may have led to a focus on logistics only. As
logistics are mainly governed by procedures, current practices may reinforce a procedure-
oriented culture. Adding more detailed information on the performance boundary still led to
predominantly logistic discussions of specific trains and concentrated discussions on the
procedures followed (or deviated from) and the entities communicated with (Siegel &
Schraagen, 2016). When the team was given access to information on safety, performance and
workload together, it started searching for interesting phenomena, trying to explain the
reasoning behind these phenomena. Personal subjective workload scoring was used as a
reminder of details and was explained to the whole team. These differences in discussion topics
are in line with the information to which the team of signallers had access to. However, the
variety of topics grew, when reflecting on all three boundaries, among which new-fangled
relations were established by the team members.
The usage of weak resilience signals (Siegel & Schraagen, 2014c) stimulated discussion and
understanding as opposed to strong signals, where action is needed and false alarms undermine
the trust in the system (Breznitz, 1984). Weak signals appear earlier than strong signals and
allow time to learn, anticipate and prevent escalation (Lekka, 2011). The discussion topics were
initiated by OS movements towards the boundaries, together with correlation of these
components. For example, a change with respect to the performance boundary was not always
correlated with a workload change, or vice-versa, which caused discussion. Within boundaries,
variables also did not always correlate, such as with the relation between subjective and
objective workload. Any discrepancy related to their understanding of the system, was seen as a
weak signal, triggered a discussion and made related knowledge explicit.
The rail operational system produces a very large amount of data on a daily basis, which is not
easily accessible. This fact looks like mastering big data (Kezunovic, Xie, & Grijalva, 2013),
with the addition that operators at control rooms, as opposed to dedicated analysts, must be able
to use and understand the tools and figures. Moreover, it is important that each worker on the
floor is able to translate the system level to his own context. In our concept we required such a
function, in order for team members to identify with figures on an OS level and reveal related
information to their colleagues. During the reflection the RSs were able to refer to technical
details and share supporting information. A side effect of this feature was a growing trust in the
system (Pfautz, Carlson, Farry, & Koelle, 2009), since they could verify the information with
their memory.
120 A. W. Siegel
The scope of this research was the explicit knowledge beyond procedures revealed through team
reflection. A next research step can focus on learning and anticipation using this knowledge
enhancing the performance by preventing occurrences or avoiding escalation of incidents. That
research would first require the necessary organisational adjustments to be implemented to feed
the knowledge back into the organisation. A longer period of observation would then be needed
to identify links between the knowledge made explicit and fed back and future prevented or
deescalated incidents.
The team-reflection with a WRS reflection-tool described in this paper is also applicable to other
domains with control rooms. The control operators from the team, reflecting at the end of their
shift. System analysis and software development are needed for a dedicated reflection tool. The
generic design method using the AH analysis (section 2.1.) can be applied. The top two level of
the AH will look similar, but the level from the purpose related functions and below are specific
– the quantification of relative OS movements towards the boundaries. The quantification of
each boundary does not, and cannot, be comprehensive but needs to be sufficient for the
operators to trust and identify with. This was also the case in our prototype. Another challenge
for the analysts is to find a search mechanism for the operators, to translate values from system
level to personally identifiable components. We found a mechanism with three button presses to
go from high level to specific. Observations are needed to verify similar results in other
domains.
We have chosen for a naturalistic observation to study our theoretically derived proposition.
This method allows for a diverse collection of observations in situ, resulting in findings to be
verified in a more controlled environment (Woods, 2003). This is also the case for our study
where components need to be verified in a controlled setting. Especially the creation of a WRS
is a challenge since its occurrence is not necessarily related to a future event, thus hard to
simulate. However, a much longer observation period may strengthen the patterns, where natural
WRSs occur. The adaptation of the methodology is also a good candidate to verify during a
longer period. With these caveats and limitations in mind, our tentative conclusion is that team
reflection with the right tooling complements the experience and knowledge residing in
procedures, enhancing the handling of the unforeseen and unexpected.
A. W. Siegel 121
Acknowledgement
We would like to thank the post Alkmaar for their openness and cooperation. The good
atmosphere and their enthusiasm has contributed a lot to the success of the research. We greatly
appreciate the guidance and review comments by Alfons Schaafsma. Last but not least, we
would like to thank Bert Bierman and Victor Kramnik for the software development and
graphical design of the Resiliencer. This research was conducted within the RAILROAD project
and was supported by ProRail and the Netherlands organization for scientific research (NWO)
(under grant 438-12-306).
122 A. W. Siegel
A. W. Siegel 123
SUMMARY AND CONCLUSIONS
124 A. W. Siegel
A. W. Siegel 125
CHAPTER 6
SUMMARY AND CONCLUSIONS
1. Summary
Sociotechnical systems, interrelating people and technology, are becoming more complex while
safety expectations grow. Examples of such complex sociotechnical system are civil aviation,
process industry, nuclear industry, and rail systems. Existing methods of system development,
system maintenance, and system control have achieved very low failure rates within
sociotechnical systems, but have reached a limit. New paradigms are sought to overcome this
limit. This dissertation makes a step in this direction and researches this matter in the rail
domain. A search for new ways of viewing and understanding the rail sociotechnical system
dynamics, new ways of analysis and tooling, enabling operators and their environment to
identify new aspects to learn, anticipate and act on, to achieve the sought breakthrough.
Our research objective was to investigate means to improve the abilities of rail operators to
enhance the performance of their sociotechnical system, when unexpected or unforeseen events
occur. The approach taken was to view at the behaviour of a rail sociotechnical system through
the lens of resilience engineering. The concept of ‘weak resilience signal’ was introduced as a
change of the system needing further investigation to make underlying resilience-related
knowledge explicit. Complementary, team reflection was modified and further developed for
rail operators, in our case rail signallers, to reflect on the weak resilience signals. Team
reflection, with help of tooling for weak resilience signal presentation and analysis, was
hypothesized to enhance the resilience of the sociotechnical system by exposing related
knowledge. The whole theory based concept was designed, prototyped and verified in real rail
operations. This is the basis for answering the three main research questions: (How) can
indicators of resilience be measured during real-time operations (chapters 2 and 3)? What is a
method for control room teams to learn from their recent operational experiences (chapters 4 and
5)? Which skills of the operator need to be developed due to increasing automation so as to
enhance system resilience (chapters 4 and 5)? In the next section 2.-Conclusions, are answers
given on these question, while in the next paragraphs is a summary given of each chapter and of
the whole dissertation.
In chapter 2, a ‘weak resilience signal’ was defined as a distinguishable relative movement, with
a possible variable discrepancy, of the operating system towards its boundaries. A framework
was developed with a resilience-state model for a railway sociotechnical system, resulting in a
126 A. W. Siegel
generic, quantifiable, weak resilience signal model. The model posited three system boundaries:
performance, workload and safety. The weak resilience signal was initially studied separately in
the workload and performance domains, through observations and off-line data. The workload
weak resilience signal was modelled and quantified through the measurement of both subjective
and objective workload (derived from task-load measures). Heart rate variability was used for
correlation and verification. A new metric, called stretch, was introduced to compare the
workload types and identify discrepancies, which were viewed as a workload weak resilience
signal. During an observation study, the weak resilience signal helped in revealing obstacles
jeopardizing the resilience state.
Subsequently, the objective workload algorithm was implemented in a real-time tool and
verified against behavioural observation, a physiological measurement of electrodermal activity
and a measure of subjective workload. Two cases were analysed and showed that the system
information for communication, manual actions and switch cost were discriminating for
workload.
Chapter 3 dealt with the performance weak resilience signal. A multimethod, ethnographic
observation and resilience questionnaire, was used to determine resilience baseline conditions at
an operational rail traffic control post. The performance weak resilience signal was developed
and implemented using off-line data. At this point the weak resilience signal analysis function
was introduced to interpret underlying factors of the performance weak resilience signals and to
serve as a method revealing potential sources of future resonance that could comprise system
resilience. Results indicated that performance weak resilience signals can successfully be
implemented to accentuate relative deviations from resilience baseline conditions. A weak
resilience signal analysis function can help to interpret these deviations and can be used to
reveal (creeping) change processes and unnoticed initiating events that potentially degrade rail-
system resilience.
In chapter 4, team reflection (Ellis et al., 2014) was proposed as a macrocognitive function to
make resilience-related knowledge explicit. This knowledge is implicitly available with
individual team members active at the sharp end but is never explicitly shared due to invisibility,
in the work environment, of goal-relevant constraints. An observational study at a rail control
post was conducted to assess the value of team reflection in making resilience-related
knowledge explicit. For this purpose, we developed a real-time prototype application with the
performance weak resilience signal algorithm combined with the weak resilience signal analysis
function. The application allowed a team of rail traffic signallers to analyse movements toward
system boundaries and share knowledge on these movements. A naturalistic observation of a
team during a week showed how they reflected at the end of their shift on salient system
changes. A global content analysis of talk-aloud data was used to show the relevance of the
content to resilience and to test a potential increase in making resilience related knowledge
explicit throughout the observation period. A specific case of a human approach to the rail
tracks, as a potential suicide, was analysed in detail. The results showed the value of team
A. W. Siegel 127
reflection on system movements towards their boundaries, thus showing the value of making
goal-relevant constraints explicit within the operational rail environment.
Chapter 5, the final stage of the research, described an observation study of team reflection
using a newly developed tool, called the Resiliencer. The Resiliencer provided weak resilience
signals on all three boundaries and weak resilience signal analysis functions to connect system
level signals to personal identifiable details. We hypothesized that the Resiliencer would
complement proceduralization within complex sociotechnical systems. Proceduralization aims at
defining precise and quantified system objectives, and at defining a process that describes and
prescribes how to achieve those objectives. Although proceduralization has been successfully
implemented to capture knowledge and experience, it is limited when the unexpected and
unforeseen occurs. A team reflection process was proposed to enhance resilience of a rail
sociotechnical system, complementing its proceduralization. In this study, rail signallers
experimentally acted as a team reflecting with the Resiliencer, allowing in-depth post-shift
inspection of train movements. A near accident, occurring during the one-week observation, was
described and used for two purposes. First, it was used as an example to explain the usage of the
Resiliencer. Second, it was used as a reference case of topics playing a role in evolving
accidents. The analysis showed that the topic categories discussed during the team reflections
were similar to the incident categories. This means that relevant topics are available, when
things go right, to learn from and anticipate on. In addition, it was shown that rail signallers,
over the course of the observations, increasingly analysed and reasoned about their work. This
enriched knowledge beyond procedures, enhancing the ability to cope with the unexpected and
unforeseen. Last, but not least, the rail signallers increasingly saw the value of the Resiliencer
during team reflections: initial scepticism gave way to moderate enthusiasm.
Figure 1 depicts the summary of the whole concept. The left circle portrays the course of the
shift with events during operations, which is input to the central circle. In the central circle is the
course of the shift presented by the Resiliencer, providing weak resilience signals together with
analysis functions. The team reflects on the Resiliencer, resulting in explicit knowledge on top
of procedures, visualized in the right circle. The heart of the matter, in the central circle, is the
theory and algorithms of the Resiliencer and the team reflection methodology, which were
observed and verified in real operations.
128 A. W. Siegel
FIGURE 1 Summary of the total concept (graphical abstract, Siegel & Schraagen, 2017)
Our research contributed to three fields of theory and practice related to answers on the three
main research questions: resilience quantification, safety-II and rail signaller evolution. The
field of resilience engineering quickly expands with many new concepts to support
sociotechnical systems to become more resilient, to manage its resources dealing with
unexpected events. However, the field currently lacks methods to quantify the resilience state in
real-time operations. Through the introduction of the weak resilience signal, it is possible to
quantify weak signals related to the resilience state, supporting the process of making resilience
related knowledge explicit. This may be viewed as a step in the direction of resilience
quantification, an indicator of resilience measured during real-time operation. Safety-II is a
paradigm shift from the traditional safety-I management philosophy, which focuses on the
elimination of safety risks. Safety-II propagates focusing on what goes right instead of what
goes wrong. Team reflection on weak resilience signals at the end of each control room shift,
learning from what went right during their recent operational experiences, is our
operationalization of the Safety-II concept. This process also influences the evolution of the rail
signaller, the third and final field to which we contributed. The next generation of rail signallers
need analysis abilities to cope with increasing automation and enhance system resilience. Rail
signallers of the future need to reflect on their own thinking and acting through weak resilience
signals, they need to analyse signals at a system and details level, and communicate their
thoughts and understanding while listening to others. In short, the importance of non-technical
skills will increase, relative to technical skills.
The proposed design introduces several novelties compared to existing practice in rail
operations. First, the proposed method stimulates simultaneous reflection on the three system
domains - performance, workload, and safety while most analysis approaches focus on a single
domain. Second, the emphasis of the design is on “here-and-now”, immediately reflecting with
people after their shift on what they have experienced, on basis of actual data, is unique in rail
systems. Third, weak signals are no longer under the radar and get on regular basis attention.
A. W. Siegel 129
Last, the reflection tooling supports translation from system level to identifiable details, which is
not possible today with system KPI’s (key performance indicators) displayed in the control
rooms. Our research has shown the potential of the Resiliencer in this area.
2. Conclusions
In the introduction of this dissertation we defined the problem domain as the growing
complexity of sociotechnical systems. Our research objective was to investigate for means for
improving the abilities of rail operators to enhance the resilience of their sociotechnical system.
Three background areas were presented leading to knowledge gaps, with associated research
questions, addressed within the current research. The first was the need for resilience
quantification, the second the need for operationalization of Safety-II and the third is the need
for the next evolution step of the rail signaller. Our research addressed these gaps in
combination with the development of related theory.
2.1. Resilience real-time quantification during operations

The first overall research question is about measuring indicators of resilience, during real-time
operations, and the specification of those measurements. In theory, we would like to measure the
resilience state of a sociotechnical system during operations and close the loop in real-time with
operators taking actions to sustain the requested level of resilience. However, in practice this is
difficult, if not impossible, to achieve. The difficulty lies in the prediction of the sociotechnical
system behaviour around its boundaries. The area beyond the boundaries is defined here as the
collapse or non-functioning of the system. Woods (Woods et al., 2014) describes modes of
sociotechnical system degradation in relation to the boundary and moving it further to postpone
the collapse. Since the exact situation of a future event is unknown, the behaviour of the system
cannot be precisely predicted and thus difficult to quantify. Another difficulty is the
measurement unit. Resilience is an abstract construct and needs a model to measure it indirectly.
In chapter 2 we describe a model based upon Rasmussen’s (1997) safe operating envelope and
the Stress-Strain model (Woods & Wreathall, 2008). For the purpose of weak signals, it is
sufficient to measure relative changes and measuring them in the base competence area, where
no out of control situations occur. These assumptions enabled measurement of values in the
boundary axis, without knowing the exact position of the boundary itself. Again, the boundary
axis needed to be modelled to identify the measurable variables. This has been done for three
boundaries: performance, workload and safety. This quantification of changes within the
boundary axes was sufficient for identifying weak resilience signals to reflect on. This approach
has been used and its value has been verified in real rail operations.
130 A. W. Siegel
2.2. Operationalizing of Safety-II during operations

Safety-II needs methods to deal with what goes right during operations. In the recent past,
methods have been proposed but all may be characterized as analysis tools to be employed by
researchers (Hollnagel, 2014). The method proposed here, team reflection on weak resilience
signals, gives an answer to the second overall research question on how the Safety-II concept
can be operationalized. The adapted team reflection (Ellis et al., 2014; West, 2000) enables
control room teams to learn from what goes right during their recent operational experiences, as
part of their duty. During their reflection, they are guided by support tools that make weak
resilience signals explicit within their daily successful operations. This is a way to focus on
relevant topics, risks as well as opportunities, as advanced by Safety-II. This state of mind
influences the operators during their work, having the tooling continuously available. At the end
of their shift, when everything is fresh in their mind, they can analyse their work, make relevant
knowledge explicit, and anticipate and learn from their daily successes. This method is actually
an operationalization of Safety-II during operations executed by the operators themselves (rather
than researchers telling them).
2.3. Evolution of the rail signaller and others

Based on our research, we foresee the following step in the evolution of the rail signaller’s job:
from “setting a rail-switch mechanically at a pre-planned moment” through “handling an
automated train-plan” to “an analyst making resilience-related knowledge explicit”. This is a
deductive premise introducing the answer on the overall research question on the skills the
operator needs to develop to enhance the resilience of the system. Team reflection requires
analysis and communication abilities of the rail signaller. This is a new ability for the signallers
at the sharp end, which is today mainly exercised by analysts having a staff function, the blunt
end of the sociotechnical system. During the observations, the rail signallers enjoyed using their
craftsmanship. Even during a week of applying the new method, a change in their attitude was
observed. They started the week in a defensive mode stating the ease of work and following the
rules as much as possible. At the end of the week, they looked at themselves in a more open
manner and spotted interesting cases for analysis.
These results will have an influence on recruiting and training requirements. The future rail
signaller needs abilities to reflect on his own thinking and acting through weak resilience
signals. He needs to understand these signals at a system level, analyse them to basic details and
deduce it back to the generic system level, beyond his own responsibility. This will be done in a
team context where he will need to communicate his thoughts and understanding, while listening
to those of the others. Through the dissipation of their knowledge on the sociotechnical system
in operation, the operators will become the resilience enhancement engine. In addition, it
influences the abilities of others in their work environment. For example, the team leader needs
A. W. Siegel 131
to develop his skills to lead the team reflection and the management needs to absorb the
knowledge into the organisation and provide feedback to stimulate the process. The setting, the
culture and the depth of understanding are of great influence on the reflection results. The
organisation as a whole needs to adapt as well. The knowledge made explicit during the
reflection needs to be incorporated and sustained not only at the individual level but also at the
organisational level. The Dutch rail infrastructure manager, ProRail, currently has 13 rail control
posts and one national centre where all rail parties, train operating companies and maintenance
companies, are seated. In the future, we foresee that as team reflection will take place after each
shift at all the control posts, the knowledge and understanding need to be shared and
accumulated to the national level. While this process is performed during operations another
process needs to be designed for the staff level. The staff level needs to incorporate the
knowledge into procedure updates and take structural actions for the medium and long term,
with all the rail parties. This implies an organisational structure change which will influence the
evolution of needed skills in the whole organisation and eventually will result in resilience
enhancement of the rail sociotechnical system.
3. Implications
Two main theories and their combination have been developed in the context of resilience
engineering and safety management. The first is the weak resilience signal which is a novelty
and the second team-reflection, expanding existing theory.
The introduction of the weak resilience signal has an impact on the way we think about signals.
The signal is thought of as a trigger which needs an action and when in hindsight no action is
needed, we talk about a false signal. A low ratio of true/false signals will introduce the cry wolf
effect (Breznitz, 1984) and causes distrust in the signals. The weak signal, meant here, is not
mainly a signal that may become a normal signal requiring action, but a trigger to investigate
and understand better the sociotechnical system. The cry wolf effect is in this case not relevant
anymore, because the weak signal merely stimulates a deep understanding of what is happening.
It must trigger interest and cause an "aha" exclamation during team reflection. Where a generic
weak signal reveals understanding of any subject, the weak resilience signals aim to stimulate
resilience related understanding. This is not easy to prove. The ultimate proof would be a trace
from a weak resilience signal to specific knowledge made explicit and used in an event,
preventing escalation or failure of the system. An escalation or system failure is a complex
interrelated process where a specific piece of knowledge is hard to trace and demonstrate its
unique contributing role in the event. Even if this would be possible it would be difficult to
prove its strong relation to the weak resilience signal. Maybe the knowledge existed already
without the weak resilience signal. The assumption, used in this research, was that a weak
resilience signal is resilience related because it is a derivative of a movement towards
boundaries, which are theoretically resilience related. The validity of this assumption has not
132 A. W. Siegel
been proven. Moreover, knowledge made explicit during reflection can be of any nature and is
strongly dependent on the connotation of the team members. It seems that it is a skill of team
members to set and derive relevant relations, while the weak resilience signal provides the
trigger to do so. Still, this fragility does not undermine the novelty of the weak signal to
stimulate thought and understanding. It is this understanding, which can compensate the low
amount of action, due to high and successful automation.
The theory of team reflection has been expanded to utilize the potential of the weak resilience
signal. Whereas the original focus of team reflection was on the team itself reflecting on its
mission, strategy and performance (Ellis et al., 2014; West, 2000), our instantiation of the
reflection concept focuses here on the whole sociotechnical system through weak resilience
signals. The data-frame theory of sensemaking (Klein et al., 2006) has been used to explain the
macrocognitive process within the team. Although the theory explains the process it does not
provide an answer on the coverage of exposing all resilience related knowledge. It is obvious
that more conditions are needed to enlarge the coverage. The individual cognition, mentioned
above, setting relations between weak resilience signals and other aspects are important. The
culture within the team and organisation, promoting openness without penalties, is essential and
not part of the theory developed in this dissertation. However, the added value of team reflection
on weak resilience signals is that it stimulates the deeper understanding of the operator during
normal work and is a kind of daily exercise. This approach is an extension to training with
simulators, which is done with a low frequency.
The weak signal and weak resilience signal concept is generic and is applicable to other
domains. Its usage through team reflection implies a specific nature of the domains. As
described above, it is meant for operators working with highly automated systems, providing
them primarily a monitoring function. This is the case of operators in a control room. This can
be a nuclear centre, a civil aviation centre or a logistics transport centre. Each domain needs
modelling of parameters of its boundaries. The difficulty lies in the ability to develop algorithms
based on real-time data and to develop a simple search engine which supports operators to set
the link between system level variables and specific identifiable details. This is probably not a
rigid relation but an evolutionary one. While working with the tooling the understanding grows,
which needs to be modelled as well. The weak resilience signal is not a static model in each
domain but a dynamic one, continuously triggering the interest of the operators.
4. Novelties in operations
The proposed design introduces several novelties compared to existing practice in rail
operations. First, most analysis approaches focus only on a single domain, while the proposed
method stimulates simultaneous reflection on the three system domains - performance,
workload, and safety. This approach causes an initial understanding and awareness on the
interrelationship between the different areas. Second, the emphasis of the design is on “here-
A. W. Siegel 133
and-now”. Immediately reflecting with people after their shift on what they have experienced,
on basis of actual data, is unique in rail systems. Post-event analysis is done, however mostly a
few days later, not with all the people of the shift together and with data, which was off-line
processed. Third, weak signals are no longer under the radar. Mainly clear events or strong
signals get attention, while the design stimulates attention on drifts and unseen processes. Last,
the reflection tooling supports translation from system level to identifiable details. Today system
KPI’s (key performance indicators) are presented in the control rooms, but the individual is not
able to translate the figures to his own actions. With help of the Resiliencer they can identify
specific and personal details connected to a system figure. They can identify for example, the
delay of a specific train, the workload on a specific moment at a specific workstation, or a
specific safety instruction given to a specific train driver.
5. Future research
The strength of naturalistic observation studies used in this study is that it deals with the
complexity in real-world situations and allows a descriptive, ethnographic, and exploratory
approach. At the same time, the naturalistic approach has some limitations. Events occur
spontaneously and reference settings or groups operate under different circumstances and
events. A simulated environment can solve this drawback by a more controlled setting. This is a
next research step as explained by Woods (2003). The naturalistic observation is a first step to
understand and theorize what is happening in reality but afterwards needs validation through
controlled experiments. Different areas are candidates for validation. For example, the
algorithms of weak resilience signals around the boundaries can be evaluated in a simulated
environment. Are similar resilience related knowledge elements identified by different teams?
And is there a significant difference in revealed knowledge when team reflection takes place
with or without weak resilience signal tooling?
The duration of a week observation is constrained with events occurring during that particular
week. I was “lucky” that during the last observation a near-accident occurred. A longer period of
exercising team reflection with weak resilience signal tooling would include more significant
events to analyse and verify its effectiveness and allow comparison to a team operating without
the team reflection. Again, a longer period of testing would provide similar conditions over time
to compare and demonstrate a difference in performance. However, it is not only the exposure of
knowledge, making the performance difference. As discussed is the sub-paragraph above (2.3.),
the knowledge needs to be fed back into the organisation to effectuate the performance. The
closure of this loop, either in real operations or in simulation needs further research and is
essential to realize the potential of the theories, methods and design developed in this
dissertation.
134 A. W. Siegel
The theory developed was of a generic nature and relevant for sociotechnical systems with a
control room. In this research the algorithms have been developed in detail for a rail
environment. It is of importance to go through a similar process for another domain, like civil
aviation, process industry and nuclear industry, to verify the concept there as well. Focussing on
only the rail domain may have limited findings. Other domains may expand the theory and
methods.
This research focussed on the relation between the team reflection and weak resilience signals
without taking into account the influence of the surrounding environment. The role of culture, to
make the knowledge explicit and converting it into learning and anticipation, is an important
research direction. Culture may limit or stimulate the openness needed during the reflection
sessions. For example, a blame culture will be much more restricted than an open supportive
culture. These aspects were discussed during the observation, but were not systematically
investigated. The role of culture has been studied generically as influence on cognition
(DiMaggio, 1977) and specifically in rail operations, showing the influence of risk, trust, and
safety culture on performance (Jeffcott et al., 2006). These findings should be tested on the
proposed team reflection to evaluate their influence on resilience enhancement of the
sociotechnical system.
A. W. Siegel 135
REFERENCES
1. Publication list (in chronological research order)
Siegel, A. W., & Schraagen, J. M. C. (2014a). Developing resilience signals for the Dutch
railway system. In 5th REA Symposium managing trade-offs; 24th-27th June 2013,
Soesterberg, Netherlands (pp. 191–196). Retrieved from http://hdl.handle.net/1811/60454
Siegel, A. W., & Schraagen, J. M. C. (2014b). A method to reveal workload weak-resilience-

signals at a rail control post. In D. Harris (Ed.), HCII 2014 (pp. 82–93). Springer Berlin
Heidelberg. http://doi.org/10.1007/978-3-319-07515-0
Siegel, A. W., & Schraagen, J. M. C. (2014c). Measuring workload weak resilience signals at a
rail control post. IIE Transactions on Occupational Ergonomics and Human Factors, 2(3–
4), 179–193. http://doi.org/10.1080/21577323.2014.958632
De Regt, A., Siegel, A. W., & Schraagen, J. M. C. (2016). Towards quantifying metrics for rail-
system resilience: Identification and analysis of performance weak resilience signals.
Cognition, Technology & Work, 18(2), 319–331. http://doi.org/10.1007/s10111-015-0356-9
Siegel, A. W., & Schraagen, J. M. C. (2014d). Can weak-resilience-signals (WRS) reveal

obstacles compromising (rail-) system resilience? In D. de Waard, J. Sauer, S. Röttger, A.
Kluge, D. Manzey, C. Weikert, … H. Hoonhout (Eds.), Human Factors and Ergonomics
Society Europe Chapter 2014 Annual Conference (Vol. 2333–4959, pp. 187–191).
Retrieved from http://www.hfes-europe.org/human-factors-high-reliability-industries-2/
Siegel, A. W., & Schraagen, J. M. C. (2015). Can team reflection of rail operators make
resilience-related knowledge explicit? - an observational study design. In 6th REA
Symposium managing resilience; 22th-25th June 2015, Lisbon, Portugal (pp. 126–131).
Retrieved from http://www.resilience-engineering-association.org/download/resources/
symposium/ symposium_2015/Siegel_W.-Schraagen_J.M.-Can-team-reflection-of-rail-
operators-make-resilience-related-knowledge-Paper.pdf
Siegel, A. W. (2015). Design to enhance resilience of (rail-)control rooms [poster]. Human

Factors and Ergonomics Society Europe Chapter 2015 Annual Conference. Poster.
Retrieved from http://doc.utwente.nl/97347/
136 A. W. Siegel
Van Broekhoven, R., Siegel, A. W., Schraagen, J. M. C., & Noordzij, M. L. (2016). Comparison
of real-time relative workload measurements in rail signallers. In B. Milius & A.
Naumann (Eds.), Rail Human Factors Proceedings of the 2nd Germany Workshop
March, 8th and 9th, 2016, Stadthalle Braunschweig (pp. 30–40). Braunschweig: ITS
automotive nord. Retrieved from http://doc.utwente.nl/99353/
Siegel, A. W., & Schraagen, J. M. C. (2016). Team reflection makes resilience-related

knowledge explicit through collaborative sensemaking. Manuscript submitted for
publication.
Siegel, A. W., & Schraagen, J. M. C. (2017). Beyond procedures: Team reflection in a rail
control centre to enhance resilience. Safety Science, 91, 181–191.
http://doi.org/10.1016/j.ssci.2016.08.013
A. W. Siegel 137
2. References
Alderson, D. L., & Doyle, J. C. (2010). Contrasting views of complexity and their implications
for network-centric infrastructures. IEEE Transactions on Systems, Man, and Cybernetics
- Part A: Systems and Humans, 40(4), 839–852.
http://doi.org/10.1109/TSMCA.2010.2048027
Amalberti, R. (2001). The paradoxes of almost totally safe transportation systems. Safety
Science, 37(2–3), 109–126. http://doi.org/10.1016/S0925-7535(00)00045-X
American Association for Public Opinion Research. (2015). Standard Definitions: Final
Dispositions of Case Codes and Outcome Rates for Surveys. 8th edition. AAPOR. 8th
edition. AAPOR. Retrieved from
http://www.aapor.org/AAPORKentico/Communications/
AAPOR-Journals/Standard-Definitions.aspx
Back, J., Furniss, D., Hildebrandt, M., & Blandford, A. (2008). Resilience markers for safer
systems and organisations. In M. D. Harrison & M. A. Sujan (Eds.), SAFECOMP (Vol.
5219 LNCS, pp. 99–112). Berlin Heidelberg: Springer-Verlag. http://doi.org/10.1007/978-
3-540-87698-4_11
Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775–779.

http://doi.org/10.1016/0005-1098(83)90046-8
Bartone, P. T. (2006). Resilience under military operational stress: can leaders influence
hardiness? Military Psychology, 18(Suppl.), S131–S148.
http://doi.org/10.1207/s15327876mp1803s_10
Bass, B. M. (1990). From transactional to transformational leadership: Learning to share the

vision. Organizational Dynamics, 18(3), 19–31. http://doi.org/10.1016/0090-
2616(90)90061-S
Baysari, M. T., Caponecchia, C., McIntosh, A. S., & Wilson, J. R. (2009). Classification of
errors contributing to rail incidents and accidents: A comparison of two human error
identification techniques. Safety Science, 47(7), 948–957.
Baysari, M. T., McIntosh, A. S., & Wilson, J. R. (2008). Understanding the human factors
contribution to railway accidents and incidents in Australia. Accident Analysis and
Prevention, 40(5), 1750–1757. http://doi.org/10.1016/j.aap.2008.06.013
Belmonte, F., Schön, W., Heurley, L., & Capel, R. (2011). Interdisciplinary safety analysis of
complex socio-technological systems based on the functional resonance accident model:
An application to railway trafficsupervision. Reliability Engineering & System Safety,
96(2), 237–249. http://doi.org/10.1016/j.ress.2010.09.006
138 A. W. Siegel
Benedek, M., & Kaernbach, C. (2010). Decomposition of skin conductance data by means of
nonnegative deconvolution. Psychophysiology, 47(4), 647–658.
http://doi.org/10.1111/j.1469-8986.2009.00972.x
Berche, B., von Ferber, C., Holovatch, T., & Holovatch, Y. (2009). Resilience of public
transport networks against attacks. The European Physical Journal B - Condensed Matter
and Complex Systems, 71(1), 125–137. http://doi.org/10.1140/epjb/e2009-00291-3
Bieder, C., & Bourrier, M. (Eds.). (2013). Trapping safety into rules: how desirable or
avoidable is proceduralization? Farnham, Surrey: Ashgate Publishing Limited.
Billman, G. E., & Billman, George, E. (2011). Heart rate variability - a historical perspective.
Frontiers in Physiology, 2, 86. http://doi.org/10.3389/fphys.2011.00086
Blandford, A., & Furniss, D. (2006). DiCoT: A methodology for applying distributed cognition
to the design of teamworking systems. In S. W. Gilroy & M. D. Harrison (Eds.), DSVIS
2005 (Vol. 3941 LNCS, pp. 26–38). Berlin Heidelberg: Springer-Verlag.
http://doi.org/10.1007/11752707_3
Boucsein, W. (2012). Electrodermal activity (2nd ed.). New York, USA: Springer.
http://doi.org/10.1007/978-1-4614-1126-0
Bourrier, M., & Bieder, C. (2013). Trapping safety into rules: An introduction. In C. Bieder &
M. Bourrier (Eds.), Trapping safety into rules: how desirable or avoidable is
proceduralization? (pp. 1–25). Farnham, Surrey: Ashgate Publishing Limited.
Branlat, M., & Woods, D. D. (2010). How do systems manage their adaptive capacity to
successfully handle disruptions? A resilience engineering perspective. In AAAI Fall
Sympoisum (pp. 26–34). Retrieved from http://www.aaai.org/ocs/index.php/FSS/
FSS10/paper/viewPaper/2238
Breznitz, S. (1984). Cry wolf : the psychology of false alarms. Hillsdale N.J.: Lawrence Erlbaum
Associates.
Brookhuis, K. A., & de Waard, D. (2010). Measuring physiology in simulators. In D. L. Fisher,

J. D. Lee, J. K. Caird, & M. Rizzo (Eds.), Handbook of Driving Simulation for
Engineering, Medicine and Psychology (pp. 233–241). London: Taylor & Francis.
Retrieved from http://worldcat.org/isbn/9781420061000
Bryant, S. E. (2003). The role of transformational and transactional leadership in creating ,

sharing and exploiting organizational knowledge. The Journal of Leadership and
Organisational Studies, 9(4), 33–44. http://doi.org/10.1177/107179190300900403
Burns, C. M., & Hajdukiewicz, J. R. (2004). Ecologocal interface design. Boca Raton, FL: CRC
Press.
CBS - Statistics Netherlands. (2015). Rail suicides 2003-2012. Retrieved September 11, 2015,
from http://statline.cbs.nl/Statweb/publication/?DM=SLNL&PA=7022GZA&D1=7&D2=
A. W. Siegel 139
a&D3=a&D4=53-62&HDR=G2&STB=T,G1,G3&P=T&VW=T
CBS - Statistics Netherlands. (2016). Reizigers kilometer 2010-2014. Retrieved from

http://statline.cbs.nl/Statweb/publication/?DM=SLNL&PA=82512ned&D1=a&D2=18-
20&D3=(l-0)-l&VW=T
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal
of the Learning Sciences, 6(3), 271–315. http://doi.org/10.1207/s15327809jls0603_1
Collis Lynne, Schmid, F. (2012). Managing Interdependencies A Railway Case Study, 1–23.
Cook, R., & Rasmussen, J. (2005). “Going solid”: a model of system dynamics and
consequences for patient safety. Quality & Safety in Health Care, 14(2), 130–134.
http://doi.org/10.1136/qshc.2003.009530
Cooke, N. J., Stout, R. J., & Salas, E. (2001). A knowledge elicitation approach to the
measurement of the team situation awareness. New Trends in Cooperative Activities:
System Dynamics in Complex Settings. http://doi.org/10.1177/014233129301500405
Cowley, S., & Borys, D. (2014). Stretching but not too far : Understanding adaptive behaviour
using a model of organisational elasticity. Journal of Health and Safety, Research and
Practice, 6(2), 18–22.
Davis, D. R., & Parasuraman, R. (1982). The psychology of vigilance. New York: Academic
Press.
Day, R. E. (2005). Clearing up “implicit knowledge”: Implications for knowledge management,

information science, psychology, and social epistemology. Journal of the American
Society for Information Science and Technology, 56(February), 630–635.
http://doi.org/10.1002/asi.20153
De Carvalho, P. V. R. (2011). The use of Functional Resonance Analysis Method (FRAM) in a

mid-air collision to understand some characteristics of the air traffic management system
resilience. Reliability Engineering & System Safety, 96(11), 1482–1498.
http://doi.org/10.1016/j.ress.2011.05.009
De Regt, A., Siegel, A. W., & Schraagen, J. M. C. (2016). Towards quantifying metrics for rail-
system resilience: Identification and analysis of performance weak resilience signals.
Cognition, Technology & Work, 18(2), 319–331. http://doi.org/10.1007/s10111-015-0356-9
Dekker, S. (2003). Failure to adapt or adaptations that fail: Contrasting models on procedures
and safety. Applied Ergonomics, 34(3), 233–238. http://doi.org/10.1016/S0003-
6870(03)00031-0
Dekker, S. (2011). Drift into failure - from hunting broken components to understanding
complex systems. Farnham, Surrey: Ashgate Publishing Limited.
Dekker, S., Hollnagel, E., Woods, D. D., & Cook, R. (2008). Resilience Engineering : New
directions for measuring and maintaining safety in complex systems. Retrieved from
140 A. W. Siegel
https://msb.se/Upload/Kunskapsbank/Forskningsrapporter/Slutrapporter/2009 Resilience
Engineering New directions for measuring and maintaining safety in complex systems.pdf
DiMaggio, P. (1977). Culture and cognition. Annual Review of Sociology, (23), 263–287.
Doyle, J. C., Alderson, D. L., Li, L., Low, S., Roughan, M., Shalunov, S., … Willinger, W.
(2005). The “robust yet fragile” nature of the Internet. Proceedings of the National
Academy of Sciences of the United States of America, 102(41), 14497–14502 .
http://doi.org/10.1073/pnas.0501426102
Doyle, J. C., & Csete, M. (2011). Architecture, constraints, and behavior. Journal of the
National Academy of Sciences, 108(suppl. 3), 15624–15630.
Doyle, J. C., Francis, B. A., & Tannenbaum, A. R. (2013). Feedback control theory. New York:
Macmillan Publishing Company.
Dutch Ministry of Infrastructure and Environment. (2015). STS-passages 2014. Retrieved from
https://www.ilent.nl/Images/jaarrapportage-stoptonend-sein-passages-2014-van-de-
inspectie-leefomgeving-en-transport-ilt_tcm334-370877.pdf
Ellis, S., Carette, B., Anseel, F., & Lievens, F. (2014). Systematic reflection: Implications for
learning from failures and successes. Current Directions in Psychological Science, 23(1),
67–72. http://doi.org/10.1177/0963721413504106
European Railway Agency. (2014). Railway safety performance in the European Union 2014.
Retrieved from http://www.era.europa.eu/Document
-Register/Documents/SPR2014.pdf
Farrington-Darby, T., Wilson, J. R., Norris, B. J., & Clarke, T. (2006). A naturalistic study of
railway controllers. Ergonomics, 49(12–13), 1370–1394.
http://doi.org/10.1080/00140130600613000
Ferreira, P., Wilson, J. R., Ryan, B., & Sharples, S. (2011). Measuring resilience in the planning
of rail engineering work. In E. Hollnagel, J. Paries, D. D. Woods, & J. Wreathall (Eds.),
Resilience in practice (pp. 145–156). Aldershot, UK: Ashgate Publishing Limited.
Fiore, S. M., Rosen, M. A., Smith-Jentsch, K. E., Salas, E., Letsky, M., & Warner, N. (2010).
Toward an understanding of macrocognition in teams: predicting processes in complex
collaborative contexts. Human Factors, 52(2), 203–224.
http://doi.org/10.1177/0018720810369807
Flin, R., O’Connor, P., & Crichton, M. (2008). Safety at the sharp end - A guide to non-
technical skills. Aldershot, Hampshire: Ashgate Publishing Limited.
Fowler, D. (2013). Proceduralization of safety assessment - A barrier to rational thinking. In C.

Bieder & M. Bourrier (Eds.), Trapping safety into rules: how desirable or avoidable is
A. W. Siegel 141
Frappaolo, C. (2008). Implicit knowledge. Knowledge Management Research & Practice,

6(October 2007), 23–25. http://doi.org/10.1057/palgrave.kmrp.8500168
Fucks, I., & Dien, Y. (2013). “No rule, no use”? The effects of over-proceduralization. In C.
Bieder & M. Bourrier (Eds.), Trapping safety into rules: how desirable or avoidable is
Furniss, D., Back, J., Blandford, A., Hildebrandt, M., & Broberg, H. (2011). A resilience
markers framework for small teams. Reliability Engineering & System Safety, 96(1), 2–
10. http://doi.org/10.1016/j.ress.2010.06.025
Gao, Q., Wang, Y., Song, F., Li, Z., & Dong, X. (2013). Mental workload measurement for
emergency operating procedures in digital nuclear power plants. Ergonomics, 56:7, 1070–
1085. http://doi.org/10.1080/00140139.2013.790483
Goedhart, A. D., van der Sluis, S., Houtveen, J. H., Willemsen, G., & de Geus, E. J. C. (2007).
Comparison of time and frequency domain measures of RSA in ambulatory recordings.
Psychophysiology, 44(2), 203–215. http://doi.org/10.1111/j.1469-8986.2006.00490.x
Goverde, R. M. P. (2005). Punctuality of railway operations and timetable stability analysis.

PhD thesis. Delft University of Technology. Retrieved from
http://repository.tudelft.nl/view/ir/uuid:a40ae4f1-1732-4bf3-bbf5-fdb8dfd635e7/
Goverde, R., & Odijk, M. (2002). Performance evaluation of network timetables using PETER.
In J. Allan, R. J. Hill, C. A. Brebbia, & G. Sciutto (Eds.), Computers in Railways VIII (pp.
731–740). Southampton: WIT press.
Grote, G. (2014). Promoting safety by increasing uncertainty – Implications for risk

management. Safety Science. http://doi.org/10.1016/j.ssci.2014.02.010
Hale, A., & Borys, D. (2012). Working to rule or working safely? Part 2: The management of
safety rules and procedures. Safety Science, 55, 222–231.
Hansen, I. A. (Ed.). (2010). Timetable Planning and Information Quality. Southampton: WIT
Press.
Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results
of empirical and theoretical research. Advances in Psychology, 52, 139–183. Retrieved
from http://humanfactors.arc.nasa.gov/groups/TLX/downloads/NASA-TLXChapter.pdf
Haunschild, P. R., & Sullivan, B. N. (2002). Learning from Complexity: Effects of Prior
Accidents and Incidents on Airlines’ Learning. Administrative Science Quarterly, 47(4),
609–643. http://doi.org/10.2307/3094911
Healey, J. A., & Picard, R. W. (2005). Detecting stress during real-world driving tasks using
physiological sensors. IEEE Transactions on Intelligent Transportation Systems, 6(2),
156–166. http://doi.org/10.1109/TITS.2005.848368
142 A. W. Siegel
Heaslip, K., Louisell, W., Collura, J., & Urena Serulle, N. (2010). A Sketch Level Method for
Assessing Transportation Network Resiliency to Natural Disasters and Man-Made Events.
In Transportation Research Board 89th Annual Meeting, Washington, D.C (Vol. 10).
Heath, C., & Luff, P. (2000). Technology in action. Cambridge: Cambridge University Press.
Hendy, K. C., Hamilton, K. M., & Landry, L. N. (1993). Measuring subjective workload: When
is one scale better than many? Human Factors, 35(4), 579–601.
Herrmann, T., Hoffmann, M., Kunau, G., & Loser, K.-U. (2004). A modelling method for the
development of groupware applications as socio-technical systems. Behaviour &
Information Technology, 23(2), 119–135. http://doi.org/10.1080/01449290310001644840
Hoffman, R. R., & Woods, D. D. (2011). Beyond Simon’s Slice: Five Fundamental Trade-Offs
that Bound the Performance of Macrocognitive Work Systems. Intelligent Systems, IEEE,
26(6), 67–71. Retrieved from http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=
6096576
Hollnagel, E. (2004). Barriers and accident prevention. Aldershot, UK: Ashgate Publishing
Limited.
Hollnagel, E. (2009). The four cornerstones of resilience engineering. In C. P. Nemeth, E.

Hollnagel, & S. Dekker (Eds.), Resilience Engineering Perspectives. Volume 2:
Preparation and restoration (pp. 117–134). Farnham, Surrey: Ashgate Publishing
Limited.
Hollnagel, E. (2011a). Prologue: the scope of resilience engineering. In E. Hollnagel, J. Pariès,

D. D. Woods, & J. Wreathall (Eds.), Resilience engineering in practice: A guidebook (pp.
xxix–xxxix). Farnham, Surrey: Ashgate Publishing Limited.
Hollnagel, E. (2011b). RAG - The Resilience Analysis Grid. In E. Hollnagel, J. Paries, D. D.

Woods, & J. Wreathall (Eds.), Resilience engineering in practice: A guidebook (pp. 275–
296). Farnham, Surrey: Ashgate Publishing Limited.
Hollnagel, E. (2012). FRAM: The Functional Resonance Analysis Method: Modelling. Farnham,
Surrey: Ashgate Publishing Limited.
Hollnagel, E. (2014). Safety-I and Safety–II: The Past and Future of Safety Management.
Farnham, Surrey: Ashgate Publishing Limited.
Hollnagel, E., Woods, D. D., & Leveson, N. G. (Eds.). (2006). Resilience engineering: concepts
and percepts. Hampshire: Ashgate Publishing Limited.
Hoover, A., Singh, A., Fishel-Brown, S., & Muth, E. (2012). Real-time detection of workload
changes using heart rate variability. Biomedical Signal Processing and Control, 7(4),
333–341. http://doi.org/10.1016/j.bspc.2011.07.004
Hovden, J., Størseth, F., & Tinmannsvik, R. K. (2011). Multilevel learning from accidents -
Case studies in transport. Safety Science, 49(1), 98–105 .
A. W. Siegel 143
Jeffcott, S., Pidgeon, N., Weyman, A., & Walls, J. (2006). Risk, trust, and safety culture in U.K.
train operating companies. Risk Analysis, 26(5), 1105–1121.
http://doi.org/10.1111/j.1539-6924.2006.00819.x
Jenkins, D. P., Farmilo, A., Stanton, N. A., Whitworth, I., Salmon, P. M., Hone, G., … Walker,
G. H. (2007). The CWA Tool V0.95. Human Factors Integration Defence Technology
Centre (HFI DTC), Yeovil Somerset UK.
Jorna, P. G. A. M. (1992). Spectral analysis of heart rate and psychological state: A review of its
validity as a workload index. Biological Psychology, 34(2), 237–257.
Kezunovic, M., Xie, L., & Grijalva, S. (2013). The role of big data in improving power system
operation and protection. Proceedings of IREP Symposium: Bulk Power System
Dynamics and Control - IX Optimization, Security and Control of the Emerging Power
Grid, IREP 2013. http://doi.org/10.1109/IREP.2013.6629368
Kichenside, G., & Williams, A. (1998). Two Centuries of Railway Signalling. Yeovil: Oxford
publishing co.
Klein, G. A., Moon, B., & Hoffman, R. R. (2006). Making sense of sensemaking 2: A
macrocognitive model. IEEE Intelligent Systems, 21(5), 88–92.
http://doi.org/10.1109/MIS.2006.100
Klein, G. A., Phillips, J. K., Rall, E. L., & Peluso, D. A. (2007). A Data-Frame Theory of
Sensemaking. In R. R. Hoffman (Ed.), Expertise out of context: Proceedings of the sixth
international conference on naturalistic decision making (pp. 113–155). New York:
Lawrence Erlbaum Associates.
Langer, E. J. (1989). Mindfulness. Reading, MA: Addison-Wesley.
Lekka, C. (2011). High reliability organisations: A review of the literature. Health and Safety
Executive. Retrieved from http://www.hse.gov.uk/research/rrpdf/rr899.pdf
Leveson, N. G. (2004). A new accident model for engineering safer systems. Safety Science,
42(4), 237–270.
Leveson, N. G. (2016). Rasmussen’s legacy: A paradigm change in engineering for safety.

Applied Ergonomics, 1–11. http://doi.org/10.1016/j.apergo.2016.01.015
Leveson, N. G., Dulac, N., Zipkin, D., Cutcher-Gershenfeld, J., Carroll, J., & Barrett, B. (2006).
Engineering Resilience into Safety-Critical Systems. In E. Hollnagel, D. D. Woods, & N.
G. Leveson (Eds.), In Resilience Engineering: Concepts And Precepts (pp. 95–123).
Hampshire: Ashgate Publishing Limited.
Lintern, G. (2009). The foundations and pragmatics of cognitive work analysis: A systematic
approach to design of large-scale information systems. Open source. Cognitive Systems
Design. Retrieved from www.cognitivesystemsdesign.net
144 A. W. Siegel
Lowe, E., & Pickup, L. (2008). Network rail signaller’s workload toolkit. Contemporary
Ergonomics, 558–563. Retrieved from
http://ergotools.co.uk/Content/Documents/Ergonomic Society Conference 08 - Workload
Toolkit paper.pdf
Lundberg, J., & Johansson, B. J. (2015). Systemic resilience model. Reliability Engineering and
System Safety, 141, 22–32. http://doi.org/10.1016/j.ress.2015.03.013
Lundberg, J., Rollenhagen, C., & Hollnagel, E. (2009). What-You-Look-For-Is-What-You-Find

- The consequences of underlying accident models in eight accident investigation
manuals. Safety Science, 47(10), 1297–1311. http://doi.org/10.1016/j.ssci.2009.01.004
Mach, E. I. (1905). Erkenntnis und Irrthum. Skizzen zur Psychologie der Forschung. Leipzig:
Barth.
Madni, A. M., & Jackson, S. (2009). Towards a conceptual framework for resilience
engineering. IEEE Systems Journal, 3(2), 181–191.
http://doi.org/10.1109/JSYST.2009.2017397
Malik, M. (1996). Heart Rate Variability. Annals of Noninvasive Electrocardiology, 1(2), 151–
181. http://doi.org/10.1111/j.1542-474X.1996.tb00275.x
Marsden, G., & Bonsall, P. (2006). Performance targets in transport policy. Transport Policy,
13(3), 191–203. http://doi.org/10.1016/j.tranpol.2005.09.001
Millen, D. R. (2000). Rapid Ethnography: Time Deepening Strategies for HCI Field Research.
Proceedings of the Conference on Designing Interactive Systems: Processes, Practices,
Methods and Techniques, 280–288. http://doi.org/10.1145/347642.347763
Minister van Verkeer & Waterstaat. (2010). Veiligheid van het railvervoer - aan de Tweede
Kamer der Staten-Generaal. Retrieved from https://zoek.officielebekendmakingen.nl/kst-
29893-100.pdf
Morgan, D. (2016). FAA fails to ensure pilots’ manual flying skills: government report.
Retrieved from http://www.reuters.com/article/us-usa-faa-pilots-
idUSKCN0UP21N20160111
Morrison, J. E., & Meliza, L. L. (1999). Foundations of the after action review process.
Retrieved from http://oai.dtic.mil/oai/oai?
verb=getRecord&metadataPrefix=html&identifier=
ADA368651
Muckler, F. A., & Seven, S. A. (1992). Selecting performance measures: “Objective” versus
“subjective” measurement. Human Factors, 34(4), 441–455.
Murphy, P. (2001). The role of communications in accidents and incidents during rail
possessions. In Engineering Psychology and Cognitive Ergonomics Volume Five.
A. W. Siegel 145
Aerospace and Transportation Systems. (pp. 447–454). Aldershot, UK: Ashgate

Publishing Limited.
Naikar, N. (2013). Work Domain Analysis. CRC Press. http://doi.org/10.1201/b14774
Nakamura, D. (2013). Operational use of flight path management systems. Retrieved from
http://www.faa.gov/about/office_org/headquarters_offices/avs/offices/afs/afs400/parc/
parc_reco/media/2013/130908_parc_fltdawg_final_report_recommendations.pdf
Neerincx, M. A. (2003). Cognitive task load analysis: allocating tasks and designing support. In
E. Hollnagel (Ed.), Handbook of cognitive task design (pp. 283–305). Mahwah, NJ:
Lawrence Erlbaum Associates.
Nemeth, C. P., & Herrera, I. (2015). Building change: Resilience Engineering after ten years.
Reliability Engineering and System Safety, 141, 1–4.
http://doi.org/10.1016/j.ress.2015.04.006
Nikandros, G., & Tombs, D. (2007). Measuring railway signals passed at danger. In
Proceedings of the twelfth Australian conference on safety, critical systems and software
(Vol. 86, pp. 41–46).
Norman, D. A. (1994). Things that make us smart: Defending human attributes in the age of the
machine. New York: Basic Books.
Norros, L., Liinasuo, M., & Savioja, P. (2014). Operators’ orientations to procedure guidance in
NPP process control. Cognition, Technology and Work, 16(4), 487–499.
http://doi.org/10.1007/s10111-014-0274-2
Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: what can be
done? Assessment & Evaluation in Higher Education, 33(3), 301–314 .
http://doi.org/10.1080/02602930701293231
Perin, C. (2005). Shouldering risks: the culture of control in the nuclear power industry.
Princeton, NJ: Princeton University Press.
Pfautz, J. D., Carlson, E. C., Farry, M. P., & Koelle, D. M. (2009). Enabling operator/analyst
trust in complex human socio- cultural behavior models. In Proceedings of Human
Behavior-Computational Intelligence Modeling Conference (HB-CMI).
Pickup, L., Wilson, J. R., Nichols, S., & Smith, S. (2005). A conceptual framework of mental
workload and the development of a self-supporting integrated workoad scale for railway
signallers. In J. R. Wilson, B. J. Norris, T. Clarke, & A. Mills (Eds.), Rail human factors
(pp. 319–329). Surrey: Ashgate Publishing Limited.
Pickup, L., Wilson, J. R., Norris, B. J., Mitchell, L., & Morrisroe, G. (2005). The Integrated
Workload Scale (IWS): a new self-report tool to assess railway signaller workload.
Applied Ergonomics, 36(6), 681–693. http://doi.org/10.1016/j.apergo.2005.05.004
146 A. W. Siegel
Poh, M.-Z., Swenson, N. C., & Picard, R. W. R. W. (2010). A wearable sensor for unobtrusive,
long-term assessment of electrodermal activity. IEEE Transactions on Biomedical
Engineering, 57(5), 1243–1252. http://doi.org/10.1109/TBME.2009.2038487
Polanyi, M. (1969). Knowing and being: In M. Grene (Ed.), Essays by Michael Polanyi (p. 264).
Chicago: University of Chicago Press.
Pretorius, A., & Cilliers, P. J. (2007). Development of a mental workload index: a systems
approach. Ergonomics, 50(9), 1503–15. http://doi.org/10.1080/00140130701379055
Qureshi, Z. H. (2008). A review of accident modelling approaches for complex critical

sociotechnical systems. In 12th Australian Workshop on Safety Related Programmable
Systems (SCS’07), Adelaide (Vol. 86, pp. 47–59).
Rankin, A., Lundberg, J., Woltjer, R., Rollenhagen, C., & Hollnagel, E. (2013). Resilience in
everyday operations: A framework for analyzing adaptations in high-Rrisk work. Journal
of Cognitive Engineering and Decision Making, 8(1), 78–97 .
http://doi.org/10.1177/1555343413498753
Rasmussen, J. (1983). Skills, rules, and knowledge; signals, signs, and symbols, and other
distinctions in human performance models. IEEE Transactions on Systems, Man and
Cybernetics, SMC-13(3), 257–266.
Rasmussen, J. (1985). The role of hierarchical knowledge representation in decisionmaking and

system management. IEEE Transactions on Systems, Man, and Cybernetics, SMC-15(2),
234–243. http://doi.org/10.1109/TSMC.1985.6313353
Rasmussen, J. (1997). Risk management in a dynamic society: a modelling problem. Safety

Science, 27(2/3), 183–213.
Resilience engineering association. (2016). Retrieved May 1, 2016, from http://www.resilience-

engineering-association.org/
Reymen, I. M. M. J. (2003). Research on design reflection: overview and directions. In

Proceedings of ICED 03, the 14th International Conference on Engineering Design,
Stockholm (p. 33–34 (exec.summ.), full paper DS31_1148FPB).
Scheffer, M., Hosper, S. H., Meijer, M. L., Moss, B., & Jeppesen, E. (1993). Alternative
equilibria in shallow lakes. Trends in Ecology & Evolution, 8(8), 275–279.
http://doi.org/10.1016/0169-5347(93)90254-M
Schippers, M. C., Den Hartog, D. N., & Koopman, P. L. (2007). Reflexivity in teams: A
measure and correlates. Applied Psychology, 56(2), 189–211.
http://doi.org/10.1111/j.1464-0597.2006.00250.x
Schippers, M. C., Edmondson, A. C., & West, M. A. (2014). Team reflexivity as an antidote to
team information - processing failures. Small Group Research, 45(6), 731–769.
http://doi.org/10.1177/1046496414553473
A. W. Siegel 147
Schraagen, J. M. C. (2015). Resilience and networks. In 6th REA Symposium managing

resilience; 22th-25th June 2015, Lisbon, Portugal (pp. 1–6). Retrieved from
http://www.resilience-engineering-association.org/download/resources/symposium/
symposium_2015/
Schraagen_J.M.-Resilience-and-networks-Paper.pdf
Schulman, P. (2013). Procedural paradoxes and the management of safety. In C. Bieder & M.
Bourrier (Eds.), Trapping safety into rules: how desirable or avoidable is
Shanahan, P., Gregory, D., Shannon, M., & Gibson, H. (2007). The role of communication
errors in railway incident causation. In J. R. Wilson, B. Norris, T. Clarke, & A. Mills
(Eds.), People and rail systems—human factors at the heart of the railway. (pp. 427–35).
Farnham, Surrey: Ashgate Publishing Limited.
Siegel, A. W., & Schraagen, J. M. C. (2014a). A method to reveal workload weak-resilience-

signals at a rail control post. In D. Harris (Ed.), HCII 2014 (pp. 82–93). Springer Berlin
Heidelberg. http://doi.org/10.1007/978-3-319-07515-0
Siegel, A. W., & Schraagen, J. M. C. (2014b). Can weak-resilience-signals (WRS) reveal

obstacles compromising (rail-) system resilience? In D. de Waard, J. Sauer, S. Röttger, A.
Kluge, D. Manzey, C. Weikert, … H. Hoonhout (Eds.), Human Factors and Ergonomics
Society Europe Chapter 2014 Annual Conference (Vol. 2333–4959, pp. 187–191).
Retrieved from http://www.hfes-europe.org/human-factors-high-reliability-industries-2/
Siegel, A. W., & Schraagen, J. M. C. (2014c). Developing resilience signals for the Dutch
railway system. In 5th REA Symposium managing trade-offs; 24th-27th June 2013,
Soesterberg, Netherlands (pp. 191–196). Retrieved from http://hdl.handle.net/1811/60454
Siegel, A. W., & Schraagen, J. M. C. (2014d). Measuring workload weak resilience signals at a
rail control post. IIE Transactions on Occupational Ergonomics and Human Factors, 2(3–
4), 179–193. http://doi.org/10.1080/21577323.2014.958632
Siegel, A. W., & Schraagen, J. M. C. (2015). Can team reflection of rail operators make
resilience-related knowledge explicit? - an observational study design. In 6th REA
Symposium managing resilience; 22th-25th June 2015, Lisbon, Portugal (pp. 126–131).
Retrieved from http://www.resilience-engineering-association.org/download/resources/
symposium/
symposium_2015/Siegel_W.-Schraagen_J.M.-Can-team-reflection-of-rail-operators-
make-resilience-related-knowledge-Paper.pdf
Siegel, A. W., & Schraagen, J. M. C. (2016). Team reflection makes resilience-related

knowledge explicit through collaborative sensemaking. Manuscript submitted for
publication.
Siegel, A. W., & Schraagen, J. M. C. (2017). Beyond procedures: Team reflection in a rail
control centre to enhance resilience. Safety Science, 91, 181–191.
148 A. W. Siegel
Simon, H. A. (1996). The architecture of complexity: Hierachic systems. In The Sciences of the
Artificial (3rd ed., pp. 183–216). Cambridge, Massachusetts: MIT Press.
Staw, B. M. (1981). The escalation of commitment to a course of action. Academy of

Management Review, 6(4), 577–587. http://doi.org/10.5465/AMR.1981.4285694
Steenhuisen, B. (2012). Cutting dark matter: professional practice after institutional reform. In
IRSPM XVI: 16th Annual Conference of the International Research Society for Public
Management, Rome, Italy, 11-13 April 2012 (pp. 1–17). Retrieved from
http://repository.tudelft.nl/assets/uuid:3f7bb6b1-8bf1-49a0-bd4c-337d28615e9d/
285905.pdf
Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics. Boston, Massachusetts:
Allyn and Bacon.
Tamvakis, P., & Xenidis, Y. (2013). Comparative evaluation of resilience quantification

methods for infrastructure systems. Procedia - Social and Behavioral Sciences, 74, 339–
348. http://doi.org/10.1016/j.sbspro.2013.03.030
Taylor, F. W. (1911). The principles of scientific management. New York: Harper. Retrieved
from http://www.gutenberg.org/ebooks/6435
Togo, F., & Takahashi, M. (2009). Heart rate variability in occupational health -a systematic
review. Industrial Health, 47(6), 589–602.
Top, J. Van Den, & Steenhuisen, B. (2009). Understanding ambiguously structured rail traffic
control practices. International Journal of Technology, Policy and Management,
9(2), 148–161.
Underwood, P., & Waterson, P. (2013). Systemic accident analysis: Examining the gap between
research and practice. Accident Analysis and Prevention, 55, 154–164.
http://doi.org/10.1016/j.aap.2013.02.041
Van Broekhoven, R., Siegel, A. W., Schraagen, J. M. C., & Noordzij, M. L. (2016). Comparison
of real-time relative workload measurements in rail signallers. In B. Milius & A.
Naumann (Eds.), Rail Human Factors Proceedings of the 2nd Germany Workshop
March, 8th and 9th, 2016, Stadthalle Braunschweig (pp. 30–40). Braunschweig: ITS
automotive nord. Retrieved from http://doc.utwente.nl/99353/
Van der Beek, D. F. A., & Schraagen, J. M. C. (2015). ADAPTER: Analysing & Developing
Adaptability & performance in Teams to enhance resilience. Reliability Engineering &
System Safety, 141, 33–44. http://doi.org/10.1016/j.ress.2015.03.019
Vaughan, D. (2002). Signals and interpretive work: The role of culture in a theory of practical
action. In K. A. Cerulo (Ed.), Culture in mind: Toward a sociology of culture and
cognition (pp. 28–54). New York: Routledge.
A. W. Siegel 149
Veltman, J. A., & Gaillard, A. W. K. (1993). Indices of mental workload in a complex task
environment. Neuropsychobiology, 28, 72–75.
Veltman, J. A., & Gaillard, A. W. K. (1996). Pilot workload evaluated with subjective and
physiological measures. In K. Brookhuis, C. Weikert, J. Moraal, & D. Waard de (Eds.),
Human factors and ergonomics society (pp. 107–128). Haren: University of Groningen.
Vicente, K. J. (1999). Cognitive work analysis: Towards safe, productive and healthy computer-
based work. Mahwah, New Jersey: Lawrence Erlbaum Associates.
Vicente, K. J., & Rasmussen, J. (1992). Ecological interface Design: Theoretical Foundations.
IEEE Transactions on Systems, Man and Cybernetics, 22(4), 589–606.
Walker, B., Holling, C. S., Carpenter, S. R., & Kinzig, A. (2004). Resilience, adaptability and
transformability in social-ecological systems. Ecology and Society, 9(2), 5.
Waterson, P., Robertson, M. M., Cooke, N. J., Militello, L., Roth, E., & Stanton, N. A. (2015).
Defining the methodological challenges and opportunities for an effective science of
sociotechnical systems and safety. Ergonomics, 139(April), 1–35 .
http://doi.org/10.1080/00140139.2015.1015622
Weick, K. E., & Sutcliffe, K. M. (2001). Managing the unexpected. San Francisco, CA: Jossey-
Bass.
West, M. A. (2000). Reflexivity, revolution and innovation in work teams. In M. M. Beyerlein,

D. A. Johnson, & S. T. Beyerlein (Eds.), Product development teams (Vol. 5, pp. 1–29).
Stanford, CT: JAI Press.
Wiedow, A., & Konradt, U. (2010). Two-dimensional structure of team process improvement:
Team reflection and team adaptation. Small Group Research, 42(1), 32–54.
http://doi.org/10.1177/1046496410377358
Wilms, M. S., & Zeilstra, M. P. (2013). Subjective mental workload of Dutch train dispatchers:
Validation of IWS in a practical setting. In 4th International Conference on Rail Human
Factor (pp. 641–650).
Wilson, J. R., Farrington-Darby, T., Cox, G., Bye, R., & Hockey, G. (2007). The railway as a
socio-technical system: human factors at the heart of successful rail engineering.
Proceedings of the Institution of Mechanical Engineers, Part F: Journal of Rail and
Rapid Transit. http://doi.org/10.1243/09544097JRRT78
Wilson, J. R., & Norris, B. J. (2005). Rail human factors: Past, present and future. Applied
Ergonomics, 36(6), 649–660. http://doi.org/10.1016/j.apergo.2005.07.001
Woods, D. D. (2003). Discovering how distributed cognitive systems work. In E. Hollnagel

(Ed.), Handbook of Cognitive Task Design (pp. 37–53). Mahwah, New Jersey: Lawrence
Erlbaum Associates.
150 A. W. Siegel
Woods, D. D. (2006a). Essential characteristics of resilience. In E. Hollnagel, D. D. Woods, &

N. G. Leveson (Eds.), Resilience engineering: concepts and precepts (pp. 21–34).
Aldershot, UK: Ashgate Publishing Limited.
Woods, D. D. (2006b). Resilience engineering: Redefining the culture of safety and risk
management. Human Factors and Ergonomics Society Bulletin, 49(12), 1–8.
Woods, D. D. (2011). Resilience and the ability to anticipate. In E. Hollnagel, J. Pariès, D. D.

Woods, & J. Wreathall (Eds.), Resilience engineering in practice: A guidebook (pp. 121–
126). Farnham, Surrey: Ashgate Publishing Limited.
Woods, D. D., & Branlat, M. (2010). Hollnagel’s test: Being “in control” of highly
interdependent multi-layered networked systems. Cognition, Technology and Work, 12(2),
95–101. http://doi.org/10.1007/s10111-010-0144-5
Woods, D. D., Chan, Y. J., & Wreathall, J. (2014). The stress-strain model of resilience
operationalizes the four cornerstones of resilience engineering. Retrieved from
http://hdl.handle.net/1811/60454
Woods, D. D., & Cook, R. (2006). Incidents–markers of resilience or brittleness. In E.

Hollnagel, D. D. Woods, & N. Leveson (Eds.), Resilience Engineering. Concepts and
Precepts (pp. 69–75). Aldershot, Hampshire: Ashgate Publishing Limited.
Woods, D. D., & Patterson, E. S. (2001). How unexpected events produce an escalation of
cognitive and coordinative demands. In P. A. Hancock & P. A. Desmond (Eds.), Stress,
workload, and fatigue (pp. 290–304). Hillsdale, N.J.: Lawrence Erlbaum Associates.
Woods, D. D., Patterson, E. S., & Cook, R. I. (2007). Behind human error: Taming complexity
to improve patient safety. In P. Carayon (Ed.), Handbook of human factors and
ergonomics in health care and patient safety (pp. 459–476). Mahwah, NJ: Lawrence
Erlbaum Associates.
Woods, D. D., Schenk, J., & Allen, T. (2009). An initial comparison of selected models of
system resilience. In Resilience engineering perspectives (pp. 73–94). Surrey: Ashgate
Publishing Limited.
Woods, D. D., & Wreathall, J. (2008). Stress-Strain plots as a basis for assessing system
resilience. In E. Hollnagel, C. P. Nemeth, & S. Dekker (Eds.), Resilience engineering
perspectives, volume 1: Remaining sensitive to the possibility of failure (pp. 145–161).
Aldershot, UK: Ashgate Publishing Limited.
Woolley, A. W., Gerbasi, M. E., Chabris, C. F., Kosslyn, S. M., & Hackman, J. R. (2008).
Bringing in the experts: How team composition and collaborative planning jointly shape
analytic effectiveness. Small Group Research, 39(3), 352–371 .
http://doi.org/10.1177/1046496408317792
Xie, B., & Salvendy, G. (2000). Review and reappraisal of modelling and predicting mental
workload in single- and multi-task environments. Work & Stress, 14(1), 74–99.
A. W. Siegel 151
http://doi.org/10.1080/026783700417249
Young, M. S., Brookhuis, K. A., Wickens, C. D., & Hancock, P. A. (2015). State of science:
mental workload in ergonomics. Ergonomics, 58(1), 1–17 .
http://doi.org/10.1080/00140139.2014.956151
Zagoršek, H., Dimovski, V., & Škerlavaj, M. (2009). Transactional and transformational
leadership impacts on organizational learning. Journal for East European Management
Studies, 14(2), 144–165.
152 A. W. Siegel
A. W. Siegel 153
SAMENVATTING
Teamreflectie op zwakke veerkrachtsignalen
Veerkrachtverhoging van een socio-technisch spoorvervoersysteem
Socio-technische systemen worden steeds complexer terwijl het niveau van vereiste veiligheid
steeds verder toeneemt. Onder een socio-technisch systeem wordt de interactie verstaan tussen
mens en techniek om gedefinieerde doelen te halen. Voorbeelden van complexe socio-
technische systemen zijn de burgerluchtvaart, proces- en nucleaire industrie en het vervoer over
het spoor. Bestaande ontwikkel-, onderhouds- en bestuurs-methodes zijn succesvol gebleken in
het behalen van een zeer kleine faalkans, maar hebben een bovengrens bereikt. Nieuwe
paradigma’s zijn nodig om deze grenzen te doorbreken. In dit proefschrift wordt een stap gezet
in deze zoektocht naar nieuwe paradigma’s. De focus ligt daarbij op het domein van het
spoorvervoer en in het bijzonder de functie van treindienstleider. Het is een poging om nieuwe
wegen te vinden om socio-technische spoorsystemen te observeren, te analyseren en te
doorgronden. Hierdoor worden de treindienstleider en zijn omgeving in staat gesteld dreigende
verstoringen eerder op te merken met als doel toenemende complexiteit het hoofd te kunnen
bieden.
De doelstelling van de onderzoeken in dit proefschrift is om de vermogens van treindienstleiders

te verhogen, zodanig dat de veiligheid van het socio-technisch spoorsysteem toeneemt of gelijk
blijft als zich onverwachte en onvoorziene situaties voordoen. In de gekozen aanpak keken we
naar het gedrag van het socio-technisch spoorsysteem door de bril van de resilience (veerkracht)
engineering. Resilience engineering houdt zich bezig met het managen van het vermogen om
om te gaan met verstoringen, calamiteiten en externe druk. In het proefschrift wordt het concept
van een ‘zwak signaal van veerkracht’ geïntroduceerd, gedefinieerd als een signaal dat
vervolgonderzoek behoeft om onderliggende veerkracht-gerelateerde kennis expliciet te maken.
Complementair aan het zwakke signaal van veerkracht is een proces dat de kennis expliciet
maakt. Teamreflectie is een dergelijk proces, in deze studie aangepast en ontwikkeld voor
treindienstleiders om te kunnen reflecteren op zwakke signalen. De hypothese van het
onderzoek is dat teamreflectie tot een verhoogde veerkracht van het socio-technisch systeem zal
leiden doordat impliciete kennis expliciet zichtbaar, geanalyseerd en toegepast wordt. Met
andere woorden: het inzichtelijk maken van zwakke signalen van veerkracht leidt tot kennis die
er wel is maar niet wordt onderkend en toegepast. In dit proefschrift wordt dit concept
ontwikkeld, omgezet naar een prototype en getoetst in een operationele spoorinfra omgeving.
Dit is de basis voor antwoorden op drie overkoepelende onderzoeksvragen: (Hoe) kunnen
veerkracht indicaties gemeten worden tijdens real-time operatie (Hoofdstukken 2 en 3)? Wat is
een methode voor control room teams om te leren van hun recente operationele ervaringen
(Hoofdstukken 4 en 5)? Welke vaardigheden dienen control room teamleden te ontwikkelen als
154 A. W. Siegel
gevolg van voortschrijdende automatisering teneinde de veerkracht te behouden of te verhogen

(Hoofdstukken 4 en 5)?
Hoofdstuk 2 beschrijft een observatiestudie op een controlepost van de spoorinfra om zwakke

signalen van de veerkracht te meten die betrekking hebben op het naderen van de grens van de
acceptabele werklast. Een zwak signaal van veerkracht wordt hier gedefinieerd als een
onderscheidende relatieve beweging, met een mogelijke variabele discrepantie, van het
besturingssysteem naar zijn grenzen. We ontwikkelden een raamwerk om de veerkracht van een
socio-technisch spoorsysteem modelmatig in kaart te brengen. Dit raamwerk resulteerde in een
generiek kwantificeerbaar model dat zwakke signalen van veerkracht beschrijft. Het beschreven
model gaat uit van drie systeemgrenzen: prestatie, werklast en veiligheid. Op basis van
prestatiematen en gegevens die offline beschikbaar waren, werd het zwakke signaal van
veerkracht eerst afzonderlijk in het werklastdomein bestudeerd. We modelleerden het zwakke
werklastsignaal en kwantificeerden het vervolgens door subjectieve en objectieve
werklastmetingen, die weer afgeleid waren van taaklastmetingen. We gebruikten
hartslagvariabiliteit, als objectieve fysiologische maat, om veranderingen in de werklast
algoritmen te kunnen correleren en verifiëren. Vervolgens introduceerden we in het onderzoek
een nieuwe maat om variaties in werklast te kunnen kwantificeren en onderling te vergelijken.
We noemden deze maat Stretch, een maat die de werkinspanning omtrent een gebeurtenis
behelst. Een discrepantie tussen objectieve en subjectieve Stretch werd gezien als een zwak
werklastsignaal van de veerkracht, omdat we ervan uit gaan dat normaliter objectieve en
subjectieve Stretches synchroon lopen. In het observatieonderzoek heeft een dergelijk signaal
geholpen bij het blootleggen van een probleem omtrent rangeren, dat buitenproportioneel veel
aandacht opeiste, de beschikbare capaciteiten van treindienstleiders verminderde en daarmee de
veerkracht toestand aantastte. Een volgende stap was om het gekwantificeerde werklastmodel te
gaan toepassen in een real-time applicatie. We hebben het model getoetst aan
gedragsobservaties, fysiologische meting van huidgeleiding en subjectief ervaren werklast. In
een tweetal real-time situaties vonden wij dat de verschillen in werklast vooral tot uiting
kwamen bij communicatie, handmatige acties en kosten gemoeid met het schakelen tussen
taken.
Hoofdstuk 3 beschrijft een observatiestudie op een controlepost van de spoorinfra om zwakke

prestatiesignalen van veerkracht te meten. Om de uitgangspositie van de veerkracht vast te
stellen hebben we een meervoudige methode toegepast die bestond uit etnografische observatie
en een vragenlijst over de veerkracht. Op basis van het generieke model dat zwakke signalen
van veerkracht beschrijft, ontwikkelden we een prestatie-signaalmodel, dat we met behulp van
offlinedata implementeerden. We introduceerden een analysefunctie van deze zwakke signalen,
als een methode om potentiële bronnen te identificeren, die de veerkracht van het systeem
kunnen ondermijnen. De resultaten laten zien dat een dergelijke analysefunctie inderdaad
gebruikt kan worden om afwijkingen in veerkracht ten opzichte van de uitgangssituatie bloot te
leggen. Deze afwijkingen kunnen vervolgens worden geïnterpreteerd en gebruikt om
(sluimerende) afwijkingen in de veerkracht op te sporen.
A. W. Siegel 155
In hoofdstuk 4 introduceren wij teamreflectie (Ellis et al., 2014) als een macro-cognitieve
functie die veerkracht-gerelateerde kennis expliciet maakt. Deze kennis is impliciet aanwezig bij
individuele teamleden uit de operationele omgeving, maar wordt niet expliciet gedeeld omdat de
beperkingen waaronder treindienstleiders moeten werken niet altijd zichtbaar zijn. We voerden
een observatiestudie uit op een spoorinfra controlepost om de waarde in te schatten van
teamreflectie voor het expliciet maken van veerkracht-gerelateerde kennis. Voor dit doel is een
real-time prototype ontwikkeld. Deze applicatie toonde zwakke prestatiesignalen van veerkracht
gecombineerd met analysefuncties voor deze signalen. De applicatie maakte het voor een
treindienstleidersteam mogelijk om bewegingen naar de grenzen van het systeem te analyseren
en kennis daaromtrent te delen. Een observatie van een team, gedurende een week, toonde aan
hoe zij reflecteerden aan het einde van hun dienst over opmerkelijke systeemveranderingen. We
gebruikten een globale inhoudelijke analyse van hun gesprekken om de relevantie aan te tonen
van de inhoud van het besprokene voor veerkracht. De analyse toonde tevens de toename aan
van veerkracht-gerelateerde kennis tijdens de observatieperiode. Een specifieke casus van een
persoon die het treinspoor naderde, een potentiele zelfdoding, werd in detail geanalyseerd. De
resultaten toonden de waarde van teamreflectie voor systeembewegingen naar de grenzen van
het socio-technische systeem aan.
Hoofdstuk 5, de finale fase van ons onderzoek, beschrijft een observatiestudie over
teamreflectie. Daarbij maakte een team van treindienstleiders gebruik van een door ons
ontwikkeld real-time prototype, de Resiliencer genaamd. De Resiliencer verschaft zwakke
signalen van veerkracht op alle drie systeemgrenzen. De Resiliencer biedt eveneens
analysefuncties om signalen op systeemniveau te koppelen aan persoonlijke identificeerbare
details. Onder het begrip proceduralisatie verstaan wij de activiteiten om systeemdoelen exact en
kwantificeerbaar te definiëren, processen te beschrijven en voor te schrijven hoe deze doelen
bereikt kunnen worden. Proceduralisatie is zeer succesvol geïmplementeerd in vele organisaties
om kennis en ervaring vast te leggen, maar is beperkt toepasbaar indien zich onverwachte en
onvoorziene situaties voordoen. Wij stelden voor om proceduralisatie binnen complexe socio-
technische systemen aan te vullen met teamreflectie die gebruikt maakt van de Resiliencer. In
dit onderzoek reflecteerden treindienstleiders met behulp van de Resiliencer, waarmee ze
diepgaande inspecties konden uitvoeren over treinbewegingen na hun dienst. Een bijna-ongeval,
dat plaatsvond tijdens onze observaties, werd beschreven en gebruikt om verder onderzoek mee
te doen met een tweedelig doel. Ten eerste diende dit bijna-ongeval als real-time voorbeeld van
het gebruik van de Resiliencer. Ten tweede was het een referentiecasus over thema’s die een rol
spelen in de ontwikkeling van een ongeval. De analyse van het ongeval leverde thema-
categorieën op die overeenkwamen met de categorieën die besproken zijn tijdens de
teamreflecties. Dit betekent dat relevante thema’s beschikbaar komen als de dienst goed
verloopt, om ervan te leren en om op te anticiperen. Daarnaast werd tijdens de observatieperiode
vastgesteld dat de treindienstleiders hun werk in toenemende mate gingen analyseren en
beredeneren. Dit proces was kennis verrijkend en kwam bovenop procedures, waardoor het
vermogen om met onvoorziene en onverwachte omstandigheden om te gaan naar verwachting
156 A. W. Siegel
zal verbeteren, maar dit is nog niet aangetoond. Uiteindelijk zagen de treindienstleiders de
toegevoegde waarde van teamreflectie met de Resiliencer in en initiële scepsis maakte plaats
voor gematigd enthousiasme.
In Figuur 1 wordt het door ons ontwikkelde concept grafisch weergegeven. De linker bol geeft
het verloop van de dienst met gebeurtenissen weer en levert input aan de centrale bol. Het
dienstverloop wordt weergegeven op de Resiliencer, die zwakke signalen van veerkracht met
analysefuncties aanbiedt. Het team reflecteert daarop, hetgeen resulteert in expliciet gemaakte
kennis bovenop procedures, zichtbaar in de rechter bol. De kern van de zaak, in de centrale bol,
is de combinatie van theorie en algoritmes van de Resiliencer met de teamreflectie-
methodologie, die in het onderzoek geobserveerd en geverifieerd zijn tijdens operationeel
gebruik.
FIGUUR 1 Samenvatting van het totale concept (grafische abstract, (Siegel & Schraagen, 2017)
Het onderzoek heeft bijgedragen aan drie theorie- en praktijkgebieden, die relateren aan
antwoorden op de drie overkoepelende onderzoeksvragen: veerkracht-kwantificatie, safety
management en de evolutie van de functie van treindienstleider. Het gebied van resilience
engineering heeft zich snel ontwikkeld met vele nieuwe concepten om ondersteuning te bieden
aan de veerkrachtverhoging van socio-technische systemen, het managen van vermogens voor
de omgang met onverwachte situaties. Het onderzoeksveld miste tot op heden echter methoden
om veerkracht real-time te meten. Door de introductie van zwakke signalen van veerkracht is het
mogelijk die signalen, gerelateerd aan de veerkrachtstatus, te kwantificeren en te gebruiken voor
veerkracht-gerelateerde kennis. Deze ontwikkeling is een nieuwe stap op weg naar het
kwantificeren van veerkracht, een indicatie te meten tijdens real-time operatie. Het Safety-II
begrip is een paradigmaverschuiving ten opzichte van de traditionele Safety-I management
gedachte. Safety-I focust op eliminatie van veiligheidsrisico’s, terwijl Safety-II gericht is op wat
goed verloopt in plaats van wat er fout gaat. Teamreflectie op zwakke signalen van veerkracht,
na afloop van iedere dienst, is feitelijk leren van wat er goed verliep tijdens hun recente
A. W. Siegel 157
operationele ervaring en wij verstaan dit onder het begrip “operationaliseren van het Safety-II
concept”. Dit proces beïnvloedt ook de evolutie van de functie van treindienstleider, het derde
domein besproken in dit proefschrift. De nieuwe generatie treindienstleiders zal steeds meer
behoefte krijgen aan handvatten om analyses uit te voeren over hetgeen in zijn dienst heeft
plaats gevonden als gevolg van voortschrijdende automatisering teneinde de veerkracht te
behouden of te verhogen. Treindienstleiders behoeven reflectie op het eigen denken en acteren.
Wij geloven dat zwakke signalen van veerkracht hierbij behulpzaam zijn. Zij zullen signalen
kunnen analyseren op zowel systeem- als detailniveau, gedachten en begrippen communiceren,
in nauw contact met teamgenoten die hetzelfde doen. Kortom, het belang van niet-technische
vaardigheden zal in verhouding tot technische vaardigheden steeds meer toenemen.
Het voorgestelde ontwerp van teamreflectie op zwakke veerkracht signalen introduceert een
aantal vernieuwingen vergeleken met de huidige praktijk. Ten eerste, de voorgestelde methode
stimuleert reflectie op de drie systeemdomeinen – prestatie, werklast en veiligheid – terwijl de
meeste analysemethodieken slechts op één domein zijn gefocust. Ten tweede, de nadruk van het
nieuwe ontwerp ligt op het “hier-en-nu”. Direct ná de dienst evalueren met diegenen die de
dienst hebben uitgevoerd op basis van actuele gegevens is uniek voor de spoorwereld. Ten
derde, zwakke signalen blijven niet langer onder de radar en krijgen op reguliere basis een veel
prominentere rol toebedeeld. Ten slotte, het reflectie-instrument ondersteunt de vertaalslag van
systeemniveau naar identificeerbare details. Deze vertaalslag is tot op heden niet mogelijk
gebleken met systeem KPI’s (key performance indicators) die zichtbaar zijn op de controlepost.
Ons onderzoek heeft de potentie van de Resiliencer op dit gebied aangetoond.
158 A. W. Siegel
A. W. Siegel 159
DANKWOORD
Het was voor mij een grote stap om 23 jaar, na het behalen van mijn master Aerospace
Engineering in Israël, weer de academische wereld te betreden voor een promotietraject in
Nederland bij de afdeling Cognitieve Psychologie en Ergonomie (CPE). Ik heb goede
begeleiding nodig gehad, die mij stapje voor stapje weer inwijdde in de academische wereld en
mij deelgenoot maakte van een nieuwe discipline die ik eerder niet kende. Ik moest
ontwikkelingen doorlopen zoals het oppakken van het academisch denken, de state of de art
eigen maken via wetenschappelijke literatuur, het leren schrijven van artikelen en bovenal
academisch onderzoek doen om de wetenschap een stap verder te brengen.
Jan Maarten, jij hebt mij met alle geduld en respect in dit proces begeleid en met nog veel meer.
Je hebt op subtiele wijze de richting aangegeven, kennis gedeeld en mij geïntroduceerd bij
vooruitstrevende collega-hoogleraren. Via het vele reviewen van onze artikelen stimuleerde je
me door uitdagende vragen te stellen en je hebt voorstellen gedaan en ideeën gegeven die me
weer verder hielpen. Maar bovenal hebben we goede gesprekken gevoerd in onze
tweewekelijkse ontmoetingen waarin je waardevolle tips gaf. We maakten medio 2011 kennis
met elkaar door een introductie via prof. Hansen. Daarna heb ik jouw colleges bezocht en
hebben we in 2012 een onderzoeksvoorstel ingediend bij NWO en ProRail. De academische
onderbouwing heb jij geschreven en ik leerde zo over de academische fundamenten, die mijn
uitgangspunt waren voor mijn onderzoek. Wat bofte ik dat jij in 2013, het eerste jaar van mijn
onderzoek, het Resilience Engineering Association (REA) symposium in Nederland mede
organiseerde. Je hebt me toen gestimuleerd mijn eerste abstract te schrijven en me aan te melden
voor de Young-talent masterclass met alle leidende professoren in het domein. Je deelde met mij
jouw top-netwerk dat me inspireert tot de dag van vandaag. Je hebt goed voor me gezorgd door
een plek voor me te regelen in jouw kamer bij TNO, mij ruimte te bieden om colleges te geven
en in contact te komen met talentvolle master-studenten. Jouw inzet heeft veel bijgedragen aan
het tot stand komen van dit proefschrift. Ik ben je zeer dankbaar voor de begeleiding in de
afgelopen jaren en nu als mijn promotor.
I am grateful to my graduation committee, reviewing this dissertation and deciding that it

demonstrates my ability to conduct independent scientific research. In particular I feel very
honoured that prof. Woods and prof. Stanton have agreed to take place in the committee. Before
starting my Ph.D. study I have read articles from both of you, which have encouraged me to start
this trajectory. Prof. Stanton, thank you for enriching me through you thoroughly response. I
had the honour to participate in the young-talent program of the REA conference in 2013, where
prof. Woods was one of the professors leading the master class. Your lecture during that class
and the many articles of yours I read, have influenced my thoughts substantially. This can be
seen through the amount of your articles appearing in the reference. On top of that, you have
responded on your reading of the dissertation with your latest articles, making a new
160 A. W. Siegel
breakthrough in the way we think about the mystery of sustained adaptability. I feel privileged
to have been exposed to your novel paradigms in the area of Resilience engineering.
Een belangrijk fundament van mijn onderzoek waren de IT-middelen die ik ontworpen heb. In
mijn eerste onderzoeksjaar heb ik samen met Jaldert van der Werf een tool ontwikkeld waarmee
subjectieve werklast in een operationele omgeving gemeten kon worden (de IWS-tool). Jaldert
is met dit product afgestudeerd aan de afdeling Human Media Interaction. We hebben samen
een heel mooi proces doorlopen op de post Zwolle, waar treindienstleiders voor het eerst hun
werklast konden aangeven. De IWS-tool heb ik in alle vervolgonderzoeken gebruikt en dat heeft
me veel geholpen.
De grootste IT-ontwikkeling was het bouwen van de Resiliencer. De belangrijkste persoon
hierbij was Bert Bierman, die in een jaar tijd verschillende versies van de Resiliencer
geprogrammeerd heeft. Het was een pittige uitdaging voor Bert om zich de informatie binnen de
wereld van de spoorwegen eigen te maken en toe te passen. Bert heeft heel veel geduld gehad
met al mijn open aanmerkingen over het systeem. Bert, het was een verrijking om met je te
werken. Dankzij jou hebben we in moeilijke omstandigheden het geheel zodanig werkend
gekregen dat treindienstleiders er vertrouwen in hadden en erop konden reflecteren.
Bert heeft ondersteuning gehad van Pim Sierhuis, die geholpen heeft met de connectie naar de
operationele ProRail-systemen en die ons wegwijs heeft gemaakt in de complexe datastructuren.
Pim, we hebben zo een vervolg weten te geven aan onze jarenlange samenwerking met het
creëren van innovatieve systemen. Dankjewel voor jouw support in deze ontwikkeling.
De mooie vormgeving van de Resiliencer is verzorgd door mijn vriend Victor Kramnik, die niet
alleen mijn tekeningen grafisch heeft vertolkt, maar ook de lay-out van dit proefschrift heeft
verzorgd. Het is altijd een bijzonder proces om met jou samen te werken en tot mooie en
functionele ontwerpen te komen.
Als laatste component was het nodig om de Resiliencer aan te sluiten op het essentiële
telefoniesysteem van ProRail. Dat bleek een zeer moeilijke klus. Dit onderdeel heeft Ben
Beemster op zich genomen en met veel kennis van zaken tot een werkend geheel gebracht.
Een aantal studenten hebben hun masteronderzoek uitgevoerd onder mijn begeleiding en zo
bijgedragen aan dit promotieonderzoek. Jaldert was mijn eerste student en vervolgens heb ik
mooi onderzoek verricht samen met Anouk de Regt en in de eindfase met Rob van Broekhoven.
Met beiden heb ik een artikel geschreven: hoofdstuk 3 bevat het artikel met Anouk en het
tweede gedeelte van hoofdstuk 2 bevat het artikel met Rob. Het waren intensieve onderzoeken
met lange werkweken op de posten Zwolle en Alkmaar met het volgen van twee aansluitende
diensten (van 6 AM tot 11 PM) en overnachtingen. Met Anouk heb ik de eerste keer ook naar
nachtdiensten gekeken. Het was een intensieve periode. Jullie hebben me ook veel geholpen om
mijn gedachten op een rij te zetten. Feitelijk waren jullie, naast Jan Maarten, de enigen waarmee
ik alle details kon bespreken. Die inzichten waren zeer waardevol.
Tijdens mijn onderzoek heb ik op meerdere locaties gewerkt, waar ik gebruik kon maken van
alle faciliteiten. Als eerste bij de vakgroep CPE van Universiteit Twente (UT) als academische
A. W. Siegel 161
thuisbasis. In de eerste jaren ben ik wekelijks (6 uur met de trein) naar de UT gereisd en heb
daar een kamer gedeeld met Jonathan en Florence, die goed gezelschap waren. Ik heb ook goede
gesprekken gevoerd met de andere stafleden, zoals Matthijs, Martin, Willem, Jasmine, Jeanine
en Suzanne. Matthijs, het was fijn om samen de European HFES-chapters te bezoeken waar we
onze ideeën hebben uitgewisseld. In het eerste jaar ben ik ook wekelijks aanwezig geweest bij
de TU-Delft, afdeling Interactive Intelligent group, waar ik een aantal cursussen heb gevolgd,
waaronder een cursus van prof. Mark Neerincx en samen kon werken met Maaike, Giel en later
Ursula.
Vanaf het tweede jaar heb ik meerdere proeven uitgevoerd op de posten Zwolle en Alkmaar, die
me alle ruimte hebben geboden. De grootste support op de posten heb ik ontvangen van Dion
Boertien, die in Zwolle managementtrainee was en vervolgens Postmanager in Alkmaar is
geworden. Dion heeft de ontwikkeling van het onderzoek meegemaakt en heeft het beste inzicht
in de resultaten verkregen die hij kan vertalen naar de behoeften van ProRail. Het was heel fijn
om met Dion samen te mogen werken.
Alfons Schaafsma was de belangrijkste persoon bij ProRail voor mijn onderzoek. Alfons, wij
hebben al contact gehad in 2011 toen ik me aan het oriënteren was over de mogelijkheden van
een promotieonderzoek. Daarna was jij mijn researchcoach vanuit het Explorail programma en
heb je me bijgestaan tijdens alle fases van het traject tot en met dit proefschrift. Jij begreep tot in
detail waar het over ging, kon commentaar leveren op de inhoud, inclusief de papers. Verder
kon je als geen ander de relatie leggen met de ontwikkelingen binnen ProRail. Je hebt me veel
steun geboden tijdens het onderzoek, dank daarvoor.
Tijdens mijn hele onderzoeksperiode heb ik wekelijks gewerkt bij TNO-Soesterberg op de
afdeling HOI (Human Behaviour & Organisational Innovation) en specifiek bij het werkveld
Control Operations. Ik ben daar altijd hartelijk ontvangen en ik voelde me als collega binnen het
team en werd als zodanig overal uitgenodigd, terwijl ik feitelijk te gast was. Dank jullie wel
voor de gastvrijheid.
Ieder promotieproces kent zijn ups en downs. In geval van het laatste is de mentale support van
je omgeving cruciaal. Vele vrienden hebben mij in verschillende fasen en vormen ondersteund,
het nodige vertrouwen gegeven en zo bijgedragen aan mijn proces. Dankjewel Rob, Hans,
Marion, Joanne, Betty, Peter, ‫אריאל‬, Erik, Anton, Dick, Hugo en nog vele anderen waar ik
zinvolle gesprekken mee heb gehad. Twee daarvan zijn mijn boezemvrienden vanaf de
kleuterschool, mijn paranimfen. Ruben en Dave, onze vriendschap is van onschatbare waarde en
ik hoop nog vele jaren daarvan te genieten en veel moois samen te beleven.
Als laatst, maar wel het meest waardevol, is mijn dichtstbijzijnde familie. Mama, Dina, Yaniv,
Eyal, Itai en ook wel mijn trouwe viervoeter Diva, jullie zijn er altijd. Tijdens het
promotietraject was mijn aandacht vooral gericht op mijn onderzoek. Uiteindelijk zijn juist jullie
het belangrijkst en dat besef ik me des te meer nu het traject afgerond is. Fijn dat we elkaar
hebben en dat overstijgt de waarde van de wetenschap.
162 A. W. Siegel
CURRICULUM VITAE
Aron Wolf (Willy) Siegel was born on the 25 th of November 1956 in Amsterdam and emigrated
in 1969 to Israel. He started his bachelor’s degree in aeronautical engineering in 1976 at the
Technion in Haifa. He graduated cum laude in 1979, after which he joined the Israeli Airforce,
as aeronautical software engineer and thereafter as department head, developing and
implementing F-16 Avionics algorithms and human machine interfaces (HMI). In 1986 he
started his master’s degree in aerospace engineering at the Technion, specializing in Optimal
control, Differential games, and Artificial intelligence. During his master’s study, he was
assistant in Numerical analysis. He defended his master’s thesis in 1989. After returning to the
Netherlands he worked at different companies, among which Fokker Space & Systems, IBM
consulting group, Movares/InTraffic and ProRail, as a lead engineer, architect, consultant and
project manager, innovating and developing IT-systems in the area of aerospace, robotics,
simulators, transportation and rail. Before starting his Ph.D. program in 2013, at the Department
of Cognitive Psychology and Ergonomics at the University of Twente, he worked for more than
10 years in the rail industry, developing and realizing innovative real-time systems with HMI
concepts.
A. W. Siegel 163
164 A. W. Siegel
A. W. Siegel 165

Siegel (2017 Dissertation) Team Reflection On Weak Resilience Signals

Uploaded by

Copyright:

Available Formats

You might also like

Siegel (2017 Dissertation) Team Reflection On Weak Resilience Signals

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Siegel (2017 Dissertation) Team Reflection On Weak Resilience Signals

Uploaded by

Copyright:

Available Formats

i

Aron Wolf (Willy) Siegel

Chairman/secretary: Prof. dr. Th. A. J. Toonen

Members: Prof. dr. D. K. J. Heylen

ter verkrijging van

Aron Wolf (Willy) Siegel

CTIT Ph.D. Thesis Series No. 16-416

Cover: Victor Kramnik

© 2017, Aron Wolf Siegel. All rights reserved.

CHAPTER 2: WEAK RESILIENCE SIGNAL - WORKLOAD.......................................................................

CHAPTER 3: PERFORMANCE WEAK RESILIENCE SIGNAL...................................................................

CHAPTER 4: TEAM REFLECTION IN RAIL OPERATIONS.......................................................................

CHAPTER 5: DESIGN & PROTOTYPE IN RAIL OPERATIONS................................................................

CHAPTER 6: SUMMARY AND CONCLUSIONS...........................................................................................

Operators within a sociotechnical system perform more supervisory functions compared to 50

3.1. Resilience engineering

3.2. Safety management - from Safety-I to Safety-II

3.3. The rail signaller

FIGURE 2 Function change of the rail signaller (downloaded from

4. Research approach and overview of the thesis

FIGURE 3 Research structure of the thesis with chapter numbers

WEAK RESILIENCE SIGNAL -

We introduce the concept of weak-resilience-signal (WRS), which we will use to quantify

2. A framework for rail weak resilience signal modelling

2.1. Theoretical resilience state model for a railway system

FIGURE 1 Resilience state model for a railway system

2.2. Generic quantifiable weak resilience signal model

Or, as the change of two moments in time, t1 and t2:

3. A method to measure workload WRS

FIGURE 2 Task flow of a rail signaller at his workstation

number of activations∈5 min base ­slot

Objective Stretch Subjective Stretch

4. Observational study during rail operations

FIGURE 5 Rail control-post setting with observer O

FIGURE 6 Integration of the IWS tool within operations

Before Target 35 5.30 2.61

II. Comparison of real-time relative workload

2.2.1. Integrated Workload Scale (IWS)

FIGURE 1 Integrated Workload Scale tool (Siegel & Schraagen, 2014d),

2.2.2. External Cognitive Task Load

K mon = 0.4, K plan =0.9 , K man= 1.2 and K com =1.5 .

2.2.3. Behavioural cognitive Task Load

TABLE 1 Overview categorized behaviours BTL

Monitor Planning Manual Communication

2.2.4. Electrodermal activity

3.2. Short description of case 1 and 2

3.3. IWS results

FIGURE 3 IWS scale 1 to 9 for case one and two

3.4. EDA results

3.5. BTL results

3.6. XTL results

PERFORMANCE WEAK RESILIENCE

Siegel and Schraagen (2014d) contributed to diminishing the research-practice gap by

The overall aim of this research is to enhance tangibility of resilience by facilitating

2.1. Research Setting

The population consisted of a convenience sample of 25 rail dispatchers (5 female, 20 male)

2.2. Resilience observations

1. Level 2. Generalizability 3. Quantity 4. Example

number of activations∈5 min base slot