Available online at www.sciencedirect.

com
Available online at www.sciencedirect.com
ScienceDirect
Available ScienceDirect
online
Procedia
Available atonline
CIRPwww.sciencedirect.com
00 (2019) 000–000
at www.sciencedirect.com
www.elsevier.com/locate/procedia
ScienceDirect
Procedia CIRP 00 (2019) 000–000
ScienceDirect www.elsevier.com/locate/procedia

ProcediaProcedia
CIRP 00CIRP
(2017)
81000–000
(2019) 1052–1058
www.elsevier.com/locate/procedia
52nd CIRP Conference on Manufacturing Systems
52nd CIRP Conference on Manufacturing Systems
Active Learning for Accurate Settlement Prediction Using Numerical
Active Learning for
28th Accurate
Simulations
CIRP inSettlement
Mechanized
Design Conference, Prediction
May 2018,Tunneling Using Numerical
Nantes, France
Simulations in Mechanized Tunneling
AAmal
new methodology
Saadallah a,*
, Alexey to analyze
Egorov a the functional
, Ba-Trung Caob, Steffen and physical
Freitag b architecture
, Katharina of
Morika, Günther
b
Meschke
existing
Amal products
Saadallah a,*
forEgorov
, Alexey an assembly
a
, Ba-Trungoriented product
Caob, Steffen Freitagbfamily identification
, Katharina Morika, Günther
b
Meschke
Chair of Artificial Intelligence, Department of Computer Science, TU Dortmund University, Otto-Hahn-Straße 12, 4427 Dortmund, Germany
a

InstituePaul
b
Stief
for Structural *, Jean-Yves
Mechanics, Dantan,
Ruhr University Bochum,Alain Etienne,
Universitätsstraße 150, Ali
44801Siadat
Bochum, Germany
Chair of Artificial Intelligence,
a
Department of Computer Science, TU Dortmund University, Otto-Hahn-Straße 12, 4427 Dortmund, Germany
* Corresponding author. Tel.:
b +49-231-755-6490;
Institue fax: +49-231-755-5105.
for Structural Mechanics, E-mail
Ruhr University address:
Bochum, amal.saadallah@cs.tu-dortmund.de
Universitätsstraße 150, 44801 Bochum, Germany
École Nationale Supérieure d’Arts et Métiers, Arts et Métiers ParisTech, LCFC EA 4495, 4 Rue Augustin Fresnel, Metz 57078, France
* Corresponding author. Tel.: +49-231-755-6490; fax: +49-231-755-5105. E-mail address: amal.saadallah@cs.tu-dortmund.de
* Corresponding author. Tel.: +33 3 87 37 54 30; E-mail address: paul.stief@ensam.eu
Abstract
Abstract
Finite Element simulation is a possible tool to investigate interactions between the Tunnel Boring Machine and the surrounding soil. Surface
Abstract
settlements can be predicted in real-time based on simulation results by machine learning surrogate models. However, to train such models, large
Finite Element simulation is a possible tool to investigate interactions between the Tunnel Boring Machine and the surrounding soil. Surface
amounts of computationally intensive simulations are required. To accomplish this step with minimal costs, we propose a hybrid active learning
settlements can be predicted in real-time based on simulation results by machine learning surrogate models. However, to train such models, large
Inapproach
today’s business
to select environment, the trendoftowards
the minimal amount more necessary
simulations product variety andan
to build customization
accurate model.is unbroken.
During theDuetunnel
to thisconstruction,
development,the
thereal-time
need of
amounts of computationally intensive simulations are required. To accomplish this step with minimal costs, we propose a hybrid active learning
settlements
agile prediction model
and reconfigurable will besystems
production used to analyze
emergedassociated risksvarious
to cope with to ensure safe and
products andsustainable constructions
product families. in urban
To design andareas.
optimize production
approach to select the minimal amount of simulations necessary to build an accurate model. During the tunnel construction, the real-time
systems as well as to choose the optimal product matches, product analysis methods are needed. Indeed, most of the known methods aim to
settlements prediction model will be used to analyze associated risks to ensure safe and sustainable constructions in urban areas.
analyze
© 2019aTheproduct or one
Authors. product family
Published on the
by Elsevier physical
Ltd. This islevel. Different
an open accessproduct families,
article under the however, may differ
CC BY-NC-ND largely in terms of the number and
license
© 2019
nature ofThe Authors. Published
components. This fact by Elsevier
impedes anLtd.
(http://creativecommons.org/licenses/by-nc-nd/3.0/) efficient comparison and choice of appropriate product family combinations for the production
© 2019
This is AThe
an Authors.
open access Published by Elsevier
articleisunder Ltd. This is license
the scientific
CC BY-NC-ND an open(http://creativecommons.org/licenses/by-nc-nd/3.0/)
access article under the CC BY-NC-ND license
system.
Peer-review new methodology
under responsibility proposed
of the to analyze existing products
committee of the 52ndin view
CIRPofConference
their functional and physical Systems.
on Manufacturing architecture. The aim is to cluster
(http://creativecommons.org/licenses/by-nc-nd/3.0/)
Peer-review under responsibility of the scientific committee of the 52nd CIRPofConference on Manufacturing Systems.
these products in new assembly oriented product families for the optimization existing assembly lines and the creation of future reconfigurable
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference on Manufacturing Systems.
assembly
Keywords: systems. Based on
Active learning; Datummodel;
surrogate Flow FE-simulations;
Chain, the physical structure
mechanised of the risk
tunnelling; products is analyzed. Functional subassemblies are identified, and
analysis.
a functional analysis is performed. Moreover, a hybrid functional and physical architecture graph (HyFPAG) is the output which depicts the
Keywords: Active learning; surrogate model; FE-simulations; mechanised tunnelling; risk analysis.
similarity between product families by providing design support to both, production system planners and product designers. An illustrative
example of a nail-clipper is used to explain the proposed methodology. An industrial case study on two product families of steering columns of
1. Introduction
thyssenkrupp Presta France is then carried out to give a first industrial evaluation tunnelingof theprocesses
proposed are most often used as tools to investigate
approach.
©1.2017 The Authors. Published by Elsevier B.V. the interactions between the TBM, the surrounding soil and
Introduction tunneling processes are most often used as tools to investigate
Over the
Peer-review lastresponsibility
under decades, theof world-wide trend of increased
the scientific committee of the 28th CIRP provide reliable pre-calculated estimates of the expected
theDesign Conference
interactions 2018. the TBM,
between the surrounding soil and
urbanization creates problems for expanding and newly- surface settlements and associated risks of damage for existing
Over the last decades, the world-wide trend of increased provide reliable pre-calculated estimates of the expected
developing
Keywords: cities, Design
Assembly; whichmethod;
increases theidentification
Family demand for reliable and structures [5,6]. However, these simulations need to take into
urbanization creates problems for expanding and newly- surface settlements and associated risks of damage for existing
sustainable infrastructures. The use of underground consideration the complexity of the tunneling process which
developing cities, which increases the demand for reliable and structures [5,6]. However, these simulations need to take into
infrastructures such as tunnels can help cities to meet these includes the advance of the TBM, the application of the face
sustainable infrastructures. The use of underground consideration the complexity of the tunneling process which
increased demands [1,2]. However, in addition to their pressure, the soil excavation, etc. A realistic consideration of all
infrastructures
1.advantages,
Introduction such as tunnels can help cities to meet these includes the advance of the TBM, the application of the face
tunnels may have undesirable effects such as of the components
these product
pressure, the
range
soil
andandthe characteristics
excavation,
interaction between
etc. A realistic
manufactured
them in the
consideration
and/or
of
3D
all
increased demands [1,2]. However, in addition to their assembled
ground surface deformations, especially in soft grounds or calculationinconstitutes
this system.a In bigthis context, the
challenge, main challenge
because of the largein
advantages,
Due to tunnels
the fastmay have
developmentundesirable
in theeffects
domain such as
of these
modellingcomponents
and and
analysis the
is interaction
now not between
only to them
cope in
with the 3D
single
densely constructed areas, where damages to surface structures effort for numerical modelling and the large uncertainties
ground surface and deformations, especially calculation constitutes a big challenge, because of the large
communication
and utilities should beanprevented
ongoing trend
[3]. ofin digitization
Mechanized
soft groundsand or
tunneling is products,
concerning a limited productbetween
the interaction range ortheseexisting product [5-7].
components families,
As
densely constructed areas, where damages to surface structures effort for numerical modelling and the large uncertainties
digitalization, manufacturing
an efficient technology for theenterprises
construction areoffacing
tunnelsimportant
in a wide but also tothe
a result, be use
ableofto 3D
analyze and to simulations
numerical compare products to define
for studying or
and utilities should be prevented [3]. Mechanized tunneling is concerning the interaction between these components [5-7]. As
challenges
range of in today’s market
geological conditions environments:
without inducinga continuing large new product the
monitoring families.
tunnelItprocess
can be observed
design isthat veryclassical existing
demanding. In
an efficient technology for the construction of tunnels in a wide a result, the use of 3D numerical simulations for studying or
tendency
disturbance towards reduction
to the of product
surrounding developmentDuring
environment. times and addition,families
the product due to their
are computational
regrouped in time, of
function these simulations
clients or are
features.
range of geological conditions without inducing large monitoring the tunnel process design is very demanding. In
shortened product lifecycles. In addition,
machinesthere is an the
increasing also inappropriate
surface However, assembly for real-time use and the online prediction of
advancement of tunnel boring
disturbance to the surrounding environment. During the
(TBM), addition, due to theiroriented product
computational families
time, thesearesimulations
hardly to find.
are
settlements
demand of can be controlled
customization, being at by
the adjusting
same time the
in a process
global theOn tunnel-soil
the interaction.
product family level, products differ mainly in two
advancement of tunnel boring machines (TBM), the surface also inappropriate for real-time use and the online prediction of
parameters such
competition as the face support pressure and the
Thistail void main Incharacteristics:
contrast, models (i) thebased on ofmachine learning (ML)
settlements with can competitors
be controlled all over the world.
by adjusting the trend,
process the tunnel-soil
approaches
interaction.
can be
number
evaluated in
components
real-time and
and (ii)
offer
the
the
grouting
which is pressure
inducing [4].
the 3D Finite-Element
development from (FE) simulations
macro to micro of typeInof contrast,
components (e.g. mechanical, electrical,
models based on machine learning (ML) electronical).
parameters such as the face support pressure and the tail void
markets,
grouting results
pressurein[4].
diminished lot sizes due
3D Finite-Element (FE)tosimulations
augmenting of Classical methodologies
approaches can be evaluated considering mainly single
in real-time products
and offer the
product varieties (high-volume to low-volume production) [1]. or solitary, already existing product families analyze the
2212-8271 © 2019 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
To cope with this augmenting variety as well as to be able to
(http://creativecommons.org/licenses/by-nc-nd/3.0/) product structure on a physical level (components level) which
identify
2212-8271 possible
© 2019 The optimization
Authors. Publishedpotentials
by Elsevier in
Ltd. the
This is existing
an open access
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference causes on difficulties
article under regarding
the CC BY-NC-ND
Manufacturing Systems.license an efficient definition and
(http://creativecommons.org/licenses/by-nc-nd/3.0/)
production system, it is important to have a precise knowledge comparison of different product families. Addressing this
Peer-review under responsibility of the scientific committee of the 52nd CIRP Conference on Manufacturing Systems.
2212-8271 © 2019 The Authors. Published by Elsevier Ltd.
This is an©open
2212-8271 2017access article Published
The Authors. under theby CC BY-NC-ND
Elsevier B.V. license (http://creativecommons.org/licenses/by-nc-nd/3.0/)
Peer-review
Peer-review under
under responsibility
responsibility of scientific
of the the scientific committee
committee of the of theCIRP
28th 52ndDesign
CIRPConference
Conference2018.
on Manufacturing Systems.
10.1016/j.procir.2019.03.250
 Amal Saadallah et al. / Procedia CIRP 81 (2019) 1052–1058
2 Amal Saadallah et al. / Procedia CIRP 00 (2019) 000–000
possibility to predict upcoming unknown events with a
reasonable accuracy based on the analysis of past observations
of a set of explanatory variables. Therefore, new trends in
applied machine learning aim to replace simulations with
machine learning surrogate models that are more appropriate
for real-time application [8-10]. However, to train such models
a high amount of data is required, whose acquisition through
3D numerical simulations is a highly resource-consuming task.
Hence, it is necessary to reduce the amount of data required to
train an accurate ML model. The same idea was broached in the
literature with Active Learning (AL). Starting from a small and
non-optimal training set, AL procedures aim to select
iteratively unlabeled data points whose inclusion in the training
set improves the performance of the ML model [11,12].
In this paper, we consider one simulation scenario (i.e. a
given combination of simulation input parameters) as a data
point characterized with a set of features, which are the
corresponding input parameters. If we run the simulation for a
given scenario, then the scenario is treated as “labelled” data
point (i.e. If we run one simulation scenario, we have the output
which is the surface point movements in time. So, we get the
label for the learning task which is the time series of the
settlement). In the opposite case, it is considered as “unlabeled”
data point. Our main goal is to reduce the cost of running the
3D FE-simulation for “labelled” training data points generation
using an AL method. Hence, we propose a hybrid active
learning approach which combines both error-based with
distance-based AL procedures, to select the minimal number of
simulation scenarios necessary to build an accurate ML model
for surface settlement prediction. Our procedure starts first with
a small initial number of randomly selected “labelled” scenarios
and iteratively selects the “unlabeled” scenarios to run next and
add to the training data set. The error-based procedures aim to
query data points features space in the regions where our ML
model is less accurate. The distance-based approach aims to
query unseen regions while avoiding outliers. The main
contributions of this paper are summarized as follows:
 We replace the computationally intensive 3D FE-simulation
with a ML surrogate model, namely the Vector
Autoregressive with Exogenous variables (VARX).
 We combine two AL procedures to iteratively select
simulation scenarios whose inclusion in the training set
improves the accuracy of the ML model.
 We present an experimental study using a 3D FE-simulation
designed specifically for process-oriented computational
simulations of shield tunnelling processes [8]. Our study
shows the ability of our framework in reducing the number
of training scenarios required for training up to 60% while
improving the prediction accuracy up to 20%.
The remainder of this paper is structured as follows. Section
2 revises related works in surface settlement predictions in
tunnelling and active learning methods while Section 3
formalizes the proposed methodology. Section 4 describes the
3D simulation model and the data acquisition process. Section
5 shows the experimental set-up, the evaluation metric and the
discussion of the results. Finally, concluding remarks are drawn
in the final Section 6.
1053
2. Literature Review
The control of surface settlement during the construction of
urban tunnels is essential for the success of tunnel projects [8],
[13]. The FE method is gradually used to predict the expected
tunnel induced settlements. A number of 2D and 3D models
have been proposed using different simplifications of the
components involved in mechanized tunneling [6,15,16].
Ercelebi et al. [13] performed simulation examples
for settlement predictions using data from Istanbul Metro
project with Plaxis FE commercial program. Components of
the shield tunneling process such as soil excavation, ground
support and face support are simulated in the numerical model
following the real construction steps. More recently, a process-
oriented FE model has been developed, which includes all
components required for the interaction mechanism in realistic
mechanized tunneling [17].
As an alternative approach, ML methods have been
employed to predict surface movements based on the analysis
of sensor data. Kim et al. [18] have used an artificial neural
network for predicting ground surface settlement based on
collected field data. They performed a feature selection for
optimizing the performance of their model in order to monitor
ground surface profiles for a total of 113 monitoring lines.
Dindarloo et al. [19] proposed a shallow tunnel classification
system, based on decision tree method to predict maximum
settlement class for 14 tunnelling projects.
Surface settlements records are not always available or easy
to collect with high-frequency, especially when the
measurements are performed manually. Consequently, other
research works took advantage of the modelling power and
accuracy of FEM based simulations and applied ML models to
represent the capabilities of these expensive complex
simulations for real-time usage [8,9]. In this context, ML
methods are used as tools to learn simulation models and
accomplish the step from the offline computationally intensive
analysis to real-time predictions. In [8], the authors have used
a recurrent neural network (RNN) to predict upcoming
settlements for the defined monitoring surface points using data
generated by a 3D numerical model. They have employed
additionally a Gappy proper orthogonal decomposition
(GPOD) to approximate the settlements for the remaining
points in the monitoring field. However, to train accurate ML
models, a large amount of training data is usually required [20,
21]. The acquisition of these data through the FE simulation is
expensive in terms of time and resources.
In this context, Active Learning (AL) appears to be a good
solution to approach this issue since it formulates an iterative
process to select the minimal amount of data required to
achieve the same or better accuracy obtained using more data
points [11]. The selection is given by a problem-dependent
heuristic defined by the informativeness of the candidate data
points to the learner, for example, the decrease of the prediction
error on a test set. AL approaches can be divided into two main
classes. The first class of active learning methods relies on
geometrical representation of data instances. Two main
approaches can be mentioned in this class. The first one uses
the SVM specificities, where the closest samples to the
dividing hyperplane are considered to be the most informative
as their probability of becoming support vectors is higher [22].
The second approach relies on the distance between the current
training data points and the candidate instances. The main idea
1054 Amal Saadallah et al. / Procedia CIRP 81 (2019) 1052–1058
Amal Saadallah et al. / Procedia CIRP 00 (2019) 000–000 3

behind this approach is to make the selection in favour of 3.1. Notations

samples placed in areas which are not covered by training
samples [23]. Douak et al. [24] select for each labelled instance Let 𝑆𝑆𝐿𝐿𝑖𝑖 , 𝑆𝑆𝐿𝐿 and 𝑆𝑆𝑈𝑈 be the initial set of randomly chosen
the closest unlabeled one to avoid selecting outliers. Unlabeled scenarios to run, the set of labelled scenarios at a given iteration
instances with the highest minimum distances are then selected. and the set of unlabeled scenarios, respectively. We denote
This “min-max” distance-based method allows also to avoid with 𝑁𝑁𝐿𝐿 the number of selected labelled scenarios and 𝑁𝑁𝐴𝐴𝐴𝐴 the
clustering around seen regions. number unlabeled scenarios to add at each iteration. 𝑠𝑠𝑖𝑖 is a
The second class is based on the estimation of the learning given simulation characterized by the vector of features 𝜃𝜃𝑖𝑖 (i.e.
model performance on a set of candidates. Expected model
change through expected error or variance reduction Sele
e
estimations, if one or more candidates are added to the current th
i n el ML Model
HDAL ct Q
u er
Tra od Procedure y
training set, can be used to monitor the performance of the m
learning model with each iteration [25-26]. The performance
can also be measured for one or a set of meta-models. For
Unlabelled
instance, a single meta-model can be used to predict a loss Labelled
Scenarios Scenarios
value on a set of candidates and put the selection in favour of
candidates with highest predicted losses [27]. A set of meta
for
models can also be used within the so-called Query-by- on s
Upd Finite-Element u lati ario
a Simulation sim cen
Committee (QBC) paradigm, where meta-models are trained to train te the
ing s nt
he ted s
et Ru selec
predict the label of candidate instances and the selection is put the
in favour of instances where the prediction disagreement
between them is maximal [10,28]. Fig. 1. HDAL Framework.
Most of AL approaches were applied to classification the corresponding set of the simulation input parameters).
problems and few of them were oriented to regression or time 𝐸𝐸𝐸𝐸(𝑠𝑠𝑖𝑖 , 𝑠𝑠𝑗𝑗 ) denotes the Euclidean distance between the
series prediction problems [11]. Dealing with time series data scenarios 𝑠𝑠𝑖𝑖 and 𝑠𝑠𝑗𝑗 .
is quite challenging in our case. In case we do not run the
simulation for one scenario, the historical values of the target
3.2. VARX Model
variable are not then available. This makes the use of AL
approaches from the second class not possible. For example,
The tunnel surface consists of a set monitoring point.
Saadallah et al. [10] use a ML method to learn a simulation
model for milling processes. Even though they refined their Settlement calculations over time are made for each point. As
model with AL approach based on QBC, they were not able to a result, the collected data is a multivariate time series. VAR is
save the costs of running the simulation since their committee known to be one of the most commonly applied tools to
of learners is a set of forecasting models which use the multivariate time series for forecasting purposes [29]. Even
historical data of candidate scenarios to predict upcoming though many works in literature have used neural networks
events. In case we want to focus on saving the cost of running based models for the surface settlement prediction task [8, 9],
the simulation, the light should be put on the space of input [18], VAR model has the benefit of the fast training, especially
parameters. Therefore, we consider the scenarios themselves as in our framework where model retraining is required with each
instances characterized by their input parameters as features update of the training set. A comparison of VAR with a hybrid
and we combine error-based with distance-based AL RNN-GPOD framework from [8,30] shows that VAR has a
approaches. considerably lower training time while maintaining
approximately similar accuracy. The comparison results are
3. Methodology included in Section 5.3.
VAR models a random process of order 𝑝𝑝 as follows:
Before starting with the AL procedure, we select a ML 𝑝𝑝
Ŷ𝑡𝑡+1 = Y𝑡𝑡𝑇𝑇 𝛽𝛽0 + ⋯ + Y𝑡𝑡−𝑝𝑝+1
𝑇𝑇 𝑇𝑇
𝛽𝛽𝑝𝑝−1 + 𝝐𝝐𝑡𝑡 = ∑𝑗𝑗=1 Y𝑡𝑡−𝑗𝑗+1 𝛽𝛽𝑗𝑗−1 + 𝝐𝝐𝑡𝑡
model which will be used to learn the FE-simulation model.
Building an accurate ML model based on data generated by the where 𝜷𝜷 = (𝛽𝛽0 , … , 𝛽𝛽𝑝𝑝−1 ) ∈ ℝ𝑝𝑝×𝕂𝕂 are the parameters of the
simulation is a crucial task since it will be employed to model while 𝝐𝝐𝑡𝑡 is a vector of white noise. VAR represents each
represent the predictive capabilities of the FE-simulation in value of the target variable as a linear combination of its past
real-time. In this work, we chose to use the Vector observations. Additional variables can be considered when
Autoregressive model with exogenous variables (VARX). The explanatory variables (e.g., grouting and support pressures) are
main motivation behind this choice and more details about this expected to have a place. In order to incorporate such potential
model are included in the next Subsection. First, we train knowledge in the model, the original VAR model can be
VARX on small number of randomly selected simulation extended to VARX with exogeneous time series features,
scenarios. Then, we apply our framework to iteratively select denoted as X. The resulting feature space can be defined as
the scenarios to run next by the simulation. The training set is Y ′ = Y + X. Naturally, the inclusion of additional exogenous
updated and the model is retrained until a stopping criteria is variables leads to higher dimensions and can result in sparse
met. Our framework is illustrated in Fig. 1. solution. To mitigate this problem, a L1-regularization schema
is applied while learning the VAR process [31].
 Amal Saadallah et al. / Procedia CIRP 81 (2019) 1052–1058 1055
4 Amal Saadallah et al. / Procedia CIRP 00 (2019) 000–000

3.3. Hybrid Distance-based AL Approach: HDAL

Input : 𝑆𝑆𝐿𝐿𝑖𝑖 , 𝑆𝑆𝑈𝑈 , 𝑁𝑁𝐿𝐿 , 𝑁𝑁𝐴𝐴𝐴𝐴
The main motivation behind AL is that a ML model trained on Output : 𝑆𝑆𝐿𝐿
a small training set is able to perform efficiently as a model 1 Initialization: 𝑆𝑆𝐿𝐿 = 𝑆𝑆𝐿𝐿 𝑖𝑖
trained on a larger number of examples randomly chosen, while
being computationally smaller [11]. Following this idea, AL 2 Stage 1: Train the ML model on 𝑆𝑆𝐿𝐿
exploits the user-machine interaction to increase the model
accuracy using an optimized training set. Starting from a small Stage 2: Model evaluation on the current training set
and non-optimal training set, the AL procedure aims to select 3 for 𝑠𝑠𝑙𝑙 ∈ 𝑆𝑆𝐿𝐿 do
iteratively unlabeled data points whose inclusion in the training 4 Compute the training error 𝑒𝑒𝑙𝑙
set improves the accuracy of the ML model. With HDAL, we 5 end
start by randomly selecting a small set of scenarios 𝑆𝑆𝐿𝐿𝑖𝑖 to run. Select 𝑁𝑁𝐿𝐿 scenarios from 𝑆𝑆𝐿𝐿 with the highest error
6
Then, our framework operates in three stages. First, it trains the values: 𝑆𝑆𝑁𝑁𝐿𝐿
ML model on the available labelled scenarios 𝑆𝑆𝐿𝐿 . Second, it Stage 3: Selection of the simulation scenarios to run
computes the training errors of the ML model on each next
individual scenario in 𝑆𝑆𝐿𝐿 . This step allows us to identify regions 7 for 𝑠𝑠𝑙𝑙 ∈ 𝑆𝑆𝑁𝑁𝐿𝐿 do
where the model is weak and probably needs to investigate 8 for 𝑠𝑠𝑢𝑢 ∈ 𝑆𝑆𝑈𝑈 do
more samples from these regions. We select the 𝑁𝑁𝐿𝐿 labelled 9 Compute 𝐸𝐸𝐸𝐸(𝑠𝑠𝑢𝑢 , 𝑠𝑠𝑙𝑙 )
scenarios with the highest error rates. In the third stage, we 10 end
consider as a result the closest unlabeled scenarios to these 𝑁𝑁𝐿𝐿 11 𝑠𝑠𝑢𝑢𝑙𝑙,𝑚𝑚𝑚𝑚𝑚𝑚 = 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑠𝑠𝑢𝑢∈𝑆𝑆𝑈𝑈 𝐸𝐸𝐸𝐸(𝑠𝑠𝑢𝑢 , 𝑠𝑠𝑙𝑙 )
labelled scenarios based on the assumption that two close 12 end
instances probably share similar characteristics. However, to
avoid clustering problems around these labelled scenarios (i.e. Select from (𝑠𝑠𝑢𝑢1,𝑚𝑚𝑚𝑚𝑚𝑚 , … , 𝑠𝑠𝑢𝑢𝑁𝑁 ,𝑚𝑚𝑚𝑚𝑚𝑚 ) the 𝑁𝑁𝐴𝐴𝐴𝐴 scenarios
13
seen regions), we select from these closest unlabeled scenarios 𝐿𝐿

the 𝑁𝑁𝑈𝑈 furthest ones. The entire process is iterated until a with the highest 𝐸𝐸𝐸𝐸(𝑠𝑠𝑢𝑢𝑙𝑙,𝑚𝑚𝑚𝑚𝑚𝑚 , 𝑠𝑠𝑙𝑙 ), 𝑙𝑙 ∈ (1, … , 𝑁𝑁𝐿𝐿 )
stopping criterion is met. The steps of HDAL framework are 14 Run the simulation for the 𝑁𝑁𝐴𝐴𝐴𝐴 selected scenarios: 𝑆𝑆𝑠𝑠𝑠𝑠𝑠𝑠
presented in Algorithm 1. 15 𝑆𝑆𝐿𝐿 = 𝑆𝑆𝐿𝐿 ∪ 𝑆𝑆𝑠𝑠𝑠𝑠𝑠𝑠
The stopping criteria for AL procedures is still an open research 16 Go to Stage 1 until the stopping condition is satisfied
question [11]. It can be set according to a maximum budget of
iterations or when the model accuracy improvement on an Algorithm 1. HDAL algorithm.
independent calibration/validation set over the last iterations
becomes insignificant. In our experiment, we stop the caused by the excavation process and to reduce the volume loss
procedure once the reduction of the error over the initial behind the conical shield. The tail void grout is modelled as a
recorded error is more than 20% or a user-defined maximum fully saturated two-phase material. The interaction between the
of iterations is reached. existing structures at the surface and the ground is modelled
using the mortar method [35].
4. Numerical Simulation Model The simulation results from the advanced numerical model
ekate has been compared with registered data from several
For the simulation of shield tunnelling in soft soils, a three- tunnel projects such as the Wehrhahn metro line in Düsseldorf,
Germany [36], the L9 tunnel in Barcelona [33]. In general, the
dimensional FE model ekate described in [32,33] is employed.
simulation results and measurement data show a good
The model is developed within the object-oriented FE
agreement that enables the application of the numerical FE
framework KRATOS [34] and takes into account all relevant
model to real-world tunnel problems.
components of the mechanised shield tunnelling and their
mutual interactions. After each TBM advance, by deactivation
5. Experiments
of soil elements and adjusting all boundary conditions to the
new situation, the excavation at the cutting face, the tail void
5.1. Experimental Set-up
grouting and the installation of a new lining ring are simulated.
The soil is modelled as a three or two phase material for
The simulation scenarios were conducted for 160 different
partially or fully saturated soils considering the solid, the pore
combinations of grouting and support pressures. Each scenario
water and the pore air as individual phases. The Tunnel Boring
results in a time series with 64 steps of settlements
Machine (TBM) is modelled as a deformable body connected
observations for 154 surface points. 20 scenarios were
with frictional contact to the soil along the (conical) shield skin.
initially selected for training. 130 scenarios were considered as
With this modelling approach, the volume loss considers the
unlabeled and 10 were used for testing. 𝑁𝑁𝐿𝐿 the number of
real, tapered geometry and the over-cutting of the shield
selected labelled scenarios with highest error rates is set to 10
machine. The TBM is pushed forward by the hydraulic jacks,
and 𝑁𝑁𝐴𝐴𝐴𝐴 the number of scenarios to add at each AL iteration is
which are represented by truss elements and connecting the
set to 5 . The maximum number of iterations is set to 12 .
element nodes of the tunnel lining with the pressure bulkhead
However, early stopping can happen if the error is reduced by
of the machine. At the tunnel face and the tail void gap, the
20% over the initial recorded error. We used the same data set
support pressure and the grouting pressure are applied
in [30]. For instance, for each scenario we predict the 23𝑟𝑟𝑟𝑟
respectively to ensure the tunnel face stability due to distortions
time step. For VARX setting, the input data consists of the set
 Amal Saadallah et al. / Procedia CIRP 00 (2019) 000–000 5
1056 Amal Saadallah et al. / Procedia CIRP 81 (2019) 1052–1058

Table 1. Comparison between the test error and training time of VARX and
RNN-GPOD for the 10 test scenarios.

Method Test error Training time

RNN-GPOD 5,36% ± 0,94% 1,5 ℎ𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜
VARX 5,71% ± 1.03% 8 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚
Table 2. Comparison between the test error of VARX with different training
sets for the 10 test scenarios.

Method Test error

VARX with initial set (20 scenarios) 7,83% ± 2,37%
VARX with full set (150 scenarios) 6,70% ± 3,22%
VARX with RS 7,16% ± 2,92%
VARX with DAL 7,28% ± 3,57%
VARX with HDAL 𝟔𝟔, 𝟒𝟒𝟒𝟒% ± 𝟐𝟐, 𝟖𝟖𝟖𝟖%
of time series of settlements of the surface monitoring points
(i.e. measured in millimeters), plus the time series records of
Fig. 2. Prediction error progress with each iteration for three
the grouting and the support pressures (i.e. measured in Pa) of
different sampling methods for two different scenarios (Init. Perf.
the machines which are treated as exogenous variables. A data and Full Perf. denote error values using the initial randomly
normalization process is performed to bring all the variables to selected scenarios and all the training scenarios, respectively.
the same scale. we set the order 𝑝𝑝 to 5 and we used the grouting
and the support pressures as exogenous variables using the 5.4. Discussion

The discussion of the experimental results is made in three

same order 𝑝𝑝 . The 𝐿𝐿1 -regularization parameter was tuned
using cross validation over the training set. VARX model is
retrained with each update of the training set at each AL
iteration.
5.2. Evaluation Metric
In terms of baselines methods, we compared two methods in
the state-of-the art with our HDAL framework: random
selection (RS) [11] and distance-based AL (DAL) [27] (i.e. it
uses the min-max approach and consider all the labelled
samples in the selection process). To estimate the loss of each
method at each iteration for each test scenario, a relative
percentage error was used [37]. For each scenario 𝑠𝑠𝑖𝑖 , the
prediction error was measured by:
∑𝑁𝑁
𝑠𝑠 𝑠𝑠
̂𝑡𝑡 𝑖𝑖 − 𝑦𝑦𝑡𝑡 𝑖𝑖 )2
𝑡𝑡=1 (𝑦𝑦
𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑠𝑠𝑖𝑖 = √ 𝑠𝑠
∑𝑁𝑁 𝑖𝑖 2
𝑡𝑡=1 (𝑦𝑦𝑡𝑡 )
𝑠𝑠
where 𝑦𝑦𝑡𝑡 𝑖𝑖 is the true settlement value at a given instant 𝑡𝑡 and
𝑠𝑠
𝑦𝑦̂𝑡𝑡 𝑖𝑖 is the predicted values and 𝑁𝑁 is the total number of steps.
5.3. Results
The results are presented over three distinct perspectives: 1)
Table 1 shows a comparison between the two surrogate models
VARX and RNN-GPOD [30] in terms of accuracy and training
time; 2) Table 2 encloses descriptive statistics over the ͳͲ test
scenarios of the experimental results on the prediction error per
method. Initial and final errors using the initial random set of
scenarios and the ͳͷͲ scenarios (i.e. full set) respectively for
training are also shown; 3) Fig. 2 depicts two illustrative
examples of the prediction error progress within each iteration
for the three methods.
different folds across this Section: (i) Comparison of the ML
surrogate models for settlements prediction, (ii) Generalisation
error and performance and (iii) Potential Deployment.
(i) The first conclusion drawn from Table1 is that VARX
has approximately similar performance to the RNN-GPOD in
[30], while having considerably lower training time. In
addition, the parameters tuning of the RNN in the RNN-GPOD
requires at least 10 to 15 repetitions of the experiments, which
requires at least 15 hours to get the whole framework ready for
training and test. Oppositely. VARX parameters tuning and the
L1-regularization are done automatically during the training
phase which makes it adaptive to new characteristics in time
series observations. These results highlight the robustness and
the usefulness of our surrogate ML model for these
experiments, especially that the model has to be retrained with
each update of the training set using the AL procedures.
(ii) Table 2 shows that VARX combined with HDAL
presents a lower generalization error than all other methods.
Using HDAL, we were able to outperform in average the model
trained using all the available training scenarios. The reduction
of error between the model trained used HDAL sampling and
state-of-the art including RS and DAL is on a ratio of 11%.
Our stopping criteria enabled us to achieve at least 20% of
improvement over the initial performance and also to stop early
the iterations using less than 40% of the total number of
candidate scenarios for 8 out of the 10 test scenarios. Since
each scenario requires up to 68 hours to run, we were able to
save 4800 hours (200 days) in average needed for simulation
runs.
Fig. 2 illustrates well the robustness of HDAL in sampling
“informative” scenarios for the ML model. With HDAL, we
are able to outperform the full performance in both cases and
earlier than DAL and RS in the second case. Our stopping
criteria enables us to stop the query after the Ͷ–Š and the ͺ–Š
 Amal Saadallah et al. / Procedia CIRP 81 (2019) 1052–1058
6 Amal Saadallah et al. / Procedia CIRP 00 (2019) 000–000
iterations for the two cases, respectively. In both cases, a
satisfactory performance is achieved. However, if the user has
enough budget of time and resources, more iterations can be
added to win more in terms of accuracy.
(iii) Both sampling and prediction schemas in our work are
able to produce on-demand predictions of surface settlements
for short-term horizons thanks to their self-adaptive
capabilities. It can be deployed in a standalone fashion, as a
white box decision support model where the TBM drivers have
explicit information about the future settlements values to then
decide which is the best driving strategy or the best input
parameters to use next. Alternatively, its outputs can serve as
input to any sensitivity analysis model for risk-management
systems.
6. Conclusion
3D FE-simulations for mechanized tunneling processes are one
of the most powerful tools for investigating the safety and the
sustainability of tunnels. However, they are expensive in terms
of time and resources and inadequate of real-time usage. ML
methods emerge as promising solutions to learn the capabilities
of these simulations and make them useful for real-time
applications, where critical decisions have to be made. In this
paper, we propose HDAL framework where we use AL to
select simulation scenarios for building optimized training set
for the ML model. The ML model is used later as surrogate
model to predict a field of surface settlements in real-time. In
the future, we plan to deploy this framework in the real world
to further validate it, as well as to improve its robustness.
Acknowledgements
This work is supported by the DFG within the Collaborative
Research Centers SFB876 and SFB837 and MERCUR Project.
References
[1] W. Broere, “Urban underground space: Solving the problems of today’s
cities,” Tunn. Undergr. Space Technol., vol. 55, pp. 245–248, 2016.
[2] D. Kaliampakos, “Underground development: a springboard to make city
life better in the 21st century,” Procedia Eng., vol. 165, pp. 205–213, 2016.
[3] E. Telioni, “Ground Surface Settlements due to Underground Works,” in
Geodetic Deformation Monitoring: From Geophysical to Engineering
Roles, Berlin, Heidelberg, 2006, pp. 285–292.
[4] B. Maidl, M. Herrenknecht, U. Maidl, and G. Wehrmeyer, Mechanised
shield tunnelling. John Wiley & Sons, 2013.
[5] H. Mroueh and I. Shahrour, “A simplified 3D model for tunnel construction
using tunnel boring machines,” Tunn. Undergr. Space Technol., vol. 23,
no. 1, pp. 38–45, 2008.
[6] T. Kasper and G. Meschke, “A 3D finite element simulation model for TBM
tunnelling in soft ground,” Int. J. Numer. Anal. Methods Geomech., vol.
28, no. 14, pp. 1441–1460, 2004.
[7] G. Galli, A. Grimaldi, and A. Leonardi, “Three-dimensional modelling of
tunnel excavation and lining,” Comput. Geotech., vol. 31, no. 3, pp. 171–
183, 2004.
[8] B.-T. Cao, S. Freitag, and G. Meschke, “A hybrid RNN-GPOD surrogate
model for real-time settlement predictions in mechanised tunnelling,” Adv.
Model. Simul. Eng. Sci., vol. 3, no. 1, p. 5, Mar. 2016.
[9] S. Freitag, B. Cao, J. Ninic, and G. Meschke, “Surrogate modeling for
mechanized tunneling simulations with uncertain data,” in Sixth
International Workshop on Reliable Engineering Computing (REC),
Chicago, IL, May, 2014, pp. 25–28.
1057
[10] A. Saadallah, F. Finkeldey, K. Morik, and P. Wiederkehr, “Stability
prediction in milling processes using a simulation-based Machine Learning
approach,” Procedia CIRP, vol. 72, pp. 1493–1498, 2018.
[11] B. Settles, “Active learning literature survey,” Univ. Wis., vol. 52, no. 11,
pp. 55–66, 2010.
[12] Y. Fu, X. Zhu, and B. Li, “A survey on instance selection for active
learning,” Knowl. Inf. Syst., vol. 35, no. 2, pp. 249–283, May 2013.
[13] S. Ercelebi, H. Copur, and I. Ocak, “Surface settlement predictions for
Istanbul Metro tunnels excavated by EPB-TBM,” Environ. Earth Sci., vol.
62, pp. 357–365, 2011.
[14] M. Karakus and R. J. Fowell, “Effects of different tunnel face advance
excavation on the settlement by FEM,” Tunn. Undergr. Space Technol.,
vol. 18, no. 5, pp. 513–523, 2003.
[15] M. Melis, L. Medina, and J. M. Rodriguez, “Prediction and analysis of
subsidence induced by shield tunnelling in the Madrid Metro extension,”
Can. Geotech. J., vol. 39, no. 6, pp. 1273–1287, 2002.
[16] J. N. Franzius, D. M. Potts, and J. B. Burland, “The influence of soil
anisotropy and K0 on ground surface movements resulting from tunnel
excavation,” Geotechnique, vol. 55, no. 3, pp. 189–199, 2005.
[17] G. Meschke, F. Nagel, and J. Stascheit, “Computational simulation of
mechanized tunneling as part of an integrated decision support platform,”
Int. J. Geomech., vol. 11, no. 6, pp. 519–528, 2010.
[18] C. Y. Kim, G. J. Bae, S. W. Hong, C. H. Park, H. K. Moon, and H. S. Shin,
“Neural network based prediction of ground surface settlements due to
tunnelling,” Comput. Geotech., vol. 28, no. 6, pp. 517–547, 2001.
[19] S. R. Dindarloo and E. Siami-Irdemoosa, “Maximum surface settlement
based classification of shallow tunnels in soft ground,” Tunn. Undergr.
Space Technol., vol. 49, pp. 320–327, 2015.
[20] H. Sug, “The Effect of Training Set Size for the Performance of Neural
Networks of Classification,” W Trans Comp, vol. 9, no. 11, pp. 1297–1306,
Nov. 2010.
[21] D. R. B. Stockwell and A. T. Peterson, “Effects of sample size on accuracy
of species distribution models,” Ecol. Model., vol. 148, no. 1, pp. 1–13,
2002.
[22] Q. Pan and D. Dias, “An efficient reliability method combining adaptive
support vector machine and Monte Carlo simulation,” Struct. Saf., vol. 67,
pp. 85–95, 2017.
[23] F.-L. Liu, F. Min, L.-Y. Wen, and H.-J. Wang, “Representative-Based
Active Learning with Max-Min Distance,” Springer, Cham, 2016, pp. 365–
375.
[24] F. Douak, F. Melgani, and N. Benoudjit, “Kernel ridge regression with
active learning for wind speed prediction,” Appl. Energy, vol. 103, pp. 328
– 340, 2013.
[25] W. Cai, M. Zhang, and Y. Zhang, “Batch mode active learning for
regression with expected model change,” IEEE Trans. Neural Netw. Learn.
Syst., vol. 28, no. 7, pp. 1668–1681, Jul. 2017.
[26] W. Cai, Y. Zhang, and J. Zhou, “Maximizing Expected Model Change for
Active Learning in Regression,” in 2013 IEEE 13th International
Conference on Data Mining, 2013, pp. 51–60.
[27] F. Douak, F. Melgani, and N. Benoudjit, “Kernel ridge regression with
active learning for wind speed prediction,” Appl. Energy, vol. 103, pp.
328–340, 2013.
[28] R. Burbidge, J. J. Rowland, and R. D. King, “Active Learning for
Regression Based on Query by Committee,” in Intelligent Data
Engineering and Automated Learning - IDEAL 2007, Berlin, Heidelberg:
Springer Berlin Heidelberg, 2007, pp. 209–218.
[29] R. S. Tsay, Multivariate time series analysis: with R and financial
applications. John Wiley & Sons, 2013.
[30] B. T. Cao, S. Freitag, and G. Meschke, “A fuzzy surrogate modelling
approach for real-time predictions in mechanised tunnelling,” Int. J. Reliab.
Saf., vol. 12, no. 1–2, pp. 187–217, 2018.
[31] M. Y. Park and T. Hastie, “L1-regularization path algorithm for
generalized linear models,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 69,
no. 4, pp. 659–677, 2007.
[32] G. Meschke, F. Nagel, and J. Stascheit, “Computational simulation of
mechanized tunneling as part of an integrated decision support platform,”
Int. J. Geomech., vol. 11, no. 6, pp. 519–528, 2010.
[33] A. Alsahly, J. Stascheit, and G. Meschke, “Advanced finite element
modeling of excavation and advancement processes in mechanized
tunneling,” Adv. Eng. Softw., vol. 100, pp. 198–214, 2016.
1058 Amal Saadallah et al. / Procedia CIRP 81 (2019) 1052–1058
Amal Saadallah et al. / Procedia CIRP 00 (2019) 000–000 7

[34] P. Dadvand, R. Rossi, and E. Oñate, “An object-oriented environment for
developing finite element codes for multi-disciplinary applications,” Arch.
Comput. Methods Eng., vol. 17, no. 3, pp. 253–297, 2010.
[35] A. Popp, “Mortar methods for computational contact mechanics and
general interface problems,” PhD Thesis, Technische Universität
München, 2012.
[36] J. Ninić, “Computational strategies for predictions of the soil-structure
interaction during mechanized tunneling,” 2016.
[37] G. H. Golub and C. F. Van Loan, “Matrix computations. 1996,” Johns
Hopkins Univ. Press Baltim. MD USA, pp. 374–426, 1996.