Data-Driven Modeling To Predict The Load vs. Displacement Curves of Targeted

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 14

Composite Structures 258 (2021) 113207

Contents lists available at ScienceDirect

Composite Structures
journal homepage: www.elsevier.com/locate/compstruct

Data-driven modeling to predict the load vs. displacement curves of targeted


composite materials for industry 4.0 and smart manufacturing
Monzure-Khoda Kazi a,⇑, Fadwa Eljack a, E. Mahdi b
a
Qatar University, Department of Chemical Engineering, College of Engineering, Doha 2713, Qatar
b
Qatar University, Department of Mechanical and Industrial Engineering, College of Engineering, Doha 2713, Qatar

A R T I C L E I N F O A B S T R A C T

Keywords: This work presents an approach for smart manufacturing focusing on Industry 4.0 to predict the load vs. dis-
Machine learning placement curve of targeted cotton fiber/Polypropylene (PP) composite materials while complying with the
Artificial neural network required intended properties. Experimental data for varying composite fiber percentage to characteristic load
Data‐driven modeling and earlier built artificial neural network (ANN) models are used as the feed. A newly built ANN model is being
Industry 4.0
trained and tested on the TensorFlow backend using the Keras library in Python to predict the load vs. displace-
Smart manufacturing
Fiber‐reinforced polymer
ment curves for any in‐between values of the experimental range (e.g., 0–50% cotton fiber filler content in PP)
without doing any further experiment. Finally, a Python package for the sparse identification of nonlinear
dynamical (PySINDy) systems is used to identify the exact data‐driven ANN model through the system identi-
fication, which will facilitate the effective implementation of the control algorithms, smart internet of things
(IoT), and high‐tech automated system.

1. Introduction systems or genetic algorithms when the conceptual model of the ana-
lyzed industrial system is complicated, but at the same time, a compu-
The fourth industrial revolution or Industry 4.0 refers to the mod- tationally less efficient method is necessary [4]. ANNs are advanced
ern industries' intellectualization by integrating innovative fitting and pattern recognition algorithms of machine learning that
approaches, technologies, and instruments on the manufacturing floor allow users to extract complex relationships among nonlinear variables
[1]. Artificial Intelligence (AI), the Internet of Things (IoT), Big Data [7,8]. As the training progresses, the neural network‐based identifica-
analysis, Cyber‐Physical Systems (CPS), and Additive Manufacturing tion model (identifier) will learn the unknown dynamics of the plant/
are some key components associated with this revolution [2]. Smart process [6]. It can eliminate the necessity of physical modeling by
manufacturing is one of the key drives of Industry 4.0 to improve learning through examples, which is known as data‐driven modeling.
the efficiency and responsiveness of a production system. Application This advantage makes ANN computational tools very appealing in
of machine learning and data‐driven modeling can extract useful engineering applications, in cases where there is little or incomplete
knowledge and information from a large volume of structured and understanding of the problem, yet experimental measurements are
unstructured data to make appropriate smart manufacturing [1]. Nev- readily available [9–24].
ertheless, to apply the acquired useful information into real‐life appli- Data‐driven predictive models can facilitate the computer‐aided
cations, a mathematical model describing the system behavior is manufacturing of natural fiber‐reinforced polymer composites
crucial. This modeling is still very challenging to accomplish [3], as (NFRPC). NFRPC is now widely used for the aerospace and automotive
most industrial systems exhibit nonlinear behavior [3]. Therefore, industries due to their inherent advantages of mechanical properties
identifying those complex nonlinear models plays an essential role in and energy absorption capabilities over traditional materials [25,26].
the design and implementation of advanced human–machine interac- It has intrinsic environmental benefits with an abundance of raw mate-
tion (HMI) or smart manufacturing initiatives. rials from renewable sources in comparison to fossil sources [27].
Application of Artificial Neural Networks (ANNs) is considered one Many researchers have investigated the mechanical properties of cot-
of the popular nonlinear system identification approaches [4–6]. ANNs ton fiber (CF) as filler content in the thermoplastic composites (e.g.,
outperform other nonlinear system identification methods like fuzzy CF/Polypropylene, CF/epoxy, CF/Polyvinyl Chloride, etc.), and

⇑ Corresponding author.
E-mail address: kazi0001@qu.edu.qa (M.-K. Kazi).

https://doi.org/10.1016/j.compstruct.2020.113207
Received 31 August 2020; Revised 12 October 2020; Accepted 21 October 2020
Available online 28 October 2020
0263-8223/© 2020 Elsevier Ltd. All rights reserved.
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

increased use of those materials is documented in several other techni- regression technique will be combined with a machine learning tech-
cal applications requiring high strength, hardness and low weight nique to discover governing equations for the nonlinear dynamical sys-
materials [28–35]. In this work, the smart manufacturing of cotton tem of smart manufacturing of NFRPC.
fiber‐reinforced Polypropylene (CF/PP) composite is considered for The paper here extends the works by Kazi et al. on CF/PP compos-
the automotive application, namely for a collapsible energy absorber ite, where a new ANN model along with the exact mathematical model
device [35]. Polypropylene (PP) is chosen as the composite matrix are identified to predict the exact load vs. displacement curves for the
because it is widely used, relatively low‐cost thermoplastic with good optimal amount of cotton fiber filler content in PP. This work aims to
mechanical properties. Its low melting point allows processing below show how machine learning and nonlinear system identification‐based
the degradation temperature of the cotton fiber (about 370 °C). approach can help building a smart manufacturing system for produc-
While investigating the natural fiber‐reinforced polymer compos- ing a fiber‐reinforced composite material of desired quality.
ites, most researchers work through a series of experiments and mate-
rial characterization. The material manufacturer must achieve several
2. Research statement
mechanical properties that are appropriate for the intended end‐use.
Many random sample experiments usually do this function. There is
The given for this study are (1) a set of experimental data of cotton
varying filling content in a particular range. The product designers
fiber/PP composite for varying filler content (i.e., 0%, 10%, 20%,
seek to find the correct filling content that measures the mechanical
30%, 40%, & 50% cotton fiber content), (2) ANN model (Model‐1 as
properties like initiation energy, propagation energy, ductility index,
showed in Fig. 1) capable of predicting the optimal filler content for
tensile strength, modulus of elasticity, elongation for some sample cot-
CF/PP composite to achieve anticipated mechanical properties, and
ton fiber/PP composites. The procedure is tedious and costly. The crit-
(3) a robust methodology of ANN models development (Model‐2 as
ical challenge is the lack of linearity of the relation between the
showed in Fig. 1) for predicting the load vs. displacement curves of
mechanical properties considered. The ability to design such experi-
the exact experimental filler content. The goal here is to develop the
mental work would be more cost‐effective and time‐efficient; and that
final ANN model (Model‐3, as shown in Fig. 1), that will predict the
is a clear scope of smart manufacturing. The role of a predictive model
load–displacement curve for the targeted filler content (value obtained
(i.e., ANN) here is to determine the required filler content necessary to
by Model‐1) using the developed ANN model set (Model‐2) as the
achieve the targeted mechanical properties and its characteristic load
building block. This step is needed because the targeted filler content
vs. displacement curve for the composites. The daunting task is in pre-
amount may be a value that falls in between the existing experimental
dicting the nonlinear behavior of the composite materials such as the
data points. Here, the mathematical equations of the final nonlinear
load–displacement curves for CF/PP composite. It requires several
ANN model (Model‐3) needs to be identified so that it can be
assumptions to obtain the mechanical properties of the considered
implanted into any smart device such as in a 3D printer for smart man-
composite [36]. Keep in mind that the mechanical properties of com-
ufacturing along with a suitably designed controller for rejecting any
posite structures are multivariable, depend to a great extent on multi-
unknown disturbances.
ple factors, and change as a result of loads placed upon them [7].
A systematic approach is followed to accomplish the above‐
Hence, the mechanical properties of composite materials change dur-
mentioned tasks, as shown in Fig. 1. The approach presented in
ing life and thus are not fixed. Such mechanical responses make com-
Fig. 1 consists of four major steps to perform data‐driven modeling
posite materials intriguing for researchers and engineers, and a
and system identification for Industry 4.0 and smart manufacturing.
comprehensive predictive approach using modern novel techniques
In the first step, the composite materials (e.g., CF/PP) experimental
(e.g., artificial intelligence (AI), deep learning) is desirable. There is
data needs to be collected. The data will be used as the input for devel-
an ample variety of AI algorithms such as fuzzy logic, artificial neural
oping the first two ANN models (Model‐1 and Model‐2) to predict the
network, neuro‐fuzzy (combining both ANN and fuzzy logic), and
optimal filler content of targeted properties and predict the load vs.
logistic regression; each excels in a specific area that aims to offer
displacement curves of the exact experimental data points. Usually,
the best fitting idea to a problem and strives to provide the most pre-
load–displacement curves are used to understand the behavior of the
cise correlation among variables (input and output data) [7].
composite fibers. The output of those two ANN models will act as input
There is very little literature that presents the use of ANN models to
and reference for the third ANN model (Model‐3) to predict the
predict the cotton fiber content in composite polypropylene. And the
load–displacement curve of the targeted CF/PP composite materials.
ANN models in composite materials have been used to predict specific
In step 3, a nonlinear system identification method is used to identify
properties and not to predict composite materials' configuration. Kazi
the governing equations for the third ANN model. These equations will
et al. recently demonstrated that the ANN model could predict opti-
provide insights in the future to predict the load–displacement curves
mum cotton fiber composite filling content while meeting the target
for any amount of filler content without conducting any further exper-
mechanical properties [37]. They also proposed an approach for devel-
iments. The smart product designer can leverage such tools to make
oping the predictive ANN model to identify the experimental data
additive manufacturing devices and automatically producing the CF/
points of the load–displacement curve [38]. These load–displacement
PP composite materials with the desired mechanical properties. A
curves for varying loading patterns show the complicated behavior of
description of the proposed systematic approach is provided in the fol-
the considered composite materials. However, this kind of data‐driven
lowing sections.
ANN model is kind of black‐box approximation [4]. Therefore, the dis-
covery of governing equations is also crucial in designing and manag-
ing complex dynamic systems. 3. Data collection
Governing equations are usually derived from the knowledge of
first principles; however, nowadays, data‐driven modeling motivates Two types of experimental data are needed to develop the data‐
algorithmic and reproducible approaches for automated model discov- driven ANN models of the considered CF/PP composite materials:
ery. A number of such approaches have been developed in recent (1) mechanical property values through material characterizations,
years, including linear methods [39], dynamic mode decomposition and (2) energy values through experimental load vs. displacement
(DMD) [40,41], nonlinear autoregressive algorithms [17,42], neural curves and load vs. specific energy curves for the varying CF/PP com-
networks [39], Gaussian process regression [39], nonlinear Laplacian posite. The data used in developing first and second ANN models
spectral analysis [43], genetic programming [44], and sparse regres- (Model‐1 & Model‐2) are obtained from the experimental efforts by
sion [45–47]. Different approaches have their own working principle Mahdi et al., 2020 [30]. Throughout their study, the effects of the filler
and advantages depending on the problem formulation. Here, a sparse material (CF) on the CF/PP composite's mechanical properties were

2
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

Fig. 1. A systematic approach for data-driven modeling to predict the load vs. displacement curve of targeted composite materials for Industry 4.0 and smart
manufacturing.

discussed. Experimental tensile testing and load–displacement experi- ing method in TensorFlow is used to find the correct ANN Model‐3
ments were performed on dog bone specimens, cut from square plates architecture.
of varying fiber weight content formed with hot and cold presses. The Developing the ANN Model‐3 follows three necessary steps just like
specimens of cotton fiber composites were prepared using a Thermo any machine learning practice: (1) data pre‐processing, (2) building
Haake PolyDriver extruder with Rheomix R600/610 (see Fig. 2). The and training the prediction models, and (3) model validation by
melt blending was carried out at 180 °C for PP, and the rotor speed observing the validation accuracy of the training data set and the eval-
was set to 100 rpm. Complete experiment details are available in uation of the pre‐trained test data set model.
Mahdi et al., 2020 [30].
The data used to create ANN Model‐1 and Model‐2 for this work are 4.1.1. Data pre-processing
obtained from Mahdi et al., 2020 and summarized in Table 1 and The data collected (Table 1) and used as input for ANN Model 3
Fig. 3. They include details regarding the CF/PP composite formula- were pre‐processed to render them in format free of anomalies, includ-
tion, mechanical properties and the experimental load–displacement. ing incomplete, outbound, incorrect data values in order to avoid any
Table 2 lists the desired mechanical properties. discrepancies. The following step includes feature extraction to define
the independent models for input models. In this case, three distinct
4. ANN model development features (e.g., amount of varying filler content, normalized specific
energy, and displacement values of varying filler content from ANN
4.1. Configuration of ANN Model-3 Model‐2) are identified to predict the dependent variable, which is
the output load for the whole experimental range for the CF/PP com-
While developing Model‐1 and Model‐2 in previous works, it was posite materials. Therefore, three input nodes are available in the
observed that the ANN model performed satisfactorily in predicting input layers and one in the output layers. Later, the output from
the optimal filler content and the experimental load–displacement ANN Model‐1 will be used to find the exact load–displacement curve
curves. Therefore, a similar approach is also used here for Model‐3 for the optimal filler content with the targeted mechanical properties
development. ANN is a supervised machine learning method consists (see Fig. 4).
of single or multiple hidden layers, where an artificial neuron array After data pre‐processing and feature selection, the data set needs
of connected nodes is aggregated. Such neurons function as a biologi- to be divided into a training set and test set. In this case, the 'train test
cal brain receiving a signal, and then processing it and transmitting split' function (part of the Scikit‐Learn python library) is used for sep-
associated neurons. Usually, neurons have an adjustable weight while arating the dataset. The training subset is intended for model construc-
learning. The weight increases or decreases the signal strength at any tion, and the test subset is used to assess model output using unknown
connection. Neurons can have a threshold so that only the aggregate data.
signal crosses the threshold will send a signal. Neurons are usually
aggregated in layers. Different layers can transform their input in dif- 4.1.2. Building and training the prediction models
ferent ways. The key challenge of designing this model is to find the ANN Model‐3 consists of three layers, namely input, hidden, and
right architecture (i.e., the number of hidden layers, epoch, batch size, output layer. Fig. 4 shows the layered structure of the ANN Model‐3.
etc.). The parameter values (i.e., node weights, bias, etc.) in ANN typ- The hidden layer contains one or more layer for its logical use, and
ically are learned from the forward and backpropagation algorithms. each layer is made up of several neural elements. The lower part of
Nevertheless, several other parameters have been used to guide the Fig. 4 shows a neural element and a relation between the input vector
mþ1
phase of learning, known as the hyperparameter. Such hyperparame- xm
i and output vector y j . The model for ANN is made with the Ten-
ters must be tuned so that the model can better solve the given sorFlow's Keras‐Dense layer. The procedure is carried out by Dense in
machine learning problem. Here, the grid search hyperparameter tun- the following way:

3
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

Fig. 2. Set of PP dumbbell-shaped test coupons [30].

Table 1
Mechanical properties and features of CF/PP composites with various filler content.

Formulation Properties
Filler Net Weight Net Weight of Initiation Propagation Total Ductility Tensile Strength Modulus of Percent
Content (CF) of PP Fiber Energy (Ei) Energy (Ep) Energy(Et) Index (DI) (UTS) Elasticity (E) Elongation (PE)

wt% g g kJ/kg m2 kJ/kg m2 kJ/kg m2 MPa MPa %


0 44 0 12.955 0.35 13.305 0.027 35.8 388 850
10 39.6 4.4 0.079 0.061 0.139 0.766 28 440 15.7
20 35.2 8.8 0.049 0.036 0.085 0.747 24.6 640 15.7
30 30.8 13.2 0.048 0.003 0.051 0.069 27 641 10
40 26.4 17.6 0.027 0.021 0.048 0.766 23.8 700 10
50 22 22 0.0198 0.007 0.027 0.364 18.8 709 10

Output ¼ activationðdot ðinput; kernelÞ þ bias ð1Þ construction. ReLU is a well‐used, robust estimator that can simultane-
  ously activate all neurons. Here defines the ReLU activation function:
m
y mþ1
j ¼ φ ∑i¼1 wnij xm
i þ bij ð2Þ f ðxÞ ¼ 0; for x < 0 ð3Þ

where the input is an input‐data function, activation means an element‐ f ðxÞ ¼ x; for x ≥ 0 ð4Þ
wise activation function passed as the activation argument, the kernel is
a weights matrix created by the layer, the dot represents the numpy dot During training the model, both ‘adam’ and ‘rmsprop’ optimizers of
product of all input and its corresponding weights, and bias is a bias TensorFlow are tested, and 'mean_squared_error' (MSE) was used as
vector created by the layer (only applicable if use_bias is True). The rec- the loss function. The optimal iteration and regression are taken into
tified linear unit (ReLU) activation function was used during the layers’ account with MSE values.

4
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

(a) (b)

(c) (d)

(e) (f)
Fig. 3. Experimental load–displacement curves for (a) PP 100%, (b) PP 90% + CF 10%, (c) PP 80% + CF 20%, (d) PP 70% + CF 30%, (e) PP 60% + CF 40%, and
(f) PP 50% + CF 50% [30].

1 N @MSE
MSE ¼ ∑ ðy  b

2
ð5Þ Δwnij ¼ η ð6Þ
N i¼1 @wnij

where y is the measured output and b y is the predicted output of the where η is the learning rate, the optimum number of the ANN structure
training or test dataset. The error is minimized by the shift in weights was achieved by the hyperparameter tuning method. The Dropout
as follows: method will be used during the training phase to avoid overfitting reg-

5
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

Table 2 dynamical system. A Python package named PySINDy is used for iden-
Targeted values of the mechanical properties for an tifying the governing equations of the developed ANN Model‐3 for pre-
automotive application. dicting the load–displacement curve of optimal filler content. PySINDy
Targeted mechanical properties Values provides tools for applying the sparse identification of nonlinear
dynamics (SINDy) approach to model discovery. The only assumption
Ductility Index 0.08
Tensile Strength 25 MPa
concerning the model is that the dynamics only rule a few essential
Modulus of Elasticity 400 MPa terms, so the equations are sparse in the space of possible functions;
Percent Elongation 0.8% this assumption holds for many physical systems on a reasonable basis.
Initiation Energy 0.015 kJ/kg m2 The approach used leads to prudent models that balance accuracy and
Propagation Energy 0.05 kJ/kg m2
model complexity to prevent overfitting [45–47].
Total Energy 0.065 kJ/kg m2

5.1. Features of PySINDy

ulations. Table 3 summarizes the optimal structure of the ANN Model‐3


The core objective in the PySINDy package is the SINDy model
obtained using the above definition. Section 6 discusses the effects and
class, which is implemented as a scikit‐learn estimator. It also
usefulness of the model.
expresses the SINDy model object at the appropriate level of abstrac-
tion so that users can embed it into more sophisticated pipelines in
4.1.3. Model validation
scikit‐learn, such as for parameter tuning and model selection.
The derived trained ANN model in the training stage must pass the
PySINDy provides an implementation of the SINDy method to discover
validation process using the testing dataset. 20% of the data set (ran-
governing dynamical systems models of the form
domly chosen) is used for testing, and the remainder is used for train-
ing. Also, k‐fold cross‐validation is used to measure the performance d
xðt Þ ¼ f ðxðt ÞÞ ð7Þ
with randomly generated training and testing data set to avoid the pos- dt
sible overfitting of a prediction function's parameter learning on the Given data in the form of state measurements xðtÞ ∈ Rn , SINDy
same data set. This approach can be computationally expensive but identifies a model for the dynamics given by the function f, which
does not waste too much data, which is of significant benefit when describes how the state of the system evolves in time. In particular,
the sample count is limited, such as the inverse inference. Both the SINDy sparsely approximates the dynamics in a library of candidate
training loss function and the validation loss function values are basis functions θðxÞ ¼ ½θ1 ðxÞ; θ2 ðxÞ;    ; θl ðxÞ, so that most coefficients
observed here to evaluate the ANN model's performance. These values ξk are zero, and nonzero entries identify active terms in the dynamics.
need to be near zero compared to their initial values because it repre-
l
sents the summation of each data point's errors. f ðxÞ≈ ∑ θk ðxÞξk ð8Þ
The developed ANN model is checked during the prediction process k¼1

to find the load–displacement curves for the optimal cotton fiber filler To pose SINDy as a regression problem, time‐series measurements
content, previously defined by ANN Model‐1. The developed ANN mod- of x and their time derivatives x_ are arranged into matrices as follows:
els were used several times to check the accuracy, robustness, and reli- 0 1 0 1
x1 ðt 1 Þ x2 ðt 1 Þ xn ðt 1 Þ x_ 1 ðt 1 Þ x_ 2 ðt 1 Þ x_ n ðt 1 Þ
ability of the obtained characteristic curve for the optimal filler content.  
B x1 ðt 2 Þ x2 ðt 2 Þ xn ðt 2 Þ C B x_ 1 ðt 2 Þ x_ 2 ðt 2 Þ x_ n ðt 2 Þ C
Also, the measured mean squared error (MSE) values for the training B C B C
X¼B
B .. .. ..
C _ B
.. C; X ¼ B .. .. .. .. C
C
and validation phase over the epochs are presented to show how the @ . . . . A @ . . . . A
developed models perform to reduce the loss function and to acquire x1 ðt m Þ x2 ðt m Þ  xn ðt m Þ x_ 1 ðt m Þ x_ 2 ðt m Þ  x_ n ðt m Þ
the intended load–displacement curves. The path to find the accurate
prediction of the load–displacement curve is a challenging one. The con- The derivatives can be approximated numerically or measured
figuration of the ANN model plays a vital role in the prediction phase. directly. The library functions are evaluated on the data, resulting in
The detailed results and discussion are presented in Section 6.3. ΘðXÞ ¼ ½θ1 ðX Þ; θ1 ðX Þ;    ; θl ðX Þ. Sparse regression is then performed
to approximately solve
5. Discovering governing equations for system identification
_
X≈Θ ðX ÞΞ ð9Þ
System identification is a tool for defining or calculating the sys- where Ξ is a set of coefficients that determines the active terms in f,
tem's mathematical model using inputs and output measurements. here, the SINDy formulation solves Eq. (9) using a sequentially thresh-
Neural networks display a great promise in nonlinear system identifi- olded least squares algorithm [45]. Nevertheless, any sparse regression
cation, as they can approximate any nonlinear function by means of algorithm such as least absolute shrinkage and selection operator
appropriate architecture and weighting factors. Traditional (LASSO), sparse relaxed regularized regression (SR3), stepwise sparse
approaches to nonlinear systems rely on the generation, at some oper- regression (SSR), or Bayesian methods [48] can also solve this problem.
ating point, of a linear system model. However, for neural networks, The implementation of PySINDy involves three primary steps,
no linearization is necessary. The conventional mathematical model- resulting in three modeling decisions:
ing approach is also prone to modeling errors, parameter variation,
noise, and disturbance. Neural networks can make intelligent decisions 1. The numerical differentiation scheme used to approximate X_ from
in cases of noisy or corrupted data. The ANN Model‐3 developed in X
this study (as described in Section 4) can identify the nonlinear rela- 2. The candidate functions constituting the feature library Θ
tionship between the CF/PP composite manufacturing system's input 3. The sparse regression algorithm that is applied to solve Eq. (9) to
and output data. Often this type of ANN model acts as the black‐box find Ξ
model. Yet, it is essential to identify the governing equations (grey/
white model) of that nonlinear system for designing a robust control 5.2. Application of PySINDy
algorithm and embedding the developed nonlinear model into the
heart of any smart manufacturing devices. PySINDy package is used to identify the governing equations for
Here, a sparse regression technique is combined with a machine the load–displacement curve derived from the nonlinear system iden-
learning technique to discover governing equations for the nonlinear tification method ANN. The above‐mentioned (in Section 5.1) three‐

6
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

Fig. 4. The layered structure of ANN Model-1 and Model-3 for predicting the load–displacement curve for optimal filler content of CF/PP composite.

step procedure of PySINDy implementation will be used here with learned model's quality is likely to suffer. In the experimental data,
appropriate functions and arguments in Python. it was found that the load–displacement curve is very nonlinear, and
it is tough to remove all noises from the data since it was accomplished
5.2.1. Numerical differentiation (for computing X_ from X) in a discrete‐time period with human intervention. Note that even a
Numerical differentiation will be one of the core components for small amount of noise in the data can produce noticeable degradation
the SINDy constructor, for model approximation. Derivatives of mea- in the numerical derivative quality. One way to mitigate the effects of
surement variables provide the targets for the sparse regression prob- noise is to smooth the measurements before computing derivatives.
lem [48]. If care is not taken in computing these derivatives, the The ‘SmoothedFiniteDifference’ method is used for this purpose, which

7
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

Table 3 specified in Table 1. Later, this value is used to predict the exact
The optimum configuration of the ANN Model-3 for CF/PP. load–displacement curve for the CF/PP composite material.
Parameters Configurations
6.2. Prediction of experimental load–displacement curves using ANN
Number of neurons in the input layer 3
Model-2
Number of hidden layers 4
Number of the output neuron 1
Training data 80% The modeling approach for CF/PVC composite materials is used
Test data 20% here for CF/PP composite to find the load–displacement curves for
Number of folding in cross-validation 3 six different cotton fiber filler content (i.e., 0 wt%, 10 wt%, 20 wt%
Numbers of neurons in each hidden layer 300
Number of epochs 2000
30 wt%, 40 wt%, 50 wt%). Between the predicted and the experimen-
Batch size 32 tal load carrying capacity, mean squared errors of as low as
Optimizer adam 0.000246 N were achieved, showing an excellent agreement with
Activation function ReLU the experimental data (see Fig. 5 and Table 4). For each prediction,
Loss function Mean squared error (MSE)
the ANN model was run at least five times to check the model
reliability.
The model's robustness was measured using the MSE values of
performs differentiation by smoothing input data, then applying a training and validation loss function (see Table 4). Loss is the quanti-
finite difference method. tative measure of deviation or difference between the predicted output
and the actual output in anticipation. It gives the measure of mistakes
5.2.2. Feature library (for constructing Θ) made by the network in predicting the output. In Table 4, the loss val-
The SINDy method assumes that dynamics can be represented as a ues for both training and test dataset are reported for all experimental
sparse linear combination of library functions. If this assumption is curves. It should be noted that initially, the value of training loss and
violated, the method is likely to exhibit poor performance. This issue validation loss was higher, later the ANN models were able to mini-
tends to manifest itself as numerous library terms are active, often with mize the loss function values that means the model can minimize
weights of vastly different magnitudes, still resulting in low model the errors. In Fig. 6, the changes in the loss functions over the epochs
error. Typically, prior knowledge of the system of interest and its are shown. It can be noticed that there are some small fluctuations in
dynamics should be used to make a judicious choice of essential func- the loss function values (see Fig. 6a) after a certain number of epochs.
tions. When such information is unavailable, the default class of This could be due to ANN model's stochastic nature; nonetheless the
library functions, polynomials, are an excellent place to start, as algorithm was able to minimize those fluctuations. It is crucial to
smooth functions have rapidly converging Taylor series. The polyno- check the model performance and the stopping criteria carefully and
mial feature library is also used as ANN Model‐3 is to some extent, it should not be stopped before reaching the stable results.
black box in nature. The displacement values for corresponding varying load obtained
from this ANN Model‐2 will be used as the trained data for ANN
5.2.3. Optimizer (for performing sparse regression) Model‐3.
PySINDy uses various optimizers to solve the sparse regression
problem. A sequentially thresholded least squares (STLSQ) algorithm 6.3. Prediction of targeted load–displacement curve using ANN Model-3
is used to minimize the objective function to solve the sparse regres-
sion in this work. The threshold value is set at 0.00002, and the The main task of this work is to develop the ANN Model‐3 based on
optional L2 (ridge) regularization weight vector value is set at 0.05. the knowledge of previous models. In this task, the load–displacement
These values are chosen based on the data input values to the ANN curve of 14 wt% (obtained from ANN Model‐1) cotton fiber targeted
Model‐3. The optimizer will do the normalization before regression filler content in PP materials needs to be found. Thus, the 10 wt%
by subtracting the mean and dividing by the L2‐norm. and 20 wt% modeling curves of ANN Model‐2 were considered as
The final governing equations of the ANN Model‐3 obtained by the reference value; as the characteristic curve should fall between
PySINDy, its importance, and future potential applications are those two curves.
reported in Section 6.4. The basic structure of the ANN Model‐3 is shown in Fig. 4, and the
final configuration of the ANN Model‐3 is presented in Table 3. The
6. Results and discussion training method, hyperparameter tuning also showed crucial impacts
on the performance, speed, and quality of the machine learning mod-
The methodology described in Sections 4 and 5 is applied for the els, and hence those parameters were optimized for the ANN Model‐3.
data‐driven modeling of targeted CF/PP composite materials (follow- The optimal number of hidden layers, neurons, and epochs were found
ing the approach mentioned in Fig. 1) using the experimental data using the grid‐search tuning process. The hyperparameter tuning was
reported in Section 3. The ANN Model‐3 and its governing equations performed using the grid search method for the following parameters
are identified for Industry 4.0 and smart manufacturing. The last step and ranges, as reported in
of smart manufacturing will be accomplished via a robust control con- Table 5.
cept; that can be used for developing a model predictive controller to During parameter tuning, it was observed that the prediction of the
facilitate the AI‐powered Industry 4.0 innovation. The outcomes of this targeted load–displacement curve was very sensitive to the parameters
work are presented in the subsequent sections. due to the multifaceted relationship of the considered input parame-
ters. In Fig. 7, the transition of the trained model is shown in some par-
6.1. Prediction of targeted optimal filler content using ANN Model-1 ticular cases. Initially, it was very tough to predict the desire
load–displacement curve as the nodes could not capture the accurate
The ANN Model‐1 was saved to a disk using the Keras script along relationship. Nevertheless, the right level of parameter values and
with its corresponding structure information, and the saved model was accurate training method predicted the curve precisely. If Fig. 7(a)
loaded with the load script without restarting the entire operation and (b) are compared with Fig. 7(c) and (d), it can be seen that opti-
[37]. The optimal cotton fiber filler content amount was determined mizer ‘adam’ outperformed ‘rmsprop’ in this case. Similarly, it was
to be 14.38 wt% to achieve the targeted mechanical characteristics seen that unit size 300 (compare Fig. 7(a)–(i) to Fig. 7(j)–(l)), number
of hidden layers 4 (compare Fig. 7(f) and (i) to Fig. 7(j)‐7(l)), batch

8
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

(a) (b)

(c) (d)

(e) (f)
Fig. 5. Experimental and predicted load–displacement curves for (a) PP 100%, (b) PP 90% + CF 10%, (c) PP 80% + CF 20%, (d) PP 70% + CF 30%, (e) PP
60% + CF 40%, and (f) PP 50% + CF 50%

Table 4
Loss and score values of training and testing data set for varying cotton fiber filler contents.

Filler Content (CF) wt% MSE Model Evaluation


Training loss (kN) Validation loss (kN) Train score (kN) Test score (kN)
−4 −7 −4
0% 1.54 × 10 6.09 × 10 1.23 × 10 1.94 × 10−5
10% 1.99 × 10−6 1.28 × 10−6 1.60 × 10−6 4.94 × 10−6
20% 1.66 × 10−6 8.11 × 10−7 5.76 × 10−7 4.39 × 10−7
30% 3.56 × 10−6 7.38 × 10−6 4.24 × 10−6 4.43 × 10−6
40% 2.93 × 10−7 5.62 × 10−7 3.14 × 10−7 4.06 × 10−7
50% 2.46 × 10−7 3.84 × 10−7 2.84 × 10−7 3.45 × 10−7

size 32 (compare Fig. 7(f) and (h) to Fig. 7(j)–(l)) and epoch size 2000 essary biases by varying the number of active neurons in the input
work better. layer. The ANN model was run at least five times, for every prediction,
In parameter tuning, it was observed that the model's accuracy to test the model reliability.
required a high number of neurons (300 in each hidden layer) and As the objective of this work is to use this predictive load–displace-
epochs (2000). Sometimes, higher neuron numbers with restricted ment curve to acquire the nature of elasticity of the composite materi-
data points will cause the model to overfit or underfit. The dropout als, the shape of the targeted characterization curve should be in line
method, along with the dense layer, was employed to eliminate unnec- with the nature of the experimental curve. Fig. 7(j)–(l) are showing a

9
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

(a) (b)

(c) (d)

(e) (f)
Fig. 6. Improvement of loss functions values over the epochs for (a) PP 100%, (b) PP 90% + CF 10%, (c) PP 80% + CF 20%, (d) PP 70% + CF 30%, (e) PP
60% + CF 40%, and (f) PP 50% + CF 50%

Table 5 region (3), the load gradually increases to the maximum load point
Parameters and ranges of hyperparameter tuning for ANN Model-3. as the specimen is further displaced. The maximum load point is
Parameter Ranges 0.01 kN. Eventually, the load is gradually reduced to a point where
the specimen has been damaged destructively. This is indicated in
Number of hidden layers 1–10
region(4). The randomness in the predictive curves depicts that the
Number of neurons/units in each layer 1–1000
Batch size 32, 64, 128, 256, 512
curves are not over or under fitted; hence, the developed ANN
Epoch 100–20,000 Model‐3 was in excellent agreement with the experimental data; and
Activation function ‘RELU,’ ‘SIGMOID’ the predictive mechanical values were consistent.
Optimizer ‘rmsprop,’ ‘adam.’

6.4. Discovering governing equations through PySINDy

consistent behavioral pattern as their parameter was optimized. How- Table 6 shows the corresponding models recovered using the
ever, the predictive curves were not precisely the same due to the high PySINDy method, as described in Section 5.
nonlinearity in training and testing set data. The k‐fold cross‐ Here, x represents displacement, y represents load, x_ denotes the
validation method was used to choose the random set of training first derivative of displacement value, and y_ denotes the first deriva-
and testing set from the obtained experimental data. Therefore, there tive of load value concerning discrete‐time values. Please note that
should be some randomness in the predicted output, but that random- all these three are the predictive curves for 14 wt% cotton fiber filler
ness has minimal impact on the final acquired elasticity nature. Look- content, which satisfy the required mechanical properties of CF/PP
ing at Fig. 7(k) & (l), they suggest that the load–displacement curve composite fiber. But the precise solution will depend on the experi-
obtained for PP86%/CF14% shows linear characteristics within area mental validation in the later stage. However, as mentioned in Sec-
of load–displacement curve (1). The elastic limit is observed to be tion 6.3, the difference between these curves is very little (see Fig. 7
approximately (0.375, 0.02); when the load increases over this stage, (j), (k) and (l)). This curve's nonlinearity can be eliminated further if
the specimen is permanently deformed. With rising displacement, a a more experimental data point is available for the training. However,
steady, flat charge line can be viewed within the region (2). Within the identified models are enough for dynamic control, while a more

10
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

(a) (b)

(c) (d)

(e) (f)
Fig. 7. Improvement of predicted load–displacement curve for targeted filler content during hyperparameter tuning (selected evolutions along with their
parameters are shown here).

accurate model is identified. In this work, the proposed PySINDy the dynamic control system accordingly. The integration of different
framework proved to promise the real‐time recovery of nonlinear mod- IoT tools is also a possibility with such smart HMI to act as the sensors
els of ANN predictive models. Future work will be required to incorpo- for the complicated process. Usage of AI and data‐driven dynamic con-
rate it into controllers with real experiments. trol system can remove many barriers in systems where the first prin-
cipal modeling is challenging to achieve or where the traditional
6.5. The future implication of derived models and equations for smart controller fails to perform well due to the inherent high nonlinearity.
manufacturing
7. Conclusion
The acquired ANN models (ANN Model‐1, 2 & 3) and their govern-
ing equations can work as building blocks for designing a data‐driven In this work, a new ANN model (ANN Model‐3) has been developed
model predictive controller (MPC). Fig. 8 shows a schematic diagram in the TensorFlow backend using the Keras library in Python. The
of a data‐driven modeling and control concept of the targeted compos- models can be an integral part of industry 4.0 and smart manufactur-
ite materials for Industry 4.0 and smart manufacturing. The control ing of targeted CF/PP composite materials. It can predict the load–dis-
algorithm of this work will be developed further using sparse identifi- placement curve for any in‐between values of the experimental range
cation of nonlinear dynamics with control (SINDYc) in the future. This (e.g., 0–50% cotton fiber filler content in PP) without doing any fur-
kind of smart manufacturing can expedite the production of the ther experiment. The hyperparameter tuning method (i.e., the grid
desired composite materials and control the output's quality precisely. search method) was applied to find the best ANN pattern, and during
The development of a completely automated human–machine inter- the training stage, dropout regulations were used to prevent overfit-
face (HMI) is also possible with developed models and equations; ting. It was found that for CF/PP composite, ANN modeling performs
where they can assist the actuators and sensors in executing and tuning better when there are four hidden layers and 300 neurons in each layer

11
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

(g) (h)

(i) (j)

(k) (l)
Fig. 7 (continued)

Table 6 sistent while finding the required mechanical properties from the
Governing equations for three different predictive load–displacement curves by load–displacement curve. The consistency of the output results was
ANN Model-3. also reflected in the identified governing equations of ANN Model‐3.
Predictive curve Equations Incorporating these developed ANN models and their governing equa-
tions can facilitate the implementation of the data‐driven model pre-
Model of Fig. 7 x_ ¼ 0:01
dictive controller in the real‐life smart manufacturing process.
(j) y_ ¼ 0:001  0:001x þ 0:161y  0:081xy  0:267y 2
Model of Fig. 7 x_ ¼ 0:01
(k) y_ ¼ 0:001  0:002x þ 0:162y þ 0:001x2  0:084xy  0:287y 2 CRediT authorship contribution statement
Model of Fig. 7 x_ ¼ 0:01
(l) y_ ¼ 0:076y  0:099xy þ 0728y 2
Monzure‐Khoda Kazi: Conceptualization, Methodology, Software,
Data curation, Validation, Writing ‐ original draft. Fadwa Eljack:
Writing ‐ review & editing, Supervision, Funding acquisition. E.
on the ANN architecture, and the ‘adam’ optimizer is used. The usabil- Mahdi: Writing ‐ review & editing, Supervision, Resources, Validation.
ity of the developed ANN models is demonstrated by achieving a tar-
geted set of mechanical properties with the optimal cotton fiber filler Declaration of Competing Interest
content in PP. The newly developed ANN Model‐3 is used to predict
the load–displacement curve for 14 wt% CF/PP composite. It was The authors declare that they have no known competing financial
found that the predicted characteristic curve was very much aligned interests or personal relationships that could have appeared to influ-
with the previous experimental patterns. The prediction was very con- ence the work reported in this paper.

12
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

Fig. 8. Execution of data-driven modeling and control of targeted composite materials for Industry 4.0 and smart manufacturing.

Acknowledgment [15] Mahdi E-S, El Kadi H. Crushing behavior of laterally compressed composite
elliptical tubes: experiments and predictions using artificial neural networks.
Compos Struct 2008;83:399–412.
This paper was made possible by NPRP grant No 10‐0205‐170347 [16] Mareš T, Janouchová E, Kučerová A. Artificial neural networks in the calibration of
from the Qatar National Research Fund (a member of Qatar Founda- nonlinear mechanical models. Adv Eng Softw 2016;95:68–81.
[17] Medi B, Monzure-Khoda K, Amanullah M. Experimental implementation of
tion). The statements made herein are solely the responsibility of the
optimal control of an improved single-column chromatographic process for the
author[s]. separation of enantiomers. Ind Eng Chem Res 2015;54:6527–39.
[18] Sabiston T, Inal K, Lee-Sullivan P. Application of Artificial Neural Networks to
predict fibre orientation in long fibre compression moulded composite materials.
References
Compos Sci Technol 2020;190.
[19] Vassilopoulos A, Georgopoulos E, Dionysopoulos V. Artificial neural networks in
[1] Ahuett-Garza H, Kurfess T. A brief discussion on the trends of habilitating spectrum fatigue life prediction of composite materials. Int J Fatigue
technologies for Industry 4.0 and Smart manufacturing. Manuf Lett 2018;15:60–3. 2007;29:20–9.
[2] Ruiz-Sarmiento J-R, Monroy J, Moreno F-A, Galindo C, Bonelo J-M, Gonzalez- [20] Yan S, Zou X, Ilkhani M, Jones A. An efficient multiscale surrogate modelling
Jimenez J. A predictive model for the maintenance of industrial machinery in the framework for composite materials considering progressive damage based on
context of industry 4.0. Eng Appl Artif Intell 2020;87. 103289. artificial neural networks. Compos Part B: Eng 2020:194.
[3] Chakravorti T, Satyanarayana P. Non linear system identification using kernel [21] Zakaulla M, Parveen F, Amreen H, Ahmad N. Artificial neural network based
based exponentially extended random vector functional link network. Appl Soft prediction on tribological properties of polycarbonate composites reinforced with
Comput 2020;89:106117. graphene and boron carbide particle. Mater Today: Proc 2020.
[4] Kumar R, Srivastava S, Gupta JRP, Mohindru A. Comparative study of neural [22] Zarra T, Galang MG, Ballesteros Jr F, Belgiorno V, Naddeo V. Environmental odour
networks for dynamic nonlinear systems identification. Soft Comput management by artificial neural network – a review. Environ Int
2019;23:101–14. 2019;133:105189.
[5] Kumar R, Srivastava S, Gupta JRP. Modeling and adaptive control of nonlinear [23] Veeresh Kumar GB, Pramod R, Rao CSP, Gouda PSS. Artificial neural network
dynamical systems using radial basis function network. Soft Comput prediction on wear of Al6061 alloy metal matrix composites reinforced with -
2017;21:4447–63. Al2o3. Mater Today: Proc 2018;5:11268–76.
[6] Gupta M, Jin L, Homma N. Static and dynamic neural networks: from [24] Laban O, Gowid S, Mahdi E. Experimental investigation and uncertainty prediction
fundamentals to advanced theory. Wiley; 2004. of the load-carrying capacity of composite double hat for lattice core sandwich
[7] Rahmanpanah H, Mouloodi S, Burvill C, Gohari S, Davies HMS. Prediction of load- panels using artificial neural network. In: 2020 IEEE International Conference on
displacement curve in a complex structure using artificial neural networks: a study Informatics, IoT, and Enabling Technologies (ICIoT). p. 67–72.
on a long bone. Int J Eng Sci 2020;154. [25] Ray K, Patra H, Swain AK, Parida B, Mahapatra S, Sahu A, et al. Glass/jute/sisal
[8] Sivanandam SN, Sumathi S, Deepa SN. Introduction to neural networks using fiber reinforced hybrid polypropylene polymer composites: fabrication and
MATLAB 6.0. 2010. analysis of mechanical and water absorption properties. Mater Today: Proc 2020.
[9] Afram A, Janabi-Sharifi F, Fung AS, Raahemifar K. Artificial neural network (ANN) [26] Mohammed L, Ansari MNM, Pua G, Jawaid M, Islam MS. A review on natural fiber
based model predictive control (MPC) and optimization of HVAC systems: a state reinforced polymer composite and its applications. Int J Polym Sci
of the art review and case study of a residential HVAC system. Energy Build 2015;2015:243947.
2017;141:96–113. [27] Kerni L, Singh S, Patnaik A, Kumar N. A review on natural fiber reinforced
[10] Al-Assaf Y, Kadi HE. Fatigue life prediction of composite materials using composites. Mater Today: Proc 2020.
polynomial classifiers and recurrent neural networks. Compos Struct [28] Dean A, Safdar N, Rolfes R. A co-rotational based anisotropic elasto-plastic model
2007;77:561–9. for geometrically non-linear analysis of fibre reinforced polymer composites:
[11] Gowid S, Mahdi E, Alabtah F. Modeling and optimization of the crushing behavior formulation and finite element implementation. Materials (Basel) 2019;12.
and energy absorption of plain weave composite hexagonal quadruple ring [29] Mahdi E, Hamouda AMS, Sahari BB, Khalid YA. On the collapse of cotton/epoxy
systems using artificial neural network. Compos Struct 2019;229:111473. tubes under axial static loading. Appl Compos Mater 2003;10:67–84.
[12] Khan MI. Predicting properties of high performance concrete containing composite [30] Mahdi E, Dean A. The effect of filler content on the tensile behavior of
cementitious materials using artificial neural networks. Autom Constr polypropylene/cotton fiber and poly(vinyl chloride)/cotton fiber composites.
2012;22:516–24. Materials (Basel) 2020;13.
[13] Khandakar A, Chowdhury MEH, Kazi M-K, Benhmed K, Touati F, Al-Hitmi M, et al. [31] Jia Z, Luo G, Wu H, Li Z, Ni T, Ai M. Cotton fiber-biotemplated synthesis of Ag
Machine Learning Based Photovoltaics (PV) power prediction using different fibers: catalytic reduction for 4-nitrophenol and SERS application. Solid State Sci
environmental parameters of qatar. Energies 2019;12:1–19. 2019;94:120–6.
[14] Li X, Liu Z, Cui S, Luo C, Li C, Zhuang Z. Predicting the effective mechanical [32] Moradi E, Zeinedini A. On the mixed mode I/II/III inter-laminar fracture
property of heterogeneous materials by image based modeling and deep learning. toughness of cotton/epoxy laminated composites. Theor Appl Fract Mech
Comput Methods Appl Mech Eng 2019;347:735–53. 2020;105:102400.

13
M.-K. Kazi et al. Composite Structures 258 (2021) 113207

[33] Sanhawong W, Banhalee P, Boonsang S, Kaewpirom S. Effect of concentrated [42] Billings SA. Nonlinear system identification: NARMAX methods in the time,
natural rubber latex on the properties and degradation behavior of cotton-fiber- frequency and spatio-temporal domains. Wiley; 2013.
reinforced cassava starch biofoam. Ind Crops Prod 2017;108:756–66. [43] Giannakis D, Majda AJ. Nonlinear Laplacian spectral analysis for time series with
[34] Bajwa SG, Bajwa DS, Holt G, Coffelt T, Nakayama F. Properties of thermoplastic intermittency and low-frequency variability. Proc Natl Acad Sci 2012;109. 2222.
composites with cotton and guayule biomass residues as fiber fillers. Ind Crops [44] Daniels BC, Nemenman I. Automated adaptive inference of phenomenological
Prod 2011;33:747–55. dynamical models. Nat Commun 2015;6:8133.
[35] Laban O, Mahdi E. Energy absorption capability of cotton fiber/epoxy composite [45] Brunton SL, Proctor JL, Kutz JN. Discovering governing equations from data by
square and rectangular tubes. J Nat Fibers 2016;13:726–36. sparse identification of nonlinear dynamical systems. Proc Natl Acad Sci 2016;113.
[36] Mouloodi S, Rahmanpanah H, Burvill C, Davies HMS. Prediction of load in a long 3932.
bone using an artificial neural network prediction algorithm. J Mech Behav [46] Kaiser E, Kutz JN, Brunton SL. Sparse identification of nonlinear dynamics for
Biomed Mater 2020;102:103527. model predictive control in the low-data limit. Proc R Soc A: Math, Phys Eng Sci
[37] Kazi M-K, Eljack F, Mahdi E. Optimal filler content for cotton fiber/PP composite 2018;474. 20180335.
based on mechanical properties using artificial neural network. Compos Struct [47] Brunton SL, Proctor JL, Kutz JN. Sparse identification of nonlinear dynamics with
2020;251:112654. control (SINDYc)**SLB acknowledges support from the U.S. Air Force Center of
[38] Kazi M-K, Eljack F, Mahdi E. Predictive ANN models for varying filler content for Excellence on Nature Inspired Flight Technologies and Ideas (FA9550-14-1-0398).
cotton fiber/PVC composites based on experimental load displacement curves. JLP thanks Bill and Melinda Gates for their active support of the Institute of
Compos Struct 2020;254:112885. Disease Modeling and their sponsorship through the Global Good Fund. JNK
[39] Nelles O. Nonlinear system identification: from classical approaches to neural acknowledges support from the U.S. Air Force Office of Scientific Research
networks and fuzzy models. Springer; 2001. (FA9550-09-0174). IFAC-PapersOnLine 2016;49:710–5.
[40] Schmid PJ. Dynamic mode decomposition of numerical and experimental data. J [48] Brunton SL, Kutz JN. Data-driven science and engineering: machine learning,
Fluid Mech 2010;656:5–28. dynamical systems, and control. Cambridge University Press; 2019.
[41] Kutz JN, Brunton SL, Brunton BW, Proctor JL. Dynamic mode decomposition: data-
driven modeling of complex systems. Society for Industrial and Applied
Mathematics; 2016.

14

You might also like