See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/331730377

Estimating Compressive Strength of Concrete Using Deep Convolutional Neural

Networks with Digital Microscope Images

Article  in  Journal of Computing in Civil Engineering · May 2019

DOI: 10.1061/(ASCE)CP.1943-5487.0000837

CITATIONS READS

25 1,719

3 authors, including:

Youjin Jang Yong Han Ahn

North Dakota State University Hanyang University
19 PUBLICATIONS   176 CITATIONS    138 PUBLICATIONS   1,076 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Impact of Social and Psychological factors on workers productivity during Covid-19 Pandemic View project

Modular Construction View project

All content following this page was uploaded by Yong Han Ahn on 08 May 2019.

The user has requested enhancement of the downloaded file.

 Estimating Compressive Strength of Concrete
Using Deep Convolutional Neural Networks
with Digital Microscope Images
Youjin Jang 1; Yonghan Ahn 2; and Ha Young Kim 3
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.

Abstract: Compressive strength is a critical indicator of concrete quality for ensuring the safety of existing concrete structures. As an alter-
native to existing nondestructive testing methods, image-based concrete compressive strength estimation models using three deep convolu-
tional neural networks (DCNNs), namely AlexNet, GoogLeNet, and ResNet, were developed for this study. Images of the surfaces of specially
produced specimens were obtained using a portable digital microscope, after which the samples were subjected to destructive tests to evaluate
their compressive strength. The results were used to create a dataset linking the experimentally determined compressive strength with the image
data recorded for each. The results of training, validation, and testing showed that DCNN models largely outperformed the recently proposed
image processing–based ANN model. Overall, the ResNet-based model exhibited greater compressive strength estimation accuracy than either
the AlexNet- or GoogLeNet-based models. These finding indicate that image data obtained using a portable digital microscope contain patterns
that can be correlated with the concrete’s compressive strength, enabling the proposed DCNN models to use these patterns to estimate com-
pressive strength. The results of this study demonstrate the applicability of DCNN models using microstructure images as an auxiliary method
for the nondestructive evaluation of concrete compressive strength. DOI: 10.1061/(ASCE)CP.1943-5487.0000837. © 2019 American Society
of Civil Engineers.
Author keywords: Concrete; Compressive strength; Deep convolutional neural network; Estimation model; Digital microscope image.

Introduction
Concrete is one of the world’s most widely used building materials.
It is obtained by mixing aggregates, cement, water, and any addi-
tives required to achieve the desired properties. Due to its easy
availability, low cost, convenient handling, and the option to shape
it into any desired form, concrete is ubiquitous in the construction
industry, and most buildings today contain RC elements. One of the
main indicators used for evaluating the condition of existing con-
crete structures is the compressive strength of the concrete from
which they are constructed (Tiberti et al. 2015; Baygin et al. 2018).
The compressive strength is generally defined as the failure load of
the concrete under specific loading conditions. Evaluating the com-
pressive strength of concrete is vital for assessing the deterioration
of concrete structures and ensuring their safety (Steenbergen and
Vervuurt 2012).
There are two main approaches to evaluating the compres-
sive strength of concrete: destructive and nondestructive testing.
Destructive test methods measure the compressive strength in a lab-
oratory environment using a concrete core sample obtained from
1
Postdoctoral Researcher, School of Architecture and Architectural
Engineering, Hanyang Univ., 55 Hanyangdaehak-ro, Sangrok-gu, Ansan-si,
Gyeonggi-do 15588, Republic of Korea. Email: uzjang@gmail.com
2
Associate Professor, School of Architecture and Architectural Engi-
neering, Hanyang Univ., 55 Hanyangdaehak-ro, Sangrok-gu, Ansan-si,
Gyeonggi-do 15588, Republic of Korea. Email: yhahn@hanyang.ac.kr
3
Assistant Professor, Dept. of Financial Engineering, Ajou Univ., 206
Worldcupro, Yeongtong-gu, Suwon, Gyeonggi-do 16499, Republic of
Korea. (corresponding author). Email: hayoungkim@ajou.ac.kr
Note. This manuscript was submitted on July 17, 2018; approved on
November 6, 2018; published online on February 28, 2019. Discussion
period open until July 28, 2019; separate discussions must be submitted
for individual papers. This paper is part of the Journal of Computing
in Civil Engineering, © ASCE, ISSN 0887-3801.
the actual concrete structure being tested. The compressive strength
corresponds to the nominal stress at which that specimen or con-
crete core fails under uniaxial loading. However, taking concrete
cores is costly and can lead to safety problems because it can
easily damage the concrete structure being tested. Nondestructive
test methods such as the rebound hammer (RH) test, ultrasonic
pulse velocity (UPV) test, and SonReb seek to avoid these
problems by estimating the compressive strength of the concrete
using empirical formulas. However, although these methods give
approximate results for the compressive strength of the concrete
on site without damaging the actual concrete structure, they re-
quire expensive equipment and careful instrument maintenance,
as well as trained and certified personnel with a high degree of
skill and integrity. Recently, a number of image processing–based
methods for the estimation of concrete compressive strength have
been proposed as alternative nondestructive test methods (Başyiğit
et al. 2012; Dogan et al. 2017). The use of images for estimating
the compressive strength of concrete has a number of advantages
because it potentially reduces both the time and cost required
to conduct the tests. Several studies have thus sought to estimate
concrete compressive strength using statistical analysis and artifi-
cial neural networks (ANN) based on data obtained from image
processing and the results of experimental tests conducted using
traditional destructive testing methods (Başyiğit et al. 2012;
Dogan et al. 2017). Unfortunately, as yet these proposed image
processing–based studies suffer from limitations when applied in
situ, and feature engineering is challenging for complex concrete
images.
To address these issues, this study developed a new image-based
model for estimating concrete compressive strength using deep
convolutional neural networks (DCNNs) to analyze images
collected with a portable digital microscope. DCNN is a deep learn-
ing technique that can be used to autonomously extract complex
discriminative features via a learning procedure, thus reducing

© ASCE 04019018-1 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2019, 33(3): 04019018

 the need for human input to identify features of interest. DCNNs
have already achieved highly accurate results in the field of image
recognition for applications such as face recognition (Derman and
Salah 2018), autonomous vehicles (Nugraha et al. 2017), and medi-
cal diagnosis (Sun et al. 2017). In particular, a number of successful
applications of DCNNs have been reported in the field of construc-
tion engineering, including pavement crack detection (Zhang et al.
2017; Gopalakrishnan et al. 2017), concrete crack detection (Cha
et al. 2017), and structural damage detection (Lin et al. 2017).
Building on this earlier work, we therefore applied DCNNs for the
estimation of the compressive strength of concrete based on con-
crete surface images. The performance of DCNN models varies con-
siderably depending on their architecture, with factors such as the
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.
number of layers, units per layer, and size of the convolutional mask
all affecting the results (Ferreira et al. 2018). We therefore used
modified versions of three representative DCNN models, namely
AlexNet, GoogLeNet, and ResNet, to estimate concrete compres-
sive strength and then compared their performance. A perfor-
mance comparison with the image processing–based ANN model
recently proposed by Dogan et al. (2017) was also conducted. The
training, validation, and testing operations were accomplished us-
ing datasets created using concrete specimens prepared in the lab-
oratory. The estimation accuracy achieved by each DCNN model
was evaluated in terms of the value of its coefficient of determi-
nation (R2 ), its mean absolute percentage error (MAPE), and its
root-mean-square error (RMSE). The results provide a useful
reference with which to assess the suitability of DCNN as a non-
destructive test method to estimate concrete compressive strength
based on image data.
The remainder of this paper is organized as follows. The next
section reviews the literature on estimating concrete compressive
strength and DCNN models. Then the research methodology is
described including the dataset creation process, performance
evaluation measures, and the experimental settings. Lastly, the ex-
perimental results and discussions are presented, and the paper con-
cludes with a summary of the findings and suggestions for further
study.
Literature Review
Estimation of Concrete Compressive Strength
With regard to the safety management of existing concrete struc-
tures, compressive strength is considered the most critical indicator
of concrete quality (Ju et al. 2017). Due to the complex degradation
mechanisms involved and the multiple factors governing each,
evaluating and estimating the compressive strength of concrete re-
mains a challenging issue. To determine the compressive strength
of concrete, destructive test methods are the most reliable, but it is
not feasible to examine the in situ concrete properties without dam-
aging the structure. As a result, nondestructive test methods offer an
attractive alternative, and researchers are constantly seeking to
develop better nondestructive test methods for estimating the com-
pressive strength of concrete.
Existing nondestructive test methods include the rebound ham-
mer test, ultrasonic pulse velocity test, pull-out test, penetration re-
sistance test, magnetic test, and radioactive test. Among these, the
ultrasonic pulse velocity test, rebound hammer test, and a method
that combines the RH test and the UPV test, known as SonReb, are
the most widely accepted nondestructive test methods, largely due
to their simplicity and effectiveness. To improve their accuracy and
reliability, researchers have attempted to develop better estimation
methods for the RH, UPV, and SonReb tests using regression
© ASCE 04019018-2
J. Comput. Civ. Eng., 2019, 33(3): 04019018
analysis, artificial neural networks, and support vector machines
(SVMs) (Trtnik et al. 2009; Atici 2011; Wang et al. 2014; Shih
et al. 2015; Ju et al. 2017; Rashid and Waqas 2017). However,
all these methods require expensive equipment and instrument
maintenance as well as trained and certified personnel with a high
degree of skill.
In recent years, a number of image processing techniques have
been proposed for estimating the compressive strength of concrete
as an alternative to the relatively costly nondestructive test methods
described previously. Images of the surface of the concrete contain
information on the spatial structure and content of its components,
which govern the compressive strength of the concrete. Başyiğit
et al. (2012) performed regression analyses (linear, multilinear, and
nonlinear) to estimate the compressive strength of concrete based
on image processing values obtained from the surface images of
the concrete specimens using a digital camera, whereas Dogan
et al. (2017) estimated concrete compressive strength using image
processing and artificial neural networks. However, both these
studies captured concrete images in ideal laboratory environments
such as photo-shooting tents and cabins. Under a fixed light inten-
sity, the digital camera was held above each sample and an image of
the entire surface of the concrete specimen was captured from the
same height. These very restricted image capture methods clearly
suffer from serious practical limitations when applied to actual
structures on site as an alternative to existing nondestructive test
methods. Moreover, both studies used the statistical properties
(arithmetic mean, standard deviation, and median values) extracted
from a gray-level histogram diagram as inputs. Input features sig-
nificantly influence subsequent estimates of the concrete’s com-
pressive strength and it is very possible that the manually defined
features used in previous studies may lose much of the spatial struc-
ture and component content. In an attempt to address these short-
comings, this study used DCNNs to avoid the need for manual
feature identification because they can be used to extract the fea-
tures from the images directly, thus facilitating concrete compres-
sive strength estimations.
Deep Convolutional Neural Networks
A deep convolutional neural network is a deep learning algorithm
that is designed to process data that comes in the form of multiple
arrays, making it feasible to extract relevant features even in the
presence of noise, shifting, rescaling, and other types of data dis-
tortions (LeCun et al. 1998). DCNNs consist of three types of layers,
namely convolution, pooling, and fully connected layers (Fig. 1).
The general function of a DCNN includes feature extraction, clas-
sification, and regression. For feature extraction, the convolution
and pooling layers are stacked to transform the raw data into a rep-
resentation at a higher level. Fully connected layers are then used to
classify the transformed representation into a specific class. DCNNs
can learn features autonomously by updating the weights of recep-
tive fields (Cha et al. 2017), contributing to major advances in object
detection and recognition in the computer-vision domain. Over the
past few years, several effective DCNN models have been proposed.
Among these, three different DCNN models, namely AlexNet,
GoogLeNet, and ResNet, all of which have won the ImageNet Large
Scale Visual Recognition Challenge (ILSVRC), were selected for
this study.
AlexNet
AlexNet, developed by Krizhevsky et al. (2012), exhibited signifi-
cantly better performance than the other non-deep learning meth-
ods at ILSVRC 2012. The main innovation of AlexNet lies in the
way it uses rectified linear units (ReLU) as the activation function,
which is normally associated with the principle of incentive neuron
J. Comput. Civ. Eng.
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.

Fig. 1. DCNN architecture.

Fig. 2. Architecture of AlexNet.

signaling. Compared with the nonlinear function sigmoid, this sim- ResNet
ple linear activation function achieves better and faster training The residual neural network (ResNet), developed by He et al.
under large data conditions. Dropout and data augmentation are (2016), which won ILSVRC 2015, is a 152-layer deep convolu-
used to prevent overfitting, thus reducing the complex interadapta- tional neural network. It was inspired by the idea that networks
tion relationship of neurons and enhancing the robustness of the should perform better as they grow in depth, as demonstrated by
model. AlexNet is composed of five convolutional layers (CONV) GoogLeNet. ResNet uses a residual network in order to deal with
and three fully connected layers (FC) (Fig. 2). The first convolu- the degradation problem and uses deeper networks to solve com-
tional layer, CONV 1, has 96 kernels of size 11 × 11 × 3; CONV 2 plicated problems. The residual network is composed of residual
has a size of 55 × 55 × 96, which represents the result of CONV 1, learning building blocks (Fig. 4); HðxÞ is the originally expected
and contains 356 kernels of size 5 × 5 × 96; CONV 3 is composed mapping output of a certain layer, and x is the input. The use of
of 38 kernels of size 3 × 3 × 256; and CONV 4 and CONV 5 have shortcut connections means that a self-mapping operation in the
384 and 256 kernels, respectively, of size 3 × 3 × 384. The results network is equivalent to opening a channel from the input side,
from each convolution layer are expressed in ReLU, and CONV 1, so that the input can go straight to the output. The optimization
CONV 2 and CONV 5 have a max 3 × 3 pooling size. CONV 1 and target then changes from HðxÞ to HðxÞ − x, and for an opti-
CONV 2 also apply local response normalization (LRN) to the mized mapping its residuals can also be easily optimized to 0. This
result of the max pooling. The FC 6, FC 7, and FC 8 stages follow- means that a residual network solves the degradation problem and
ing the convolution layers have 4,096, 4,096, and 1,000 neurons, reduces the difficulty of optimizing the parameters of a deep net-
respectively. work. Shortcut connections can improve the recognition accuracy,
and the resulting reduction in network complexity is a major
GoogLeNet advantage of using a residual network.
GoogLeNet, which won ILSVRC 2014, was developed by Szegedy
et al. (2015) and is a 22-layer deep convolutional neural network
architecture based on nine Inception modules (Fig. 3). The salient
feature of GoogLeNet is that it not only increases the depth of the
network, but also broadens the network width without increasing
the amount of computation required. GoogLeNet can extract fea-
tures from different scales at the same time to enhance its learning
ability. The inclusion of the Inception module means that although
GoogLeNet has 12 times fewer parameters than AlexNet, its accu-
racy is higher. The Inception module consists of parallel 1 × 1,
3 × 3, and 5 × 5 convolution layers and a max pooling layer to ex-
tract a variety of features in parallel; 1 × 1 convolution layers are
then added to reduce the parameter quantity and accelerate the cal-
culation. Finally, a filter concatenation layer links the outputs of all
Fig. 3. Inception module of GoogLeNet.
these parallel layers.

© ASCE 04019018-3 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2019, 33(3): 04019018


Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.
Fig. 4. Residual learning module of ResNet.
Methodology and Experimental Settings
Dataset Creation
To estimate the compressive strength of concrete using a DCNN, it
is first necessary to create a dataset that can be used to train, val-
idate, and test the DCNN model. A suitable dataset was therefore
created for this study using the four-step process in Fig. 5.
The first step was to produce the concrete specimens. Cylindri-
cal concrete specimens with dimensions of Ø100 × 200 mm were
fabricated by mixing, curing, and polishing the samples in a labo-
ratory environment. The compressive strength of concrete is deter-
mined by its curing age as well as its water:cement ratio (Baygin
et al. 2018). In general, increasing the water:cement ratio reduces
the compressive strength of the concrete. In this study, ordinary
portland cement (OPC) with water:cement ratios of 0.68, 0.50,
and 0.33 was used to provide an appropriate range of concrete com-
pressive strengths (Table 1). A total of 27 specimens were prepared,
with 3 samples aged for 3, 7, and 28 days for each water:cement
ratio.
The second step was to capture the concrete images. We used a
portable digital microscope to capture images of the upper and
lower sides of each concrete specimen, both of which had flat
Fig. 5. Dataset creation process.
© ASCE 04019018-4
J. Comput. Civ. Eng., 2019, 33(3): 04019018
surfaces. The portable digital microscope used in this study
recorded images composed of approximately 2 million pixels
(1,920 × 1,080) with a resolution of approximately 5,400 dots
per in. (dpi). Because a 9- × 5-mm region can be photographed
by the portable digital microscope, an image of a portion of
the entire concrete specimen surface (Ø100 × 200 mm) was cap-
tured, unlike previous studies in which the entire surface of con-
crete was captured simultaneously using a digital camera. Using a
portable digital microscope allowed us to capture more-detailed
images of the microstructure features associated with the compres-
sive strength of concrete and also enabled us to acquire multiple
different images for each specimen. In particular, the images were
captured without the need for any special environmental settings
such as a photo-shooting tent or cabin to facilitate the proposed
image capture method in a realistic environment such as those
found on site. To increase the robustness of the estimation, con-
crete images were taken under a range of environmental condi-
tions, including different illumination levels (under natural,
direct, and indirect lighting), different photographers, and different
portable digital microscopes, albeit of the same specification.
Between 150 and 200 photos were taken of each specimen, and
a video record of the specimens was made using the same portable
digital microscope and the same settings as those used for the pho-
tographs in order to collect as many images of the concrete spec-
imens as possible.
The third step was to perform a concrete compressive strength
test based on the provisions of KS F 2405 Korean Standard Asso-
ciation (2010) using a 200-ton universal mechanical tester (UMT).
A concrete compressive strength test was conducted for each of the
selected curing ages of 3, 7, and 28 days and for each water:cement
ratio. The test results therefore consisted of 27 performance values
ranging from 8.89 to 41.48 MPa (Table 2), and the values obtained
indicated that the concrete samples constructed for this study con-
sisted of low- to normal-strength concrete.
The final step was to construct a concrete image dataset. DCNN
models require labeled information for all data because they are
supervised learning models. In the labeling process, the images
of the concrete samples collected prior to the compressive test were
used as the input and the results of the compressive strength test
J. Comput. Civ. Eng.
 Table 1. Mix proportions for concrete samples excludes duplicate values of the same image through the pooling
Water: Sand: Fine Corse Air and convolutional layers and self-trains the features from training
Mix cement aggregate Water Cement aggregate aggregate content data. In this respect, the proposed estimation model using DCNN
type (%) (%) (L=m3 ) (kg=m3 ) (kg=m3 ) (kg=m3 ) (%) has a decided advantage for detecting features from complex con-
Mix 1 68 51 170 250 932 944 4.5 crete surface images.
Mix 2 50 48 165 330 852 973 4.5 This study used three different DCNN models, AlexNet, Goog-
Mix 3 33 45 160 480 749 965 4.5 LeNet, and ResNet, for estimating concrete compressive strength.
Each DCNN model has different convolution kernel sizes. AlexNet
uses relatively large convolution kernels, such as 11 × 11, 5 × 5,
and 3 × 3, and 8 layers; whereas GoogLeNet uses an Inception
Table 2. Compressive strength test results (MPa) module consisting of parallel 1 × 1, 3 × 3, and 5 × 5 convolution
Experimental compressive strength kernels and 22 layers; and ResNet uses a 3 × 3 convolution kernel
Water:cement for all 50 layers to ensure that the receptive field provides adequate
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.

(%) 3 days 7 days 28 days

cover across the entire input size (Luo et al. 2016). For the ILSVRC
68 10.29 10.57 17.09 competition, these DCNN models were trained on an ImageNet
8.89 10.84 18.83 dataset containing hundreds of thousands of images with 1,000
9.21 10.29 18.70 classes for image classification. However, because the purpose of
50 16.67 18.41 28.29 this study was to estimate concrete compressive strength using
17.29 17.73 27.97 concrete surface images, the final output layers of AlexNet, Goog-
16.77 18.08 20.27 LeNet, and ResNet were modified to use a Euclidean loss function
33 25.96 34.99 40.59 instead of the Softmax function. Weights of three different DCNN
26.28 30.58 38.99 models were learned such that the loss function was minimized by
27.58 33.83 41.48 using a back propagation algorithm.
The three DCNN models in this study all used rectified linear
units for the nonlinear activation functions associated with the input
and output of the convolution layers and the fully connected layers.
were used as the output. Overall, 5,145 concrete image datasets Dropout was applied to fully connected layers to minimize over-
from the photographs and 299,291 concrete image datasets from fitting, in which the neural network is overly adaptive to the train-
individual frames of the video recording were acquired. ing dataset and thus cannot properly respond to validation and
testing datasets. Dropout minimizes any overfitting by selecting
and learning a part of the neural network randomly without learning
Estimation Model
the entire neural network. This study also used data augmentation
This study applied DCNN to autonomously extract features in to prevent overfitting. Data augmentation increases the cardinality
the hidden layers of deep neural networks, unlike previous stud- of the training set for all the classes, overcomes the problem of
ies using manually extracted features from the concrete images overfitting, and reduces any divergence from the test dataset by
(e.g., Başyiğit et al. 2012; Dogan et al. 2017). As mentioned pre- adding random noise or translation to the training dataset because
viously, DCNN exhibits good performance in classification and rec- the training dataset differs from the test dataset in reality. We used
ognition, especially in the case of images used directly as the input both random cropping and horizontal flipping (Fig. 6). Specifi-
of the neural network. DCNN has the capacity to learn features cally, an image with an original size of 1,920 × 1,080 was scaled
through weight sharing and convolution regardless of the image to 112 × 112, and a random seed was generated in the 18 × 18
coordinates, giving it a robust performance in terms of translation segment in the upper left of the image. An 84 × 84 portion of the
invariance. It is also well suited to image analysis because it image was then selected using the random seed as the upper-right

Fig. 6. Examples of data augmentation: (a) random cropping; and (b) horizontal flipping.

© ASCE 04019018-5 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2019, 33(3): 04019018

 coordinate. The resulting 84 × 84 image could also be rotated 180°
about the x-axis.
Performance Evaluation Measures
The estimation accuracies of the three DCNN models created
for this study were evaluated and compared using three frequently
used performance measurements: R2 , RMSE, and MAPE. The
R-squared coefficient, R2 , is a measure of how well the independent
variables being considered account for the measured dependent
variable; the higher the R-squared value, the better the estimation
power. Root-mean-square error is the square root of the mean
square error and is thus a measure of the average distance of a data
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.
point from the fitted line measured along a vertical line. RMSE rep-
resents the absolute value difference between the experimental com-
pressive strength and the estimated compressive strength (i.e., the
amount of error). The mean absolute percentage error is a statistical
measure of estimation accuracy and expresses the error between the
experimental compressive strength and the estimated compressive
strength as a percentage. MAPE is commonly used in quantitative
forecasting methods because it indicates the relative overall fit. The
three measures are given by the following equations:
P P P
2 ðn i yi yi0 − i yi0 i yi Þ2
R ¼ P 02 P P P ð1Þ
ðn i yi − ð i yi0 Þ2 Þðn i y2i − ð i yi Þ2 Þ
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
1X
RMSE ¼ ðy − yi0 Þ2
n i i
 
1 X yi − yi0 
MAPE ¼ × 100
n i  yi 
where yi and yi0 = experimental compressive strength and estimated
compressive strength, respectively; and n = number of data samples.
Experimental Settings
When recording the video of the concrete images with a portable
digital microscope, lens shake tends to produce out-of-focus im-
ages. This was also the case here, with blurred concrete images
being acquired from individual frames of the video recording. If
blurred concrete images are used in training, it may be difficult
to get good accuracy. We therefore used blurred concrete images
for pretraining before conducting the training, validation, and test-
ing. In pretraining, the amount of computation required for learning
may increase, and a degree of overfitting may be generated because
the number of training data images is less than the number of
parameters to be learned. To prevent overfitting, data augmentation
is required in such cases, and here it was conducted via minibatch
learning using a set of 52 images, consisting of a combination of 48
photographic images and 4 images from the video recording, even
where the learning process used photographic images only. In other
words, images from the video recording were added to the photo-
graphic images for the minibatch learning every time to augment
the data.
A total of 5,145 datasets were used for training, validation, and
testing. To present the generalized performance fairly, the DCNN
models were learned as the training dataset and selected as the val-
idation dataset, and the estimation error was measured by the test
dataset. The training, validation, and testing datasets were classified
into 3,601 (70%), 515 (10%), and 1,029 (20%) images, respec-
tively. The initial learning rate, weight decay, parameters of mo-
mentum, and dropout rate were set to 0.01, 0.0005, 0.9, and 0.5,
respectively. The deep learning framework CAFFE version 1.0
© ASCE
J. Comput. Civ. Eng., 2019, 33(3): 04019018
(Jia et al. 2014) was used, and the learning was conducted using
a workstation equipped with four GPUs (CPU: Intel Xeon E5-2620
v4 @2.1GHz; RAM: 64GB; and GPU: Nvidia GTX 1080Ti × 4).
Once the three customized DCNN models (AlexNet, GoogLeNet,
and ResNet) had been created, the training operation was repeated
80 million times (number of iterations) in order to obtain the opti-
mum structure. The resulting training loss and validation curves for
each DCNN model all tended to converge, showing that all three
networks achieved an excellent fitting performance for both the
training and validation sets.
Results and Discussion
Experimental Results
DCNN algorithms autonomously learn features through data-
intensive analysis, whereas traditional machine learning algorithms
such as ANN rely on hand-engineered features based on domain
knowledge. Because it is important to construct an appropriate
model architecture that will enable the model itself to learn the fea-
tures well, this study applied AlextNet-, GoogLeNet-, and ResNet-
based models to estimate the compressive strength of the concrete
samples. The resulting dataset was then used to compare the results
of these DCNN models with the results of the image processing–
based ANN model recently proposed by Dogan et al. (2017).
Following Dogan et al. (2017), we extracted the features from the
ð2Þ
image by applying appropriate statistical properties (arithmetic
mean, standard deviation, and median) to develop an ANN model
that estimates concrete compressive strength using the extracted
ð3Þ statistical properties as inputs. Table 3 summarizes the R2 , RMSE,
and MAPE results for the three different DCNN models and the
image processing–based ANN model. As noted previously, R2 is a
measure of how well the independent variables approximate the
estimated dependent variable, whereas RMSE and MAPE are used
as a measure of the differences between the values estimated by the
models. High R2 values and low values of RMSE and MAPE are
generally indicative of good performance. The results showed that
three DCNN models largely outperformed the image processing–
based ANN model on all three indicators.
The best model for determining R2 among the DCNN models
was found to be the ResNet-based model (R2 ¼ 0.764), whereas
the AlexNet-based model exhibited the worst estimation capabil-
ities (R2 ¼ 0.745). This is shown in the plots of the relationships
between the experimental and estimated compressive strength ob-
tained by the three DCNN models (Fig. 7). Table 3 also gives a
direct relationship between R2 and RMSE; the best model for min-
imizing RMSE was again the ResNet-based model (4.46 MPa),
and the worst was the AlexNet-based model (4.64 MPa). However,
the ranking of the three models was not the same for MAPE. Here,
the best result was obtained for the AlexNet-based model (17.67%),
and the worst was the GoogLeNet-based model (18.40%). Figs. 8(a
and b) show the RMSE and MAPE values, respectively, for the
Table 3. Performance comparison of DCNN models and image
processing–based ANN model
DCNN models Image
processing–based
Performance AlexNet GoogLeNet ResNet ANN model
R2 0.745 0.748 0.764 0.200
RMSE (MPa) 4.641 4.612 4.463 8.223
MAPE (%) 17.675 18.403 17.765 38.039
04019018-6 J. Comput. Civ. Eng.
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.
Fig. 7. Experimental versus estimated compressive strength: (a) AlexNet-based model; (b) GoogLeNet-based model; and (c) ResNet-based model.
three DCNN models in terms of the actual experimental compres-
sive strength. In the case of RMSE, the maximum values esti-
mated by the AlexNet-based model, GoogLeNet-based model, and
ResNet-based model were 7,82, 7.03, and 7.75 MPa, respectively,
and the minimum values were 1.70, 2.54, and 1.60 MPa, respec-
tively. For MAPE, the AlexNet-, GoogLeNet-, and ResNet-based
models achieved maximum values of 51.62%, 46.90%, and 45.57%,
respectively, for an experimental compressive strength value of
10.84 MPa. The minimum values estimated were 6.30%, 6.97%,
and 6.67%, respectively.
We also examined the estimation errors, namely the value dif-
ference between the experimental compressive strength and the
estimated strength value. The lowest experimental compressive
strength was estimated to be lower than the value actually measured,
and the highest compressive strength was estimated to be higher
than the observed experimental compressive strength (Fig. 9).
The AlexNet-, GoogLeNet-, and ResNet-based models had maxi-
mum error values of 23.68, 22.48, and 18.4 MPa, respectively,
and minimum error values of −24.59, −21.13, and −17.58 MPa,
respectively. The ResNet model (35.98 MPa) had the smallest differ-
ence between the maximum and minimum errors, and the AlexNet
based model (48.27 MPa) had the largest difference. We also ex-
plored the frequency distribution of the estimation error rate for
the compressive strength (Fig. 10). Here, the error rate was divided
into 5% intervals and the frequency and ratio of the error rate was
analyzed for each of the DCNN models. Overall, an average of 83%
of the total data was estimated to have an error rate of less than 30%.
The model with the lowest error rate was the AlexNet-based model,
© ASCE 04019018-7
J. Comput. Civ. Eng., 2019, 33(3): 04019018
whereas the worst performing model was the GoogLeNet-based
model. The AlexNet-based model estimated 83.7% of the total data
with an error rate of less than 30%, and only 3.98% of the total data
had an error rate of over 50%. Even the worst performing model
turned in a reasonable performance: the GoogLeNet-based model
estimated 81.3% of the total data with an error rate of less than
30%, and 6.6% with an error rate of over 50%. The analysis of the
compressive strength estimation accuracy revealed that the estima-
tion accuracy of each of the DCNN models was slightly different
depending on the evaluation criteria applied, although overall the
ResNet-based model had excellent compressive strength estimation
accuracy. Fig. 11 shows examples of good and bad results for the
ResNet-based model.
Discussion
This study investigated the applicability of estimating concrete
compressive strength using microstructure images analyzed with
DCNN. The thickness of the interfacial transition zone (ITZ), and
the existence of microcracks and pores largely determine the com-
pressive strength of concrete structures. This means that there is
a relationship between the concrete microstructure images and
compressive strength. Therefore, this study attempted to estimate
compressive strength by examining images of the concrete micro-
structure. To obtain these images of the concrete surface, we used a
portable digital microscope capable of recording images com-
posed of approximately 2 million pixels (1,920 × 1,080) with a res-
olution of approximately 5,400 dpi. Because each image captured a
J. Comput. Civ. Eng.
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.

Fig. 8. RMSE and MAPE for each experimental compressive strength: (a) RMSE; and (b) MAPE.

9 × 5 mm region, meaning that each pixel represented 4.66 μm

of the surface, it was possible to observe concrete microstructure
components such as the matrix, aggregate, pores, and ITZ with the
portable digital microscope. However, Because it is difficult to
manually extract features affecting compressive strength, as Fig. 11
demonstrates, this study applied DCNN models to autonomously
extract complex discriminative features through a learning pro-
cedure. The experimental results showed that the proposed DCNN
models were indeed capable of providing good estimates of the
compressive strength using concrete microstructure images. This
indicates that the proposed DCNN model learned the relation-
ship between the concrete microstructure images and compres-
sive strength, thus confirming that the concrete images obtained
using a portable digital microscope include patterns can be used
to estimate the compressive strength of the concrete samples
and that the DCNN models developed for this study can learn these
patterns.
The ultimate goal of this study was to provide a new image-
based method for estimating concrete compressive strength as an
auxiliary method for nondestructive testing. It is therefore impor-
tant to be able to apply the proposed compressive strength estima-
tion method on site to examine an actual structure. The method
proposed here facilitates the capture of images from actual concrete
structures because the images of the concrete surfaces are collected
without any particular constraints on the environment, in contrast to
the methods used in previous studies, which used concrete images
Fig. 9. Errors in the estimated compressive strength: (a) AlexNet-
obtained in an ideal laboratory environment. This study used a port-
based model; (b) GoogLeNet-based model; and (c) ResNet-based
able digital microscope that enables users to capture images of ac-
model.
tual concrete structures safely and conveniently. A portable digital

© ASCE 04019018-8 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2019, 33(3): 04019018

Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.

Fig. 10. Frequency distribution of error rate: (a) AlexNet-based model; (b) GoogLeNet-based model; and (c) ResNet-based model.

Fig. 11. Examples of good and bad results for ResNet-based model: (a) good results; and (b) bad results.

microscope is easy to use in the field because the microscope cam-
era attaches directly to the USB port of a computer. Eyepieces are
not required and the images are shown directly on the monitor.
We acquired multiple microstructure images for the same con-
crete specimen with the same compressive strength because a port-
able digital microscope can only capture a relatively small part of a
large concrete specimen surface in each image. This actually im-
proves the estimation accuracy because DCNN models can be ef-
fectively trained by large datasets (Halevy et al. 2009). However,
the range of the dataset created for this study is relatively small and
the opportunity to learn various types of images is therefore very
limited. If a new image with a compressive strength that is outside
the range of the dataset created for this study is tested, the results
obtained will not be an accurate estimate of the material’s compres-
sive strength. In future studies, datasets that include concrete spec-
imens with a greater range of compressive strength should be
created. Nevertheless, the results of this preliminary study are sig-
nificant in that our findings demonstrate the potential utility of our
proposed method and confirm the applicability of the proposed
model as an auxiliary to existing nondestructive methods.

© ASCE 04019018-9 J. Comput. Civ. Eng.

J. Comput. Civ. Eng., 2019, 33(3): 04019018

 Conclusions
Concrete compressive strength, which is considered to determine
the quality of concrete, is a crucial indicator in facility manage-
ment. This study proposed a new model for estimating concrete’s
compressive strength that uses DCNNs to analyze images captured
with a portable digital microscope, and investigated its applicabil-
ity. The experimental results obtained showed that all three DCNN
models developed for this study provide a better performance than
the recently proposed image processing–based ANN model, and
overall, the ResNet-based model in particular demonstrated excel-
lent compressive strength estimation accuracy, outperforming both
the AlexNet- and GoogLeNet-based models. These findings con-
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.
firm that the concrete images obtained using a portable digital mi-
croscope do indeed include patterns that can be used to estimate the
compressive strength of the concrete samples and that the DCNN
model is capable of learning these patterns. Concrete compressive
strength can indeed be estimated by using DCNNs to analyze mi-
crostructure images and the image capture method proposed in this
study is easy to apply to actual concrete structures because it uses a
portable digital microscope that can conveniently capture images of
the concrete surface with few, if any, constraints. This makes our
proposed method an attractive alternative to existing nondestructive
testing methods.
The next step in developing this promising new approach will be
to extend the range of compressive strengths included in the analy-
sis, because the range of the dataset created in this preliminary
study was very limited. We created a dataset consisting of just 27
compressive strength values, ranging from 8.89 to 41.48 MPa,
which correspond to low-normal-strength concrete, for this study.
To improve the applicability of this approach, images of a variety of
high-normal-strength concrete mixes must be included for training
and testing. The dataset created using concrete specimens produced
in a laboratory environment should also be supplemented with mea-
surements of actual concrete structures in the field in order to pro-
vide better estimates of the in situ concrete strength. Using the
method proposed in this study, an extensive dataset should be con-
structed by taking images of actual concrete structures and using the
compressive strength values from samples collected from core drill-
ing. Once a sufficiently large dataset has been constructed, it is ex-
pected that immediate estimates of concrete compressive strength in
the field will become possible simply by capturing images of the
concrete surfaces.
Acknowledgments
This research was supported by a grant (18CATP-C129782-02)
from the Technology Advancement Research Program funded by
the Korean Ministry of Land, Infrastructure and Transport.
References
Atici, U. 2011. “Prediction of the strength of mineral admixture concrete
using multivariable regression analysis and an artificial neural network.”
Expert Syst. Appl. 38 (8): 9609–9618. https://doi.org/10.1016/j.eswa
.2011.01.156.
Başyiğit, C., B. Çomak, S. Kilinçarslan, and I. S. Üncü. 2012. “Assessment
of concrete compressive strength by image processing technique.” Const.
Build. Mater. 37: 526–532. https://doi.org/10.1016/j.conbuildmat.2012
.07.055.
Baygin, M., S. G. Ozkaya, M. A. Ozdemir, and I. Kazaz. 2018. “A new
approach based on image processing for measuring compressive
strength of structures.” Int. J. Intell. Syst. Appl. Eng. 6 (4): 21–25.
© ASCE 04019018-10
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Cha, Y., W. Choi, and O. Buyukozturk. 2017. “Deep learning-based crack
damage detection using convolutional neural networks.” Comput.
-Aided Civ. Infrastruct. Eng. 32 (5): 361–378. https://doi.org/10
.1111/mice.12263.
Derman, E., and A. A. Salah. 2018. “Continuous real-time vehicle driver
authentication using convolutional neural network based face recogni-
tion.” In Proc., 13th IEEE Int. Conf. on Automatic Face and Gesture
Recognition, 577–584. Washington, DC: IEEE Computer Society Press.
Dogan, G., M. H. Arslan, and M. Ceylan. 2017. “Concrete compressive
strength detection using image processing based new test method.”
Measurement 109: 137–148. https://doi.org/10.1016/j.measurement
.2017.05.051.
Ferreira, M. D., D. C. Correa, L. G. Nonato, and R. F. de Mello. 2018.
“Designing architectures of convolutional neural networks to solve
practical problems.” Expert Syst. Appl. 94: 205–217. https://doi.org/10
.1016/j.eswa.2017.10.052.
Gopalakrishnan, K., S. K. Khaitan, A. Choudhary, and A. Agrawal. 2017.
“Deep convolutional neural networks with transfer for computer vision-
based data-driven pavement distress detection.” Constr. Build. Mater.
157: 322–330. https://doi.org/10.1016/j.conbuildmat.2017.09.110.
Halevy, A., P. Norvig, and F. Pereira. 2009. “The unreasonable effective-
ness of data.” IEEE Intell. Syst. 24 (2): 8–12.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for image
recognition.” In Proc., IEEE Conf. on Computer Vision and Pattern
Recognition, 770–778. Washington, DC: IEEE Computer Society Press.
Jia, Y., E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick,
S. Guadarrama, and T. Darrell. 2014. “Caffe: Convolutional architecture
for fast feature embedding.” In Proc., 22nd ACM Int. Conf. on
Multimedia, 675–678. New York: ACM.
Ju, M., K. Park, and H. Oh. 2017. “Estimation of compressive strength
of high strength concrete using non-destructive technique and con-
crete core strength.” Appl. Sci. 7 (12): 1249. https://doi.org/10.3390
/app7121249.
Korean Standard Association. 2010. Standard test method for compressive
strength of concrete. KS F 2405. Seoul, South Korea: Korean Standard
Association.
Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2012. “ImageNet classifi-
cation with deep convolutional neural networks.” Adv. Neural Inf.
Process. 25 (2): 1097–1105.
LeCun, Y., L. Bottou, Y. Bengio, and P. Haffner. 1998. “Gradient-based
learning applied to document recognition.” Proc. IEEE 86 (11):
2278–2324. https://doi.org/10.1109/5.726791.
Lin, Y. Z., Z. H. Nie, and H. W. Ma. 2017. “Structural damage detection
with automatic feature-extraction through deep learning.” Comput.
-Aided Civ. Infrastruct. Eng. 32 (12): 1025–1046. https://doi.org/10
.1111/mice.12313.
Luo, W., Y. Li, R. Urtasun, and R. Zemel. 2016. “Understanding the
effective receptive field in deep convolutional neural networks.”
In Proc., 29th Conf. on Neural Information Processing Systems,
4905–4913. Red Hook, NY: Curran Associates.
Nugraha, B. T., S. F. Su, and Fahmizal. 2017. “Towards self-driving car
using convolutional neural network and road lane detector.” In Proc.,
2nd Int. Conf. on Automation, Cognitive Science, Optics, Micro
Electro-Mechanical System, and Information Technology, 65–69.
Red Hook, NY: Curran Associates.
Rashid, K., and R. Waqas. 2017. “Compressive strength evaluation by
non-destructive techniques: An automated approach in construction
industry.” J. Build. Eng. 12: 147–154. https://doi.org/10.1016/j.jobe
.2017.05.010.
Shih, Y. F., Y. R. Wang, K. L. Lin, and C. W. Chen. 2015. “Improving
non-destructive concrete strength tests using support vector machines.”
Materials 8 (10): 7169–7178. https://doi.org/10.3390/ma8105368.
Steenbergen, R. D. J. M., and A. H. J. M. Vervuurt. 2012. “Determining the
in situ concrete strength of existing structures for assessing their struc-
tural safety.” Struct. Concr. 13 (1): 27–31. https://doi.org/10.1002/suco
.201100031.
Sun, W., T. L. Tseng, J. Zhang, and W. Qian. 2017. “Enhancing deep
convolutional neural network scheme for breast cancer diagnosis with
unlabeled data.” Computerized Med. Imaging Graphics 57: 4–9. https://
doi.org/10.1016/j.compmedimag.2016.07.004.
J. Comput. Civ. Eng.
 Szegedy, C., W. Liu, and Y. Jia. 2015. “Going deeper with convolution.”
In Proc., IEEE Conf. on Computer Vision and Pattern Recognition,
1–9, Washington, DC: IEEE Computer Society Press.
Tiberti, G., F. Minelli, and G. Plizzari. 2015. “Cracking behavior in rein-
forced concrete members with steel fibers: A comprehensive experi-
mental study.” Cem. Concr. Res. 68: 24–34. https://doi.org/10.1016/j
.cemconres.2014.10.011.
Trtnik, G., F. Kavčič, and G. Turk. 2009. “Prediction of concrete
strength using ultrasonic pulse velocity and artificial neural networks.”
Downloaded from ascelibrary.org by HANYANG UNIVERSITY on 03/01/19. Copyright ASCE. For personal use only; all rights reserved.
Ultrasonics 49 (1): 53–60. https://doi.org/10.1016/j.ultras.2008
.05.001.
Wang, Y. R., W. T. Kuo, S. S. Lu, Y. F. Shih, and S. S. Wei. 2014. “Applying
support vector machines in rebound hammer test.” Adv. Mater. Res.
853: 600–604. https://doi.org/10.4028/www.scientific.net/AMR.853.600.
Zhang, A., et al. 2017. “Automated pixel-level pavement crack detection on
3D asphalt surfaces using a deep-learning network.” Comput. -Aided
Civ. Infrastruct. Eng. 32 (10): 805–819. https://doi.org/10.1111/mice
.12297.

© ASCE 04019018-11 J. Comput. Civ. Eng.

View publication stats J. Comput. Civ. Eng., 2019, 33(3): 04019018