2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
17-20 October, 2021. Melbourne, Australia
A Robust Completed Local Binary Pattern (RCLBP) for Surface Defect
Detection
2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC) | 978-1-6654-4207-7/21/$31.00 ©2021 IEEE | DOI: 10.1109/SMC52423.2021.9659140
Nana Kankam Gyimah∗ , Abenezer Girma∗ , Mahmoud Nabil Mahmoud∗ ,
Shamila Nateghi∗ , Abdollah Homaifar∗ , Daniel Opoku+
∗ North Carolina A&T State University, Greensboro, North Carolina, US, 27411
+ Kwame Nkrumah University of Science & Technology, Kumasi, Ghana
Abstract— In this paper, we present a Robust Completed
Local Binary Pattern (RCLBP) framework for a surface defect
detection task. Our approach uses a combination of Non-Local
(NL) means filter with wavelet thresholding and Completed
Local Binary Pattern (CLBP) to extract robust features which
are fed into classifiers for surface defects detection. This
paper combines three components: A denoising technique based
on Non-Local (NL) means filter with wavelet thresholding
is established to denoise the noisy image while preserving
the textures and edges. Second, discriminative features are
extracted using the CLBP technique. Finally, the discriminative
features are fed into the classifiers to build the detection model
and evaluate the performance of the proposed framework. The
performance of the defect detection models are evaluated using
a real-world steel surface defect database from Northeastern
University (NEU). Experimental results demonstrate that the
proposed approach RCLBP is noise robust and can be applied
for surface defect detection under varying conditions of intra-
class and inter-class changes and with illumination changes.
Index Terms - Surface defect, Intra-class defect differ-
ences, Inter-class defect similarities, Non-local means filter
with wavelet thresholding, Completed Local Binary Pattern
(CLBP)
I. I NTRODUCTION
Metal planar materials (steel, aluminum, copper plates,
and strips) are widely used in aerospace, automobile man-
ufacturing, bridge construction, and other pillar industries.
These industries have made immense contributions to mod-
ern social development and the improvement of life [1]. The
occurrence of surface defects (corrosion, cracks, scratches or
dents) on these metal planar materials during the manufac-
turing process and use of these industrial products can cause
huge economic losses when these defects are left undetected
[2].
These surface defects have traditionally been detected
by professionally trained human inspectors [3] who judge
the quality of the defect by observing and inspecting the
differences of the surface appearance of the product with
the naked eyes. This method however does not meet the
needs of modern industrial production since the criteria for
human vision is not quantified but rather dependent on the
subjective evaluation of the human inspectors. The quality
Corresponding author: Tel.: +1-336-285-3271; fax: +1-336-334-7716; e-
mail: homaifar@ncat.edu (Abdollah Homaifar)
978-1-6654-4207-7/21/$31.00 ©2021 IEEE
Authorized licensed use limited to: Zhengzhou University. Downloaded on January 01,2023 at 16:18:18 UTC from IEEE Xplore. Restrictions apply.
of this inspection is not fully guaranteed and less efficient
since human inspectors can experience ocular fatigue due to
the high intensive and repetitive nature of work which leads
to less reliable defect detection.
Automatic Visual Inspection System (AVIS), however,
has gained greater acceptance with the development of
automated Computer Vision (CV) based inspection [4]. CV
based inspection methods have significantly increased the
defect detection speed, reduced labor costs, and improved the
quality of inspection through effective damage localization
[5]. CV based inspection have improved the productivity [5]
of defect detection as they provide a competitive advantage to
the traditional human inspection process for detecting surface
defects.
The CV-based surface defect detection however has three
main challenges which tends to diminish the recognition
performance of defect detection approaches and affects the
stability of the extracted defect features as outlined below:
1) Distinguishing between the inter-class defect similarities
and intra-class defect differences. Inter-class defects
similarities refer to different defect classes that appear
nearly identical in their appearance as shown between
the defect classes of rolled-in scale and crazing in Fig.
1. Intra-class defect differences refer to the defect class
whose images are random and highly diverse in their
appearance as in the defect classes of scratches and
patches in Fig. 1.
2) Background interference due to the influence of differ-
ent illumination and material changes tend to reduce
the recognition performance of the defect detection
approaches.
3) Most of the employed features, the Local Binary Pat-
tern (LBP) [6, 7] and Multi-Scale Geometric Analysis
(MGA) features [8] are vulnerable to noise which
reduces the recognition performance of these detection
approaches.
In [7], the Completed Local Binary Pattern (CLBP) was
proposed to characterize discriminant information that de-
scribes defect classes. CLBP is able to distinguish between
the interclass defects, the intraclass defects and also address
the problem of background interference due to its grayscale
and rotation invariance. However, it is sensitive to noise and
1927
 Rolled-in-scale Patches
Fig. 1: Samples of the defect classes of the Northeastern University (NEU) surface defect dataset[9]. The columns define
the defect class with the rows showing the sample images that belongs to each defect class.
has limited ability to represent miscellaneous textures. In [8],
multi-scale geometric analysis (MGA) features was proposed
which was effective for classifying surface defects. However,
it is highly sensitive to noise. In [10], LEDNet based on Con-
volutional Neural Network (CNN) was proposed to detect
and classify defects on LED chips. This method achieved
high defect detection accuracy but sensitive to noise.
To overcome the challenges in existing defect detection
approaches, this paper presents a robust defect detection
framework which aims to address the three challenges that
affects the stability of the extracted features and the recog-
nition performance of the defect detection approaches. The
contributions of this framework are as follows:
1) The denoising technique based on the combination of
Non-Linear (NL)-means filter with wavelet thresholding
is proposed as a pre-processing step to remove the noise
while retaining the input image’s textures and edges as
much as possible to address the vulnerability of these
features to noise.
2) CLBP is explored to address the intraclass defect dif-
ferences, interclass defect similarities, and background
interference challenges because of its grayscale and
rotation invariance.
3) Extensive experiments on the NEU-DET dataset demon-
strate the effectiveness of the proposed method in the
presence of additive Gaussian noise compared to Local
Binary Pattern (LBP), and Completed Local Binary
Pattern (CLBP).
The rest of this paper is organized as follows: Section
II presents related works on defect detection. Section III
describes the proposed methodology. Section IV describes
Authorized licensed use limited to: Zhengzhou University. Downloaded on January 01,2023 at 16:18:18 UTC from IEEE Xplore. Restrictions apply.
Crazing Pitted Surface Inclusion Scratches
the dataset, experiment setup, hyper-parameter settings and
the evaluation Metrics. Section V presents the experimental
results and discussion. Finally, the conclusion and future
work are provided in Section VI.
II. R ELATED W ORK
This section describes relevant literature that focuses on
defect detection methods. Automated Surface Inspection
(ASI) task has been modelled as a texture analysis problem
where the surface defects are generally described as local
anomalies in homogeneous textures. In general, surface de-
fect detection methods based on texture analysis are decom-
posed into statistical-based methods [11], spectrum-based
methods [12], model-based methods [13], and emerging
machine learning-based methods [14].
Statistical methods: For this approach, the regular and
periodic distribution of pixel strength is investigated by
measuring the statistical characteristics of pixel spatial dis-
tribution to detect the defects on the metal planar materials
surface. These methods are based on edge detection [15],
hough transform [16], gray-level statistics [17], local binary
pattern [9] and Generalized Completed LBP (GCLBP) [18].
In [15], edge detection based on eight directional Sobel
operators was utilized to detect Backfin defects. This was
robust to noise and protected the edge shape but was only
suitable to low resolution images. In [16], Hough Transform
(HT) was utilized to detect holes, scratches, coil breakage,
and corrosion on a cold-rolled steel strip. HT has strong
anti-interference ability which suppresses the influence of
noise and incomplete edges. However, the uncertainty of
the defect’s shape of most metal planar materials often
1928
leads to unsatisfactory detection. In [17], multi-directional
gray fluctuation was used to characterize multi-type defects.
This method was suitable for low-resolution images but
cannot automatically select the threshold to extract the defect
features. In [18], the Generalized Completed LBP (GCLBP)
was utilized to explore the non-uniform pattern hidden in
the uniform pattern to detect the multi-class defect types.
This method has strong anti-interference ability but it cannot
suppress noise and is also unable to simultaneously adapt to
scale variation at the same time.
Structural methods: These approaches model the texture
elements and spatial arrangements that characterize the de-
fects. These methods are based on Fourier Transform [19],
Gabor filter [20], and Wavelet Transform [21]. In [19],
a combination of fourier transform and curvelet transform
was proposed to detect longitudinal cracks. This method is
invariant to translation, expansion, and rotation. However, the
background and defect information in frequency domain can
easily be mixed to cause interference. In [20], traditional
Gabor filter was utilized to detect periodic defects. It is
suitable for high-dimensional feature space, but quite difficult
to determine the optimal filtering parameters which provides
no rotation invariance. In [21], the undecimated wavelet
transform was used to detect horizontal scratch defects. It
is suitable for multi-scale image analysis and can compress
images effectively. However, it is difficult to select the proper
wavelet base to extract the defect features.
Model-based methods: These approaches projects the
original texture distribution of the image block to the low-
dimensional distribution through the special structure model
enhanced by parameter learning to detect various defects.
These approaches are based on Markov random field [8],
visual saliency model [13] and fractal dimension model [22].
In [8], the hidden Markov tree model was utilized to detect
the multi-type defects. This method is able to reflect the
underlying structure of the image but is not suitable for
global texture analysis and small size defects. In [13], the
double low-rank and sparse decomposition was utilized to
detect multi-type defects. It is robust to noise and uneven
illuminations but limited to gradient strength or low contrast
defects. In [22], multi-fractal decomposition was used to
detect multi-type defects. The global information can be
represented by the local features but is only applicable to
images with adaptability.
Emerging machine learning-based methods: These
methods are data driven approaches that have been catego-
rized into supervised learning ([23]), unsupervised learning
([24]) and semi-supervised learning ([25]). These approaches
are effective however, they require a relatively high number
of training data and are computationally expensive.
The aforementioned methods addresses either one or two
of the challenges to detect the surface defects. To solve
these challenges, this paper presents a noise robust defect
detection framework which is rotation and grayscale invariant
for surface defect detection. The proposed framework differs
from LBP [6] and CLBP [7] through the use of the denoising
technique.
1929
Authorized licensed use limited to: Zhengzhou University. Downloaded on January 01,2023 at 16:18:18 UTC from IEEE Xplore. Restrictions apply.
III. P ROPOSED M ETHODOLOGY
The proposed framework consists of two distinct stages
as shown in Fig 2. The first stage is the denoising stage
which is a combination of Non-Local (NL) means filter with
wavelet thresholding [26] to remove noise from the images.
The second stage is the feature extraction process where
we extract the features which characterize the defect classes
using CLBP [7].
A. Denoising
The goal of image denoising is to remove the noise while
retaining the image textures and edges as much as possible.
From Fig. 2, the input image V is first denoised by the NL-
means filter [27]. The NL-means filter is effective in remov-
ing the noise at high Signal-to-Noise Ratio (SNR) (less noise)
while retaining the image’s textures and edges. However as
the noise increases (low SNR), its performance deteriorates
because it wrongly considers the image’s textures and edges
as noise and removes them from the features [26]. To
preserve these important information while removing the
noise, the difference between the input image (V ) and the
NL-means filtered image (IF ) is computed similar to the
Method Noise (MN) [26] as MN contains the input image’s
textures and edges. The MN however contains noise. To
∧
estimate the input image’s textures and edges D, the method
noise undergoes wavelet transformation to obtain subbands
Y which are thresholded with BayesShrink filter to estimate
∧
the true wavelet subbands W without noise. These denoised
wavelet subbands are subsequently reconstructed to estimate
∧
the image’s textures and edges D. The estimated image’s
∧
textures and edges D is finally fused with the NL-means
filtered image IF to generate B which is the denoised image
with restored textures and edges. The denoising framework
is as discussed in Section III-A.1.
1) Denoising Framework: Given a discrete noisy image
V = {V (i)|i ∈ I} on a discrete grid I ∈ R2 , the estimated
restored intensity NL(i) [27] for a pixel (i), is computed as
a weighted average of the pixel intensities v( j) of the image
V as defined in Eq. (1).
NL(i) = ∑ w(i, j)v( j) (1)
j∈I
where w(i, j) is the weight assigned to v( j) for restoring the
pixel i. The number of pixels ( j) in the weighted average
is restricted to a neighborhood search window Si centered at
the pixel i.
To compute the similarity between image pixels, a neigh-
borhood patch Nk of fixed square size centered at a pixel
k and within the search window Si . The weights w(i, j)
evaluates the similarity between the intensities of the local
neighborhoods centered on pixels i and j.
To compute the similarity of the intensity gray level
vectors v(Ni ) and v(N j ), we compute a Gaussian weighted
Euclidean distance, kv(Ni ) − v(N j )k22,σ , where σ > 0 is the
standard deviation of the Gaussian kernel. The Euclidean
distance is the traditional L2 -norm convolved with a Gaussian
 Sign Parameter CLBP_S
Local
Difference
Non-Local IF B CLBP Map
Input Image (NL) Means
v filter + Magnitude
Parameter
CLBP_M
Classifier
+

Center Gray CLBP

Wavelet Decomposition CLBP_C
Level Histogram

-
+ Thresholding
Wavelet
Reconstruction
MN Y

NL-means Filter with Wavelet Thresholding Completed Local Binary Pattern

Fig. 2: The Robust Completed Local Binary Pattern (RCLBP) Framework.

∧
kernel of standard deviation σ as it preserves the order of To estimate W , the denoised subbands textures from Y , the
similarity between pixels. The weights w(i, j) is computed subbands of Y are thresholded with the adaptive BayesShrink
as in Eq. 2 threshold [28] to remove the noise N by minimizing
∧
2
1 − kv(Ni ) − v(N j )k2,σ the Mean Squared Error (MSE) between W and Y. The
w(i, j) = e (2) BayesShrink threshold (T ) which is adaptive to each wavelet
Z(i) h2
sub-band yields a data driven estimate of the threshold T
where Z(i) is the normalizing factor as in Eq. 3
through soft thresholding as defined in Eq. 7 to remove the
kv(Ni ) − v(N j )k22,σ noise.
Z(i) = ∑ e− (3)
h2
Tessa Miller
(
CFO
Email
j sgn(Y )(|Y | − T ), |Y | > T σ2
Wso f t = where T = (7)
which ensures that ∑ j w(i, j) = 1 and h is the smoothing
manages
0, |Y | ≤ T σw
kernel width which controls the decay of the exponential
manages
Alison Donovan
manages
where sgn depicts the signum function, Wso f t is the wavelet
function. System Admin
Email

To estimate IF , all the noisy pixels in V are restored with coefficient after the shrinkage of the soft threshold, and σ 2 is
the estimated intensity NL(i) as defined in Eq. 4. the noise variance estimated from subband HH1 by a robust
Edward Morrison Evan Valet
median estimator [29] as defined in Eq. 8.
Brand Manager HR Director
Email
IF = {NL(i)|i ∈ I}
Email
(4)
Median(|Yi, j |) 2
The application of the NL-means filter on the noisy image σ2 = ( ) ,Yi, j ∈ HH1 (8)
0.6745
removes the noise and cleans the edges without losing too For the wavelet model given in Eq. 6, then
many textures and edges.
At low SNR, the NL-means filtered image IF considers the σY2 = σW2 + σ 2 (9)
input image’s textures and edges as noise and removes them ∧
from the features. To restore IF with its lost textures and where σY2 is the variance of Y, σW2 is the variance of W ,
edges, the method noise (MN) is computed as the difference and σ 2 is the variance of the Gaussian noise N. Since Y
between the noisy image (V ) and the NL-means filtered is modelled as zero mean, its variance σY2 can be found
image (IF ) which is defined in Eq. 5. empirically as defined in Eq. 10.
MN = V − IF (5) 1
σy2 =
Y2 (10)
MN ∑
where v is the original image, IF is the NL-means filtered
image of the input image V and MN is the Method Noise. where M and N is the size of the subband of Y under
The method noise undergoes a wavelet transformation consideration. The variance, σw2 of the wavelet sub-bands
∧
where MN is decomposed into k number of sub-bands; HHk , W is finally computed as defined in Eq. 11.
HLk , LHk , LLk where k={1, 2, · · · , J} is the scale, with J
σw2 = max(σy2 − σ 2 , 0) (11)
being the largest scale of the decomposition. The sub-bands
HHk , HLk , and LHk represents the textures and edges sub- ∧
band of MN whiles LLk represents the approximate sub-band The denoised subbands of W are wavelet reconstructed to
∧
of MN. These noisy wavelet sub-bands referred to as Y in give an estimate of the detail image D containing the lost
Fig. 2 is a combination of the sub-bands textures and some textures and edges.
Gaussian noise N as defined in Eq. 6. To estimate the denoised image with the restored textures
∧
∧ and edges B, the detailed image D from the wavelet thresh-
Y =W +N (6) olding is summed up with the NL-means filtered image IF

1930

Authorized licensed use limited to: Zhengzhou University. Downloaded on January 01,2023 at 16:18:18 UTC from IEEE Xplore. Restrictions apply.
Fig. 3: Central pixel (gc ) with its circularly and evenly spaced
neighbors P with radius R of the neighborhood [7].

as defined in Eq. 12 as it restores IF with the lost textures

and edges. Fig. 4: Surface defect images with Gaussian noise of different
SNR [9].
∧
B = IF + D (12)
B. Feature Extraction with CLBP_S and CLBP_M, CLBP_C is defined in Eq. 16
Completed Local Binary Pattern (CLBP) [7] is used to ex- to encode the center pixel (C) information.
tract the discriminative features that characterize the features (
of the denoised image B as defined in Eq. 12. It represents 1, x ≥ cI
CLBP_C = t(gc , cI ), t(x, cI ) = (16)
the local region of the denoised image as shown in Fig. 3 0, x < cI
by the local difference sign-magnitude transform (LDSMT)
where the threshold cI is set as the average gray level of the
and its center pixel as shown in Fig. 2.
denoised image.
Given a central pixel gc with P circularly and evenly
The three operator ma[s, CLBP_S, CLBP_M and CLBP_C
spaced neighbors g p , p = 0, 1, · · · , P − 1 in Fig. 3, the local
are combined jointly to generate the CLBP histogram as the
difference between gc and g p is computed as d p = g p − gc .
input to the classifiers as shown in Fig. 2.
The local difference characterizes the local structure at gc
and can be decomposed into the sign (s) and magnitude (m) IV. DATASET, E XPERIMENT S ETUP, H YPER - PARAMETER
components as defined in Eq. 13. TUNING AND E VALUATION M ETRICS
(
s = sign(d p ) This section describes the dataset and the Gaussian noise
d p = s ∗ m, and (13) addition process. It also describes the experiment setup,
m = |d p |
the hyper-parameter tuning and the performance evaluation
To encode the sign component (s) of the local difference metrics of the defect detection model.
(d p ), CLBP_S is proposed to extract the sign information by
A. Dataset
comparing the pixel with its neighboring pixels as defined
in Eq. 14. The dataset used for the experiments is the NorthEastern
( University (NEU) surface defect dataset [9]. It comprises of
P−1
1, x ≥ 0 six defect classes: rolled-in scale (Rs), patches (Pa), crazing
CLBP_S = ∑ s(g p − gc )2 p , s(x) = (14)
p=0 0, x < 0 (Cr), pitted surface (Ps), inclusion (In) and scratches (Sc).
The collected defects are on the surface of the hot-rolled
where gc is the gray value of the central pixel, g p is the pixel steel strip. The dataset consists of 1800 gray-scale images:
value of its neighbors, P is the total number of neighboring 300 labelled samples for each defect class. Fig. 1 shows some
pixels and R is the radius of the neighborhood. sample images of the six defect classes.
To extract the local variance information contributed by To evaluate the robustness of the proposed approach
the magnitude (m) component of the local difference (d p ), against noise, noisy defect images are generated with the
CLBP_M is proposed to code the magnitude component of addition of Gaussian noise of different SNR. Fig. 4 shows
the local information as defined in Eq. 15. some defect images with some added Gaussian noise of
(
P−1
1, x ≥ c different SNR. It can be observed from Fig. 4 that, the texture
p
CLBP_M = ∑ t(m p , c)2 , t(x, c) = (15) appearance of the defect samples with SNR of 50dB and
p=0 0, x < c
40dB are rarely affected by the noise. At SNR of 30dB,
Where, c is an adaptive threshold which is set to be the mean there is moderate change in the textural appearance of the
value of m from the whole denoised image. defect samples. However, there is a significant change in the
The center pixel (c) which expresses the image local gray textural appearance of the defect samples when the SNR of
level also has discriminant information. To make it consistent the Gaussian noise is at most 20dB.

1931

Authorized licensed use limited to: Zhengzhou University. Downloaded on January 01,2023 at 16:18:18 UTC from IEEE Xplore. Restrictions apply.
B. Experiment Setup
The dataset used for the experiments is described in 2t p
∑m
i=1 |yi | 2t pi + f pi + f ni
i

section IV-A. This dataset is split into 80% training and Weighted Average F1 − score = (19)
∑m
i |yi |
20% testing set. To evaluate the robustness of the proposed
method against additive Gaussian noise, the proposed model V. R ESULTS AND D ISCUSSION
is trained with the noise free training dataset, and tested on The performance of the proposed defect detection frame-
the Gaussian noise induced test dataset. Different SNR level work is investigated and reported from two aspects. The
of noise from 50dB to 20dB is introduced on the test set. first set of results analyzes the performance of RCLBP with
To evaluate the performance of the proposed framework, K- the different classifiers on the noise free NEU dataset and
Nearest Neighbor (KNN), Support Vector Machine (SVM), compares them with that of the LBP and CLBP frameworks.
Decision Tree (DT), Gaussian Naive Bayes (GNB) and These results are presented in Table II in terms of the three
Random Forest (RF) classifiers [30] are used to evaluate the evaluation metrics described in section IV-D.
performance of the defect detection model. Comparing the results of LBP, CLBP and RCLBP in terms
C. Hyper-parameter Tuning of the weighted average precision, recall and F1-score mea-
The hyper-parameters of the classifiers used for the sures in Table II illustrates that RCLBP outperforms LBP and
comparison of the defect detection models are determined CLBP on the noise free NEU dataset. LBP when combined
through a grid search and a K fold cross validation with a K with the classifiers has a relatively moderate performance, as
value of 10. The K-fold cross validation is implemented on opposed to CLBP because the LBP features have a relatively
the 80% training dataset.The hyper-parameters of the various lower discriminative power since only the sign component
classifiers are as summarized in Table I. is encoded in the feature extraction. LBP thus is not able to
clearly distinguish between inter-class defect similarities and
TABLE I: Classifiers Hyper-parameter Settings. intra-class defect differences. RCLBP, however with the clas-
sifiers has a relatively higher defect detection performance
Classifier Hyper-parameter Values
in terms of the evaluation metrics. This can be explained by
n− neighbors 9
KNN p 1 the combination of the denoising technique which removes
lea f− size 10 the natural noise already within the NEU defect dataset and
weights distance the CLBP which extracts the discriminative defect features
C 0.1
SVM Gamma 1 to clearly distinguish between the defect classes. All the
Kernel Linear classifiers KNN, SVM, DT, GNB and RF show comparable
Criterion Gini performance in building the defect detection models. The
max− depth 9
DT max− f eatures sqrt
better performance of the proposed framework shows that the
min− samples− lea f 1 framework is clearly able to distinguish between the inter-
min − samples− split 2 class defect similarities and intra-class defect differences in
GNB var− smoothing 3.51e−08 the presence of the background interference.
Criterion Entropy
max− depth 9
RF max− f eatures sqrt TABLE II: Performance comparison of LBP, CLBP and
min− samples− lea f 7 RCLBP on noise-free NEU-DET dataset.
min − samples− split 4
n − estimators 10
Classifier LBP CLBP RCLBP
Weighted Average Precision Comparison
D. Evaluation Metrics KNN 0.89 0.95 0.98
SVM 0.87 0.95 0.98
The weighted average values of Precision, Recall and F1- DT 0.84 0.88 0.95
score measures as defined in Eq. 17 through Eq. 19 are GNB 0.78 0.95 0.97
RF 0.89 0.95 0.97
adopted as the metrics to evaluate the performance of the
Weighted Average Recall Comparison
detection models[31]. These metrics are defined by means KNN 0.88 0.94 0.98
of the true positive (t pi ), true negative (tni ), false positive SVM 0.87 0.94 0.98
( f pi ) and false negative ( f ni ) of each defect class Ci with DT 0.83 0.88 0.95
GNB 0.77 0.95 0.97
i = 1, · · · , m where m is the total number of defect classes in RF 0.88 0.95 0.97
the dataset. |Yi | is the total number of samples assigned to Weighted Average F1-score Comparison
each defect class. KNN 0.88 0.94 0.98
t pi SVM 0.87 0.94 0.98
∑m
i=1 |Yi | t pi + f pi DT 0.83 0.88 0.95
Weighted Average Precision = (17)
∑mi |Yi |
GNB 0.77 0.95 0.97
RF 0.88 0.95 0.97
tp
∑m
i=1 |Yi | t pi + f ni
i

Weighted Average Recall = (18) The second set of results analyzes the performance of LBP,
∑m
i |Yi | CLBP and RCLBP on their robustness on the different SNR

1932

Authorized licensed use limited to: Zhengzhou University. Downloaded on January 01,2023 at 16:18:18 UTC from IEEE Xplore. Restrictions apply.
 TABLE III: Performance comparison of LBP, CLBP and RCLBP on noisy data with different SNR value.
Weighted Average Precision Weighted Average Recall Weighted Average F1-score
Classifier
LBP CLBP RCLBP LBP CLBP RCLBP LBP CLBP RCLBP
SNR=50
KNN 0.45 0.90 0.98 0.58 0.89 0.98 0.48 0.88 0.98
SVM 0.55 0.91 0.98 0.54 0.90 0.98 0.45 0.89 0.98
DT 0.37 0.89 0.92 0.53 0.87 0.92 0.42 0.86 0.92
GNB 0.43 0.69 0.96 0.54 0.76 0.96 0.45 0.71 0.96
RF 0.43 0.90 0.98 0.59 0.84 0.97 0.49 0.80 0.97
SNR=40
KNN 0.39 0.78 0.92 0.57 0.79 0.91 0.46 0.74 0.91
SVM 0.37 0.88 0.98 0.49 0.81 0.97 0.40 0.75 0.97
DT 0.35 0.83 0.87 0.49 0.80 0.87 0.40 0.77 0.87
GNB 0.42 0.66 0.93 0.53 0.75 0.93 0.44 0.69 0.93
RF 0.42 0.86 0.94 0.57 0.81 0.94 0.48 0.75 0.94
SNR=30
KNN 0.39 0.65 0.84 0.38 0.55 0.81 0.28 0.48 0.81
SVM 0.39 0.50 0.78 0.36 0.48 0.75 0.26 0.38 0.74
DT 0.32 0.56 0.76 0.31 0.52 0.67 0.23 0.47 0.65
GNB 0.26 0.56 0.71 0.36 0.61 0.65 0.26 0.56 0.59
RF 0.37 0.66 0.82 0.35 0.62 0.74 0.26 0.57 0.72
SNR=20
KNN 0.03 0.03 0.41 0.17 0.17 0.43 0.05 0.05 0.33
SVM 0.03 0.03 0.40 0.17 0.17 0.40 0.05 0.05 0.28
DT 0.03 0.19 0.33 0.17 0.18 0.34 0.05 0.07 0.24
GNB 0.06 0.23 0.38 0.14 0.20 0.37 0.09 0.10 0.27
RF 0.03 0.19 0.42 0.17 0.17 0.42 0.09 0.10 0.27

value of noise in terms of the three evaluation metrics. These
results are presented in Table III.
At SNR = 50dB, the weighted average precision, recall
and F1-score of the LBP and CLBP drops comparably to
the performance of the noise free experiment. The pro-
posed approach RCLBP however maintains a comparable
performance to the performance on the noise free defect
dataset. At SNR = 40dB, RCLBP achieves comparable per-
formance at SNR = 50dB. However, the performance of LBP
drops considerably with the CLBP providing a relatively
moderate performance. At SNR = 30dB, the performance
of the LBP and CLBP diminishes in terms of the three
evaluation metrics. RCLBP however achieves considerable
performance with a weighted average precision and recall of
about 0.78 and 0.72 respectively with the weighted average
F1-score of 0.70. At SNR = 20dB, RCLBP still achieves
considerable performance with a weighted average precision
and recall of about 0.39 and 0.39 respectively with the
weighted average F1-score of 0.29 under the influence of
the highest additive Gaussian noise influence. The robustness
of the proposed framework to noise can be attributed to the
denoising framework which which removes the signal while
retaining the textures and edges characterizing the defect
classes.
VI. C ONCLUSION AND F UTURE W ORK
In this paper, we presented the Robust Completed Local
Binary Pattern (RCLBP) framework which is a combination
of the NL-means filter with wavelet thresholding and CLBP
which extracts the noise robust features to characterize the
defect classes. The RCLBP framework not only detected
surface defects under the influence of the feature variations
of the intraclass and interclass changes, but also achieved
a relatively moderate recognition accuracy in the toughest
situations with additive Gaussian noise. Our future work
will seek to investigate the use of surface defect dataset
with a varying scale and a high number of defect classes to
build more robust and generalized defect detection models.
Further research will also seek to explore synthetic data
augmentation to introduce more diversity and increase the
size of the training samples.
VII. ACKNOWLEDGEMENT
This work is supported by the Lockheed Martin cor-
poration under fund number 234363 and the Air Force
Research Laboratory (AFRL) and Office of Secretary of
Defense (OSD) under agreement number FA8750-15-2-
0116. This work is also partially supported by the NASA
University Leadership Initiative (ULI) under grant number
80NSSC20M0161. The authors would also like to thank the
support from the OUSD(R&E)/RT &L under Cooperative
Agreement Number W911NF-20-2-0261.
R EFERENCES
[1] Xiaoxin Fang et al. “Research Progress of Automated
Visual Surface Defect Detection for Industrial Metal
Planar Materials”. In: Sensors 20.18 (2020), p. 5136.
[2] Xiaohong Sun et al. “Research progress of visual
inspection technology of steel products—A review”.
In: Applied Sciences 8.11 (2018), p. 2195.
[3] Qinbang Zhou et al. “An automatic surface defect in-
spection system for automobiles using machine vision
methods”. In: Sensors 19.3 (2019), p. 644.
[4] Carlos Mera et al. “Automatic visual inspection: An
approach with multi-instance learning”. In: Computers
in Industry 83 (2016), pp. 46–54.

1933

Authorized licensed use limited to: Zhengzhou University. Downloaded on January 01,2023 at 16:18:18 UTC from IEEE Xplore. Restrictions apply.
 [5] Elias N Malamas et al. “A survey on industrial vision
systems, applications and tools”. In: Image and vision
computing 21.2 (2003), pp. 171–188.
[6] Timo Ojala, Matti Pietikainen, and Topi Maenpaa.
“Multiresolution gray-scale and rotation invariant tex-
ture classification with local binary patterns”. In: IEEE
Transactions on pattern analysis and machine intelli-
gence 24.7 (2002), pp. 971–987.
[7] Zhenhua Guo, Lei Zhang, and David Zhang. “A
completed modeling of local binary pattern operator
for texture classification”. In: IEEE transactions on
image processing 19.6 (2010), pp. 1657–1663.
[8] Ke Xu, Yong-hao Ai, and Xiu-yong Wu. “Application
of multi-scale feature extraction to surface defect
classification of hot-rolled steels”. In: International
Journal of Minerals, Metallurgy, and Materials 20.1
(2013), pp. 37–41.
[9] Kechen Song and Yunhui Yan. “A noise robust method
based on completed local binary patterns for hot-
rolled steel strip surface defects”. In: Applied Surface
Science 285 (2013), pp. 858–864.
[10] Hui Lin et al. “Automated defect inspection of LED
chip using deep convolutional neural network”. In:
Journal of Intelligent Manufacturing 30.6 (2019),
pp. 2525–2534.
[11] Xue–Wu Zhang, Fang Gong, and Li–Zhong Xu. “In-
spection of surface defects in copper strip using mul-
tivariate statistical approach and SVM”. In: Interna-
tional journal of computer applications in technology
43.1 (2012), pp. 44–50.
[12] Ali Rebhi, S Abid, and Farhat Fnaiech. “Fabric defect
detection using local homogeneity and morphological
image processing”. In: 2016 International Image Pro-
cessing, Applications and Systems (IPAS). IEEE. 2016,
pp. 1–5.
[13] Shiyang Zhou et al. “Double low-rank and sparse
decomposition for surface defect segmentation of steel
sheet”. In: Applied Sciences 8.9 (2018), p. 1628.
[14] YJ Duan et al. “Deep learning for control: the state
of the art and prospects”. In: Acta Automatica Sinica
42.5 (2016), pp. 643–654.
[15] Tian Shi et al. “Improved Sobel algorithm for defect
detection of rail surfaces with enhanced efficiency and
accuracy”. In: Journal of Central South University
23.11 (2016), pp. 2867–2875.
[16] M Sharifzadeh et al. “Detection of steel defect using
the image processing algorithms”. In: The Interna-
tional Conference on Electrical Engineering. Vol. 6.
6th International Conference on Electrical Engineer-
ing ICEENG 2008. Military Technical College. 2008,
pp. 1–7.
[17] Yunpeng Ma et al. “A surface defects inspection
method based on multidirectional gray-level fluctua-
tion”. In: International Journal of Advanced Robotic
Systems 14.3 (2017), p. 1729881417703114.
[18] Qiwu Luo et al. “Generalized completed local binary
patterns for time-efficient steel surface defect classi-
1934
Authorized licensed use limited to: Zhengzhou University. Downloaded on January 01,2023 at 16:18:18 UTC from IEEE Xplore. Restrictions apply.
fication”. In: IEEE Transactions on Instrumentation
and Measurement 68.3 (2018), pp. 667–679.
[19] Yong-hao Ai and Ke Xu. “Surface detection of con-
tinuous casting slabs based on curvelet transform and
kernel locality preserving projections”. In: Journal of
Iron and Steel Research International 20.5 (2013),
pp. 80–86.
[20] Doo-chul Choi et al. “Pinhole detection in steel slab
images using Gabor filter and morphological features”.
In: Applied Optics 50.26 (2011), pp. 5122–5129.
[21] Xiu-yong Wu, Ke Xu, and Jin-wu Xu. “Application
of undecimated wavelet transform to surface defect
detection of hot rolled steel plates”. In: 2008 Congress
on Image and Signal Processing. Vol. 4. IEEE. 2008,
pp. 528–532.
[22] Mohammadreza Yazdchi, Mehran Yazdi, and Arash
Golibagh Mahyari. “Steel surface defect detection us-
ing texture segmentation based on multifractal dimen-
sion”. In: 2009 International Conference on Digital
Image Processing. IEEE. 2009, pp. 346–350.
[23] Ge-Wen Kang and Hong-Bing Liu. “Surface defects
inspection of cold rolled strips based on neural net-
work”. In: 2005 International Conference on Ma-
chine Learning and Cybernetics. Vol. 8. IEEE. 2005,
pp. 5034–5037.
[24] Francisco G Bulnes et al. “Vision-based sensor for
early detection of periodical defects in web materials”.
In: Sensors 12.8 (2012), pp. 10788–10809.
[25] He Di et al. “Surface defect classification of steels
with a new semi-supervised learning method”. In: Op-
tics and Lasers in Engineering 117 (2019), pp. 40–48.
[26] BK Shreyamsha Kumar. “Image denoising based on
non-local means filter and its method noise thresh-
olding”. In: Signal, image and video processing 7.6
(2013), pp. 1211–1227.
[27] Antoni Buades, Bartomeu Coll, and Jean-Michel
Morel. “A review of image denoising algorithms, with
a new one”. In: Multiscale Modeling & Simulation 4.2
(2005), pp. 490–530.
[28] S Grace Chang, Bin Yu, and Martin Vetterli. “Adaptive
wavelet thresholding for image denoising and com-
pression”. In: IEEE transactions on image processing
9.9 (2000), pp. 1532–1546.
[29] David L Donoho and Jain M Johnstone. “Ideal spatial
adaptation by wavelet shrinkage”. In: biometrika 81.3
(1994), pp. 425–455.
[30] Fayaz Itoo, Satwinder Singh, et al. “Comparison and
analysis of logistic regression, Naıve Bayes and KNN
machine learning algorithms for credit card fraud
detection”. In: International Journal of Information
Technology (2020), pp. 1–9.
[31] Jiajun Zhang, Georgina Cosma, and Jason Watkins.
“Image enhanced mask R-CNN: A deep learning
pipeline with new evaluation measures for wind tur-
bine blade defect detection and classification”. In:
Journal of Imaging 7.3 (2021), p. 46.