2018 23rd International Scientific-Professional Conference on Information Technology (IT)
Image Processing Based Anomaly Detection
Approach for Synchronous Movements in
Cyber-Physical Systems
Hasan Yetis, and Mehmet Karakose, Senior Member, IEEE

Abstract — Today cyber-physical systems are becoming
more and more popular with the industry 4.0. Given the size
and criticality of the systems they are used, it is very
important that the cyber physical systems work properly. In
this study, image-processing based anomaly movement
detection method, which can be integrated to systems
anytime, is proposed. With detection of anomaly movements
and preventing unexpected situations, the cost arises from
hitches, breakdowns or damaged productions is reduced. This
study consists of two main section, first is implementing
remote-controlled simple industrial robot system, second is
the detection of unwanted movements and controlling the
system.
Keywords — anomaly detection, cyber-physical systems,
image processing, industry 4.0.
I. INTRODUCTION
C YBER-PHYSICAL systems (CPS), which is a concept
of Industry 4.0, consist of cyber layer and physical
layers. While cyber layer refers to the layer that algorithms
run and calculations done, physical layer refers to the layer
that the sensors and the actuators are. Although these two
layers are separated, they interact tightly thanks to the
communication technologies [1]. A simple block diagram
for CPS structure is given by fig. 1.
Observation
Sensors
Algorithms
Controllers
Actions
Cyber Layer Physical Layer
Figure 1. Simple CPS Structure
Although the CPS is relatively new concept, it has been
subjected to many researches [2]. The CPS applications
Corresponding Hasan Yetis. Author is with the Faculty of Computer
Engineering, Firat University, Elazig, 23119, Turkey (phone: 424-237-
0000/6336; e-mail: h.yetis@firat.edu.tr).
Mehmet Karakose. Author is with the Faculty of Computer
Engineering, Firat University, Elazig, 23119, Turkey; (phone: 424-237-
0000/6320, e-mail: mkarakose@firat.edu.tr).
978-1-5386-3620-6/18/$31.00 ©2018 IEEE
Žabljak, Montenegro
includes many areas such as health, environment, industry,
transportation, and agriculture. One of the studies, which is
done in environment, is about monitoring the
environmental variables such as temperature, relative
humidity, blackdamp, absolute pressure, and illumination
[3]. Another study is done in agricultural. In the study, the
data collected by the sensors on the potato fields are
transported to servers. After the analyses actualized on
server, the suggestions such as fertilization or soil
ventilation are notified to users. So the possible loss of
new farmers is reduced [4]. Transportation is another area
which CPS is preferred [5-7]. In one of these studies, the
data is obtained with protecting the privacy by the help of
the sensors on roads. According the data, the traffic jam is
estimated by the algorithms on server [7]. The applications
of CPS in health have a wide area from creation of the
platforms that patients use to the development of medical
imaging tools [8,9].
Because the CPS has multilayered structure and the
communication between the layers are actualized by
network technologies, the systems may be subjected to
network or physical sensor attacks. So, the studies, which
related with security, are important for making the CPS
widespread and using it with safe, especially in industrial
applications [10,11]. Another area, which CPS is used, is
industry because of the advantages it provides. Because of
the easy use of advanced controllers such as fuzzy logic
controller [12], the CPS is preferred rather than embedded
systems [13]. Furthermore, because of the advantage of
separated cyber and physical layers, any update in control
algorithm is performed much more easily than embedded
systems. Because it makes the system living, it is preferred
by industrial applications in which up-to-dateness is
important. Industry 4.0, which uses CPS in industry, is the
most significant evidence of that [14]. There are many
studies in literature which aims to develop CPS-based
industry [15-20]. One of these studies aims verification of
the data which is sent from physical layer to cyber layer in
real-time [16].
The success of CPS is related with how the physical
layer is modelled in cyber layer. But regardless how a
system well modeled, unexpected situation such as
uncertainty are always a threat to the system [21]. Because
the CPS is used in critical system, working of system
properly is very important [22]. In this study, it is aimed to
2018 23rd International Scientific-Professional Conference on Information Technology (IT) Žabljak, Montenegro

detect anomaly movements, which arise from possible

uncertainties. After detecting an anomaly movement, the
Cyber Space
system is halted. So, possible costs can be prevented.
V = grayscale(frame – reference)

II. PROPOSED METHOD Eleminate low pixels, histogram equalization

The study consists of two part. First of one is
establishment of sample industrial CPS system, which is Find Connected Companents
mentioned in previous section. But because of some
reasons, the system might not work properly. Some of Eleminate small objects, create mask
these are:
x Synchronization errors Im = mask * grayscale (frame)
x Incorrect or incomplete transmission of data
S[i] = sumofpixels(Im), i++
x Finding different objects in the environment
If the system makes movements except for the expected No
i=last?
movements, this may cause permanent damages on parts of Yes
the system. So the second part of this study is about
C = correlation (S, refS), i=0
anomaly movement detection. For this aim, we fixed a
camera which can see the system first. To succeed our
method, we should be sure that the camera is fixed and the Yes
C>k?
illumination is stable. No
First, the reference background is obtained and the fixed
Alert Anomaly and stop system
camera started to record. After, the system performs the
movements which the current command needs. Each frame
Fetch and send the next command
of this movement is subtracted from reference image to
detect moving objects. Later, to increase the robustness of
the moving objects, elimination of low pixels, which are
probably noise is applied. Then, histogram equalization is Physical Space
used to increase the contrast in image. The image
converted to black and white (binary), and connected
Actuator 1

Actuator 2

components are found on the binary image. The small
components are eliminated from binary image, and binary
mask is obtained. After it is obtained, the mask is subjected
to dot product with the original grayscale image and robust
moving objects are left in the image. After obtained the
grayscale moving objects, we used the pixel values of each
frame to calculate the feature. The feature for each frame is
calculated by sum of the all pixel values in robust moving
grayscale images. Because the video has sequential frames,
this process is repeated until the movement is done, and
the features for each frame are stored in an array. When the
process is done, a signal, which is formed by the features
of each frames is created. With correlation of the obtained
signal and reference signal, which is created at beginning,
we obtain a correlation coefficient. The correlation
coefficient is between -1 and 1. And the biggest value of
this means that the signals are similar to each other more.
When it is near to 0, it means there are no relation between
these signals. We use a threshold value to control that if
the movement is expected one or not. If the movement is
not expected, the system is halted, and the next command
isn’t sent to the physical components. But if there is no
problem, the next command is fetched and sent to the
components in physical layer. And the steps are realized
again, but the reference signal is changed with the signal
form of the features of current expected movement. A
detailed block scheme, which summarizes the steps
explained above, is given by fig. 2.
Actuator n
...
Take The Image
Figure 2. Block Scheme for Proposed Method
III. EXPERIMENTAL RESULTS
In this study, to model industrial production
environment, Arduino Mega development board, Arduino
Wi-Fi Shield, Arduino Braccio robotic arm, camera and
personal computer are used. The Arduino Braccio robotic
arm, which has 6 degrees of freedom, is controlled by the
Arduino Braccio Shield. The robotic arm, which come in
pieces, is mounted. To control and send introductions to
the Braccio adaptor, Arduino development boards are
used. No algorithmic operations are run on this board, they
are just used for executing the commands and control the
Braccio robotic arm. Arduino ATMega 2560 development
board is used in this study. Arduino Wi-Fi shield is
required for wireless communication of the board. The Wi-
Fi shield is the elementary unit which provides the
communication between the cyber and physical layers. It
2018 23rd International Scientific-Professional Conference on Information Technology (IT) Žabljak, Montenegro

transports the commands which come from the personal
computer in the cyber layer to physical layer. The Braccio
Shield, Wifi Shield and Arduiono Mega can be seen
respectively top to bottom in fig. 3.
to which the control commands are sent and anomaly
motion detection process is applied on the images. As a
result, the end result of anomaly movement is acquired.
The command loop which is sent from personal computer
is given by table 1. The system makes continuous take-the-
sponge movement with repetition of the commands.

Table 1. The commands for take-the-sponge movement

Step M1 M2 M3 M4 M5 M6
1 0° 45° 180° 180° 90° 10°
2 0° 90° 180° 180° 90° 10°
3 0° 90° 180° 180° 90° 60°
4 0° 45° 180° 45° 0° 60°
5 180° 45° 180° 45° 0° 60°
6 0° 90° 180° 180° 90° 60°
7 0° 90° 180° 180° 90° 10°
Figure 3. Arduino Mega, Wi-Fi Shield and Braccio Shield.
After establish the system, camera is started to record
After the connections between the robotic arm and and reference variables are obtained according to proposed
Arduino done with the help of Braccio shield, the robotic method. The fig. 5 shows an example Im variable in the
arms are made movement according the commands come method given by fig. 2.
from personal computer. In short, Arduino, Arduino
Braccio shield, Braccio, and camera form the physical
layer, Wi-Fi shield form the communication layer, and
personal computer forms the cyber layer. The general
block scheme of the installed system is given in fig. 4. It is
observed here that camera images are used to view the
system. It is also understood from this picture that the
personal computer is used as the cyber layer.

Figure 5. Detected grayscale moving objects

By the help of the Im images, the features, which are

calculated by sum of the all pixels in Im, are obtained. The
features for example 87 sec video is given by fig. 6.

Braccio Shield Braccio Shield

Arduino Mega Arduino Mega

Wi-Fi Shield Wi-Fi Shield

Figure 6. The features for 87 sec sample video

Control Commands In fig. 6, the system does the expected movements

Image
Figure 4. Block Schema for Experimental System
In Figure 4, it is desirable to display a sample behavior
and the physical components have been successfully
executed in response to commands from the cyber layer.
Then the images obtained are transmitted to the computer
between the secs 2-12, 16-26 and 30-40. Between the secs
40-48 and 55-59, a foreign matter is included in video.
Between the secs 59-69, the system does the movements
other than expected movements. After the sec 69, all the
objects in video is removed and only reference background
image is left in video. The features between the secs 83-87
is belong to the reference background image, and no
moving object is detected there. To have a closer look, the
signals between 2-12 and 16-26 is extended in fig. 7 and
fig. 8 respectively.
2018 23rd International Scientific-Professional Conference on Information Technology (IT)
Figure 7. Expected reference video features
Figure 8. The features between 16-26 secs
When we calculate the correlation coefficient between
the signals expected signal given in fig.7 and recorded
signal given fig.8., we obtained high value such as 0.9869.
This means these signals are so similar to each other. But
the coefficient is 0.2620 for the signals given between 2-12
and 59-69. And this means these signals are not similar.
With doing the comparisons for each step in table 2, we
control the system step by step.
IV. CONCLUSIONS
The robustness of CPS is highly important because they
are preferred by mission critical and expansive systems.
Because of unwanted reasons, unexpected movements may
occur in systems sometimes. In this paper, a new method
based on image processing is proposed to detect these
unwanted situations. The sum of gray level images of
moving objects are converted to signals for sequential
frames. After, the signals are subjected to correlation with
reference signal, and the similarity between these is
calculated. While the high level of correlation coefficient
means that it is expected movement, the low level of it
shows the movement is unexpected or foreign object is
included in frame. The reference signals are compared
with current signal, and it is decided the system works
properly for every step of movement. Thus, the proper
work of system is guaranteed, and the robustness of CPS is
increased. This study is also an external control mechanism
for industry 4.0, which aims totally unmanned production.
ACKNOWLEDGMENT
This work was supported by Scientific Research
Projects Coordination Unit of Firat University. Project
number: MF.17.09.
Žabljak, Montenegro
REFERENCES
[1] H. Yetis, M. Karakose, “Adaptive Vision Based Condition
Monitoring and Fault Detection Method for Multi Robots at
Production Lines in Industrial Systems,” Int. Journal of Applied
Mathematics, Electronics and Computers, vol. 4, 271-276, 2016.
[2] K. Kim, P. Kumar, “Cyber–Physical Systems: A Perspective at the
Centennial,” Proceedings of the IEEE, vol. 100,1287-1308, 2012.
[3] G. Mois, T. Sanislav, S. C. Folea, “A Cyber-Physical System for
Environmental Monitoring,” IEEE Transactions on Instrumentation
and Measurement, vol. 65, no. 6, pp. 1463-1471, 2016.
[4] C. R. Rad, O. Hancu, I. A. Takacs, G. Olteanu, “Smart Monitoring
of Potato Crop: A Cyber-Physical System Architecture Model in
the Field of Precision Agriculture,” Int. Conference "Agriculture for
Life, Life for Agriculture, 2015.
[5] R. S. Bali, N. Kumar, “Learning Automata-assisted Predictive
Clustering approach for Vehicular Cyber-Physical System,”
Computers & Electrical Engineering, vol. 52, pp. 82-97, 2016.
[6] G. Xiong, F. Zhu, X. Liu, X. Dong, W. Huang, S. Chen, K. Zhao,
“Cyber-physical-social system in intelligent transportation,”
IEEE/CAA Journal of Automatica Sinica, vol. 2-3, 320-333, 2015.
[7] Y. Zhou, Z. Mo, Q. Xiao, S. Chen, Y. Yin, “Privacy-Preserving
Transportation Traffic Measurement in Intelligent Cyber-Physical
Road Systems,” IEEE Transactions on Vehicular Technology, vol.
65, no. 5, pp. 3749-3759, 2016.
[8] Y. Zhang, M. Qiu, C. W. Tsai, M. M. Hassan, A. Alamri, “Health-
CPS: Healthcare Cyber-Physical System Assisted by Cloud and Big
Data,” IEEE Systems Journal, vol. 11, no. 1, pp. 88-95, 2017.
[9] P. Kaveh, G. Yishuang, R. C. David, B. Guanqun, M. Liang, A.
Emmanuel, K. Andrew, S. Kamran, T. Vahid, “A Novel Cyber
Physical System for 3-D Imaging of the Small Intestine in Vivo,”
IEEE Access, vol. 3, pp. 2730-2742, 2015.
[10] S. Mitra, T. Wongpiromsarn, R. M. Murray, “Verifying Cyber-
Physical Interactions in Safety-Critical Systems,” IEEE Security &
Privacy, vol. 11, no. 4, pp. 28-37, 2013.
[11] X. Jin, W. M. Haddad, T. Yucelen, “An Adaptive Control
Architecture for Mitigating Sensor and Actuator Attacks in Cyber-
Physical Systems,” IEEE Transactions on Automatic Control, vol.
PP, no. 99, pp. 1-1, 2017.
[12] M. Karakose, “An Improved Nonstationary Fuzzy System
Approach versus Type-2 Fuzzy System for the Lifting Motion
Control with Human-in-the-Loop Simulation”, Int. Journal of
Computational Intelligence Systems, vol. 11-1, pp. 183-194, 2018.
[13] A. Sargolzaei, C. D. Crane, A. Abbaspour, S. Noei, “A Machine
Learning Approach for Fault Detection in Vehicular Cyber-
Physical Systems,” 15th IEEE International Conference on
Machine Learning and Applications (ICMLA), 2016.
[14] S. I. Shafiq, C. Sanin, E. Szczerbicki, C. Toroc, “Virtual
Engineering Object / Virtual Engineering Process: A specialized
form of Cyber Physical System for Industrie 4.0,” Procedia
Computer Science, vol. 60, pp. 1146-1155, 2015.
[15] K. Thramboulidis, “A cyber–physical system-based approach for
industrial automation systems,” Computers in Industry, vol. 72, pp.
92-102, 2015.
[16] J. Lee, H. D. Ardakani, S. Yang, B. Begheri, “Industrial Big Data
Analytics and Cyber-Physical Systems for Future Maintenance &
Service Innovation,” Procedia CIRP, vol. 38, pp. 3-7, 2015.
[17] J. Morgan ve G. E. O'Donnell, “The Cyber Physical
Implementation of Cloud Manufactuirng Monitoring Systems,”
Procedia CIRP, vol. 33, pp. 29-34, 2016.
[18] X. Yue, H. Cai, H. Yan, C. Zou, K. Zhou, “Cloud-assisted
industrial cyber-physical systems: An insight,” Microprocessors
and Microsystems, vol. 39, no. 8, pp. 1262-1270, 2015.
[19] P. Leitão, A.Colombo, S. Karnouskos, “Industrial automation based
on cyber-physical systems technologies: Prototype implementations
and challenges,” Computers in Industry, vol. 81, pp. 11-25, 2016.
[20] Mehmet Karaköse and Hasan Yetiş, “A Cyberphysical System
Based Mass-Customization Approach with Integration of Industry
4.0 and Smart City,” Wireless Communications and Mobile
Computing, vol. 2017. doi:10.1155/2017/1058081
[21] M. Zhang, B. Selic, S. Ali, T. Yue, O. Okariz, R. Norgren,
“Understanding Uncertainty in Cyber-Physical Systems: A
Conceptual Model,” Lecture Notes in Computer Science, vol. 9764,
Cham, Springer, 2016, pp. 247-264.
[22] F. Pasqualetti, F. Dörfler, F. Bullo, “Attack Detection and
Identification in Cyber-Physical Systems,” IEEE Transactions on
Automatic Control, vol. 58, no. 11, pp. 2715 - 2729, 2013.