energies

Article
Unsupervised Fault Detection and Analysis for Large
Photovoltaic Systems Using Drones and
Machine Vision
Moath Alsafasfeh 1 , Ikhlas Abdel-Qader 1 , Bradley Bazuin 1 , Qais Alsafasfeh 2, * and Wencong Su 3
1 Electrical and Computer Engineering Department, College of Engineering and Applied Sciences,
Western Michigan University, Kalamazoo, MI 49001, USA; moath.al-safasfeh@wmich.edu (M.A.);
ikhlas.abdelqader@wmich.edu (I.A.-Q.); brad.bazuin@wmich.edu (B.B.)
2 Energy Engineering Departments, College of Engineering, Al Hussein Technical University,
Amman 25175, Jordan; Sabbatical leave from Tafila Technical University, Department of Electrical power and
Mechatronics, Tafila-Jordan
3 Department of Electrical and Computer Engineering, University of Michigan-Dearborn,
Dearborn, MI 48121, USA; wencong@umich.edu
* Correspondence: qsafasfeh@ttu.edu.jo




Received: 25 June 2018; Accepted: 13 August 2018; Published: 27 August 2018


Abstract: One of the most important sources of clean energy in the future is expected to be solar
energy which is considered a real time source. Research efforts to optimize solar energy utilization
are mainly concentrated on the components of solar energy systems. Photovoltaic (PV) modules are
considered the main components of solar energy systems and PVs’ operations typically occur without
any supervisory mechanisms, which means many external and/or internal obstacles can occur and
hinder a system’s efficiency. To avoid these problems, the paper presents a system to address and
detect the faults in a PV system by providing a safer and more time efficient inspection system in real
time. In this paper, we proposing a real time inspection and fault detection system for PV modules.
The system has two cameras, a thermal and a Charge-Coupled Device CCD. They are mounted on a
drone and they used to capture the scene of the PV modules simultaneously while the drone is flying
over the solar garden. A mobile PV system has been constructed primarily to validate our real time
proposed system and for the proposed method in the Digital Image and Signal Processing Laboratory
(DISPLAY) at Western Michigan University (WMU). Defects have been detected accurately in the
PV modules using the proposed real time system. As a result, the proposed drone mounted system
is capable of analyzing thermal and CCD videos in order to detect different faults in PV systems,
and give location information in terms of panel location by longitude and latitude.

Keywords: PV module; real time fault detection; thermal and CCD video processing

1. Introduction
The energy of the Sun is renewable, clean, and costs nothing. Solar energy generates green and
cheap electricity from sunlight using photovoltaic modules which generate electricity from sunlight.
A PV module has individual PV cells, typically arranged and grouped in an array. The light strikes the
solar cell, where a certain amount of the light is absorbed, which means the energy of the absorbed
light is transferred to the solar cell. Thus, the electrons will flow freely because the energy will
knock electrons loose [1]. Solar cells have electric fields which act to force the electrons freed by
light absorption to flow in a certain direction, which produces a current [1]. This current could be
used externally by placing metal contacts on the top and bottom of the solar cell. The solar cell’s
voltage, which is a result of its built-in electric fields, together with the current, provides the power

Energies 2018, 11, 2252; doi:10.3390/en11092252 www.mdpi.com/journal/energies

Energies 2018, 11, 2252 2 of 18

that can be produced by the solar cell. PV modules produce an amount of electricity that depends
on the prevailing natural factors, the type and the composition of the module, and the combined
effect of the installation [2]. One big limitation for solar (and wind) power is that it only generates
electricity when the sun is shining (or the wind is blowing)—falling short at peak demand times
like the evenings. Unlike conventional generation sources, wind and solar power outputs are highly
uncertain and unpredictable [3,4]. In addition, temperature fluctuation is an important factor in PV
modules due to changes in solar irradiance and temperature. For example, crystalline PV modules
have a 0.5% reduction in efficiency when the temperature is increased by 1 ◦ C [2]. Solar radiation is
the main component of the heat balance of PV modules. Solar radiation influences the output power
of the PV module, radiation heat exchanges and convective heat exchanges [2]. Additionally, because
of fluctuating weather conditions and/or a lack of abundant historical data, a growing generation
share of renewable energy (e.g., wind and solar) significantly complicates power system operations
hence imposing more severe system constraints (voltage, frequency, etc.). On the customer side,
“flexible/dispatchable loads” can adjust their own electric energy usage on the basis of a real-time set
point. With the growing share of these intermittent energy sources and flexible/dispatchable loads,
it becomes much more difficult to maintain the power balance [5–7].
PV systems are used to produce power and provide electricity to homes, buildings, and more.
The importance of PV systems is attracting more attention and research across the world because the
energy demand is continuously increasing. The utilization of renewable energy resources is important
in order to reduce environmental effects [8]. Solar modules have been distributed rapidly due to the
fast-technological development and decreasing production costs. In addition, recently the price of PV
systems is depreciating as well the PV system installation process takes less time and provides a better
return the investment [8]. Numerous countries have announced governmental support for the production
of solar energy [9]. For example, many countries have plans for building and using PV systems to develop
or modernize existing street lighting systems [9]. Regarding the future, the Earth is exposed to the hazards
due to global warming and the greenhouse effect, and the use of renewable energy may be the only
solution to stop the effects of these hazards on human life [10]. Therefore, in order to get more efficient
power and higher performance for PV systems, save energy, money and minimize the harmful impact
faults should be detected as soon as possible.
The common faults in the PV module types can be considered as manufacturing faults; such as
delamination that can occur when the adhesion between the glass, encapsulant, active layers, and back
layers becomes weak, back sheet adhesion loss, junction box failure that takes place when there are no
reliable soldering contacts of the string interconnects; this could cause high a resistance and consequent
heating, and frame breakage [11]. The most common failures due to external causes are clamping,
connector failures, lightning, transport and installation failures [11]. Some faults in a PV module are
categorized [12] as follows: hot spots caused by the panel act as a load; hot spots can be taken as a sign
of heat loss from the PV module due to some electrical problems, as shown in Figure 1, for example in
a bypass diode, or in a shunt resistor. Defects in the anti-reflective coating; the color of the module
can change due to a change in the coating layer. The light that reaches the cells may be lower [12],
and as a result the antireflection properties may suffer changes. Bubbles in the solar modules; when
the adherence between Tedlar and the back side of the module has been lost, bubbles will develop.
As a result, overheating will take place which means a reduction of cell life. Delamination over cells
and interconnections; this occurs when the Tedlar adherence is lost with subsequent unstuck layers,
and as a result water penetration can take place inside the module, and this is considered as a major
problem. Browning and yellowing; it is caused by water penetration inside the module, because of lack
of adherence between the Tedlar and the module. Newly cracked cells; the thickness of the solar cells
has changed from 300 µm to 200 µm [12]. In other words, the cells have become more fragile which
means they are easily breakable on manipulation or storage of the modules. Some of the considered
electrical faults in the PV modules [13–15] include the functionality of the shunting in cells’ bypass
Energies 2018, 11, 2252 3 of 18
Energies 2018, 11, 2252 3 of 18

diode, the functionality

reverse-biased of erroneous
heating, high bypass diodes,
series resistance reverse-biased
resistive heating,
solder bonds, highin
hot spots series resistance
modules, and
resistive solder bonds,
nonlinear weak diodes. hot spots in modules, and nonlinear weak diodes.

Figure
Figure 1.
1. Hot
Hot spots
spots in
in PV
PV modules
modules at
at the WMU main
the WMU main campus.
campus.

The faults in PV arrays can be divided in three categories; line-line faults, open circuit faults,
The faults in PV arrays can be divided in three categories; line-line faults, open circuit faults,
and ground faults. The faults inside PV arrays usually cause an overcurrent which damages PV
and ground faults. The faults inside PV arrays usually cause an overcurrent which damages PV
components [16]. When an accidental connection goes through a low resistance path between two
components [16]. When an accidental connection goes through a low resistance path between two
points in the PV module, a line-line fault will occur [16]. A line-line fault may occur between two
points in the PV module, a line-line fault will occur [16]. A line-line fault may occur between two adjacent
adjacent strings or between two points on the same string. The potential difference between the
strings or between two points on the same string. The potential difference between the connected points
connected points influences the magnitude of the line-line fault, so the higher potential difference
influences the magnitude of the line-line fault, so the higher potential difference means higher feedback
means higher feedback current results, and tripping the over-current protection devices (OCPD) will
current results, and tripping the over-current protection devices (OCPD) will increase [17]. A line-line
increase [17]. A line-line fault can be considered as a ground fault if one of the points is on the
fault can be considered as a ground fault if one of the points is on the equipment grounding conductor
equipment grounding conductor (EGC) [16]. A line-line fault may not be detected once the current
(EGC) [16]. A line-line fault may not be detected once the current flowing through one of the current
flowing through one of the current carrying conductors (CCC) is less than the rated current of the
carrying conductors (CCC) is less than the rated current of the OCPDs. This will reduce the system
OCPDs. This will reduce the system efficiency [18].
efficiency [18].
The electromagnetic spectrum is a scale classifying the different forms of electromagnetic
radiation electromagnetic
The [19], such as X-rays, spectrum
gamma is arays,
scale classifying
radio waves, the different visible
microwaves, forms of electromagnetic
rays, and invisible
radiation [19], such as X-rays, gamma rays, radio waves, microwaves, visible rays, and invisible
rays. Because the human vision system can see just the visible light, the use of infrared imaging is
rays.
often Because
necessary. theInfrared
human vision
imaging system
is thecan see just the
technique visible light,
of capturing the useinfrared
invisible of infraredimagesimaging
and
is often necessary. Infrared imaging is the technique of capturing invisible infrared images and
converting them into visible images. Infrared imagers and cameras have special sensors that do not
converting
need visiblethem light into
to seevisible images.
in infrared lightInfrared imagersradiation
[19]. Infrared and cameras have special
is produced from any sensors
warmthat do
object
not need visible light to see in infrared light [19]. Infrared radiation
with a temperature above absolute zero because there is no molecular and atomic activity at absolute is produced from any warm
object withMolecular
zero [19]. a temperature above absolute
and atomic zero because
activity increases when there
theistemperature
no molecular and atomic
increases, whichactivity
means at
absolute zero [19]. Molecular and atomic activity increases when
more thermal radiation and heat is produced, and more infrared radiation is emitted [19]. The the temperature increases, which
means
targetedmore
objectsthermal
or theradiation
reflected and heat is
radiation in produced,
the infraredand more emit
imaging infrared radiation
radiation. is emitted
Using sunlight[19].for
The targeted objects
illumination will causeor the reflected
reflected radiationorinthe
radiation, the imaging
infrared imaging
device may emithave
radiation. Using
infrared sunlight
illuminator
for illumination
lasers equipped with will cause
LEDs.reflected radiation,
These infrared wavesor the imaging
absorb device
or reflect mayany
from have infrared
object that illuminator
falls within
lasers equipped with LEDs. These infrared waves absorb or reflect from
the range of these invisible illuminators [19]. Using long wave thermal infrared imagers will any object that falls within
helpthe
to
range
detectof these
and pickinvisible
up theilluminators [19]. Using
infrared radiation long wave
reflected from thermal
or emitted infrared
by warmimagers will help
objects. The to detect
infrared
and
rays pick up the infrared
are directed onto anradiation reflectedsensors,
array of infrared from orand emitted
thereby warm
can be manyobjects. The infrared
thousands rays are
of sensors on
directed onto an array of infrared sensors, and there can be many thousands
the sensor array [19]. The infrared energy is transformed into electrical signals and then converted of sensors on the sensor
array
into an[19]. The infrared
image. Infrared energy
light hasis transformed
many uses because into electrical
it hassignals and then
the ability converted
to cross intocannot
areas that an image.be
Infrared light has many uses because it has the ability to cross areas that
reached by visible light, and it reveals unclear objects. The military has developed it to produce cannot be reached by visible
light,
night and
vision it reveals
camerasunclear
and gun objects.
sightsThe military
[19]. Infrared hasimaging
developed it to in
is used produce night vision cameras
many applications, such as
and
helping firefighters and police to catch criminals in the dark, or to rescue people lostfirefighters
gun sights [19]. Infrared imaging is used in many applications, such as helping at night and and
in
police to catch criminals in the dark, or to rescue people lost at night
dark places. Using infrared imaging enables technicians to locate leaking chemicals as well as and in dark places. Using infrared
imaging
overheated enables techniciansparts,
or underheated to locate
which leaking
helpschemicals
to eliminate as well as overheated
potential or underheated
hazards. Infrared imagingparts,also
which helps to eliminate potential hazards. Infrared imaging also
helps researchers in the wild to study animals in their habitat at night. On the other hand, helps researchers in theinfrared
wild to
study
imaging animals
can bein theirfor
used habitat
medicalat night.
diagnosticOn the other hand,
purposes; infrared
it is used imaging
to perform can scans.
body be used for medical
Satellites also
diagnostic purposes; it is used to perform body scans. Satellites also
use infrared imaging to study the Earth’s condition. Fault detection on PV modules plays a main use infrared imaging to studyrolethe
in getting more efficient power production from PV system and reduce maintenance costs when the
faults can be detected in their early stages. Using IR images can help to detect hot spots in PV
Energies 2018, 11, 2252 4 of 18

Earth’s condition. Fault detection on PV modules plays a main role in getting more efficient power
production from PV system and reduce maintenance costs when the faults can be detected in their
early stages. Using IR images can help to detect hot spots in PV modules which provide indications
of abnormal operations which can lead to a reduction in the power production or hazards to the PV
system or humans. The proposed method takes the advantages of using an IR imaging system with
drone capability to detect the faults in PV systems.
In 2004, a temperature-controlled stage with a vacuum chunk was used for fast shunt analysis
in solar cells. The solar cell has a layer of liquid crystal foil, and images allow for the localization
of places where the temperature changes the reflectivity of the liquid crystal overlayer [20]. In 1998,
the hot spots in circuits and devices were located using fluorescence microthermography (FMT).
FMT is based on the efficiency of the temperature-dependent luminescence of rare earth chelate
dyes [20]. In 2001, infrared methods were used for non-destructive testing, for example, thermal
emissions, IR sensors and optics, and imaging and analysis. In 2000, the applications of infrared
imaging have been summarized in a survey which included shunts in solar cells, such as reverse-bias
heating, by-pass diode functionality, resistive solder bonds, and other components of the balance
of a system [20]. In 2002, the thermographic imaging was used to inspect cracked solar cells [20].
In 2011, an IR camera was used in the field to monitor solar modules’ performance [20]. The proposed
methods for using thermal images for fault detection in PV modules are discussed. Using infrared
images via a camera mounted on a moving cart was proposed by Gao et al. to detect and recognize
defective PV modules [21]. They used an optical flow to establish frame-to-frame association in order
to count the number of panels within any given array. They recognized the panels using a Hough
Transform (HT) and the DBSCAN clustering technique to identify a hot panel by comparing it with
its neighbours. Several drawbacks were reported such as, the first column of an array cannot be
recognized when the top row is missed, or challenges with image registration in relation to optical flow
results [21]. The use of electroluminescence (EL) and independent component analysis (ICA)-based
images to detect a defective solar cell was reported by Tsai et al. [22]. Defects were presented as
some dark spots in EL images, such as finger interruptions, breaks, and microcracks. All of these
defects have been detected using EL images. To measure the distance and minimize reconstruction
errors, the cosine function has been used for these purposes. The drawback of their work is that small
localized defects can be missed because the results are all based on a global approach [22]. Canny edge
detection and the use of standard thermal image processing is proposed by Tsanakas et al. This method
was proposed for module-related faults that results in hot spot heating. The main objective was the
applicability of thermal image processing and edge detection to detect defective PV modules. The fault
types are not classified by this method. Another limitation is false alarms will occur because of the
unwanted grey-level variations, that are caused by any specular object present in the background, that
will conflict with the actual variations related to hot spots. There are further limitations referring to
emissivity uncertainties [23]. A fault detection and automatic supervision method for a PV system
based on power loss analysis is proposed by Chouder and Silvestre [24]. The method calculates the
main parameters of the PV system for monitoring data in real time while taking into account the
temperature evolution and environmental irradiance. Their method focuses on the output power
losses present in the DC side of the PV system generator and capturing losses. This model defines two
power loss types: thermal loss and miscellaneous loss. The system can generate a fault-signal indicator
based on the values of these two loss types. These indicators show the voltage and current ratios.
After analyzing both, the fault-signal and the current/voltage ratios, the type of fault can be identified.
Detecting the array faults in the PV modules can be accomplished by comparing current operation
characteristics with recorded reference characteristics that account for time, season and climate which
will affect the performance. Regarding [25], due to the complexity of these numerous conditions,
the need for reference profiles and the need for generating reference profiles, as well as the comparison
of the parameters with the reference profiles, this process can be very complicated and the result could
be inaccurate. One of the methods used to detect the fault in PV modules is reverse current faults.
Energies 2018, 11, 2252 5 of 18

The idea is to detect the abnormal current flow [13]. The PV system combines multiple power resources
to provide a single output, such as systems for combining the power generated by multiple PV module
arrays [13]. It generates power from multiple PV modules. In this case it is important to detect the
faults Energies
generated by reverse currents [13]. The fault detection based on the current flow checks
2018, 11, 2252
for the
5 of 18
current flow in the opposite direction. Using a plurality of photovoltaic elements provides a system
for sensing
checksand for interrupting
the current flow a reverse
in thecurrent.
oppositeEach conductor
direction. Usingisa associated
plurality ofwith a sensor. elements
photovoltaic The sensor
is ableprovides
to detecta the
system for sensing
current flow inandtheinterrupting
respective aconductor,
reverse current.
and itEach conductor to
is configured is associated
provide an with a
output
sensor. The sensor is able to detect the current flow in the respective conductor,
of the detected direction of the current. The method proposed in this paper presents a fault detectionand it is configured
in a PVtosystem
providewherean output of the detected
the prompt direction
identification of the PV
of faulty current.
cells The
takingmethod
placeproposed in thisand
using thermal paperCCD
presents a fault detection in a PV system where the prompt identification of faulty PV cells taking
cameras which they can detect internal and external defects. They are mounted on a drone which can
place using thermal and CCD cameras which they can detect internal and external defects. They are
fly over the PV system and recorded videos where the image processing techniques will be executed
mounted on a drone which can fly over the PV system and recorded videos where the image
in order to detect the faults in PV modules and determine the longitude and latitude of the modules
processing techniques will be executed in order to detect the faults in PV modules and determine the
whichlongitude
allows for andbetter system
latitude of theoperation and enhance
modules which safety.
allows for better system operation and enhance safety.

2. Materials and Methods

2. Materials and Methods
The system description
The system is shown
description is shown ininFigure
Figure2.2. The proposedsystem
The proposed system consists
consists of two
of two cameras,
cameras, a
a thermal
thermalandand
a CCD
a CCD one.
one.TheThetwo
two cameras
cameras are able to capture
able to capturethe thescene
sceneofofthethePVPVmodules
modules
simultaneously. The cameras
simultaneously. are placed
The cameras on a on
are placed drone which
a drone is able
which to flytoover
is able the PV
fly over the modules.
PV modules. TheThe
video
video
recorded onrecorded on both
both cameras cancameras can be monitored
be monitored at groundatstations
ground stations for the thermal
for the thermal and CCD andcameras.
CCD
cameras.videos
The recorded The recorded videos on
are processed are aprocessed
multicoreonCPUa multicore
in orderCPU in order
to divide theto divide
input the input
video video
into segments
into segments and start the running of fault detection algorithms in order to detect
and start the running of fault detection algorithms in order to detect and locate the defective panels. and locate the
defective panels. The proposed system is running on the multicore CPU using a multiprocessing
The proposed system is running on the multicore CPU using a multiprocessing module in Python.
module in Python. The main steps for the proposed system are shown in Figure 3.
The main steps for the proposed system are shown in Figure 3.
Image processing techniques are implemented in order to detect the defects in the PV module.
Image processing
We implement techniques
algorithms whichare areimplemented
able to detect in order defects
different to detect thePVdefects
in the modules.in Each
the PV
module. We implement
algorithm algorithms
is implemented separatelywhich are ableinput
on different to detect different
data, and defects is
each algorithm inexplained
the PV modules.
in the
Each algorithm is implemented separately on different input data, and each algorithm is explained in
following sections.
the following sections.

Quadcopter
Multicore CPU
Thermal Camera

CCD Camera

PV PV

Ground Station for Ground Station for

the Thermal Video the CCD Video
Processing Processing

Figure 2. System description showing main components of the hardware used.

Figure 2. System description showing main components of the hardware used.
Energies 2018, 11, 2252 6 of 18
Energies 2018, 11, 2252 6 of 18

Fly the drone over the PV modules and start recording

on thermal and CCD cameras

Read the input video

Choose the number of processes for thermal and CCD

videos processing

Call the Multiprocessing module to start the video

segmentation process

Divide the CCD Divide the thermal

video by calling video by calling
ffmpeg ffmpeg

Save the CCD video Save the thermal

segments in a specific video segments in a
path specific path

Call the Multiprocessing module and send each thermal and

CCD segment to a specific process

Each process starts fault detection operation on a

specific thermal and CCD segments simultaneously

No
Does the solar garden
has defective panels?

Yes

Analysis the frame to determine the location of the

defective panel on the map, then send an alert.

Save the output results

Exit from Multiprocessing module

Figure 3. Proposed algorithm flow chart.

Figure 3. Proposed algorithm flow chart.
Energies 2018, 11, 2252 7 of 18

2.1. Morphological Transformation and Canny Edge Algorithm

Energies 2018, 11, 2252 7 of 18
The first proposed framework is shown in Figure 4. Starting with converting the input image
2.1. Morphological Transformation and Canny Edge Algorithm
to grayscale in order to set the threshold values which help in edge detection for the binary images,
we seek to The first proposed
identify regionsframework
of interestisin shown in Figure
the input 4. Starting
thermal image.with converting
Since defective the solar
input image to
cells show a
grayscale in order to set the threshold values which help in edge detection
strong yellow or white color with sharp edges in thermal images, it should be quite straightforward for the binary images, we
seek to identify
to recognize regionsregions of interest
of interest. in thealgorithm,
In this input thermal afterimage. Since defective
converting the input solar cells to
image show a
a binary
strong yellow or white color with sharp edges in thermal images, it should be quite straightforward
image, the thresholding process is applied to each frame. The threshold value, Th, for the CCD and
to recognize regions of interest. In this algorithm, after converting the input image to a binary image,
thermal frames to be determined adaptively and at some experiments, was recomputed for each frame.
the thresholding process is applied to each frame. The threshold value, Th, for the CCD and thermal
In addition, morphological transformations were used after assigning a kernel (structuring element).
frames to be determined adaptively and at some experiments, was recomputed for each frame. In
The first morphological
addition, morphological transformation
transformations technique
were used is erosion. The main
after assigning idea (structuring
a kernel of erosion iselement).
that a pixel
will be considered 1 only if all the pixels under the kernel are 1; otherwise,
The first morphological transformation technique is erosion. The main idea of erosion is that a pixelit is eroded [26]. In other
words,
willallbethe pixels near
considered the ifboundary
1 only willunder
all the pixels be discarded
the kerneldepending upon the
are 1; otherwise, it issize
eroded of the kernel,
[26]. which
In other
means white
words, allregion decreases
the pixels near theinboundary
the image. will The second morphological
be discarded depending upontransformation
the size of the kernel,technique
which means
is dilation; the idea white region is
of dilation decreases in theofimage.
the opposite erosion:Thea second
pixel ismorphological
1 if at least one transformation
pixel under the
kernel is 1. In other words, dilation increases the white region in the image or the size ofone
technique is dilation; the idea of dilation is the opposite of erosion: a pixel is 1 if at least pixel
foreground
object increases [26]. Then the defective cells in the PV module can be detected using the Cannyofedge
under the kernel is 1. In other words, dilation increases the white region in the image or the size
foreground object increases [26]. Then the defective cells in the PV module can be detected using the
detection algorithm. The Canny edge detection algorithm is an optimal edge detection algorithm [27]
Canny edge detection algorithm. The Canny edge detection algorithm is an optimal edge detection
that has been used in many practical problems with excellent performance results.
algorithm [27] that has been used in many practical problems with excellent performance results.

Each Process reads its dedicated segment

Convert the input color frame to grayscale then to

Set the kernel size and thresholding value

apply image Morphological transformation, then

apply Canny edge detection

Save the output frame

Figure 4. Use of the morphological and canny algorithms.

Figure 4. Use of the morphological and canny algorithms.
2.2. Segmentation Based on Hot Pixels Seeds Algorithm
2.2. Segmentation Based on Hot Pixels Seeds Algorithm
The segmentation process divides an image into constitutive objects or parts [28]. Segmentation
usually allows for object
The segmentation classification,
process pattern
divides an recognition,
image into and/or the identification
constitutive of clusters
objects or parts[28]. [28].
Most segmentation algorithms are based on similarity or discontinuity features, such
Segmentation usually allows for object classification, pattern recognition, and/or the identification as edges and of
lines between different pixels [28]. The proposed segmentation based hot pixel detection
clusters [28]. Most segmentation algorithms are based on similarity or discontinuity features, such as algorithm
determines a seed pixel 𝑆𝑃 (hot pixel) selection in the input image. Figure 5 shows the main steps of
edges and lines between different pixels [28]. The proposed segmentation based hot pixel detection
the proposed algorithm. The pre-processing processes, Gaussian filter, and histogram equalization
algorithm determines a seed pixel SP (hot pixel) selection in the input image. Figure 5 shows the
for the input images could solve the low contrast problem which makes thresholding more difficult.
main steps of the proposed algorithm. The pre-processing processes, Gaussian filter, and histogram
After image pre-processing, the value of the hottest pixel is determined by finding the highest pixel
equalization
value usingforEquation
the input(1).images couldallsolve
Then check the low contrast
the neighboring pixels problem
if they arewhich
relatedmakes thresholding
to the hot pixel,
morecalling
difficult. After image pre-processing, the value of the hottest
them seed pixels 𝑆𝑃 , or to the background 𝐵𝑃 based on Equation (2):pixel is determined by finding the
highest pixel value using Equation (1). Then check all the neighboring pixels if they are related to the
(1)
𝐻𝑜𝑡𝑃𝑖𝑥𝑒𝑙
hot pixel, calling them seed pixels SP , or to the(𝑝𝑖𝑥𝑒𝑙[𝑟𝑜𝑤,
= 𝑀𝐴𝑋 background 𝑐𝑜𝑙𝑢𝑚𝑛]),
BP based on Equation (2):

Hot Pixel = MAX ( pixel [row, column]), (1)

Energies 2018, 11, 2252 8 of 18

(
Energies 2018, 11, 2252 8 of 18
pixel [row, column] ≥ ( Hot Pixel − margin), Sure_SP
SP = , (2)
pixel [row, column] < ( Hot Pixel − margin), Sure_BP (2)
𝑝𝑖𝑥𝑒𝑙[𝑟𝑜𝑤, 𝑐𝑜𝑙𝑢𝑚𝑛] ≥ (𝐻𝑜𝑡𝑃𝑖𝑥𝑒𝑙 − 𝑚𝑎𝑟𝑔𝑖𝑛), 𝑆𝑢𝑟𝑒_𝑆𝑃
𝑆𝑃 = { ,
𝑝𝑖𝑥𝑒𝑙[𝑟𝑜𝑤, 𝑐𝑜𝑙𝑢𝑚𝑛] < (𝐻𝑜𝑡𝑃𝑖𝑥𝑒𝑙 − 𝑚𝑎𝑟𝑔𝑖𝑛), 𝑆𝑢𝑟𝑒_𝐵𝑃

Load input image

Apply Gaussian Filter

Apply Histogram

Equalization

Find the hottest pixel

𝐻𝑜𝑡𝑃𝑖𝑥𝑒𝑙

Find the average of the mean

values for all pixels

>= 𝐻𝑜𝑡𝑃𝑖𝑥𝑒𝑙
𝒑𝒊𝒙𝒆𝒍 ≥ 𝑯𝒐𝒕𝑷𝒊𝒙𝒆𝒍 -margin
and choose the

Yes µ_(ℎ𝑜𝑡_𝑝𝑖𝑥𝑒𝑙𝑠)
No

Set pixel as a 𝑆𝑢𝑟𝑒_𝑆𝑃

background Calculate the

𝑆𝑢𝑟𝑒_𝐵𝑃 mean µ𝑆𝑃 for the sure

hot pixel

Calculate the
µ𝑆𝑃
mean value for Calculate the standard
≥ µℎ𝑜𝑡𝑝𝑖𝑥𝑒𝑙𝑠
background pixel deviation Ω𝐴𝑐𝑡_𝑆𝑝 for the actual

µ𝐵𝑝 hot pixel

No Yes

Minimal Deviation Distance

Delta (δ)
𝑀𝐷𝐷 = 𝑚𝑖𝑛 ((Ω𝐴𝑐𝑡_𝑆𝑝 )2 )

δ≤

MDD
No Yes

𝐵𝑃 = 0 𝐵𝑃 = 255

Figure 5. Segmentation based on hot pixels seeds.

Figure 5. Segmentation based on hot pixels seeds.
Once the seed pixel 𝑆𝑃 is detected, the mean value µ𝑆𝑃 is calculated using Equation (3). µ𝑆𝑃 is
computed
Once thefor eachpixel
seed seed region with eight neighboring
SP is detected, pixels of
the mean value uSthe
P
is𝑆𝑃calculated
: using Equation (3). uSP is
computed for each seed region with eight neighboring pixels of the SP :

(∑9,9
row=0, column=0 pixel [row, column )
uSP = , (3)
9
Energies 2018, 11, 2252 9 of 18

At the same time, an average value for all hot pixels uhot_pixels is computed for each thermal frame,
however, for the CCD frames, uhot_pixels = 127 was used which is a value that worked well in most
cases. This actually brings parameter selection as one of the issues to be investigated further and hopefully
develop an adaptive method for its selection The actual seed pixel Act_S p is determined using Equation (4):
(
uSP ≥ uhotpixels SureAct_Sp
Act_Sp = , (4)
uSP < uhot_pixels , BP

In addition, for each actual hot pixel, the Minimal Deviation Distance (MDD) is calculated using
Equation (5), where the standard deviation is computed using Equation (6):
 2
MDD = min ( Ω Act_S p ), (5)

Ω Act_S p = S p − pixel [row, column],

 
(6)

Regarding the background pixel, the mean value u B p is computed for each pixel with its’
8 neighbors, then delta value δ is computed using Equation (7). MDD value is used to determine if the
BP is defected based on Equation (8).
  
δ = mean B p − uSP  , (7)
 

(
δ ≤ MDD, defected pixel BP = 1
De f ected BP = , (8)
δ > MDD, not_defected pixel BP = 0
If MDD is greater than (δ), then the background pixel is considered as a defected (hot) pixel (255);
otherwise it is considered as a (zero) pixel.

3. Results
Images have been taken for our remote PV system in the Digital Image and Signal Processing
(DISPLAY) lab at Western Michigan University (WMU) in order to validate our proposed work.
In addition, the system has been taken outside of the college of engineering and the drone has
been flown over the system. The system has two panels of OPTIMUS 60 Cell modules (Model
OPT285-60-4-1B0, SUNIVA, GA & MI, USA); each panel produces 285 W. Table 1 shows the main
specifications of the PV modules in the DISPLAY lab. The whole system was built in our lab taking
care of all the required components, connections, measurement, and mounting.

Table 1. System specifications of the WMU PV module in the DISPLAY lab.

Electrical Specifications (Nominal)

Max Power 285 W
Open Circuit Voltage (Voc) 39.0 V
Short Circuit Current (Isc) 9.37 A
Max Power Voltage (Vmp) 32.2 V
Max Power Current (Imp) 8.86 A
Max Series Fuse Current 15 A
Maximum System Voltage 1000 VDC (UL) 1000 VDC (IEC)
Mechanical Specifications
Cells 60
Module Dimensions Aprrox. 65.40 × 39 in.
Module Thickness 1.37 or 1.57 in.
Weight 39.5 ± 0.5 lb.
Limits
Operating Module Temperature −40 F to +185 F
Storm Resistance/Static Load Tested to IEC 61215 for loads of 5400 Pa (113 psf); hail and resistance
Energies 2018, 11, 2252 10 of 18
Energies 2018, 11, 2252 10 of 18

Figure 6. Schematic
Figure 6. Schematic diagram
diagram of
of the
the WMU
WMU PV
PV module
module system
system in
in the
the DISPLAY
DISPLAYLab.
Lab.
Energies 2018,
Energies 2018, 11,
11, 2252
2252 11
11 of 18
18
Energies 2018, 11, 2252 11 of 18
Figure 6 shows the schematic diagram for the PV system while Figure 7 shows the complete
Figure
Figure 66shows
shows
installed system the schematic
the
in DISPLAY. In diagram
schematic
addition for
diagram the
for PV
to our the
ownsystem
PV while
system
data, Figure
while
we used 7 shows
Figure
images 7 the
taken complete
shows
for installed
themodules
PV complete at
system in
installed DISPLAY.
the main system In addition
campusinofDISPLAY.
WMU, while to our own
In addition
the drone data,
toisour we used
ownover
flying images
data,them,
we usedtaken for
and images PV modules
takenonline
others from for PV at the
modules
sources, mainat
with
campus
the mainofcampus
permission.WMU, while the drone
of WMU, while is flying
the droneover them, over
is flying and others
them, from online from
and others sources, withsources,
online permission.
with
permission.

Figure 7. Our PV module in the DISPLAY Lab at WMU.

Figure 7. Our PV module in the DISPLAY Lab at WMU.
Figure 7. Our PV module in the DISPLAY Lab at WMU.
The thermal camera that has been used in our system is a Vue Pro (FLIR, OR, USA). Thermal
The thermal
The thermalcamera
camera that hashasbeen used in in
our system is a is
Vue Pro (FLIR, OR, USA). Thermal video
video is recorded with thethat
NTSC been
frame used
rate, andourwithsystem
a resolutiona Vue Pro ×(FLIR,
of 336 OR, USA).
256 pixels. Thermal
This resolution
is recorded
video
is highisenough
with the
recorded towith
NTSCNTSC
showthe
frameframe
an accurate
rate, rate,
and with with
thermal and
a resolution
resolution a resolutionof 336
from the PV
×
336256
of modules.pixels.
× 256 This
pixels.
The CCD
resolution
This resolution
camera
is
that
high
is enough
high enough to show an accurate thermal resolution from the PV modules. The CCD camera that has
has been used toin show an accurate
our system is thethermal
Hero 4resolution
Black CCD from the PV
camera modules.
(GoPro, The CCD
Oakland, CA,camera
USA);thatthe
beenbeen
has usedusedin ourinsystem
our is the Hero
system is the 4 Black
Hero CCD
4 camera
Black CCD (GoPro,
camera Oakland,
(GoPro, CA, USA); CA,
Oakland, the camera
USA); has
the
camera has effective photo resolution of 12.0 MP, and the max video resolution 3840 × 2160. These
effectivehas
camera photo resolution
two cameraseffective photoof
are installed
12.0 MP, and
onresolution
the max
of 12.0
a Q500 quadcopter
video
MP,(Yuneec,
and the resolution
maxUSA)
CA, video 3840 × 2160. These
asresolution
shown in 3840Figure×two
2160.cameras
8. TheThese
FLIR
are
two installed
Vue cameras on
Pro thermal a Q500 quadcopter
are installed
camera ison a Q500
shown (Yuneec, CA,
in quadcopter
Figure 8a, theUSA) as
(Yuneec, shown
GoPro CA, camera in
USA)Figure 8. The
as shown
is shown FLIR Vue
in Figure
in Figure Pro
8b, 8. The
ST-10 thermal
FLIR
is the
camera
Vue Pro is shown
thermal in Figure
camera is 8a, the
shown GoPro
in camera
Figure 8a, is
the shown
GoPro in Figure
camera is 8b, ST-10
shown
personal ground station as shown in Figure 8c. The ST-10 station is used to monitor the CCD camera in is the
Figurepersonal
8b, ST-10 ground
is the
station
photos as
personal andshown
ground in Figure
station
videos, 8c. The the
andascontrol
shown ST-10 station
in Figure
drone. A is
8c. Theused to Black
ST-10
Flysight monitor
station theused
is
Pearl CCD tocamera
RC801 monitor photos
the CCD
FPV Monitor and videos,
camera
shown in
and control
photos and the drone.
videos, and A Flysight
control theBlack
drone.Pearl
A RC801
Flysight FPV
Figure 8d is used to monitor the scene captured by the FLIR thermal camera. Black Monitor
Pearl shown
RC801 in
FPV Figure 8d
Monitor is used
shown to
in
monitor8dthe
Figure is scene
used to captured
monitorby thethe FLIRcaptured
scene thermal by camera.
the FLIR thermal camera.

Figure 8. Drone system with thermal and CCD cameras: (a) FLIR Thermal Camera, (b) GoPro CCD
Figure
Camera,8. Drone system with thermal and CCD cameras: (a) FLIR Thermal Camera, (b) GoPro CCD
Figure 8. (c) ST-10,
Drone and (d)
system FPV
with monitor.
thermal and CCD cameras: (a) FLIR Thermal Camera, (b) GoPro CCD
Camera, (c) ST-10, and (d) FPV monitor.
Camera, (c) ST-10, and (d) FPV monitor.
We applied some external defects on our system in the DISPLAY lab, such as polystyrene
We
behind theapplied
panel, some external
adhesive paperdefects on our
on the front system
glass, a piecein of
the DISPLAY
gum, lab, of
and a piece such as polystyrene
polystyrene on the
We
behind applied some external defects on our system in the DISPLAY lab, such as polystyrene behind
back of the
the panel, adhesive
panel to mimic apaper
smallon the front
defect, and glass,
we useda piece of gum,
an input andfrom
image a piece of polystyrene
an online on the
source. Python
the panel,
back theadhesive
ofthe paper
panel toIDE
mimic on the front
a small glass, a we
defect, piece of gum, andimage
a piecefrom
of polystyrene on the Python
back of
2.7 on Eclipse platform was usedand used
in order to an input
process the input an form
data onlinethe
source.
thermal and
2.7 on the Eclipse IDE platform was used in order to process the input data form the thermal and
Energies 2018, 11, 2252 12 of 18

the panel
Energies to11,mimic
2018, a small defect, and we used an input image from an online source. Python122.7
2252 of on
18
the Eclipse IDE platform was used in order to process the input data form the thermal and CCD
cameras.
CCD Windows
cameras. 10 Home,
Windows Intel (R)
10 Home, Core
Intel (R)(TM)
Corei5-4210M CPU @2.60
(TM) i5-4210M CPUGHz
@2.60with
GHz8 Gigabyte RAM is
with 8 Gigabyte
used to
RAM is implement the offline
used to implement thesystem.
offlinePython
system.was installed
Python on a Windows
was installed 10 environment,
on a Windows and other
10 environment,
modules,
and otherextensions
modules, and librariesand
extensions were installed
libraries using
were a pip command,
installed for example,
using a pip command,Multiprocessor
for example,
module, Pillow,module,
Multiprocessor NumPy, Pillow,
and matplotlib.
NumPy, Python provides Python
and matplotlib. multipleprovides
modulesmultiple
for image processing
modules for
which used OpenCV.
image processing which used OpenCV.

3.1. Morphological
3.1. Morphological Transformation
Transformation with
with the
the Canny
Canny Edge
Edge Detector
Detector
The morphological
The morphological transformations
transformationsand
andcanny
cannyedge
edgedetector were
detector implemented
were to detect
implemented the the
to detect hot
spots in the PV module, as shown in Figure 9.
hot spots in the PV module, as shown in Figure 9.

Figure 9. WMU defective and healthy PV modules in DISPLAY lab: Morphological and canny detector
Figure
results: 9.(a)WMU defective
installed and healthy
PV module, PV modules
(b) thresholding in DISPLAY
output, lab: output,
(c) dilation Morphological andoutput,
(d) erosion canny
detector
(e) cannyresults: (a) with
algorithm installed PV module,
morphological (b) thresholding
output, and (f) cannyoutput,
without(c) dilation output,
morphological (d) erosion
transformation.
output, (e) canny algorithm with morphological output, and (f) canny without morphological
transformation.
The input thermal image (a) in Figure 9 shows that the panel on the left is healthy while the one on
the right
The is defective.
input thermalPolystyrene
image (a) have been 9applied
in Figure shows behind
that thethe panel
panel ontotheactleft
as an externalwhile
is healthy defect,the
onone
the
sides
on thecausing
right isheating to take
defective. place as hot
Polystyrene have spots.
beenThe thresholded
applied behind image is shown
the panel to act in
as Figure 9b while
an external the
defect,
morphological process result is shown in Figure 9c,d for dilation and erosion, respectively.
on the sides causing heating to take place as hot spots. The thresholded image is shown in Figure 9b The canny
edge detector
while with morphological
the morphological transformation
process result is shown inisFigure
clearly included
9c,d in theand
for dilation resulting
erosion,edges defected
respectively.
region, as shown in Figure 9e. The canny algorithm without using thresholding
The canny edge detector with morphological transformation is clearly included in the resulting and morphological
transformation
edges is shown
defected region, asinshown
Figurein9fFigure
showing 9e. aThe
typical
cannyedge and not without
algorithm isolatingusingthe hot spot. Hot spots
thresholding and
indicate that the PV module is working under abnormal conditions.
morphological transformation is shown in Figure 9f showing a typical edge and not isolating the hot
spot. Hot spots indicate that the PV module is working under abnormal conditions.
3.2. Segmentation Based Hot Pixels Detection
The last section
3.2. Segmentation BasedinHot
thePixels
results shows the use of the seed selection algorithm which is based on
Detection
determining the seed pixels (hot pixels) and using the mean and standard deviation values in order to
The last section in the results shows the use of the seed selection algorithm which is based on
determine the defective pixels and the background pixels. Figure 10 shows the results of the implemented
determining the seed pixels (hot pixels) and using the mean and standard deviation values in order
algorithm where the hot spots have been detected. The input image in Figure 10a is pre-processed using
to determine the defective pixels and the background pixels. Figure 10 shows the results of the
the Gaussian filter and histogram equalization in order to avoid the low contrast pixels as shown in
implemented algorithm where the hot spots have been detected. The input image in Figure 10a is
Figure 10b,c. Then the algorithm which chooses the seed pixels is implemented and it determines if
pre-processed using the Gaussian filter and histogram equalization in order to avoid the low
the background pixels are related to the seed (defective or not) as shown in Figure 10d. The dilation
contrast pixels as shown in Figure 10b,c. Then the algorithm which chooses the seed pixels is
implemented and it determines if the background pixels are related to the seed (defective or not) as
shown in Figure 10d. The dilation morphological transformation is applied after the algorithm to fill
the holes in the output images as shown in Figure 10e. The algorithm is applied on the WMU PV
Energies 2018, 11, 2252 13 of 18

Energies 2018,transformation
morphological 11, 2252 is applied after the algorithm to fill the holes in the output images 13as
of shown
18
Energies 2018, 11, 2252 13 of 18
in Figure 10e. The algorithm is applied on the WMU PV system that we installed in our lab, and the
system that we installed in our lab, and the algorithm was able to detect the hot spots in the PV
algorithm
systemwas thatable to detectinthe
we installed hot spots in
thethe PV module as shown in the
Figure 10. After applying
module as shown in Figureour 10.lab, andapplying
After algorithm was able
the erosion to detect
morphological hot spots in the
transformation, PV
the
the erosion
module morphological
as shown in transformation,
Figure 10. After the defected
applying the PV module
erosion is shown
morphological clear on the right
transformation,
defected PV module is shown clear on the right side of Figure 10e, and there is no significant white side of
the
defected
Figure 10e, and PV module
there is nois shown clear
significant on
whitethe right
area
area on the left side to give any indications of defects.on side
the of Figure
left side to10e,
giveand there
any is no
indicationssignificant
of white
defects.
area on the left side to give any indications of defects.

Figure
Figure 10. Results
10. Results forfor segmentation-based hot
segmentation-based hot pixels
pixels detection
detectionalgorithm:
algorithm:(a)(a)
installed PV module,
installed PV module,
Figure
(b) 10. Results
Gaussian for segmentation-based
filter output, hot pixels output,
(c) histogram equalization detection
(d)algorithm:
algorithm(a) installed
output, andPV(e) module,
erosion
(b) Gaussian filter output,
(b) Gaussian filter
(c) histogram equalization output, (d) algorithm output, and (e) erosion
output, (c) histogram equalization output, (d) algorithm output, and (e) erosion
morphological output.
morphological output.
morphological output.
On our PV module system in the DISPLAY lab, external defects have been applied. Adhesive
On our
paper On
is PVourmodule
the PV defect;
first module itsystem
system is used in
in the the DISPLAY
toDISPLAY
write lab,
theexternal
lab,onexternal
WMU defects
defects
panel glass. havehave11been
been
Figure applied.
applied.
shows Adhesive
howAdhesive
the paper
defects
paper
is thehave is
firstbeen the
defect; first defect;
it is used
detected usingtoit is
the used
write to write
WMU on the
two algorithms WMU on
andpanel the
show thepanel
glass. glass. Figure
Figure 11
comparison 11 shows
showsthem.
between how the defectshave
how the defects
been have been using
detected detected theusing
two the two algorithms
algorithms and the
and show show the comparison
comparison between
between them.
them.

Figure 11. Output results for an external defect using adhesive paper: (a) installed PV module in the
FigureFigure
DISPLAY 11.lab
11. OutputOutput results
atresults
WMU, (b)for
for an an externaldefect
external
morphological defect using adhesive
andusing
canny adhesive paper:
paper:(a)
edge algorithms, installed
(a)
and PVPV
installed module
modulein the
(c) segmentation-basedin the
DISPLAY
DISPLAY lab at
hot pixels lab at WMU, (b)
WMU, algorithm.
detection morphological and canny edge algorithms, and (c) segmentation-based
(b) morphological and canny edge algorithms, and (c) segmentation-based hot
pixelshot pixels detection
detection algorithm.
algorithm.
Energies 2018, 11, 2252 14 of 18

In2018,
Energies Figure 12,
11, 2252 to mimic a small external defect, a piece of gum was put on the PV module14glass of 18
whichEnergies 2018, 11, 2252
is marked by (D1) in and detected as a hot spot. A small piece of polystyrene behind14the of 18
panel
is placed too; this defect
In Figure is represented
12, to mimic by (D2).
a small external The apolystyrene
defect, piece of gumglued on on
was put thethe
back
PV of the panel
module glass has
In Figure
been detected. 12, to mimic a small external defect, a piece of gum was put on the PV module glass which
which is marked by (D1) in and detected as a hot spot. A small piece of polystyrene behind the panel
is marked by (D1) in and detected as a hot spot. A small piece of polystyrene behind the panel is placed
is placed too; this defect is represented by (D2). The polystyrene glued on the back of the panel has
too; this defect
been is represented by (D2). The polystyrene glued on the back of the panel has been detected.
detected.

D2
D2
D1
D1

Figure 12. Output results for for

external and internal defects produced using gumand and polystyrene: (a)
FigureFigure 12. Output
12. Output results
results external
for external andand internaldefects
internal defectsproduced
produced using
usinggumgum andpolystyrene:
polystyrene: (a) (a)
installed PV
installedmodule in
PV module the DISPLAY
in the DISPLAY lab at WMU (D1 is gum defect, D2 is polystyrene defect), (b)
installed PV module in the DISPLAY lablab
at at WMU(D1
WMU (D1isisgum
gum defect,
defect, D2
D2isispolystyrene
polystyrenedefect), (b) (b)
defect),
morphological and and
morphological cannycannyedgeedgealgorithms,
algorithms, and
and (c)
(c) segmentation-
segmentation- basedhot
based hotpixels
pixels detection
detection
morphological and canny edge algorithms, and (c) segmentation- based hot pixels detection algorithm.
algorithm.
algorithm.

An
Aninput
input thermal
An input
thermal image
thermal
image was
image
was taken
was from
taken
taken from
from an
ananonline
onlinesource
online source [29],
source [29], after
[29],after getting
aftergetting
getting the
the permission
permission
the permission from
fromfrom
the source.
the This
source. image
This has
image some
has defective
some PV
defectivemodules.
PV The
modules. image
The was
image used
was to
used test
to
the source. This image has some defective PV modules. The image was used to test the ability of the
test ability
the of
ability these
of
these algorithms
algorithms for fault for fault detection.
13Figure 13the
shows the results of applying the detection algorithms
these algorithms fordetection. Figure Figure
fault detection. shows13 showsresults of applying
the results the detection
of applying algorithms
the detection on the
algorithms
input on the
image. input image.
on the input image.

Figure 13. Output results for defective PV modules: (a) input image adapted from Terrascan Labs
LTD [29], with permission from Bitton Erez, Terrascan Labs LTD, 2016; (b) morphological and canny
Figure 13. Output results for defective PV modules: (a) input image adapted from Terrascan Labs
Figureedge
13. algorithms, (c) segmentation-based
Output results for defective PVhot pixels detection
modules: (a) inputalgorithm.
image adapted from Terrascan Labs
LTD [29], with permission from Bitton Erez, Terrascan Labs LTD, 2016; (b) morphological and canny
LTD algorithms,
edge [29], with permission from Bitton Erez,
(c) segmentation-based Terrascan
hot pixels Labs LTD,
detection 2016; (b) morphological and canny
algorithm.
edge algorithms, (c) segmentation-based hot pixels detection algorithm.
Energies 2018, 11, 2252 15 of 18
Energies 2018, 11, 2252 15 of 18

Figure
Energies 1414shows
2018,
Figure 2252 the
11,shows theoutput resultsfor
output results forthe
themorphological
morphological andand canny
canny edgeedge detector
detector algorithms
15 of 18
algorithms
where
where the defects have been detected and the location of the defective panel is determinedlongitude
the defects have been detected and the location of the defective panel is determined by by
Figure
andlongitude
latitude, and 14external
shows the(E)output results for
and internal (I) the morphological
defects beenand canny edge detector algorithms
and latitude, and external (E) and internal have
(I) defects detected
have beenasdetected
shown in
as Figure
shown 14d.
in
where the defects have been detected and the location of the defective panel is determined by
Figure 14d.
longitude and latitude, and external (E) and internal (I) defects have been detected as shown in
Figure 14d.

E
I
E
E
I
E
E
E
Figure
Figure 14.14.Location
Location information
informationforfor
defective PV modules
defective (external
PV modules (E) and(E)
(external internal (I) defects):
and internal (I) (a)
defects):
input frame, (b) histogram equalization, (c) canny detector output without
(a) input frame, (b) histogram equalization, (c) canny detector output without morphological morphological
Figure 14. Location
transformation, and (d)information for defective
Morphological and cannyPV modules
detector (external
output with (E) and internal
location (I) defects): (a)
information.
transformation,
input frame,and (b)(d)histogram
Morphological and canny
equalization, (c) detector output
canny detector with
outputlocation
withoutinformation.
morphological
transformation,
Figure 15 showsandthe (d) Morphological
output and
results for thecanny detector output
segmentation basedwith
onlocation information.
hot pixels’ algorithm where
theFigure
defects15have
shows beenthe outputand
detected results for the of
the location segmentation
the defective based ondetermined
panel is hot pixels’by algorithm
longitudewhere
theand
defectsFigure
have15 shows the output results for the segmentation based on hot pixels’ algorithm where
latitude asbeen
shown detected and
in Figure the
15e. location
The use of aofCCD
the defective
camera is panel is determined
important by longitude
in order to detect some and
the defects have been detected and the location of the defective panel is determined by longitude
external
latitude defects; in
as shown such as shadows;
Figure 15e. Thethe shadow
use of a CCDcould hide some
camera hot spotsin
is important as order
showntoindetect
Figuresome
15e. external
and latitude as shown in Figure 15e. The use of a CCD camera is important in order to detect some
defects; such as shadows; the shadow could hide some hot spots as shown in Figure 15e.
external defects; such as shadows; the shadow could hide some hot spots as shown in Figure 15e.

Figure 15. Location information for defective PV modules (external defects with shadow): (a) input
frame, (b) Gaussian filter output, (c) thresholding output, (d) segmentation-based hot pixels
FigureFigure
detection 15. Location
15. Location
algorithm information
information
output, forfor
and (e) defectivePV
defective
morphologicalPVoutput.
modules (external
modules (externaldefects with
defects shadow):
with (a) input
shadow): (a) input
frame, (b) Gaussian filter output, (c) thresholding output, (d) segmentation-based hot pixelspixels
frame, (b) Gaussian filter output, (c) thresholding output, (d) segmentation-based hot detection
CCDdetection
imagesalgorithm
can be output,
used inand (e) morphological
order to determine output.
some external defects that cannot be shown as
algorithm output, and (e) morphological output.
hot spots. Figure 16 shows the output results of a Simple Linear Iterative Clustering (SLIC) super
CCD images can be used in order to determine some external defects that cannot be shown as
algorithm where the shadow can be detected. The using of SLIC for fault detection in PV modules
CCD
hot images
spots. can16
Figure beshows
used in
theorder to determine
output results of asome external
Simple Linear defects
Iterativethat cannot (SLIC)
Clustering be shown as hot
super
was discussed in [30].
spots.algorithm
Figure 16where
shows thethe outputcan
shadow results of a Simple
be detected. LinearofIterative
The using SLIC forClustering (SLIC)
fault detection in super algorithm
PV modules
wherewasthediscussed
shadow in [30].
can be detected. The using of SLIC for fault detection in PV modules was discussed
in [30].
Energies 2018, 11, 2252 16 of 18
Energies 2018, 11, 2252 16 of 18

Figure 16.Shadow
Figure16. detection
Shadow using
detection SLICSLIC
using super-pixel algorithm:
super-pixel (a) input
algorithm: (a)image;
input(b)image;
shadow(b)detection.
shadow
detection.
4. Conclusions
4. Conclusions
With the continuous increase of the energy demand, renewable energy has become a main
source With theoffers
which continuous increase of
low production andthe energy demand,
installation costs. Inrenewable energy hasenergy
addition, renewable become cana be
main
of
source
the firstwhich
choiceoffers low production
for energy and installation
usage to reduce the negative costs. In addition, renewable
environmental energydue
effects expected cantobethe
of
the first choice for energy usage to reduce the negative environmental
pollution resulting from the large amount of fuel consumed by vehicles and by power stations foreffects expected due to the
pollution generation.
electricity resulting from the large amount of fuel consumed by vehicles and by power stations for
electricity
In ordergeneration.
to use PV systems more efficiently, producing efficient power, while saving energy,
In order
money and minimizing to use PVthe systems
hazards more
of PVefficiently, producing
systems; this efficient power,
paper presented while saving
a new framework thatenergy,
is able
money
to monitorandday-to-day
minimizing the hazards
real-time of PVofsystems;
operations this paper
a PV system. presented
The system a new and
has thermal framework that is
CCD cameras
able to monitor day-to-day real-time operations of a PV system. The system
which are mounted on a drone and a ground station for each camera. The experimental work shows has thermal and CCD
cameras
that which
internal andare mounted
external faultsonwere
a drone and in
detected a ground
real timestation
which for
willeach
reducecamera. The experimental
any associated hazards
work
and showsthe
increase that PVinternal
system’sand external
efficiency and faults were detected
reliability, especiallyinforreal time
large PV which
systems. will reduce any
associated hazards and
The framework hasincrease
a drone the PV is
which system’s
able to efficiency
fly over PV andmodules
reliability,
andespecially
using faultfordetection
large PV
systems. is able to detect a variety of faults and to send an alert about the fault along with latitude
algorithms,
The framework
and longitude has a drone
information of the which
defectiveis able to fly over
PV module. WePVusedmodules and using
a multicore CPUfault
as a detection
tool that
algorithms, is able to detect a variety of faults and to send an alert about
enhances and improves the processing time for fault detection in a fully operating PV system. the fault along with latitude
and longitude information of the defective PV module. We used a multicore CPU as a tool that
Author
enhances Contributions:
and improves Conceptualization,
the processingI.A.-Q.
time and M.A.; detection
for fault Methodology,
in aM.A.
fullyand I.A.-Q.; Funding
operating Acquisition,
PV system.
I.A.-Q., B.B. and M.A.; Writing, M.A. and I.A.-Q.; Software, M.A. and B.B.; Project Administration, Visualization
and Supervision, I.A.-Q. and M.A.; Formal Analysis and Data Curation, M.A.; Validation, M.A., Q.A.; Reviewing
Author
and Contributions:
Editing, M.A., I.A.-Q.,Conceptualization,
Q.A. and W.S. I.A.-Q. and M.A.; Methodology, M.A. and I.A.-Q.; Funding
Acquisition, I.A.-Q., B.B. and M.A.; Writing, M.A. and I.A.-Q.; Software, M.A. and B.B.; Project Administration,
Funding: This research was partially funded from Consumer Energy, USA.
Visualization and Supervision, I.A.-Q. and M.A.; Formal Analysis and Data Curation, M.A.; Validation, M.A.,
Conflicts of Interest: The authors declare no conflict of interest.
Q.A.; Reviewing and Editing, M.A., I.A.-Q., Q.A. and W.S.

Funding: This research was partially funded from Consumer Energy, USA.
References
Conflicts of interest: The authors declare no conflict of interest.
1. How Solar Cells Work. Available online: http://science.howstuffworks.com/environmental/energy/solar-
cell.htm (accessed on 10 January 2017).
2. Zsiborács, H.; Pintér, G.; Bai, A.; Popp, J.; Gabnai, Z.; Pályi, B.; István, F.; Nóra, H.B.; Christian, G.;
Heidelinde, T.; et al. Comparison of thermal models for ground-mounted south-facing photovoltaic
technologies: A practical case study. Energies 2018, 11, 1114. [CrossRef]
3. Chen, T.; Alsafasfeh, Q.; Pourbabak, H.; Su, W. The Next-generation retail electricity market with customers
and prosumers—A bibliographical survey. Energies 2018, 11, 8. [CrossRef]
Energies 2018, 11, 2252 17 of 18

4. Adetokunbo, A.A.; Luo, J.; Liang, Z.; Qais, A.; Su, W. Intelligent Home Energy Management System for
Distributed Renewable Generators, Dispatchable Residential Loads and Distributed Energy Storage Devices.
In Proceedings of the 2017 8th IEEE International Renewable Energy Congress (IREC), Amman, Jordan,
21–23 March 2017; pp. 1–6.
5. Louy, M.; Tareq, Q.; Al-Jufout, S.; Alsafasfeh, Q.; Wang, C. Effect of dust on the 1-MW photovoltaic power
plant at Tafila Technical University. In Proceedings of the 2017 8th International Renewable Energy Congress
(IREC), Amman, Jordan, 21–23 March 2017; pp. 1–4.
6. Liang, Z.; Alsafasfeh, Q.; Jin, T.; Pourbabak, H.; Su, W. Risk-Constrained Optimal Energy Management
for Virtual Power Plants Considering Correlated Demand Response. IEEE Trans. Smart Grid 2017, 12, 1.
[CrossRef]
7. Arfoa, A.; Alsafasfeh, Q.; Al, S.M. Evaluation of Electric Energy Losses in Southern Governorates of Jordan
Distribution Electric System. Int. J. Energy Eng. 2015, 5, 2.
8. Zsiborács, E.; Pályi, B.; Pintér, G.; Popp, J.; Balogh, P.; Gabnai, Z.; Pető, K.; Farkas, I.; Baranyai, N.H.; Bai, A.
Technical-economic study of cooled crystalline solar modules. Sol. Energy 2016, 140, 227–235. [CrossRef]
9. Pintér, G.; Baranyai, N.H.; Wiliams, A.; Zsiborács, H. Study of Photovoltaics and LED Energy Efficiency:
Case Study in Hungary. Energies 2018, 11, 790. [CrossRef]
10. Zsiborács, H.; Baranyai, N.H.; Vincze, A.; Háber, I.; Pintér, G. Economic and Technical Aspects of Flexible
Storage Photovoltaic Systems in Europe. Energies 2018, 11, 1445. [CrossRef]
11. Review of Failures of Photovoltaic Modules. Available online: http://www.solarquarter.com/index.php/
resources/82-white-papers/6051-review-of-failures-of-photovoltaic-modules-final (accessed on 13 August 2018).
12. Munoz, M.A.; Alonso-García, M.C.; Vela, N.; Chenlo, F. Early degradation of silicon PV modules and
guaranty conditions. Sol. Energy 2011, 85, 2264–2274. [CrossRef]
13. WO/2014/011593 Reverse Current Fault Prevention Insolar Panel. Available online: https://patentscope.
wipo.int/search/en/result.jsf (accessed on 13 August 2018).
14. King, D.L.; Kratochvil, J.A.; Quintana, M.A.; McMahon, T.J. Applications for infrared imaging equipment in
photovoltaic cell, module, and system testing. In Proceedings of the Conference Record of the Twenty-Eighth
IEEE Photovoltaic Specialists Conference-2000 (Cat. No.00CH37036), Anchorage, AK, USA, 15–22 September
2000; pp. 1487–1490.
15. Kendig, D.; Christofferson, J.; Alers, G.; Shakouri, A. Application of thermoreflectance imaging to identify
defects in photovoltaic solar cells. In Proceedings of the 2010 26th Annual IEEE Semiconductor Thermal
Measurement and Management Symposium (SEMI-THERM), Santa Clara, CA, USA, 21–25 February 2010;
pp. 672–676.
16. Zhao, Y.; de Palma, J.; Mosesian, J.; Lyons, R.; Lehman, B. Line-line fault analysis and protection challenges
in solar photovoltaic arrays. IEEE Trans. Ind. Electron. 2013, 60, 3784–3795. [CrossRef]
17. Alam, M.; Khan, F.; Johnson, J.; Flicker, J. PV faults: Overview, modeling, prevention and detection techniques.
In Proceedings of the 2013 IEEE 14th Workshop on Control and Modeling for Power Electronics (COMPEL),
Salt Lake City, UT, USA, 23–26 June 2013; pp. 1–7.
18. Ye, Z. Fault Analysis in Solar Photovoltaic Arrays. Master’s Thesis, Northeastern University, Boston, MA,
USA, December 2010.
19. What is Infrared Imaging? (with pictures). Available online: http://www.wisegeek.com/what-is-infrared-
imaging.htm (accessed on 14 December 2015).
20. Anwar, S.; Efstathiadis, H.; Qazi, S. Handbook of Research on Solar Energy Systems and Technologies; IGI Global:
Hershey, PE, USA, 2012.
21. Gao, X.; Munson, E.; Abousleman, G.; Si, J. Automatic solar panel recognition and defect detection using
infrared imaging. Proc. SPIE 2015, 9476. [CrossRef]
22. Tsai, D.M.; Wu, S.C.; Chiu, W.Y. Defect detection in solar modules using ICA basis images. IEEE Trans.
Ind. Inform. 2013, 9, 122–131. [CrossRef]
23. Tsanakas, J.A.; Chrysostomou, D.; Botsaris, P.N.; Gasteratos, A. Fault diagnosis of photovoltaic modules
through image processing and canny edge detection on field thermographic measurements. Int. J.
Sustain. Energy 2015, 34, 351–372. [CrossRef]
24. Chouder, A.; Silvestre, S. Automatic supervision and fault detection of PV systems based on power losses
analysis. Energy Convers. Manag. 2010, 51, 1929–1937. [CrossRef]
Energies 2018, 11, 2252 18 of 18

25. Luo, Y. Method and Apparatus for Detecting a Fault in a Solar Cell Panel and an Inverter. U.S. Patent Application
12/904,965. US2011090089A1, 21 April 2011.
26. OpenCV: Morphological Transformations. Available online: http://docs.opencv.org/trunk/d9/d61/
tutorial_py_morphological_ops.html (accessed on 22 August 2016).
27. Canny Edge Detection Tutorial. Available online: http://masters.donntu.org/2010/fknt/chudovskaja/
library/article5.htm (accessed on 22 August 2016).
28. Abdel-Qader, I.; Yohali, S.; Abudayyeh, O.; Yehia, S. Segmentation of thermal images for non-destructive
evaluation of bridge decks. NDT E Int. 2008, 41, 395–405. [CrossRef]
29. Solar Panels Thermal and Visual Inspection—Terrascan Labs LTD, Terrascan Labs LTD, 2016. Available online:
http://tmt.co.il/product/solar-panels-thermal-visual-inspection (accessed on 10 August 2016).
30. Alsafasfeh, M.; Abdel-Qader, I.; Bazuin, B. Fault detection in photovoltaic system using SLIC and thermal
images. In Proceedings of the 2017 8th International Conference on Information Technology (ICIT), Amman,
Jordan, 17–18 May 2017; pp. 672–676.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).