International Journal of Computer Integrated

Manufacturing

ISSN: 0951-192X (Print) 1362-3052 (Online) Journal homepage: http://www.tandfonline.com/loi/tcim20

A visual inspection system of glass ampoule

packaging defects: effect of lighting configurations

Nathir A. Rawashdeh, Jumana M. Abu-Khalaf, Walid Khraisat & Shireen S. Al-

Hourani

To cite this article: Nathir A. Rawashdeh, Jumana M. Abu-Khalaf, Walid Khraisat & Shireen
S. Al-Hourani (2018): A visual inspection system of glass ampoule packaging defects: effect
of lighting configurations, International Journal of Computer Integrated Manufacturing, DOI:
10.1080/0951192X.2018.1447145

To link to this article: https://doi.org/10.1080/0951192X.2018.1447145

Published online: 07 Mar 2018.

Submit your article to this journal

Article views: 30

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=tcim20
INTERNATIONAL JOURNAL OF COMPUTER INTEGRATED MANUFACTURING, 2018
https://doi.org/10.1080/0951192X.2018.1447145

ARTICLE

A visual inspection system of glass ampoule packaging defects: effect of lighting

configurations
Nathir A. Rawashdeha, Jumana M. Abu-Khalafa, Walid Khraisatb and Shireen S. Al-Houranic
a
Department of Mechatronics Engineering, German Jordanian University, Amman, Jordan; bDepartment of Industrial Engineering, The University of
Jordan, Amman,Jordan; cDepartment of Industrial Engineering, Faculty of Engineering, King Abdulaziz University, Jeddah, Saudi Arabia

ABSTRACT ARTICLE HISTORY

In any pharmaceutical packaging process, error rates must be kept very low to prevent defective or Received 18 March 2017
incorrect medicine from reaching the consumer. Published articles include a focus on the inspection of Accepted 18 February 2018
ampoule content for the presence of contaminants. Methods of inspection include optical cameras that KEYWORDS
record sequential images to detect the movement of floating objects via feature extraction and Process; vision; inspection;
classification. Other methods employ spectral imaging and dying of ampoules to detect cracks. defect; ampoule; vial;
Thermal imaging is also used to inspect structural integrity and mechanical stress; however, this camera; lighting
approach is marked by relatively high cost. This paper describes the enhancement of an optical
vision-based inspection system for a glass ampoule packaging process. The developed solution involves
a reconfiguration of the illumination system of the ampoule trays as well as reprogramming of a real-
time optical camera. The camera is linked to the process controller and triggers the ejection of trays
containing faulty ampoules. Eight defect types are of interest, namely: missing ampoules; ampoules
without a label; crushed ampoules; ampoules with a broken tip – present or missing; ampoules with a
broken base; cracked ampoules; empty ampoules. The original inspection system performance was
inadequate and only detected 2 of the 8 defect types reliably. The improved system which is described
in this paper has adequately detected all aforementioned defect types. Experiments show that the
overall detection rates have improved from 55% to 99.6% on average. Average camera detection
accuracy has improved from 38% to 99.5%. In addition, the false positives, i.e. rejection rates, were
reduced from 45% to 7%.

1. Introduction Specifically, using intelligent camera systems to detect the

presence of contaminating particles such as glass chips, hair,
The pharmaceutical industry is globally growing at a fast pace,
and air bubbles (Ge et al. 2017; Yao and Bo-Xiong 2011; Yao
where production processes have become more complex
et al. 2012). This solution was based on classification of defects
allowing for higher production rates. Nevertheless, potential
by taking particle trajectories into account, which required
risks of drug inconsistent packaging (J Watts 2016), mislabel-
agitation of the ampoule to force the container liquid to
ling (Reed & Gordon, 2016), and presence of impurities
move. Several sequential images were captured and analysed
(Muthiah, Mathews, and Sivashankar 2016) remain substantial.
for particle features including geometric invariant descriptors
In the past, inspection and monitoring of pharmaceutical
and grey level features. The performance achieved a classifica-
products were typically performed manually; however,
tion accuracy over 95% compared to manual inspection accu-
increased safety regulations and guidelines on health pro-
racy around 90%. In other work, ampoule leaks have been
ducts, along with the increased level of automation, have
detected using spectral analysis (Franklin and Charles 1981).
made it difficult to use the traditional approach of human
Here, an electronic circuit has been developed with two fibre
inspection. In particular, manual detection with or without
optic cables that channel the light emission of the ampoule
optical instruments has demonstrated slow inspection rates,
under inspection onto two threshold based photodiodes
and low accuracy (Liu et al. 2015). Thus, solutions which
equipped with spectral filters. This allows the detection of
integrate automated vision systems are now essential to guar-
ampoule cracks, once the ampoules have been treated using
antee product quality control and process optimisation
the dye bath immersion technique.
(Scholz 2016). These systems must have the ability to assess
Despite improvements in glass compositions, the methods
the production process in real-time, detect and quantify pro-
used to fabricate them, and tests which characterise their
duct defects, and make decisions which ensure product
performance, defects are still evident in glass containers
quality.
(Schaut and Weeks 2017). Since 2008, container closure integ-
Since the early nineties, camera-based automatic inspec-
rity (CCI) evaluation has replaced traditional microbial chal-
tion systems have been developed for both particle and cos-
lenge testing in validating sterile products (Simonetti and
metic inspection (Vaczek 2013; Jiedi and Jiangtao 2009).
Amari 2015). In their work, defects of predefined sizes and

CONTACT Nathir A. Rawashdeh Nathir.Rawashdeh@gju.edu.jo Department of Mechatronics Engineering, German Jordanian University, P.O.Box 35247,
Amman 11180, Jordan
© 2018 Informa UK Limited, trading as Taylor & Francis Group
2 N. A. RAWASHDEH ET AL.
types were induced in glass syringes filled with medicine.
Next, four different methods have been implemented to
detect the induced defects. These were vacuum decay, phar-
macopoeial dye ingress test, Novartis specific dye ingress test,
and high-voltage leak detection. The vacuum decay method
resulted in the best performance, reliability, and repeatability
of measurements. Victor et al. (2017) have experimentally
validated a laser-based headspace analysis methodology.
This has an advantage of measuring gas ingress rates of a
pharmaceutical container regardless of the packaging condi-
tions, i.e. fully vacuumed, partially vacuumed, or packaged at
no vacuum. Specifically, an adjustable laser diode wavelength
is transmitted through a glass container to an opposing detec-
tor. If oxygen, carbon dioxide, or water vapour are present in
the container, laser energy will be absorbed and less energy is
received by the detector. This method, however, only mea-
sures the leak rate of a container and does not specify the
geometry of the existing defect.
Besides visual inspection of ampoules for contaminants and
defects, ultrasonic measurement systems have been used to
ensure the quality of liquids bottled in glass containers by
measuring the liquid’s viscosity (Zhao, Basir, and Mittal 2004).
Examples of container defects include cracks, mislabelling,
incorrect level of filling, and inconsistency in the product
size, colour and/or shape. The implementation of such inspec-
tion systems is expected to decrease the risk of medicine
contamination and hence reduce the possibility of the product
being recalled. Other work highlights the importance of light-
ing configurations for automated visual inspection in assuring
product quality in industrial processes (West, Ellis, and
Finkelstein 1988; Johannessen 1994). Here, the lighting con-
figurations are presented with example images of ampoules
under each setup. The configurations are classified by the
illuminant diffusion, i.e. light point diameter from 0° to 360°,
as well as by the angle between the light source and the
optical camera. The two lighting schemes employed in this
paper are described in this reference as diffused front lighting
(using lens ring-light in our work), and diffused oblique lighting
from below (using the LED bar in our work).
Only these two conditions are investigated in this paper, as
the employed camera includes a ring-light around its lens. This
ring light implements the diffused front lighting shining down
on the conveyer belt and ampoule trays. This default config-
uration yielded poor detection rates of some glass-related
defects. Thus, the lighting was modified to shine through the
ampoule base to create a unique light point pattern, and
contrast regions, resulting from the presence of the liquid
inside the ampoules, while unwanted reflections are reduced
to a minimum.
Besides optical cameras, thermal imaging is also used in
industrial inspections. A thorough review is offered in
Bagavathiappan et al. (2013; Gade & Moeslund 2014). This
approach, also known as thermography, is based on visualis-
ing heat signatures or heat propagation in materials to detect
anomalies and structural defects. Example applications include
monitoring of welds, corrosion, machinery, civil structures,
deformation, thermal insulation, printed circuit board faults,
and reactor fatigue points. In the food industry, thermography
can be used to ensure the correct temperature of cooked or
frozen perishables, the filling level of liquid containers, quality
of heat-seals, or the performance of ovens or cooling installa-
tions. Thermography employs thermal imaging cameras that
are more capable, but also more expensive than optical coun-
terparts. It requires a heat source, which can result via
mechanical friction or stress in machinery or materials under
load. When inspecting workpieces however, heat has to be
induced into the material via laser, magnetic, microwave or
ultrasound energy. Then, the thermal camera picks up the
heat propagation in the workpiece, thus enabling the detec-
tion of faults.
In this paper, we did not employ thermography because
heating the ampoules with afore mentioned sources might
damage the liquid medicine contained within the ampoules.
The work presented in this paper is particularly interested in
the detection of ampoule structural defects which can be
present at the beginning of the production process caused
by shipping accidents, or might occur due to human or
machinery handling and packaging. Cracked and fragmented
containers may result in medicine contamination; therefore, it
is important to protect the consumer as well as the brand
reputation by implementing suitable inspection tools. In order
to achieve this objective, a unique lighting and real-time
image processing setup is proposed based on pre-pro-
grammed parameter thresholds and regions of interest (ROI).
This solution has been implemented and tested at Hikma
Pharmaceuticals Co. Ltd. in Amman-Jordan and has resulted
in improved detection rates compared to an older less reliable
configuration (Hikma 2015).
Compared to other medicine containers, ampoules have
the decisive advantage that medication only comes into con-
tact with glass during its entire storage period. Glass is inert,
impermeable and tamper-safe. Ampoules vary in volume from
1 to 30 mL where most ampoules are made from clear glass. In
some cases, amber glass is also used to protect sensitive
content from UV light. Snap-off ampoules can be broken with-
out any special tools. Often, an ampoule manufacturer inte-
grates a one-point-cut (OPC) on the ampoule neck so that it is
easy to break open.
Hikma Pharmaceuticals in Jordan usually produces
ampoules at their sister company in Portugal, which are then
shipped to Jordan for packaging purposes. At Hikma’s factory,
the ampoules are re-opened, filled, and re-sealed. Some
ampoules are delivered open by the manufacturer, such as
the straight-stem and funnel-type ampoules. These are
cleaned, filled and melted closed at the factory. Finally, the
ampoules are packaged in trays to be shipped to the distri-
buter or the end customer. During the manufacturing, ship-
ping, and packaging processes of ampoules several defects
may occur due to mishandling during shipping and the packa-
ging machine tolerances. In order to prevent the defected
products from reaching the end customer, several inspection
devices are usually added to the manufacturing and packa-
ging machines.
2. Ampoule packaging process
The ampoule packaging machine subject to the study was
configured to produce two trays with five ampoules each
 INTERNATIONAL JOURNAL OF COMPUTER INTEGRATED MANUFACTURING
per second; this amounts to 600 ampoules per minute
(Uhlmann 2015). The glass ampoules are lodged into a plastic
tray moulded by the machine itself at the beginning of the
packaging process. The ampoules have stick-on labels applied
automatically in a separate stage preceding the packaging
machine under study. After the trays are populated with
ampoules, they travel on a conveyer belt under an optical
camera for defect inspection. If defect free, the trays are
manually placed into a consumer carton box along with the
medical brochure.
Figure 1 shows a picture of the ampoule packaging
machine under study (Uhlmann 2015). The process starts by
vacuum thermo- forming ampoule trays onto a PVC sheet fed
from a roll. The tray sheet travels upwards under the ampoule
magazine shutter feeders. These feeders are manually filled
with labelled ampoules by operators. Ampoules are placed
with their base sliding on the metal feeder, i.e. pointing out-
wards. As the formed PVC tray sheet moves into position
under the shutter feeders, 10 ampoules are released onto
the tray sheet using gravity as a force. Unfortunately, the
shutter feeders horizontally shake the glass ampoules rather
harshly to ensure ampoules do not lodge themselves in the
Ampoule Shutter Feeders
Ampoule Press-in
Tray Puncher
Tray Sheet Molding
Figure 1. Main components of the ampoule packaging machine.
Figure 2. Ampoule magazine shutter feeders above moving PVC tray sheet.
3
feeder output funnel. This shaking can cause ampoule defects
in this machine, as ampoules may break or crack especially if
they stick together due to imperfectly applied labels, which is
a rare occurrence. The feeder may also fail to release all five
ampoules, thus cause ampoules to be missing from the tray.
A closer view of the magazine shutter feeders is given in
Figure 2. The feeders act as gravity driven planar funnels. A
horizontal link (emphasised with a bidirectional arrow) shakes
the ampoules every second to ensure they are not lodged at
the funnel outlet. It can be observed, that the PVC tray sheet
has two lanes of trays that are each fed by one of the shutter
feeders, which are spatially offset in depth. Each feeder places
five ampoules on a tray per second. The next step entails a
press-in block that moves vertically to secure the ampoules in
the tray mould.
The press-in station is shown in Figure 3. It consists of a
vertically moving pair of plates holding 10 spring-loaded rods
– one per ampoule. One such rod is highlighted with a dashed
line in the figure. These rods gently push the 10 ampoules
down into the two formed trays once every second. In addi-
tion, they are connected to the process programmable logic
controller and send a signal if an ampule is missing. This is
Separated Ampoule Trays
PVC Sheet Roll
4 N. A. RAWASHDEH ET AL.
Figure 3. Vertically moving ampoule press-in station with 10 spring-loaded
presence sensors.
achieved by monitoring the movement of the pushing rods,
since if an ampoule is missing from the tray, the correspond-
ing rod will not move, i.e. will not be pushed upwards. A tray
with missing ampoules is detected by the press-in station, and
will be rejected by a pressurised air stream actuator before the
final packaging stage.
After the ampoules are secured in the tray sheet, a cutter
punches out the populated individual trays, which travel on a
conveyer belt in sequence. The ampoule trays pass under-
neath an optical camera at a rate of two per second (Cognex
2013). The tray image frames are processed in real-time to
detect trays with defective or missing ampoules. The vision
system is also linked to the programmable logic controller to
ensure imperfect trays are removed from the conveyer belt
before final packaging.
3. Ampoule defects
In service for over two decades, the described machine forms
PVC ampoule trays and populates them with imported glass
ampoules. The process is semi-automatic where four workers
are needed to operate the machine. The imported ampoules
have to be unboxed and supplied to the shutter feeders. In
addition, the movement of the PVC roll used for forming the
trays and the computer control have to be monitored. Finally,
quality inspection and final packaging of the ampoule trays
into consumer-ready shipments is also done manually. This
process is inefficient and error prone; thus, a vision system was
installed. The initial configuration was unable to detect all the
defect types and achieve the required zero error that ensures
the end-user receives fault free ampoule packets. Figure 4
shows ampoules with some of the most challenging defects
that need to be caught before final packaging. Such defects
can be caused by: mishandling of ampoules during shipping;
Figure 4. Some ampoule defects. From left to right: broken top (small, missing);
broken top (missing); broken top (present); broken base; cracked.
Table 1. Significance of some ampoule defects and initial vision system detec-
tion performance.
Ampoule tray defect Initial detection performance and issues
Missing ampoule label 100% detection and tray rejection
Missing ampoule 100% detection and tray rejection
Broken ampoule top Detectable if 30% of the tip is missing
Broken ampoule base Not detectable; will leak fluid and contaminate
environment
Damaged label or 100% detection and tray rejection
incomplete wrap
Crack in the ampoule Not detectable; can leak fluid and contaminate
environment
below standard labels that do not adhere completely to the
ampoules; forces on the ampoules in the funnel shakers.
Table 1 shows a list of defects and how well the initial vision
system detected them. The table lists three defects that were
initially not detectable; however, the update to the lighting
configuration and vision programming solved this problem.
These defects are as follows: missing or present broken
ampoule tops; a broken ampoule base; a cracked ampoule.
The detection rates of label-related defects are satisfactory;
however, it was necessary to significantly improve the broken-
and cracked glass related defects. More detail about the
enhancement of the system for defect detection follows in
the section on experimental results.
4. Vision system hardware
As illustrated in Figure 5, the employed vision system consists of
an industrial camera with integrated real-time image processing,
a programming and monitoring software package, as well as a
programmable logic controller to manage the process and eject
Figure 5. System layout of the real-time image processor, control software, and
process controller.
 INTERNATIONAL JOURNAL OF COMPUTER INTEGRATED MANUFACTURING 5

faulty trays (Cognex 2013). The camera module is capable of

processing 100 frames, i.e. parts per second; however, the
ampoule blister machine under study only produces 2 trays per
second. The camera module memory can hold 32 different
image-processing programmes, has a resolution of 752 by 480
pixels, an 8.5mm lens, 4 discrete direct output terminals, and
weighs about 150 g. It requires a 24V power supply and comes
with a number of accessories such as ring lights and colour filters.
It has a compact size of 67 × 41 x 69 mm and weighs 150 g.
The camera module is programmed via a software pack-
age that also allows on-line monitoring of the inspected
parts, and can save images on a file transfer protocol (FTP)
server. It takes up to 100 greyscale images per second and
detects the presence of ampoule trays though programmed
reference points. The ampoule trays pass under the camera
module twice per second, and once detected, are analysed
for defect presence instantly. If a tray is faulty, the camera
module energises an indicator lights to alert the operator
though the direct I/O port. At the same time, it sends a
signal to the PLC though the PROFINET bus, upon which
the PLC activates a pressurised air nozzle to eject the faulty
tray from the conveyer belt.

5. Lighting configurations
The main contribution of this study is to compare the defect
detection performance, before and after the camera pro-
gramming and lighting system upgrade. Before the
upgrade, the camera was illuminating the ampoule trays
via a white ring light amounted around the camera lens.
This configuration complicated the camera programming
and reduced fault detection rates due to the prevalent
light reflection points from the glass ampoules and the
transparent PVC tray. The intensity of reflections could be
reduced by employing a polarising lens filter; however, such
a filter would also reduce the intensity of light points
needed to detect ampoule faults. In other words, the signal
and unwanted reflection noise are of similar intensity and a
polarising filter would not help separate the two. To over-
come unwanted reflections from the ring light, the lighting
was reconfigured by replacing the ring light with a horizon-
tal brightness-adjustable LED bar that significantly reduced
the inconvenient reflections. Figure 6 shows the camera
module and LED illumination configuration.
The LED bar shines light through the length of the
ampoules starting at the ampoule base. The liquid filled
ampoule acts as alight guide and scatters the light upwards
to the camera at certain points, most notably at the base,
neck and tip of the ampoules. A comparison of images
using the vertical ring light and horizontal LED bar is
shown in Figure 7. The shown trays each have a defective
ampoule. The ring light images suffer from reflections that
make it difficult to detect the defective ampoules. On the
other hand, the LED bar light significantly reduces reflec-
tions and shows each defective ampoule with a minimal
light point at the neck. This is exploited in the image
processing programme, described in the next section, to
detect faulty ampoules.
Figure 6. On-line inspection camera with horizontal LED lighting configuration.
6. Real-time image processing
Since the conveyer carries ampoule trays at a rate of two per
seconds, it is critical that the image processing is kept simple.
The employed camera combines, in a small industrial IP67
enclosure, a triggered ring light, real-time image processing
code using the Cognex Vision library, dimension measurement
sensors, and input/output functionality (Cognex 2013). In
order to ensure real-time processing with a cap of 500 ms/
tray, the image processing steps are very simple and rely on
contrast and brightness checking of regions of interest (ROI)
that are programmed to achieve high defect detection, as well
as low false-positive rejection rates.
The contrast and brightness thresholds are fixed at pro-
gramming time and are not set automatically. This is possi-
ble because the horizontal LED light bar, ampoule tray, and
camera are inside a dark enclosure that eliminated the
variable effects of ambient light. The regions of interest
are the light points that are produced by the label, and
the ampoule liquid when the light is applied at the base.
These regions which result from the aforementioned light-
ing configuration are shown in Figure 8. The real-time
image processing steps process two ampoule trays per
second, and are explained here.
Step 1: Detection of ampule tray presence. The ampoule trays
pass in front of the LED light bar that illuminates them from
the base at a rate of two per second. The vision programme
embedded into the camera examines about half of the whole
frame size at a rate of 100 Hz in order to detect the ampoule
trays. This sub-frame is delineated by two horizontal lines
6 N. A. RAWASHDEH ET AL.
(a)
Figure 7. Effect of lighting configuration on defect visibility. Top row: vertical ring light. Bottom row: horizontal LED bar. Columns: (a) present broken top, (b) broken
ampoule base and (c) empty ampoule.
(c)
(b)
(a)
Figure 8. Regions of interest in the vision programme: (a) presence, (b) bright-
ness and (c) contrast.
shown in Figure 8. In this sub-frame, the camera checks the
luminance of a rectangular ROI designed to fit the area of the
ampoule labels, which are white. This ROI is shown in Figure 8
(a), where the threshold, size, and location of this area are set
experimentally.
Step 2: Capturing an image frame. Once a tray is
detected, a full resolution 752 × 480 grayscale image is
taken and registration is performed since the trays are
never perfectly horizontal in the captured frame. This
step enables the registration of all the ROIs needed in
further steps, since they are all located above the indivi-
dual ampoules that are aligned and fitted inside the hold-
ing tray.
Step 3: Detection of faulty ampoules. Several brightness
(Figure 8(b)) and contrast (Figure 8(c)) ROIs are located at
the point where the liquid in the ampoules emits the light
coming from the ampoule bases. A defective ampoule will
(b) (c)
lose some of the liquid, which causes the contrast and bright-
ness thresholds in the ROIs to not be met. The ampoule label
presence is checked via the brightness threshold ROI’s shown
in Figure 8(b)). Similarly, structural damage to the glass
ampoules is detected via the pre-defined contrast threshold
ROI’s shown in Figure 8(c). The ROI thresholds are set once,
experimentally rather than automatically in real time. This is
necessary to reduce processing time and variability due to
ambient lighting. The camera had to be placed in the dark
enclosure for these reasons. If all ROI thresholds are met, the
tray is deemed fault free.
Step 4: Tray passing/ejection signalling. If one of the ROI’s
does not contain pixels that pass the set thresholds, a signal is
sent to the process controller to activate the conveyer blower
that is placed after the camera station to eject the faulty tray.
In this situation, an indicator light is turned red and the
processed image frame is saved for later offline analysis, by
the operator. If a tray passes all the ROI thresholds, the indi-
cator light turns green and the blower remains deactivated to
allow the ampoule tray to proceed on the conveyer to the
final packaging section where two ampoule trays are placed
into end-user cartons along with a brochure for shipping. A
flow diagram summarising the sequential operation of the
vision system is given in Figure 9.
Some examples of detected faulty trays are shown in
Figure 10. In each of the processed frames, the ROI corre-
sponding to a defect changes colour from green to red. In
the first column, an ampoule with a broken neck is
detected via the contrast ROI shown in Figure 8(c). An
ampoule with a missing label is present in the tray in the
centre column. The defect was detected via the brightness
ROI test shown in Fig, 8 (b). Lastly, the right column shows
a tray with a broken top that is missing from the tray. The
defect is caught via the centre contrast ROI shown in
Figure 8(c).
 INTERNATIONAL JOURNAL OF COMPUTER INTEGRATED MANUFACTURING
Figure 9. Vision algorithm flow diagram.
7. Experimental results
The performance of defective ampoule tray detection was
measured via a case study before and after the lighting system
and vision programming were modified. For each of the two
system configurations, three trials were conducted using a
total of 37,500 Midazolam 15 mg/3 mL ampoules. The
machine automatically placed, five at a time, into a total of
7500 transparent plastic trays that underwent visual inspec-
tion. The three trials had sample sizes as follows: Trial 1 used
15,000 ampoules in 3000 trays; Trial 2 used 10,000 ampoules
in 2000 trays; Trial 3 used 12,500 ampoules in 2500 trays. For
each of the eight ampoule defects listed in Tables 2 and 3, 10
defective trays were manually created and inserted into the
trial runs at random points in time to test the camera. This
approach was necessary because defects occur rarely and the
trays move too quickly to practically define the ground truth
by catching the pneumatically ejected trays and determining
whether a defect is present or not. The two tables show of the
7
percentage of detected defective trays for each of the eight
ampoule defect types. It can be seen that the detection rates
of the initial inspection system, before the modifications, were
unsatisfactory for six of the eight defect categories. In contrast,
all eight defect types were detected reliably after the inspec-
tion system upgrade, which included the previously detailed
lighting and programming changes.
In addition to the number of faulty trays detected, the total
numbers of rejected and passed ampoules were also recorded.
These enable the calculation of the Rejection Rate (RR) and
Camera Accuracy (CA) metrics for each trial run. The perfor-
mance before and after are shown in Tables 4 and 5. In
addition, the average defect detection rates over all trials are
illustrated in Figure 11. The results affirm the effectiveness of
the inspection system modifications. The derivation and sig-
nificance of these metrics follow.
The ampoule Rejection Rate metric is similar to a false-
positive measure and should be as low as possible to reduce
loss of non-defective ampoule trays, i.e. loss of good
ampoules. This metric is calculated as follows:
Nrejected
RR ¼  100; (1)
Npassed þ Nrejected
where, per trial run, Nrejected is the total number of rejected
ampoules, and Npassed is the total number of accepted
ampoules. The RR is high if the inspection system is too
sensitive and errors on the side of caution, thus rejecting
ampoule trays that do not contain any faulty ampoules. A
high RR translates to a loss of good ampoules, which poses a
financial loss and reduces yield.
The Camera Accuracy metric measures the system’s ability
to detect faulty ampoules. For example, if the total number of
inspected ampoules were 1000, and there were 30 faulty
undetected ampoules, i.e. not rejected by the system, the CA
would be 97%. Taking into consideration that the camera
should be able to detect all types of defects reliably, the
desired value of CA should be close to 100%. The metric is
calculated as follows:
Npassed  Nmissed
CA ¼  100; (2)
Npassed
where, per trial run, Npassed is the total number of accepted
ampoules, and Nmissed is the number of undetected defective
ampoules out of the passed ampoules. Tables 4 and 5 show that
the average Camera Accuracy has improved from 38% to 99.5
%, and the average Rejection Rate has been improved from 45%
to 7%, over the course of the inspection system upgrade.
The following is a discussion of knowledge gained during
tests. The Broken Ampoule Base defect was synthesised by
braking the ampoules manually and inserting them into trays
for inspection. This defect was detected reliably in the first two
trials after the inspection system has been upgraded, and thus
was omitted in the third trial. It was noticed that the Cracked
Ampoule defect was limited to the case where the ampoule
neck was broken. After the first trial, the plastic tray design
was altered to enable free movement of the broken tip in the
tray for better detection. In addition, an air blower nozzle was
added before the camera stage to remove the broken tip from
8 N. A. RAWASHDEH ET AL.

(a) (b) (c)

Figure 10. Examples of defect detection via real-time image processing using the horizontal LED bar. Columns: (a) present broken top; (b) missing label; (c) missing
broken top.

Table 2. Initial inspection system: percent detection rates of faulty ampoule causing good detection rates, but at a cost of a high rejection
trays.
rate around 15%. This caused the rejection of too many
Ampoule tray defects Trial 1 Trial 2 Trial 3 Average Acceptable? defect-free trays. Consequently, the camera programming
Missing ampoule 100 100 100 100 Yes was modified to check for filled ampoules rather than empty
Ampoule with missing label 100 100 100 100 Yes
Crushed ampoule 50 100 50 66.7 No ones. Ideally, empty ampoules should be checked at ampoule
Broken top – missing 50 40 50 46.7 No production, rather than packaging.
Broken top – present 10 0 10 6.7 No The results show that the in Trial 3 of the upgraded
Broken ampoule base 0 0 – 0 No
Cracked ampoule 60 – – 60 No system test, only 90% of trays passed when the defect was
Empty ampoule 60 – – 60 No Broken top – present as indicated in Table 3. This result is not
perfect; however, a great improvement over the initial
inspection system, where only 10% of trays with this defect
Table 3. Upgraded inspection system: percent detection rates of faulty ampoule where detected – see Table 2. This defect is challenging to
trays.
detect because the ampoule ‘looks’ intact when the top
Ampoule tray defects Trial 1 Trial 2 Trial 3 Average Acceptable?
remains present in the tray, especially considering that the
Missing ampoule 100 100 100 100 Yes
Ampoule with missing label 100 100 100 100 Yes liquid rarely leaks out of the ampoule body when the neck is
Crushed ampoule 100 100 100 100 Yes snapped – probably by design of the ampoule.
Broken top – missing 100 100 100 100 Yes Finally, in the first trial test of the modified inspection sys-
Broken top – present 100 100 90 96.7 Yes
Broken ampoule base 100 100 – 100 Yes tem, the total number of passed defective ampules, i.e. missed
Cracked ampoule 100 – – 100 Yes ampoules, was not recorded precisely. The reason is that the
Empty ampoule 100 – – 100 Yes camera was being tuned for optimal performance in this case.
The limitations of the system include the inability to detect
cracked ampoules, when the liquid has not leaked, because
Table 4. Initial inspection system: inspection performance metrics in percent.
the light point pattern will be similar to a defect-free ampoule.
Metric Trial 1 Trial 2 Trial 3 Average Acceptable?
This could occur if the crack is under the label. In addition, if
Rejection rate 50 44 41 45 No
Camera accuracy 30 40 44 38 No there is a contaminant in the liquid, or if the label has unread-
able text, it will also not be detected.

Table 5. Modified inspection system: inspection performance metrics in percent.

8. Conclusion
Metric Trial 1 Trial 2 Trial 3 Average Acceptable?
Rejection rate 13 5 3 7 Yes The presented inspection system upgrades proved very effec-
Camera accuracy – 99.4 99.6 99.5 Yes tive. Initially, only 2 out of the 8 defect types were detected
reliably. All 8 defect types were detected reliably after the
change in lighting configuration and camera programming,
the tray. The ampoule magazine shutter feeder was also mod- as well as other modifications to the process and tray design.
ified to place the ampoules onto the empty trays more gently Extensive trial runs have been conducted to measure the
to reduce the probability of cracked ampoules. inspection system’s performance before and after the
The Empty Ampoule defect was the most challenging to upgrades. The overall detection rates have improved from
detect. Initially, the inspection system was too sensitive 55% to 99.6%. Camera detection accuracy has improved
 INTERNATIONAL JOURNAL OF COMPUTER INTEGRATED MANUFACTURING
Empty ampoule
Cracked ampoule
Broken ampoule base
Broken top - present
Broken top - missing
Crushed ampoule
Ampoule with missing label
Missing ampoule
0% 10% 20% 30%
Horizontal Lighting
Figure 11. Average defect detection rates for the two lighting configurations.
from 38% to 99.5% on average. In addition, the false positives,
i.e. average rejection rates, were reduced from 45% to 7%. The
system changes included: changing the vertical lighting that
was plagued with reflections, to a horizontal lighting bar that
illuminated the ampoules from the base; reprogramming the
camera sensor to detect all defect types in question; adding an
air blower that removed any broken pieces from the tray
before visual inspection; changing the PVC ampoule tray
design and the shutter feeder mechanism to reduce the
chance of broken ampoules.
Disclosure statement
No potential conflict of interest was reported by the authors.
References
Bagavathiappan, S., B. B. Lahiri, T. Saravanan, J. Philip, and T. Jayakumar.
2013. “Infrared Thermography for Condition Monitoring–A Review.”
Infrared Physics & Technology 60: 35–55. Elsevier.
Cognex. 2013. Checker 4G Camera Module. USA: Cognex Corporation.
Franklin, M. L., and W. J. Charles 1981, ‘Dual Wavelength
Spectrophotometer for Ampoule Leak Detection and Content
Inspection’, Washington, DC: U.S. Patent and Trademark Office, U.S.
Patent 4,300,689, issued November 17.
Gade, R., and T. B. Moeslund. 2014. “Thermal Cameras and Applications: A
Survey.” Machine Vision and Applications 25 (1): 245–262. doi:10.1007/
s00138-013-0570-5.
Ge, J., S. Xie, Y. Wang, J. Liu, H. Zhang, B. Zhou, F. Weng, C. Ru, C. Zhou, M.
Tan, and Y. Sun. 2017. “‘A System for Automated Detection of Ampoule
Injection Impurities’.” IEEE Transactions on Automation Science and
Engineering 14: 1119–1128. doi:10.1109/TASE.2015.2490061.
Hikma. 2015. Hikma Pharmaceuticals PLC. Amman, Jordan.
Jiedi, S., and W. Jiangtao 2009, ‘Recognition of Impurity in Ampoules
Based on Wavelet Packet Decomposition Energy Distribution and
SVM’, Computer Science and Information Technology, ICCSIT 2009. 2nd
IEEE International Conference on, pp.162–166, 8-11 August.
Johannessen, F. 1994. “Lighting for Automated Visual Inspection: Theory
and Practice for Transparent and Glossy Objects.” In Optics for
Productivity in Manufacturing, 399–407. . Society of Photo-Optical
Instrumentation Engineers (SPIE).
Liu, X., Y. Cheng, H. Cui, C. Gui, and P. Cao. 2015. Ampoule Bottle
Detection Equipment Research and Development. In MATEC Web of
Conferences. Vol. 31. EDP Sciences. January.
9
40% 50% 60% 70% 80% 90% 100%
Vertical Lighting
Muthiah, T., L. Mathews, and K. R. Sivashankar. 2016. “Potential Threat of
Meningitis from Ampoule Impurities: Prevention Is Always Better than
Cure!” Indian Journal of Anaesthesia 60 (2): 145. doi:10.4103/0019-
5049.176280.
Reed, A. R., and P. C. Gordon. 2016. “SANS 444: 2014: A New Standard for
Small-Ampoule Labeling and A Chance to Reduce Drug Administration
Errors in South Africa.” South African Medical Journal = Suid-Afrikaanse
Tydskrif Vir Geneeskunde 106 (3): 225.
Schaut, R. A., and W. P. Weeks. 2017. “Historical Review of Glasses Used for
Parenteral Packaging.” PDA Journal of Pharmaceutical Science and
Technology 71 (4): 279–296.
Scholz, M. 2016. “How High-Speed Cameras Improve Quality Control:
Manufacturing Processes and Quality Control are Improved and
Disruptive Problems on Production Lines Eliminated.” Quality 55 (13):
8VS–8VS.
Simonetti, A., and F. Amari. 2015. “Non-Destructive Vacuum Decay Method
for Pre-Filled Syringe Closure Integrity Testing Compared with Dye
Ingress Testing and High-Voltage Leak Detection.” PDA Journal of
Pharmaceutical Science and Technology 69 (1): 108–122. doi:10.5731/
pdajpst.2015.01004.
Uhlmann. 2015. Ampoule Blister Machine, B 1240 59/1633. Laupheim,
Germany: Uhlmann Pac-Systeme GmbH & Co. KG.
Vaczek, D. 2013. “Inspection Milestones.” Pharmaceutical & Medical
Packaging News 21: 8. October 2013.
Victor, K. G., L. Levac, M. Timmins, and J. Veale. 2017. “Method
Development for Container Closure Integrity Evaluation via
Headspace Gas Ingress by Using Frequency Modulation
Spectroscopy.” PDA Journal of Pharmaceutical Science and Technology,.
Vol. 71, No. 6, pp 429-453.
Watts, J. 2016. “More on the Risks of Inconsistent Drug Packaging and
Ampoule Labelling.” Anaesthesia 71 (3): 348–349. doi:10.1111/
anae.2016.71.issue-3.
West, G., T. Ellis, and L. Finkelstein. 1988. “Image Processing and
Understanding for Automatic Inspection.” Transactions of the Institute
of Measurement and Control 10 (5): 265–272. doi:10.1177/
014233128801000505.
Yao, Q., and W. Bo-Xiong 2011, ‘Study of On-Line Inspection Technique for
Foreign Substance in Ampoule’, Artificial Intelligence, Management
Science and Electronic Commerce (AIMSEC), 2011 2nd International
Conference on. IEEE, 2011.
Yao, Q., W. Bo-Xiong, L. Wei, and Y. Chun-Yu. 2012. “‘Inspection of Small
Moving Foreign Substances in Ampoule Based on Cascade Classifiers’.”
Optics and Precision Engineering 1: 28.
Zhao, B., O. Basir, and G. Mittal. 2004. “Conceptual Design and Evaluation
of an Ultrasonic System for Bottled Beverages’ Quality Measurement
and Control.” Transactions of the Institute of Measurement and Control
26 (1): 41–54. doi:10.1191/0142331204tm0107oa.