Professional Documents
Culture Documents
Deploying Ocr Systems in Industrial Manufacturing - Whitepaperpdf.render
Deploying Ocr Systems in Industrial Manufacturing - Whitepaperpdf.render
D E S I G N
EDITORIAL DIGEST
Deploying OCR
in industrial
manufacturing
Todays manufacturing environments
2
Machine vision
system makes light of
high-speed printing
inspection
10
15
Machine Vision
Adds Traceability to
Packaging
Reprinted with revisions to format from Vision Systems Design. Copyright 2014 by PennWell Corporation
paper handling equipment is often used. Previously, paper would be fed into a
printer, printed and stacked into folds where it could be manually inspected by
an operator. Now the printed paper is rewound onto a paper handling device
at speeds up to 1000 ft per minute and beyond, further limiting visual manual
inspection.
Hence, these organizations are now faced with the challenge of inspecting full
color documents containing variable, personalized information to ensure that the
quality of the printed documents meet the needs of their customers.
Inspecting the image
To help such document services organizations address these quality control
issues, engineers at Videk (Fairport, NY, USA; www.videk.com) have developed a
vision-based inspection system specifically for the task (Figure 1).
The DocuVision 8600 inline system inspects full pages of colored printed material
in real time and at high speed to verify the print quality of the pages and the
integrity of the data printed on them. The system can be integrated directly
into printers from Canon-Oc (Melville, NY, USA; www.usa.canon.com), Kodak
(Rochester, NY, USA; www.kodak.com), Pitney Bowes (Stamford, CT, USA; www.
pb.com), Hewlett Packard (Palo Alto, CA; www.hp.com), Xerox (Norwalk, CT, USA;
www.xerox.com) and Ricoh (Malvern, PA, USA; www.ricoh-usa.com), or can be
placed downstream from a printer via a custom designed transportation stand
designed by Videk and manufactured by EMT International (Hobart, WI, USA;
www.emtinternational.com).
To inspect printed materials for errors and defects as they move continuously
along a web at speeds up to 200 ins/s, DocuVision captures images of each
printed document at a resolution of 200 dpi using up to four Piranha4 4k
color line-scan cameras from Teledyne DALSA (Waterloo, ON, Canada; www.
teledynedalsa.com) that feature 4096 x 3 pixels, a 10 m pixel size and a
maximum line rate of 70 kHz.
Most lenses available today feature automatic focus and aperture and optimized
for the smaller frame size of digital cameras. However, for the DocuVision 8600
system, Videk chose to fit the cameras with F-mount 55mm manual focus lenses
Vision Systems Design :: EDITORIAL DIGEST
from Nikon (Tokyo, Japan; www.nikon.com) which are situated two feet above the
web along which the printed material travels (Figure 2).
The number of cameras deployed in the system depends on the width of the web
and the resolution of the image that is demanded by the application. A single 4k
camera for example, might be appropriate
to capture images from a 20in web at 200
dpi resolution, while a wider 42in web
would require the use of two cameras to
capture images at the same resolution.
Figure 2: To
Most applications using line scan imaging require synchronization between the
moving object and the camera, and the DocuVision 8600 print inspection system
is no exception. By synchronizing the two,
the system ensures that the geometry of
the image will remain constant even as the
speed of the web changes. In addition, the
system must accurately identify a top of
form position mark on each page which
indicates the position at which the printer
started printing onto the page. Once the
mark is identified, it is used as a trigger
to initiate the capture of images from the
line-scan camera.
The line rates of the Piranha4 4k line-scan
cameras are determined through the use
of an incremental encoder that is coupled
to a drive unit on the web. Data from the
encoder is transferred to a proprietary set
Figure 3: Aside from capturing and analyzing data
of hardware external to the PC dubbed
on documents, DocuVision 8600 can be optionally
PageVision, which was developed to handle fitted with a separate imaging system to detect
all the I/O signals required for the interface perforations in documents, such as those that
contain a check attached to printed material.
between the DocuVison software and
the printer hardware. The top of form
position mark, on the other hand, is detected by a small spot laser reflective
sensor which also transfers the data indicating the start of the document to the
same PageVision module.
Once acquired, the PageVision system feeds both sets of data into the PC host
over a serial connection which it uses to dynamically adjust the line rate of the
cameras and determine when the cameras should be triggered to start to capture
images. The captured images are then analyzed by the DocuVision software.
Aside from capturing and analyzing data on documents, the DocuVision 8600
system can be optionally fitted with additional lighting and cameras to detect
One feature of the system is its ability to determine whether any errors on the
page have been created by a printer whose ink jets have become stuck on or off
due to clogging. Jets that stick on cause streaking resulting in a continuous line
page after page while jets that become clogged can cause a color to drop out or
result in voids in page content.
Vision Systems Design :: EDITORIAL DIGEST
Since most modern documents are printed in color, the software also measures
the correctness of the color on the paper. Brand-essential colors such as those
found in a corporate logo, can be trained into the system and subsequently
monitored for the entire print job when each document is analyzed. In addition,
the software can be set up to read 2D bar codes, linear bar codes, and postal
codes, and perform optical
character recognition (OCR) to
ensure that the printed documents
contain accurate information.
DocuVision also checks the
registration of the cyan, magenta,
yellow and black (CMYK) color
planes by inspecting a test element
such as a crosshair. If a color
moves out of register, the vision
system identifies the problem and
which color has moved. No less
important is the measurement of
the registration of the print itself
either in the vertical or horizontal
direction, which can be resolved to
an accuracy of 0.005ins.
Figure 4: Each
One of the most important attributes of the software, however, is its ability to
perform conditional execution, or executing a sequence of inspection routines on
a document based on the artifacts that are found on the document. In that way,
printed documents comprising a number of different pages can be inspected in a
custom fashion according to the nature of the material printed on them.
Each document inspected by the system can be tagged as having passed or failed
any given vision inspection task. This data can be used to trigger the system to
produce a visual indicator on a display screen to alert an operator (Figure 4) or to
sound an alarm.
If the system detects a catastrophic error in the printing process, it can also stop
Vision Systems Design :: EDITORIAL DIGEST
one or more of the printers, eliminating the expenditure of further paper, ink
and production time. To do so, the DocuVision software delivers an instruction
to halt the printer to the external PageVision module which communicates the
instruction to the printer over an optically isolated output.
Tagged and tracked
When combined with Videks ReCon Manager data collection and reporting
software tool, the 8600 system can also collect and store data relating to the print
quality and document
data fields (such as check
amounts) on an SQL-based
database.
To do so, ReCon Manager
software accepts the
inspection results passed
to it by the DocuVision
system over an Ethernet
connection. A data
Figure 5: ReCon Manager compiles all data on inspections
performed by the DocuVision system and generates production handshake protocol
between the two enables
reports based on customer-specified requirements.
the DocuVision system to
be aware that the data collection system is connected and will alert the operator
via a warning on screen if the connection is lost.
ReCon Manager compiles all data on the inspections performed by the
DocuVision system, and generates production reports based on customerspecified requirements. It allows networked access - either on-site, remote or
mobile - to the production information and serves as a gateway for inspection
data to be communicated to higher-level Automated Document Factory (ADF)
systems via Videks ADF Bridge module (Figure 5).
9
Vision Systems Design :: EDITORIAL DIGEST
10
In this process, barcode labels (or codes laser marked onto the individual boards
within a panel) are affixed to the panel and individual circuit boards before the
screen printing process. To ensure that each of these panels and the individual
circuit boards they contain can be tracked though the entire process, these
barcodes must be read after they are affixed or marked.
Vision Systems Design :: EDITORIAL DIGEST
Differing approaches
In the past, says King, this was
often accomplished by an operator
using a handheld barcode scanner -a
process that was time consuming and
subject to human error. To automate
this process, PCB manufacturers can
incorporate automatic barcode readers
that can read the panel and circuit
codes after they are affixed or marked.
One approach to performing this
task automatically, says King, is to
use an array of cameras or barcode
readers that are positioned over
specific areas of the board where the
barcode will be known to be affixed.
While this solution is effective when
Figure 1: Designed to read multiple barcodes on
large batches of the same product are
circuits on PCB panels, PanelScan is a linebeing manufactured it is not flexible
scan based vision system with a throughput of
since, should a new batch of panels
approximately one board per second.
with different-sized circuits need to be
processed, the positions of each of the
cameras or barcode readers must be reset.
To overcome this limitation, while at the same time increasing throughput and
eliminating human error, Microscan has developed a line-scan-based machine
vision system known as PanelScan (Figure 1). Capable of being mounted to most
in-feed or transport conveyors, PanelScan will be offered in two versions. The
first, PanelScan Standard, incorporates a single line-scan camera and is capable of
scanning PCBs as wide as 10 in while the second, PanelScan Wide, features a dual
line-scan camera combination capable of scanning boards of 18 in width.
Line-scan cameras
11
In the design of the PanelScan Wide system, two raL6144-16gm Racer line-scan
Gigabit Ethernet cameras from Basler (Ahrensburg, Germany; www.baslerweb.
Vision Systems Design :: EDITORIAL DIGEST
com) are mounted on a gantry 13 in from the PCB. Both cameras are fitted
with an AF 60mm Nikor lens from Nikon (Melville, NY; USA: www.nikonusa.
com) that allow each camera to scan a 10 in swatch across the PCB as it moves
though the system. To ensure that the entire
18-in swath is scanned, each camera is
positioned so that the field of view of each
camera overlaps by 1in (Figure 2). This
configuration, says King, allows the system
to read standard barcodes as small as 3.3
mil or Data Matrix 2D codes as small as 5
mil.
To illuminate the PCB as it moves under
the cameras field of view, Microscan has
incorporated two of the companys own
NERLITE HI-BRITE LL-300 Series white lineFigure 2: Two Gigabit Ethernet line-scan cameras
lights into the system. Mounted in an offare used to scan an 18-in swath of the PCB as it
axis configuration, these 300 mm lights are
moves under the field of view of the system.
butted together to form a seamlessly white
line-light that illuminates the PCB as it
moves under the field of view of the dual-camera system.
As a board moves through the system, its leading edge is detected by a retroreflective sensor from Tri-Tronics (Tampa, FL; USA: www.ttco.com), the output of
which is used to trigger the scanning process. Images are then transferred over
each cameras Gigabit Ethernet interface to a host PC. This data is buffered to the
PC in 256 line segments of each 6k x 1 scan. Transferring fifty of these 256, 6k x 1
line segments from both cameras results in an image of 12k x 12k x 8-bit pixels.
Image analysis
12
This data must then be processed to detect both the barcode on each panel and
the individual barcodes associated with each individual circuit board. To perform
this task, Microscan has incorporated its Visionscape machine vision software
package into the system. In addition to incorporating automatic identification
tools for decoding linear and 2D codes, the software also incorporates an array
of image enhancement and analysis tools. However, just as important as the
Vision Systems Design :: EDITORIAL DIGEST
Figure 3: Once
trained, the
locations of each of the
barcodes on each individual
PCB are identified, read and
stored as a concatenated
string.
software tools that are used to perform barcode detection is the graphical user
interface that Microscan has developed for the PanelScan system.
In the development of the GUI, says John Agapakis, Director of Americas Sales
at Microscan, it was necessary to provide an easy method of both configuring
the system and providing fast barcode reading without the operator requiring any
knowledge of machine vision software.
Specifically, each batch of PCBs panels may contain a number of different
circuits in an array of multiple rows and columns (Figure 3). While some panels
may contain twelve circuits configured in a 3 x 4 matrix, others may contain
many more. To identify which type of panel is desired to be inspected, the GUI
allows the operator to enter this matrix format before the panel is scanned.
After this data is entered, a single panel is scanned through the system in
teach mode. After scanning the complete image of the panel, each of the circuit
boards is automatically processed to increase the contrast and the image
displayed on the GUI.
13
After this operation is complete, the operator highlights the center portion
of the top left and bottom right barcode within the image. The system then
automatically searches in all the regions for the barcodes located on each of
the individual circuits and reads each individual barcode. Data from all of the
Vision Systems Design :: EDITORIAL DIGEST
barcodes is then concatenated into a string and stored in the host PC. After this
process is complete, multiple PCB panels of the same configuration can be run
through the system at a rate of approximately one board per second.
Throughout the teach process or the subsequent run-time operation of the
system, no specific machine vision knowledge is required from the operator.
Track and trace
Of course, says King, while barcode reading is important to perform in the
initial stage of PCB fabrication, it is equally important to monitor the panels and
circuits as they move through solder paste inspection, IC placement, automatic
optical inspection, and solder reflow. However, at these stages, it is no longer
necessary to read each of the barcodes associated with each panel and circuit
since these identifiers are all related.
Thus, it is only necessary to perform a single barcode read at each of these stages
to determine which panels and circuits are being processed. This task can be
performed with a single smart camera or barcode reader at each of these stations.
To date, the PanelScan system has been recently validated at a number of PCB
manufacturers including a Tier I automotive electronics manufacturer and a
Top Ten Global EMS (Electronics Manufacturing Services) company, according
to Agapakis. While the system is currently configured to read barcodes on PCB
panels and circuits, it could be equally well deployed in systems in semiconductor
applications where trays of ICs need to be inspected.
Not a Vision Systems Design magazine subscriber? Click here to request a free
subscription.
14
Vision Systems Design :: EDITORIAL DIGEST
15
FIGURE 2. Vision
Sort has
developed an in-line visionbased system that can
automatically identify case
types by the identification
marks on them that are used
to indicate the number and
sizes of products within. The
data are then used to affix an
individual barcode to each case.
As a case of produce moves under the inspection system, its presence is detected
by a photodetector from Banner Engineering, which triggers a pair of 12-in.
LC300 LED strobe lights from Smart Vision Lights positioned at a 30 angle and a
monochrome 2-Mpixel, 2/3-in. format, 50 frames/sec ace camera from Basler.
16
Images from the camera are transferred over a GigE interface to an Intel-based
multicore PC. To identify the types of cases, Gaddy used the HALCON 11 vision
software package from MVTec Software. For the HALCON software to identify
the boxes of produce, the system must first be trained. Images of each type of
box to be identified are first captured as they travel through the system, and
shape model files are created by analyzing the physical dimensions of the box
and artwork printed on it (see Fig. 3). In practice, some boxes require as many
FIGURE 3. Images
of cases
are analyzed by a PC to
determine the type of
box and the nature of
the contents within. The
data are combined with
additional information
about the grower,
packing facility, and
date it was packed
and used to generate a
unique barcode.
as six shape models per box to be captured by the system before one box can be
distinguished from another.
Shape models of the cases are then compared with previously trained models to
rank, or score, the potential matches that are found. All models that are found to
have a score above a set minimum are then run through a scoring algorithm that
ensures the correct case is identified by the software. Should two cases appear
similar in all but color, a grayscale analysis is performed on the image to make
the distinction between cases.
Once the case type has been identified, the location of marked checkboxes is
analyzed to determine the number and size of the fruit in the case. In instances
where the produce type in the cases has been defined by placing a sticker on the
case, or manually stamped, an optical character recognition (OCR) function reads
the characters.
17
The system can search through more than 100 shape models and can classify the
case type and determine which checkboxes have been checked by the packers
in about 65 msec, after which a label is printed and affixed to the case. The total
time taken to identify and affix a label to a case is approximately 0.5 sec.
Vision Systems Design :: EDITORIAL DIGEST
FIGURE 4. All
After the model of the case has been identified, a CLV620-0120 barcode reader
from SICK reads a barcode sticker previously affixed to the back of the case
to identify the packager of the produce. These data, together with the box
identification data, are logged into a database and can be recalled later for
productivity, report generation, or payroll purposes.
Having identified the case type, size, and number of items of produce, the data
are used to create a barcode identifier label. This is transferred over an Ethernet
link to a S84 print engine from Sato that prints the barcode label, which is affixed
to the case by a 250 label applicator from IDTechnology.
System control
A graphical user interface (GUI) written in Microsoft Visual Studio C# enables
the user to control all the parameters of the system from a single touch-panel
and handles the interface to the vision inspection software, the label printing
and application tasks, and data logging, report generation, and alarm handling
functions (see Fig. 4).
18
Custom Visual Studio C# software also monitors critical system parameters, such
as whether the printer is low or out of labels. If these parameters exceed alarm
levels, a text message is sent to maintenance personnel.
Due to the variety of different-sized cases that travel down the conveyor, it
is important that the system be able to position the barcode label at specific
locations according to the type of case. A custom-built positioning system aligns
the boxes vertically and horizontally in front of the label application tool so they
can be affixed in the exact position.
Future models
Although the systems that have currently been shipped use just one camera
to identify and label the cases, future systems will employ two, or even three,
cameras to further increase the functionality of the system. For example, a
second camera will capture an image of the front of the cases as they move down
the conveyor, enabling labels affixed to them by the packers to be automatically
identified. A third camera placed in the hood of the vision system will capture
images of the fruit in open cases. These images can then be analyzed to identify
what type of produce is contained in them, obviating the need for packers to
physically mark the checkboxes on the cases.
The Vision Sort system was developed to minimize the impact to the users
process and be fast enough that the average packing operation would only need
one system per packing line rather than several systems. Two of the current
single-camera systems have been in use since June 2012 at New Leaf Produce.
Additional units were installed later in the summer of 2012 at two other
California grower-shipper facilities. Multiple camera systems are expected to be
shipped to customers during the course of 2013.
Not a Vision Systems Design magazine subscriber? Click here to request a free
subscription.
19
Vision Systems Design :: EDITORIAL DIGEST