Research and develop on PCB defect intelligent

visual inspection robot

MA Can MAO JianXu
Department of Electrical and information Engineering Department of Electrical and information Engineering
Hunan university Hunan university
Changsha 410082, China Changsha 410082, China
Macan5@126.com maojianxu@yahoo.com.cn
MAO JianPing
Fuzhou Vocational and Technical College
Fuzhou 344000, China
mao_jianpin@163.com

Abstract: In order to realize the fast detection of the printed In the study of PCB automatic test system, because
circuit board's quality, a set of PCB defect intelligent automatic foreign research institutions and companies started relatively
detection device based on machine vision is analysed and early, at present, the AOI system of PCB in our country
designed. The paper introduced the robot system market is basically foreign brands. Because the domestic
construction,machine system,electronic control system,the related technology still stay in the initial stage and
design and realization of vision imaging system. software used a exploratory research stage, domestic equipment performance
fast iterative algorithm for image segmentation based on 2D can't compared with the similar foreign equipment. These
maximum between-cluster variance, using the template cause it stays in a passive disadvantage status in the
matching method for PCB image of the automatic detection and competition.
recognition defects. Experiment results on model machine show As the domestic PCB manufacturer can't bear the great
that the design of the intelligent visual inspection robot can economic pressure caused by the expensive price of import
detect PCB defect exactly and effectively, it also can be used in equipment and domestic operators also isn't used to
the actual PCB online detection. operating these equipment. Combining with the actual
Keywords: machine vision; PCB; detection robot; defect reco situation of our country enterprise , the paper studies a PCB
gnition defect intelligent measuring robot based on machine vision.

I. INTRODUCTION The system uses machine vision instead of the human eye for

With the application scope of modern electronic on-line measuring and sorts out the conforming product and

equipment expanding, PCB (PCB：Printed Circuit Board) as nonconforming product through the intelligent machine arm,

the carrier of an integrated, installation of electronic then it accurately detects defects that may appear in PCB

components plays a more and more important role, which production process such as hiatus,spots, burr short circuit,

makes the PCB quality become the core problem in narrow line, open circuit pinhole and so on. The robot can

electronic manufacturing industry. The current domestic improve the speed and precision of PCB quality inspection,

PCB manufacturer still adopt the method of function test and reduce the rate of false detection and missed detection.
II. SYSTEM DESIGN
artificial visual inspection in the production line, this far
couldn't keep up with the pace of PCB manufacturing According to the investigation of actual production site,

technology. Relative to the limitation of the artificial there is a variety of reasons caused PCB defects. These

detection, AOI (AOI：Automatic Optic Inspection) system defects may exist in the process of every working procedure.

has obvious advantage in speed, accuracy, repeatability, According to artificial detection and semi-automatic

reliability and so on. So the AOI system is being applied detection system, the paper designed a PCB defect intelligent

more and more in all kinds of production line. visual inspection robot and developed special-purpose

978-1-4577-0911-1/12/$31.00 ©2012 IEEE

machinery system, electrical control system and imaging
system. Under the premise of ensuring the safety of the
normal production process, it can realize the automatic visual
inspection of PCB defect.
A. System working principle
Intelligent robot visual inspection system is based on
machine vision detection and recognition. First, it gets the
Figure 1. System working principle diagram
B. Automatic detection mechanical system
The mechanical system of intelligent visual inspection
robot mainly includes three parts: machine arm automatic
feeding system, visual imaging and defects detection system
and machine arm automatically sorting material feeding
system.
1) machine arm automatic feeding system: the system
with cameras and vacuum cups is responsible for feeding the
testing PCB automatically and accurately from placed area to
the automatic transmission line. It is controlled by the master
computer. Feeding setting mainly includes three the
positioning tool, which makes sure that the testing PCB can
run according to specified way.
2) visual imaging and defects detection system:
Automatic transmission line equipped with a step into line
which installs 20 fixtures. The fixtures are used to fix PCB
ensuring that it can smoothly through the camera area. The
movement of automatic transmission line is mainly
controlled by the stepping motor and PLC.
3) machine arm automatically sorting material feeding
system: Its system construction and working principle is
same with machine arm automatic feeding system. It would
put the detected PCB into the designated area according to
the signal of image detection and recognition results, for the
purpose of sorting the defective goods .
In order to ensure that the automatic transmission line
can be operated smoothly, it is required to try our best to
reduce the vibration caused by machine operation when
designing and manufacturing the mechanical equipment.
PCB image through the CCD photographic way. Then, it
does computer processing and analysis. At last, it can
real-time automatic sort out not qualified products through
the method of the digital image processing and identification
to judge the defects. The specific working principle is shown
in fig.1 .
C. Electrical control system design
In order to realize the automatic operation of the PCB
defects detection system, a electrical control system as
shown in fig.2 has been developed. The system consists of
two parts: the main control system and production line
control system. The main control system is mainly realizing
the control of machine arm automatic feeding system and
machine arm automatically sorting material feeding system.
Besides, image acquisition and processing is also its main
task. Production line control system is mainly composed by
PLC. It completes the real-time control of visual inspection
workstation, positioning photoelectric switches, solid-state
relay, equipment indicator light alarm, visual light source,
stepping driver and so on.
Figure 2. electrical control system structure
D. Imaging and image acquisition system
In order to guarantee that the PCB surface have equal
light intensity, it is necessary to design the special-purpose
imaging system. As is shown in fig.3, the imaging and image
acquisition system consists of industrial high-speed CCD Z = { z 0 | z 0 ∈ ( 0 , L − 1)} . Any pixel gray level
camera, image processing equipment, LED photosource and f ( m , n ) in the image is the value of the set,
other components. Industrial CCD uses American cognex namely f ( m , n ) ∈ Z . We define the neighboring region
Insight micro-1403 type intelligent camera. lens uses 25 mm average gray level of the pixels in (m, n) as g ( m , n ) which
lens that made in Japanese computar company. Optical is shown as follows:
element used in this paper is stereo light splitter,which makes 1 (W − 1 ) / 2 j = (W −1 ) / 2
g (m , n ) = ∑ ∑ f ( m + i , n + j ) （2）
the PCB board has equal light intensity everywhere. W ×W i = − (W − 1 ) / 2 j = − (W −1 ) / 2

Among them, g ( m , n ) ∈ Z , W is the the width of square

neighborhood window in the pixels f ( m , n ) , which is
generally odd number.
This article apply with [ f ( m , n ), g ( m , n )] to express
C i, j
2-d histogram. We define as the frequency number
Pi j
appened to the vector ( i , j ) and define as the
Figure 3. Imaging and image acquisition system structure frequency happened to the vector ( i , j ) .
III. DEFECT DETECTION ALGORITHM C L −1 L −1
Pij =
M × N
ij
0 ≤ i, j ≤ L − 1, ∑ ∑ Pij = 1
In practical production,the grayscale between target and i= 0 j= 0
（3）
background of the detecting PCB have no obvious difference. Based on the 2d vector (t, s) as a threshold, the image is
Furthermore, uneven illumination and transport line shocks segmented into into two parts: target class and background
also would seriously influence the image quality. Therefore, class. w 0 represent the proportion of the target class in the
it is required to do a series of image processing to the whole image. w 1 represent the proportion of the background
collected image, specific process is shown in fig.4. class in the whole image. Then mean vector of target class
and background class can be shown as follows:
t L −1 L −1 s
1 1
μ 0 = ( μ 0i , μ 0 j )T = [
w0
∑ ∑ iP
i=0 j=0
ij ,
w0
∑∑
i=0 j=0
jPij ] （4）

L −1 L −1 L −1 L −1
1 1 （5）
μ 1 = ( μ 1i , μ 1 j ) T = [
w1
∑∑ iPij ,
w1
∑∑ jPij ]
Figure 4. Defect detection and recognition flow diagram i=t j=0 i=0 j=s

A. Image preprocessing Overall mean μ z is expressed as follows:

L −1 L − 1 L −1 L −1
The target's grayscale of the detecting PCB gained by
μ z = ( μ zi , μ zj ) T = [ ∑ ∑ iP , ∑ ∑ ij jPij ] （6）
image sensor is similar with its background. So it is i= 0 j= 0 i=0 j= 0

necessary to enhance image contrast. The article uses the In this paper, the trace t r δ b of discrete measurement
method of interception type linear transformation. matrix between target class and background class is used to
⎧ c ; f ( x, y ) < a their distance measurement function.
⎪ d −c
g ( x, y ) = ⎨c + [ f ( x, y ) − a ]; a ≤ f ( x, y ) < b
⎪ b−a trδb (t, s) = w0[(μ0i − μzi )2 + (μ0 j − μzj )2 ] + w1[(μ1i − μzi )2 + (μ1 j − μzj )2 ] (7)
⎩ d ; f ( x, y ) ≥ b
（1）
in formula (1), f ( x , y ) represents for the original image, The paper choose the corresponding threshold vector of
g ( x , y ) represents for the transformation image. According the maximum distance measurement function as the optimal
to transformation function formula (1), the gray level map threshold vector ( t * , s * ) of 2d Otsu method. Then we can
from [a, b] to [c, d]. get the best threshold vector from type (3) to type (7).
In addition, aimming at salt &pepper noise and gaussian 1 1 （8）
(t * , s * ) = ( ( μ 0 i + μ 1i ), ( μ 0 j + μ 1 j ) )
noise that polluted the image, this paper uses the 5 x 5 2 2

median filter to remove. Then it does the Laplace sharpness The initial values of t and s are setted to image gray
to the denoised image. level mean t 0 and neighbourhood average gray level mean s 0 .
B. Image segmentation Putting them into type (8) to iterate until both the results of t
It is suppose that M × N image gray level is 0, 1,... , L-1. and s are fixed values, namely t(i+1)=t(i),s(i+1)=s(i).
Let us use set Z to say L gray levels, then C. Defect detection and recognition
 Defects detection is using already divided PCB image
to match with the standard PCB image which does the same
image processing, then judge them qualified or not.
I ( x, y ) = f ( x, y ) ⊗ g ( x, y ) （9）
in formula (9), f ( x , y ) is the original image, g ( x , y ) is the
transformation image, I ( x , y ) is the pixel with different
grayscale value in the corresponding position of the two
images. Obviously, because of the limitation of manufacture
process, it is sure to produce some noise.The paper would
(a) (b) (c)
figure 5. image defects detection
The important and difficult point in the test is how to
get clear binary image. To test and verify the real-time
performance of the algorithm and the stability of the results,
the experiment chooses 100 pieces qualified PCB boards and
40 pieces unqualified PCB boards that have been passed
artificial detection to test. Each PCB detection takes 0.9
seconds. Robot testing result is shown in table 1
TABLE 1 TEST RESULTS
Test results
Classification Accuracy Rate
Qualified Unqualified
qualified 96 4 96%
unqualified 1 39 97.5%
In table 1, we know there are because some judgement
errors. Most production of this is because the paper used area
threshold segmentation to recognize defects at last. Therefore,
how to improve detection rates of the small defects must be
one of research contents in the future.
V. CONCLUDING REMARKS
This paper designs a new type of PCB defect intelligent
visual inspection robot , detailed describes the working
principle of the system, and comprehensive introduces its
machine structure, electric control structure and image
acquisition modes and so on.The experiments show that PCB
defect intelligent visual inspection robot system can resolve
the low detection efficiency and the rate of false detection
and missed detection brought about by artificial detection.
However, PCB defect intelligent visual inspection robot
is not perfect, it must improves in several aspects as follows:
uses area threshold processing method to get the accurate
defects figure.
IV. THE EXPERIMENT RESULTS AND ANALYSIS
The configuration of the main controlling computer
system is Intel Core 2 Duo CPU, 2 GB memory, 2.20 GHz
frequency. Software development environment applies vc
++ 6.0. The different stages of the image processing results
in this algorithm is shown in fig.5.
(d) (f) (g)
1) improving the algorithm and constantly enhancing the
precision and speed of detection. 2) improving the
performance of the machine equipment, and then further
improving the stability of the system and the speed of the
automatic visual inspection.
ACKNOWLEDGMENT
This work is supported by National Natural Science
Foundation of China (Grant no. 61072121, 60835004,
60872130), the Fundamental Research Funds for the Central
Universities of Hunan University.
REFERENCES
[1] FU Zhong-liang.The Making of Method for Image Threshold
Selection.CHINA JOURNAL OF IMAGE AND GRAPHICS[J].2000,5(6)：
466-469.
[2]GAO Na, WANG Yan Wei.Application of the improved OTSU Method in
image Segmentation.DIGITAL TECHNOLOGY AND APPLICATION[J].
2009(12)：91-92.
[3]QI Li-na,ZHANG Bo,WANG Zhan-kai.Application of the OTSU Method
in image Processing.RADIO ENGINEERING OF CHINA[J].2006,36(7)：
25-26.
[4]WU Yiquan,WU Wenyi,PAN Zhe.A fast iterative algorithm for image
segmentation based on 2D maximum between cluster variance.CHINESE
JOURNAL OF STEREOLOGY AND IMAGE ANALYSIS[J].2007,12(3)：
216-220
[5]Zhang Hui,Wang Yaonan,Zhou Bowen,Ge Ji.Research on automatic
visual inspection method and system for foreign substances in medicine
transfusion liquid.JOURNAL OF ELECTRONIC MEASUREMENT AND
INSTRUMENT[J].2010,24(2)：125-130.
[6]ZHANG Jie,WANG Yao-nan,LU Juan.Research on on-line visual system
to detect transfusion solution.COMPUTER ENGINEERING AND
APPLICATION[J].2010,46(23)：231-234.
[7]Hu Yueming,Tan Ying. State and Development of Automatic Optional
Inspection Application in China.CONTROL&AUTOMATION[J].2006,
22(4): 143-147.
[8] Duan F,Wang Y N.A Machine Vision Inspector for Beer Bottle
[J].Int.J.Engineering Applications of Artificial Intelligence, 2007, 20(7):
1013 -1021.
[9] Pun T, Kapur J N,Weszka J S.Entropic thresholding:A new approach
[J].Computer Vision,Graphics and Image Processing,1982,16(3):273-285.
[10] Abutalab A.S.Automatic Thresholding of Gray-level Picture using
two-dimensional Entropies [J]. Pattern Recognition,1989,47(1):22-32.