354

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 2, FEBRUARY 1998

Bar Code Recovery via the EM Algorithm

William Turin, Senior Member, IEEE, and Robert A. Boie
Abstract The ubiquitous supermarket checkout scanner is indeed a well engineered and effective device. There is, nevertheless, demand for better devices. Existing scanners rely on simple and indeed low-cost signal processing to interpret bar code signals. These methods, nevertheless, fundamentally limit label reading and cannot be extended. A new method based on the deterministic EM algorithm is described here. First results show a substantial improvement in label reading depth of eld, which is an important performance parameter for bar code readers. Index Terms Bar-code decoding, deconvolution, EM algorithm, inverse problems.

I. INTRODUCTION HE MACHINE-readable labeling system of choice in many applications employs bar code labels and ying spot readers (laser scanner). The label data is encoded in the relative widths of alternating light-absorbing, usually black, printed bars and light scattering (diffusely reecting) white spaces arranged like a picket fence. Relative width encoding permits the label reading to be relatively insensitive to geometric effects such as label size, reading distance and, to some degree, label orientation. Spot velocity, in principle, need only be uniform over any label scan. A laser scanner reads a bar code by projecting a bright spot of light that moves across the label while a coaxial photodetector converts the time-varying intensity of light backscattered from the label into a corresponding time-varying electrical signal current. Our ability to read a label depends directly on both the intensity since substantial noise sources are present as are dimensions of the projected spot relative to the bars. Laser power is limited to about 1 mW by eye safety considerations and, hence, provides an upper limit on label illumination and, therefore, the signal intensity. Efforts to improve bar code readers are directed at reducing the constraints on label presentation. Check-out procedures then benet by the resulting reduction in labor and faster processing. Any increase in the depth of eld within which labels are successfully read is advantageous in this regard. Depth of eld is primarily limited by blurring effects. If the beam is in focus along a surface that bisects the reading volume, then labels are minimally blurred since the projected light spot is the smallest. Labels further from the focus are increasingly blurred by the increased spot size. Reading volume then depends on the ability of the signal
Manuscript received September 17, 1996; revised August 12, 1997. The associate editor coordinating the review of this paper and approving it for publication was Dr. Xiang-Gen Xia. The authors are with AT&T Laboratories-Research, Florham Park, NJ 07932-0971 USA (e-mail: wt@research.att.com). Publisher Item Identier S 1053-587X(98)01387-6.

processing to recover signals from blurring. The degradation in bar code signals due to blurring is accurately modeled by the convolution with a Gaussian kernel with variable blurring factor , depending on the labels distance from the focal surface. The problem of bar code decoding is closely related to the problem of edge detection. Conventional methods of edge detection are based on the signal ltering and feature (e.g., zero crossings, peaks, etc.) extraction [1][13]. Alternatively, edges can be found by solving iteratively a system of nonlinear equations [14]. Edge detection can be achieved as a byproduct of reversing the blur (image deblurring). In general, the process of reversing Gaussian blur cannot be represented as a linear lter [15]. However, if we restrict the space of allowable input functions to polynomials of xed degree, then the convolution inverse exists [15], [16]. All these algorithms work well if the blurring is small relative to the bars. Performance is then limited primarily by signal-to-noise ratio (SNR). Nevertheless, as relative blurring increases, reading errors occur in conventional readers that are independent of SNR. Increased signal does not, in these cases, reduce errors. Conventional methods are primarily limited by intersymbol interference (ISI) to a range of spot sizes that produce relatively small blurring. To improve the algorithms performance, the blur factor is estimated from the blurred signal [8], [14], [16]. Recently, Vardi and Lee [17] generalized the statistical expectation-maximization (EM) algorithm [18] for solving deterministic linear inverse problems. They successfully applied this deterministic EM algorithm to image deblurring. This algorithm gives good results if the blur factor is known (see Section III-C). However, in practice, it is usually not known and must be estimated from the bar code signal. Bar code decoding is simpler than edge detection because of the bar code special structure. Bars are parallel, and their widths and gaps between them are multiple of a unit bar width (representing one bit of information). In addition, only a subset of all possible bar sequences represent legal bar codes. This additional information can be incorporated into the edge detection algorithms [7], [8]. The problem of bar code decoding can be formulated in terms of information transmission over communication channel. The original sequence of bars can be viewed as a transmitted signal, and the scanner output signal can be viewed as a received signal. The convolution distortion of the transmitted signal is called ISI. The signal deblurring is performed by the channel equalization. There are many algorithms for communication channel equalization that are

1053587X/98$10.00 1998 IEEE

TURIN AND BOIE: BAR CODE RECOVERY VIA THE EM ALGORITHM

355

presented in textbooks [19], [20] and numerous papers. The main problem of applying these methods to bar code decoding is that communication systems are usually synchronized, which is equivalent to estimating with high accuracy the position of the initial bar and the pulse duration (relative bar width). In practice, however, it is difcult to estimate directly the blur factor or the pulse width . We propose to estimate the signal parameters indirectly by tting the distorted signal to its model. By using the KullbackLeibler divergence (KLD) between the signals [21], we develop a deterministic EM algorithm for solving nonlinear inverse problems and apply this algorithm to estimating the signal parameters. The algorithm is iterative, and it monotonically decreases the KLD. As a byproduct of this algorithm, we estimate both and . The paper is organized into ve sections. In Section II, we present the bar code signal model and Gaussian blur. In Section III, we consider different algorithms for bar code recovery: deconvolution, the EM-based deblurring, and the EM-based decoding. We conclude in Section IV. Major equation derivations are given in the Appendix. II. SIGNAL MODEL A scanner system consists of a light source, optics, photodetector, and signal processing hardware. We assume that that the scanned image has two colors: black and white. While the beam is stationary, the photodetector current has the form (1) includes the white parts of the illuminated region, where and is an elemental contribution of the reected light from the point if the beam spot is centered at . The point spread function is assumed to be symmetrical and can be modeled by Gaussian distribution densities

Fig. 1. Bar code scanner model.

The label reading time is just a few microseconds, and the variation of due orientation is small enough to be ignored. Our experimental data conrm that the assumption is valid. It is convenient to use a complementary function . After the substitution, we obtain const where

If

is a binary representation of a bar code, then (2)

where to a bar

is a rectangular pulse of width if otherwise

corresponding

where is proportional to the beam width, and are the coordinates of the beam center. A label illuminated by a beam is illustrated in Fig. 1. Neither the source nor the photodetector are shown in the gure. The beam focal region is at distance above the label. If the beam center moves, and are functions of time. We assume that the beam moves with constant velocity so that

Our goal is to recover signal or

from the scanner

(3) is a parameter vector that where we need to recover from the scanner signal. This equation shows that a scanner can be modeled as a linear lter with Gaussian impulse response . After substitution of , this equation can be written as

If we further assume that bars have innite length, (1) can be simplied

where is the velocity component in the direction perpendicular to bars, at bars, and at white space (see Fig. 1). By changing the time scale, we can always have . We also assume that so that is constant. This assumption is valid for the following reasons:

After integrating the sum, we obtain (4)

356

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 2, FEBRUARY 1998

Fig. 2. Original signal (solid line) and deblurred signal with optimal

 (dotted line).

where

otherwise, the solution is indeterminate. Indeed, it follows from the identity

erf and erf is the error function.

erf

(5)

that if

is a solution, then

with

III. BAR CODE RECOVERY A. Deconvolution In the absence of noise and interference, we can recover bar code signal by solving (3). Applying the Fourier transform on both sides of (3), we obtain

where and are Fourier transforms of , respectively. Therefore

and

and

is obtained using the inverse Fourier transform of

This equation shows that the solution cannot be found because the inverse Fourier transform of by ltering does not exist. We also need to know to solve the equation that is not the case in bar code recovery because

is also a solution of (3). A trivial solution is and . However, if we limit the set of solutions to the rectangular pulse trains presented by (2), the solution is unique. The existing methods of bar code recovery assume that the blur factor is small; they treat as if it were . If is small, then slightly distorted , and the bar code can be recovered by sampling it at the moments , by using the intersections of with its mean value, or by using zero crossings of its second derivative. However, if the beam covers several bars, then we cannot neglect the contribution of the adjacent elements and to the sum, and the problem of decoding becomes more difcult. This distortion of the sample is called ISI or convolution interference. The problem becomes even more complex if the signal is distorted by other sources of interference (such as overhead lights and paper roughness) and noise. In this case, we need to recover the code from the received signal , which is usually a nonlinear function of , interference signals, and noise. To solve the problem of bar code decoding, we formulate by it as a problem of nding the coefcients tting the scanner signal model presented by (3) to the measured signal .

TURIN AND BOIE: BAR CODE RECOVERY VIA THE EM ALGORITHM

357

Fig. 3.

Original signal (solid line) and deblurred signal with modied

 (dotted line).

B. The EM Algorithm to the measured bar code signal , In order to t we need to select an optimization criterion. By proper scaling and translation, we can make and probability density functions. This allows us to interpret the problem of decoding as a problem of tting a mixture of probability distributions to the measured signal. It is convenient to t the mixture by minimizing the KLD [21], [22]

to solve (7) directly. To develop the iterative EM algorithm, we introduce an auxiliary variable with the probability density function . The augmented variable represents the complete data, whereas represents the incomplete data. Denote

Taking logarithm on both sides of this equation, we obtain Since This equation can be written as It is easy to see that and if and almost everywhere [21]. We would only if like to point out, however, that even if we nd a perfect t [ ], this does not mean that we can nd a unique parameter . If the problem is ill conditioned, it can have many different solutions. Since does not depend on , the minimization of is equivalent to maximization of log-likelihood function (6) The maximum likelihood (ML) model parameter estimates are obtained by maximizing the function (7) Note, that solutions of this equation do not change if we multiply by a positive number. Usually, it is difcult (8) does not depend on , we have

where is some xed value of the parameter vector. Muland integrating, tiplying both sides of this equation by we obtain

where

358

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 2, FEBRUARY 1998

According to Jensens inequality [21], attains its global maximum at . Therefore, if we can nd , which increases , then it will also increase . Thus, we create an iterative process that monotonically increases . The hidden variable is usually selected in such a way that it is easy to nd a maximum of . Then, the ML estimate can be obtained iteratively by the EM algorithm (9) The selection of the hidden variable is not unique. We consider several different methods of choosing the variable. The well-known statistical EM algorithm [18] is a special case of this deterministic algorithm when is a sum of functions, and integrals over in (8) are replaced with sums. It also follows from the above derivation that the algorithm can be applied to any nonnegative functions, as long as all the integrals converge. Note that since , the likelihood is upper bounded by the entropy This upper bound can be reached if and only if almost everywhere. C. Signal Deblurring Consider rst the case of tting with to the scanner signal . In this formulation, we are looking that maximize the likelihood function . for and As we pointed out before, if there is no restrictions on the set of possible solutions, the solution is indeterminate. To nd a solution, we consider two cases: is known or is piecewise constant. If is known, we can apply the EM algorithm to perform deconvolution [17]. To use the EM algorithm, we dene thus

calculations. On the other hand, we can use this phenomenon to our advantage by setting to zero in the regions where we do not expect bars to appear. Figs. 2 and 3 illustrate the application of the EM algorithm with different values of . In these gures, the solid line represents the measured bar code signal, and the dashed line represents the deblurred signal after 100 iterations. To improve conversion to the piecewise constant function after each iteration, we limited the maximum value of the deblurred signal to the maximum value of the original signal and renormalized the result. These gures show that the result is sensitive to the value of . It is reasonably good (see Fig. 2) for the value that has been obtained by the other EM algorithm described later, and it is poor if was estimated with an error of just 25% (see Fig. 3). The EM algorithm allows us to recover a general form input signal from a system output signal . In this approach, we do not explore our knowledge of the shape of . Since we are interested in reliable estimation of the model parameters (which is sometimes called the inverse problem to tting), we should increase the model sensitivity by using the smallest set of parameters. In the case of bar codes, is piecewise constant for This representation reduces the number of parameters from innity to . The new parameters are . To develop the M-step of the EM algorithm, we rewrite (10) as

(12) Now, we need to nd a maximum of this function with respect to . The maximization with respect to gives

and we have

(13) . where It is not difcult to nd the maximum with respect to

(10) where of , and . It is not difcult to nd a maximum that leads to the EM algorithm

but it is difcult to nd (11) where . Note that according to (11), if , then . Therefore, care should be taken to prevent underow during

and

TURIN AND BOIE: BAR CODE RECOVERY VIA THE EM ALGORITHM

359

Fig. 4. Measured signal (solid line) and the original t from zero crossings (dotted line).

Alternatively, we can develop the EM algorithm on the basis of representation (4) by dening

D. Signal Approximation As we pointed out before, if is small, bar codes can be decoded reliably using simple algorithms because ISI is small. Signal deblurring is necessary if is large. In this case, we can use the approximation

and

so that The ML approximation gives the relationship between parameters as where Maximizing this function, we obtain .

For , the difference between and does not exceed 2%. With this approximation, the scanner signal has the form

but the rest of the equations Let us consider rst a more general model

for the M-step are complex and are difcult to solve. In most practically important cases, we can perform the signal approximation to simplify the solution.

where and are independent variables. In the absence of interference and noise, tting this model to the measured signal is equivalent to tting the constrained model. The EM

360

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 2, FEBRUARY 1998

Fig. 5.

Fitting bar code signal after 40 iterations.

algorithm for re-estimating

and

has the form

(14)

Note the following property of the EM algorithm: If , then it follows from (16) that for all . We can use this property by setting for corresponding to gaps between the guard bars. We can also apply it to the guard bars in the complementary signal . E. Zero Crossing Fit

(15) This algorithm gives good results in the absence of the signal distortion, but for distorted signals, we should use a constrained approximation. It is easy to see that if we assume that and for

Bar codes can be uniquely identied by the sequence of zero crossings of , where is the common value of all . Using the zero crossings, we can express as

where , , and The scanner signal (4) takes the form (16) (17) (18) (19)

the EM algorithm takes the form (see Appendix)

where

where is given by (5). is small, and approximate the zero If crossings of the second derivative of . Therefore, if the blur factor is small, zero crossings of are approximately the same as the zero crossings of . This method of decoding bar codes by nding the zero crossings of the bar code signal second derivative is very popular in applications. It can be used to estimate the original values of the bar code parameters for the EM algorithm. Our procedure of decoding bar codes consists of two parts: A bar code is decoded using zero crossings of ; if the recovered code is a legal bar code, the procedure outputs

TURIN AND BOIE: BAR CODE RECOVERY VIA THE EM ALGORITHM

361

Fig. 6.

Fitted signal when the correct bar code was determined.

it. Otherwise, the estimates of the zero crossings are used to calculate initial values , and they are reestimated by (16)(19). The bar code is decoded by comparing with a threshold. Fig. 4 compares the measured bar code signal and with estimated by the zero crossings of when the zero crossing part of the algorithm failed to decode a legal bar code. Fig. 5 illustrates the t after 40 iterations. However, the correct bar code has been decoded after 11 iterations when the t was not so good (see Fig. 6). F. Discrete Approximation Usually, the received signal is given by its samples . In this case, integrals in the previous equations are replaced with sums by one of the well-known numerical methods [23]. Alternatively, we can formulate the problem of tting in terms of discrete samples. The loglikelihood function has the form

where pulse

is a discrete convolution of a rectangular of duration centered at with

Repeating the derivations of Section III-A, we obtain equations similar to (9)

where we assume that and are the normalized sequences so that they can be treated as discrete probability distributions. In this formulation, we are trying to t samples of the model to the corresponding samples of the scanner signal. If sampling rate is high, (4) can be written as

where analog of (16)(19)

. Similarly, we obtain a discrete

362

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 46, NO. 2, FEBRUARY 1998

where

It follows from (A2) that maximum of

is attained at

where

, and

can be found from the system

which has the form

(A3) where

Here, we used the rectangular approximation of the integration area, where bar denotes the normalized sequence. We can also use other rules (such as trapezoidal rule, Simpson formula, splines, Gaussian quadratures, etc.) to improve the integration accuracy or achieve the same accuracy with smaller number of samples. IV. CONCLUSION The EM algorithm shows promise for extending the performance of bar code readers. In one experiment, a UPC code label was positioned near focus where the present signal processing method read it well. As the label was moved away from focus, a distance from focus was reached where conventional bar code readers failed, but the EM algorithm decoded the bar code correctly. The EM algorithm successfully doubles the scanner depth of eld. Preliminary experiments with common label printing distortions (such as overprinting and underprinting) also show promise. APPENDIX In this Appendix, we derive (8)(10) for reestimating the constrained model parameters. We need to nd a maximum of (A1) where the logarithm in (A1), we obtain . Taking , and are dened by (14) and (15). Solving the system, we obtain the th iteration of the EM algorithm

(A2) where ACKNOWLEDGMENT The authors would like to acknowledge J. Snitow for his help in collecting bar code data and developing software for this project. Helpful suggestions and remarks of the anonymous reviewers are gratefully acknowledged.

TURIN AND BOIE: BAR CODE RECOVERY VIA THE EM ALGORITHM

363

REFERENCES
[1] L. S. Davis, Survey of edge detection techniques, Comput. Graph. Image Process, vol. 16, no. 1, pp. 248270, Apr. 1975. [2] H. Hildreth, The detection of intensity changes by computer and biological vision systems, Comput. Vision Graph. Image Process., vol. 22, pp. 127, Apr. 1983. [3] D. Marr and H. Hildreth, Theory of edge detection, Proc. R. Soc. Lond. B, vol. 207, pp. 187217, 1980. [4] R. A. Boie, I. J. Cox, and P. Rehak, On optimum edge detection, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, pp. 100108, 1986. [5] R. A. Boie and I. J. Cox, Two dimensional optimum edge recognition using matched and Wiener lters for machine vision, in Proc. IEEE First Int. Conf. Comput. Vison, June 1987, pp. 450456. [6] K. S. Shanmugan, F. M. Dickey, and J. Green, An optimal frequency domain lter for edge detection, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-2, pp. 3749, 1979. [7] T. Pavlidis, J. Swartz, and Y. Wang, Fundamentals of bar code information theory, Inst. Elec. Eng. Comput., pp. 7486, Apr. 1990. [8] E. Joseph and T. Pavlidis, Bar code waveform recognition using peak locations, IEEE Trans. Pattern Anal. Machine Intell., vol. 18, pp. 630639, June 1994. [9] J. Dybowski, Between the lines, Comput. Applications J., vol. 53, pp. 78114, 1994. [10] R. J. Tom, W. I. Hilles, B. M. Mergenthaler, and D. M. Blanford, Multiple bar code decoding system and method, U.S. Patent 5 262 625, Nov. 16, 1993. [11] M. Concetta, M. C. Morrone, and D. C. Burr, Feature detection in human vision: A phase-dependent energy model, in Proc. R. Soc. Lond. B, vol. 235, pp. 221245, 1988. [12] V. Torre and T. A. Poggio, On edge detection, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-8, no. 2, pp. 147163, 1986. [13] G. Zalman and J. Shamir, Maximum discrimination lter, J. Opt. Soc. Amer. A, vol. 8, no. 5, May 1991. [14] J. S. Chen and G. Medioni, Detection, localization, and estimation of edges, IEEE Trans. Pattern Anal. Machine Intell., vol. 11, Feb. 1989. [15] R. A. Hummel, B. Kimia, and S. W. Zucker, Deblurring Gaussian blur, in Comput. Vision Graph. Image Process., vol. 38, pp. 6680, 1987. [16] J.-B. Martens, Deblurring digital images by means of polynomial transforms, in Comput. Vision Graph. Image Process., vol. 50, pp. 157176, 1990. [17] Y. Vardi and D. Lee, From image deblurring to optimal investments: Maximum likelihood solutions for positive linear inverse problems, J. R. Statist. Soc., vol. 55, pp. 569612, 1993. [18] A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. R. Statist. Soc., vol. 76, pp. 341353, 1997. [19] J. Proakis, Digital Communications, 2nd ed. New York: McGraw-Hill, 1989.

[20] E. A. Lee and D. G. Messerschmitt, Digital Communication. Boston, MA: Kluwer, 1988. [21] S. Kullback, Information Theory and Statistics. New York: Wiley, 1959. [22] W. Turin, Fitting probabilistic automata via the EM algorithm, Stoch. Models, vol. 12, no. 3, pp. 405424, 1996. [23] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C. Cambridge, U.K.: Cambridge Univ. Press, 1992.

William Turin (SM83) received the M.S. degree in mathematics from Odessa State University, Odessa, U.S.S.R., in 1958 and the Ph.D. degree in mathematics from the Institute for Problems of Mechanics of the Academy of Sciences of the U.S.S.R., Moscow, in 1966. From 1960 to 1979, he was associated with the Moscow Institute of Electrical Engineering and Telecommunications, rst as an Assistant and later as an Associate Professor. From 1980 to 1981, he was a Senior Research Scientist, Department of Psychology, New York University, New York, NY. Since 1981, he has been a member of the Technical Staff at AT&T Bell Laboratories, Holmdel and Murray Hill, NJ. Currently, he is a Technology Consultant at AT&T Laboratories-Research, Florham Park, NJ. His research interests include digital signal processing, handwriting and speech recognition and compression, computer simulation, error correcting codes, microwave radio, and Markov processes. He has published 70 papers and is the author of four books.

Robert A. Boie was born in New York, NY, on April 24, 1939. He received the A.B. degree in physics from the University of Connecticut, Storrs, in 1965. He has held research positions at Bell Laboratories, Holmdel, NJ, Brookhaven National Laboratories, Brookhaven, NY, and, most recently, at AT&T Laboratories, Murray Hill, NJ. He is currently a Principal Member of Technical Staff in the Consumer Electronics Research Department, AT&T Laboratories, Florham Park, NJ. He is interested in devices, instrumentation, and analog design.