2019 6th NAFOSTED Conference on Information and Computer Science (NICS)

Proposing distortion compensation algorithm for

determining distance using two cameras
Phat Nguyen Huu
School of Electronics and Telecommunications
Hanoi University of Science and Technology
Hanoi, Vietnam
phat.nguyenhuu@hust.edu.vn
Tinh Tran Van and Nguyet Giap Thi
Graduate School of Electronics and Telecommunications
Hanoi University of Science and Technology
Hanoi, Vietnam
{tinh.tv144516, nguyet.gt156182}@sis.hust.edu.vn

Abstract—Imaging system is lens in the pinhole camera model.
In fact, it is an optical system with complex structure including
many different dimensions. Cheap cameras have poor quality
lens since they appear distortion while capturing image. Besides,
cameras with wide-angle or special-purpose have complex lens
that cause distortion. We find that there is a kind of distortion
that depends on structure and shape of lens (radial distortion).
Therefore, we propose a solution to eliminate it in this paper.
The results show that image after de-distortion improves quality
that is suitable for application to determine distance to object.
Index Terms—Non-linear distortion compensation, radial dis-
tortions, tangential distortion, pinhole camera, processing image.
I. I NTRODUCTION
Estimating distance of images taken from camera is an
important function in three-dimensional scanning systems. The
technique is being used in automatic robot control systems.
The distance from camera system has a lot of important
information allowing us to analyze and solve problems [1]–[8].
There are many methods for estimating distance to camera
system for example using lasers or infrared or ultrasound [7],
[9], [10]. The typical camera is Microsoft’s Kinect XBOX
360 camera. It uses an estimation method with a color image
camera, infrared camera, and projector to measure distance
according to delay time of rays. The device has opened up a
new application and will develop in the future.
The goal of paper is to estimate distance with two camera
model. There are more devices with two main cameras track-
ing by smart-phones. They have created a great potential to
handle images from two cameras with popularity of smart-
phone today. One of the most popular applications is to
determine distance from two camera model.
The article has three main contributions as follows:
First, the article focuses on theoretical issues in two-
camera model. We will present principle of image capture
from camera, structure and formula for calculating distance
with ideal two-camera model. We will analyze distortions
encountered with two camera model. We then give causes of
distortions. Next, we design real system of two-camera model.
The detailed design and algorithms for processing images that
are taken from two cameras are described. Finally, we calculate
distance for each pixel corresponds to two images.
II. R ELATED W ORK
Distortion compensation algorithms for determining dis-
tance using two cameras are investigated in [1]–[6]. The
authors [2] proposed flexible technique to easily calibrate
camera. The algorithm performs closed-form solution by a
nonlinear refinement. As a result, radial distortion is eliminated
completely. The authors [5] proposed the camera calibration
method with an iterative distortion compensation algorithm.
The results showed that the proposal improved more than 100
percent accuracy than the conventional method. The authors
[6] proposed the method for automatic estimation of two-
parameter radial distortion models. Performing on experiments
on real images with significant distortion showed the ability
of proposed approach.
The rest of the paper include ve parts and organize as
follows. Section I presents an overview of estimating the dis-
tance to the camera system. Section II presents several related
works. Section III presents the proposal system. Section IV
will evaluate the proposal model and analyze the results. In
nal section, we give conclusions and future research directions.
III. P ROPOSAL SYSTEM
A. Pinhole Camera model
The structure of camera is described by two main compo-
nents, namely lens and image sensor. The image sensor has
very complex structure considered a plane with collection of
many sub-pixels. When light comes from lens to pixels, it will
create a small current, and pixels transfer it for amplifier at
output. There are two common types of image sensors, namely
CCD and CMOS [8].
The first model is called pinhole camera [7]. The left plane
is image and middle plane is lens. X is the actual size of
object. x is the size of object in image. The focal length of
camera is f , and distance from camera to object is Z. The
relationship among distance and other camera parameters is
calculated according to the formula:
X
x = f. . (1)
Z
B. Stereo pinhole camera model
If we rotate 900 model to left side and add another pinhole
model, we will get it for two cameras (stereo camera). We

978-1-7281-5163-2/19/$31.00 ©2019 IEEE 172

 2019 6th NAFOSTED Conference on Information and Computer Science (NICS)

Fig. 2. Steps of image creation of camera.

(a) (b)
Fig. 1. Stereo pinhole camera model.
Fig. 3. The process of creating images from camera to pixel coordinates (a)
camera coordinates and (b) image plane [10].
consider this model with one-way coordinate system where
image and len planes are parallel and symmetrical across each
other. coordinate system to camera lens and from them to image
Figure 1 depicts stereo pinhole camera model that is project- sensor plane.
ing pixel P on image plane. We use virtual image as mentioned Image is created by modeling two transformations with
above. Two points pl and pr are two corresponding pixels of three coordinate systems as shown in Fig. 2. In the paper, im-
P on two left and right planes where xl and xr are their age is transformation of three-dimensional coordinate system
coordinates. Ol and Or roots are called center of camera. The into others. The transformation is characterized by camera’s
axis of Oz passes through center of projection. Orthogonal external parameters. It is transformation from three-dimension
len and image planes is called principal ray. Two cameras of to two-dimensions.
model have exact focal value f . T is called baseline [7]. Pixel is created by model that converts from camera to
We recognize the similarity of two triangles OlP Or and coordinate system when light reaches lens. As shown in Fig.
plP pr as shown in Fig. 1. Therefore, we can find Z (distance 3, we find that reference system has origin of C. Actual point
from P to center) according to the formula: taken from the reference system is (Xc , Yc , Zc ) and pixel has
coordinates (x̃, ỹ). Equation 1 allows us to convert coordinates
T − (xl − xr ) T of captured point and pixel coordinates as follows:
= . (2)
z−f z
x̃ = f xzcc ,
(5)
We have:
fT fT ỹ = f yzcc .
Z= = , (3)
(xl − xr ) d f and (Xc , Yc , Zc ) have physical units of mm while (x̃, ỹ)
where d is the difference between coordinates of two pixels are m. Therefore, they are not pixel units of digital image
on left and right of P determined by formula: system. We can find coordinates of pixel according to formula
from Eq. 5:
d = (xl − xr). (4)
x̃ f Xc ỹ f Yc
x= +x0 = +x0 , y = +y0 = +y0 . (6)
If disparity (d) is defined as difference of two pixel, the dx dx Zc dy dy Zc
above formula is not correct since f and T are real units of To analyze and calculate, we convert cartesian into homo-
meters. Therefore, it must transfer into actual size of pixel geneous coordinate system [9], [11]. Applying Eq. 6, we have:
(usually size m) in calculations to find d. When considering ⎛ ⎞ ⎛ ⎞ ⎡ ⎤⎛ ⎞
ideal one-way coordinate plane of camera in stereo pinhole Xc x fx 0 x 0 Xc
⎝ Y c ⎠ → ⎝ y ⎠ = ⎣ 0 fy y0 ⎦ ⎝ Y c ⎠ ,
model, it is easy to find distance of point to camera system
with Eq. 3. Zc 1 0 0 1 Zc
(7)
where ⎡ ⎤
C. Camera parameters fx 0 x 0
K = ⎣ 0 fy y0 ⎦ (8)
As mentioned above, image plane is two-dimensional coor-
0 0 1
dinate system (pixel coordinate system) while observed object
and camera position are considered in a three-dimensional The matrix K in Eq. 8 is called internal parameters of
model. In fact, stereo pinhole model is complex from real camera.

173
 2019 6th NAFOSTED Conference on Information and Computer Science (NICS)
Fig. 4. Proposal model of processing blocks of distance estimation method.
The external parameters will be specific to transition from
real to reference system. They determine their positions. Both
camera and real-time reference frames are three-dimensional
since they are transformed in three-dimensional space. There-
fore, we first will find out how to change coordinate axis.
D. Proposing distance estimation method
Estimating distance with two cameras is shown in Fig. 4
based on [7]. This method consists of five main steps divided
into two main processing processes as follows.
Firstly, input of processing system is two images (stereo
images). The first processing is camera calibration (on left side
of Fig. 4). Camera calibration is performed independently of
distance calculation (right of Fig. 4). The object of step is
to find parameters of two camera that are inputs for main
processing.
Secondly, it will convert original captured image into re-
constructed image since stereo camera model in section III-B
can be applied and found distance. Using parameters of two
camera will eliminate distortions that exist in each image.
Rectification uses parameters describing relationship of two
cameras. This step will get position pixels in both images
since corresponding pixels are on same row. It converts ideal
two-camera model as in section III-B. Correspondence will
take corresponding pixel in left image and find it on right.
It will then give difference in pixel called disparities (d). As
a result, distance is calculated exactly. Triangulation is last
step that needs to apply formula of ideal camera model to
174
find distance. Therefore, designing of machining model is an
important issue. This is challenge when designing hardware.
E. Hardware design analysis
Requirement in camera stereo pinhole model is that two
cameras must be exactly the same. They must have two rays
that are parallel to each other. Two cameras must be installed
in the best possible alignment. Therefore, objective of paper
is to design and explore methods of distance estimation with
two-camera model while keeping suitable cost.
We make low-cost devices but ensuring stability of system.
In this design, we use two phones.
To calculate distance, we need to find all parameters of two
camera. They include digital size of image, length of focal, size
of pixel, and space of two camera. We cannot find size of one
pixel image. Therefore, we will perform based on measuring
on image sensor directly.
In this design, the camera has size of 5M px and can be
adjusted to capture at lower resolution. Selecting image at
2M px (1224 × 1632) resolution to reduce size of memory,
we are able to capture image without needing to zoom out
directly.
Figure 5 shows actual structure of image. As mentioned
above, camera’s structure has only two main components,
namely lens and sensor. We conduct size of image using
correlated images with a ruler. We draw lines across horizontal
and vertical edges of sensor and cut through positions on ruler
to find length. Sensor size is 3.6 × 4.8 mm. The measured
dimensions are actually slightly larger than above parameters
since image has ratio of 1224 : 1632 = 3.6 : 4.8. With a size
of 3.6 × 4.8 mm and resolution of 1224 × 1632, we can find
size of pixel of images as:
3.6 4.8
α = α x = αy = = = 2.94μm. (9)
1224 1632
Next, we need to determine distance between two cameras
(baseline). Using mentioned method to measure correlation of
image, we find value of T = 5.26 cm as shown in Fig. 6.
Finally, we have the parameters as shown in Tab. I.
Therefore, formula in sect. III-B can now be rewritten as
follows:
fT 4.0 × 52.6 71, 565
Z= = −3 = .
(xl − xr ) disparity × 2.94 × 10 disparity
(10)
Equation 10 shows that the maximum distance can be found.
This is the farthest point that can capture with distance of
71,565 m.
IV. R ESULTS AND A NALYSIS
A. Scenario
We use Matlab program. We first need to prepare an object
to perform calibration of camera system. The calibration
object will be plane with white and black squares similar to
chessboard as shown in Fig. 7. We take set of chess board
images with different angles to make input image. Besides,
we use three tomatoes to measure distance.
 2019 6th NAFOSTED Conference on Information and Computer Science (NICS)

TABLE I
H ARDWARE PARAMETERS OF TWO CAMERAS .

No. Parameter Value

1 Digital image size 1224 × 1632 pixel
2 Focus 4 mm
3 Sensor size 3.6 × 4.8 mm
4 Pixel size 2.94 m
5 Length of baseline 5.26 cm

Fig. 7. Using image of chess board for camera calibration.

Fig. 5. Camera parameters are saved in each image.

Fig. 6. Determining the baseline size.

B. Results
In first step, we will determine intersection of squares using
Hariis Corner method (finding points that feature corners)
[12]–[14]. It is good method for determining angle with points
that are clearly defined as the corners of chess board squares. Fig. 8. Determining square intersections of board.
The result is shown in Fig. 8.
The corner points have the same distribution in each image.
If we do not determine enough points, it will be removed. corners in each image. The parameters of camera is found
Therefore, we have set of corresponding points of chess board according to method mentioned in sect. III. Performing for

175
 2019 6th NAFOSTED Conference on Information and Computer Science (NICS)
Fig. 9. Parameters of camera calibration.
Fig. 10. Original stereo image.
Fig. 11. Image stereo after rectification.
each camera, we obtain its parameters. From parameters of
two cameras, we perform to calculate for stereo camera model.
The result is shown in Fig. 9.
Next, we use parameters to restore left and right sides of two
images. Using two-camera system, their parameters are able to
obtain in the previous step. We will calculate the parameters
and use them for restructuring two images. The results are
shown in Figs. 10 and 11.
For two input images, the results for determining distance
between two objects are shown in Figs. 12 and Tab. II.
176
Fig. 12. Performing to calculate distance on sample images.
The obtained results have large errors in case of large
distances (more than 100 cm). Errors are less than 10 percent
with small distances (less than 100 cm). Therefore, it needs
to be improved for real applications.
V. C ONCLUSION
The article introduces two camera models and proposes
distortion compensation methods. In the article, we perform
to calculate the distance between camera system and object.
The experiment results show that the accuracy is good with
objects at close distance. There are many problems that have
not been resolved such as appearing large errors of far distance
and improving distortion compensation algorithm. The goal
of paper is to explore distance estimation method with two
cameras. To apply the method into real application, there are
still many issues to be solved.
 2019 6th NAFOSTED Conference on Information and Computer Science (NICS)

TABLE II
A LGORITHM RESULTS WITH SAMPLE PIXELS .

No. Actual distance (cm) Measured distance (cm) Error (percent)

1 18.0 18.6 3.3
2 60.0 63.7 6.2
3 85.0 90.28 6.2
4 160.0 187 17

In the future, we will design and improve two cameras [15] P. N. Huu and H. L. The, “Low-complexity image processing algorithm
system to increase accuracy of detecting and tracking objects for estimating distance and actual size of subject,” in 7th Intl Conf.
Commun. Electron., July 2018, pp. 463–468.
[15]–[18] and combine with new methods such as using CNN [16] Phat Nguyen Huu, Vinh Tran-Quang, and T. Miyoshi, “Image compres-
models [19] and robot for determining distance to obstacle on sion algorithm considering energy balance on wireless sensor networks,”
wireless sensor networks (WSNs) [20]–[27]. in 2010 8th IEEE International Conference on Industrial Informatics,
July 2010, pp. 1005–1010.
ACKNOWLEDGMENT [17] T. N. Manh, P. N. Huu, and C. V. Le, “Tracking object considering
energy balance on wireless sensor networks,” in 2016 IEEE International
This research is supported by Hanoi University of Science Conference on Consumer Electronics-Asia (ICCE-Asia), Oct 2016, pp.
and Technology and Ministry of Science and Technology 1–5.
[18] P. N. Huu, V. Tran-Quang, and T. Miyoshi, “Low-complexity motion
under the project No. 1815/QD-BGDT signed on 28/06/2019. estimation algorithm using edge feature for video compression on
wireless video sensor networks,” in 2011 13th Asia-Pacific Network
R EFERENCES Operations and Management Symposium, Sep. 2011, pp. 1–8.
[19] T. N. T. Huong, H. P. Cong, T. V. Huu, and X. H. Van, “Artificial
[1] J. Y. Bouguet, “Camera calibration toolbox for matlab,” in Camera
intelligence based adaptive gop size selection for effective wyner-ziv
calibration, 2019.
video coding,” in Intl Conf. on Advanced Technol. for Commun., 2018.
[2] Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans.
[20] G. Bradski and A. Kaehler, “Learning opencv: Computer vision with the
on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330–
opencv library,” Sebastopol, CA: O’Reilly, vol. 14, no. 2, pp. 101–110,
1334, Dec. 2000.
Feb. 2008.
[3] D. Forsyth and J. Ponce, Computer Vision: A Modern Approach. USA:
[21] P. N. Huu, V. Tran-Quang, and T. Miyoshi, “Low-complexity and
A: Prentice Hall, 2003.
energy-efficient algorithms on image compression for wireless sensor
[4] Wikipedia, “Cross product,” in Cross product, 2019.
networks,” IEICE Transactions on Communications, vol. E93-B, no. 12,
[5] Y. Xu, F. Gao, H. Ren, Z. Zhang, and X. Jiang, “An iterative distortion
pp. 3438–3447, Dec. 2010.
compensation algorithm for camera calibration based on phase target,”
[22] T. N. Duong, V. D. Than, T. H. Tran, Q. H. Dang, D. M. Nguyen, and
Sensors, vol. 2017, no. 17, pp. 1–11, Dec. 2017.
H. M. Pham, “An effective similarity measure for neighborhood-based
[6] D. Santana-Cedres, L. Gomez, M. Aleman-Flores, A. Salgado,
collaborative filtering,” in 5th NAFOSTED Conf. on Infor. and Comput.
J. Esclarn, L. Mazorra, and L. Alvarez, “An iterative optimization
Scie. (NICS 2018), Nov. 2018, pp. 250–254.
algorithm for lens distortion correction using two-parameter models,”
[23] P. N. Huu, V. Tran-Quang, and T.Miyoshi, “Multi-hop reed-solomon
SIAM Journal on Imaging Sciences, vol. 2016, no. 6, pp. 326–364, Dec.
encoding scheme for image transmission on wireless sensor networks,”
2016.
in Proceedings of 4th Intl Conf. Commun. Electron. (ICCE 2012), Aug.
[7] G. Bradski and A. Kaehler, Learning OpenCV: Computer Vision with
2012, pp. 74–79.
the OpenCV Library. USA: OReilly Media, Inc., 2008.
[24] J. K. Choi, V. D. Nguyen, H. N. Nguyen, V. V. Duong, T. H. Nguyen,
[8] Wikipedia, “Image sensor,” in Image sensor, 2019.
H. Cho, H.-K. Choi, and S.-G. Park, “A time-domain estimation method
[9] R. Hartley and A. Zisserman, Multiple View Geometry in Computer
of rapidly time-varying channels for ofdm-based lte-r systems,” Digital
Vision. UK: Cambridge University Press, 2006.
Commun. and Netw., vol. 5, no. 2, pp. 94–101, May 2019.
[10] R. J. Radke, Computer Vision for Visual Effects. York, USA: Cambridge
[25] V. Tran-Quang, P. N. Huu, and T. Miyoshi, “A collaborative target
University Press, 2013.
tracking algorithm considering energy constraint inwsns,” in 19th Intl
[11] Wikipedia, “Homogeneous coordinates,” in Homogeneous coordinates,
Conf. Software, Telecomm. Comput. Netw. (SoftCOM 2011), Sept. 2011,
2019.
pp. 1–5.
[12] C. Harris and M. Stephens, “A combined corner and edge detector,” in
[26] T. N. Duong, V. D. Than, T. H. Tran, T. H. A. Pham, V. H. A. Nguyen,
In Proc. of Fourth Alvey Vision Conference, 1988, pp. 147–151.
and H. N. Tran, “A practical solution to the acm recsys challenge 2018,”
[13] N. Dey, P. Nandi, N. Barman, D. Das, and S. Chakraborty, “A compara-
in 5th NAFOSTED Conf. on Infor. and Comput. Scie. (NICS 2018), Nov.
tive study between moravec and harris corner detection of noisy images
2018, pp. 341–343.
using adaptive wavelet thresholding technique,” 2012.
[27] V. Tran-Quang, P. N. Huu, and T. Miyoshi, “Adaptive transmission range
[14] J. Snchez, N. Monzn, and A. Salgado, “An Analysis and Implementation
assignment algorithm for in-routing image compression on wireless
of the Harris Corner Detector,” Image Processing On Line, vol. 8, pp.
sensor networks,” in Proceedings of 3rd Intl Conf. Commun. Electron.
305–328, 2018.
(ICCE 2010), Aug. 2010, pp. 1–6.

177