Professional Documents
Culture Documents
1 s2.0 S2090447922002635 Main
1 s2.0 S2090447922002635 Main
a r t i c l e i n f o a b s t r a c t
Article history: Nowadays, laser scanners are widely used to get three-dimensional modeling, but sometimes some
Received 22 January 2022 obstacles on the site or choosing the wrong scan width and the shadow areas due to different planes lead
Revised 16 August 2022 to the appearance of some gaps in the objective to be scanned. Mobile phone images (Conventional ter-
Accepted 17 August 2022
restrial photogrammetry technique) can be used to compensate for these missing data after transforming
Available online 16 September 2022
the image coordinates to ground coordinates. So, the main objective is to develop a technique to trans-
form image coordinates to ground ones through the Direct Linear Transformation model (DLT) depending
Keywords:
on a computer algorithm using the least-squares adjustment method. Four field experiments were made
Direct Linear Transformation (DLT)
Laser scanner
on AL-ASHRAFI QAITBAY. The obtained results showed that the putative technique can be applied to
3D models form a full 3D model with precision (11–22 mm) and it is compatible with the precision of a laser
Non-metric digital camera scanner.
MATLAB Ó 2022 THE AUTHORS. Published by Elsevier BV on behalf of Faculty of Engineering, Ain Shams Uni-
Documentation of monuments versity. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/
by-nc-nd/4.0/).
1. Introduction with known camera interior and exterior parameters [10]. The
collinearity equation can calculate the three-dimensional coordi-
Egypt is considered one of the richest countries with historical nates of all points in the common intersection area between
monuments globally and the national Egyptian income depends images (overlapping image region) [11]. The conventional methods
strongly on archeological tourism. Here, the importance of monu- for determining the 3D coordinates depending on the overlapping
ment monitoring and documentation appears. The 3D model for- images were recorded based on the stereographic principle, with
mation is essential for various applications. It can be used in stereometric or single metric cameras [12]. These conventional
monuments and façades documentation, maintenance of facilities, methods are substituted with modern methods in digital pho-
and many other applications [1,2], and [3]. One of the easiest ways togrammetry where there are great improvements in electronics,
to collect the 3D model data is using the laser scanner [4,5], and computers, and information technologies, such as digital cameras,
[6]. However, using the laser scanner may result in some gaps digital processing, and storage capacity [13]. Multi-digital images
due to obstacles in front of the surveyed features, incorrect width can be used to collect 3D data by linking the data from multi-
of the scan, and the presence of shadow areas due to different photos with the ground data collected by a laser scanner or a total
planes [7,8], and [9]. Terrestrial Photogrammetry has been used station based on DLT which depends on the previously mentioned
to assist with 3D information of the features. The full 3D coordi- collinearity condition [14], where the image coordinates can be
nates of the feature’s surface are obtained by overlapping images transformed to ground ones through this model. Digital cameras
are available and easy to use. Moreover, capturing photos mostly
needs no effort and no time. Therefore, data gathering using digital
⇑ Corresponding author.
photogrammetry is now the easiest [15] and [16]. From here, the
E-mail addresses: aser.eissa@eng.asu.edu.eg (A. M. Eissa), ibrahim.shaker@eng.
asu.edu.eg (I. F. Shaker), alaa.ibrahim@eng.asu.edu.eg (Alaa AL Din I. Awad). idea of integrating the data from a laser scanner and digital pho-
Peer review under responsibility of Ain Shams University. togrammetry to complete any missing data appeared, where this
integration depends mainly on the DLT algorithm to transform
the image coordinates to laser scanner-based point cloud data
coordinates. This study is concerned with the assessment of the
Production and hosting by Elsevier
https://doi.org/10.1016/j.asej.2022.101952
2090-4479/Ó 2022 THE AUTHORS. Published by Elsevier BV on behalf of Faculty of Engineering, Ain Shams University.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
A. M. Eissa, I. F. Shaker, A.M. Abdel-Wahab et al. Ain Shams Engineering Journal 14 (2023) 101952
accuracy of the above-mentioned integration and how much it can Therefore, there is a need for at least 6 common points in at
be applied. least two overlapping images having known ground and image
coordinates to get the mentioned transformation parameters.
1.1. Contribution of this research
2.2. Least-squares adjustment
The idea of the research was deduced through the practical
experience of drawing landmarks through laser scanner data. Surveying measurements are usually accompanied by errors
Sometimes it is found that the final model has some gap areas as during field observations. These errors require mathematical
a result of obstacles covering the surveyed feature or choosing adjustment. The least-squares method minimizes the sum of the
wrong scan width or due to the presence of different planes. Rather squares of differences between the observation and the estimated
than using structure from motion (SFM), conventional terrestrial values. This model has three forms which are parametric, condi-
photogrammetry is used for this research, where mobile phone tional, and combined. In this study, the parametric with its math-
cameras are used as static positions and then the coordinates of ematical model will be used and the mathematical model of this
these images are converted to the same coordinates system of method is illustrated as follows [19]:
the laser scanner and combined to provide a full 3D model.
FðXÞ ¼ w ð3Þ
2. The used mathematical model where the unknowns take terms of X. By applying Taylor series
expansion, the Jacobian matrix (A) can be calculated as follow:
2.1. Direct Linear Transformation (DLT) @FðxÞ
A¼ ð4Þ
@X
DLT depends mainly on the collinearity condition, which is
In the end, the solution matrix is calculated from the following
achieved when a certain point, its projection on the image plane,
equation:
and the projection center are collinear [17] and [18] as shown in
Fig. 1. 1
X ¼ ðAT PAÞ ðAT PWÞ ð5Þ
DLT equations are as follows:
L1 X i þ L2 Y i þ L3 Z i þ L4
xi þ ¼0 ð1Þ 3. Developed algorithm for solving the DLT model
L9 X i þ L10 Y i þ L11 Z i þ 1
The algorithm is used to solve the Direct Linear Transformation
L5 X i þ L6 Y i þ L7 Z i þ L8
yi þ ¼0 ð2Þ (DLT) model [17], where the function of this algorithm is to convert
L9 X i þ L10 Y i þ L11 Z i þ 1
the image coordinates that were obtained by the mobile phone
where: camera to cover the gap areas that may occur during the scan fea-
xi and yi : The coordinates of any image point (i). tures by using the laser scanner into 3D coordinates. The above
X i , Y i , Z i : The ground coordinates of the imaged point. step is important to let all points, both cloud points, and trans-
L1 to L11 : The eleven transformation parameters. formed image coordinates are in the same system and are compat-
2
A. M. Eissa, I. F. Shaker, A.M. Abdel-Wahab et al. Ain Shams Engineering Journal 14 (2023) 101952
ible. So, one can use the developed program by the authors through Assessment of the integration of digital photogrammetry coor-
the MATLAB (2018-a) and named Laser Data Gap Compensation dinates for gap areas and laser scanner cloud points data was made
Program (LDGCP) to do this task. This program depends on import- by comparing known laser scanner-based point cloud data coordi-
ing the data for both (ground coordinates of the common points, nates with ground coordinates after being transformed by the sug-
and the corresponding image coordinates of the required area). gested technique from the image coordinates, this procedure is
Through a script, the eleven transformation parameters are calcu- very useful when there are gaps in a 3D model created by the laser
lated for each image. After that, the ground coordinates (laser scanner only so whether the digital photogrammetry can be used
scanner-based point cloud data coordinates) are calculated from to compensate for these missing data. This was achieved through
the image coordinates by using the transformation parameters two main stages as shown in Fig. 4 for iPhone 12 Pro Max and
using the sequence of these steps in Fig. 2 which shows the algo- the same steps were repeated for iPhone 6 s evaluating two differ-
rithm used (LDGCP) to calculate the parameters and the ground ent cameras’ accuracies. The first stage was scanning a façade of
coordinates using multi-images and laser scanner data and Fig. 3 the (AL-ZAHIR ABU SAEED QANSUH AL-ASHRAFI QAITBAY)
shows the Graphical User Interface (GUI) of the program and as dome, Cairo, Egypt (See Fig. 5) using a Trimble Tx5 laser scanner
shown in the GUI the user enters the number of overlapped images with an accuracy of approximately 2 cm [6], [20] and [21], the laser
then imports the excel files of the laser scanner common points scanner specifications are shown in Table 2. In this stage, a 3D
coordinates, the images coordinates of the common points and model was created and the coordinates of the chosen points for
the image coordinates of the required points, where the output is comparison were identified. The second stage was capturing the
an XLS file of the ground coordinates of the required points. same façade using non-metric digital cameras for both two differ-
Input Data. ent accuracies (iPhone 12 Pro Max and iPhone 6 s cameras). Each
- Ground coordinates (X, Y, Z) of the common points. camera was used to capture the façade, where the conditions of
- Image coordinates (x, y) in pixel units of the common and the capturing were changed to test their effect on the accuracy. These
required points. conditions are the number of images, the number of common
Output Results. points, and the distance between the façade and the camera. In this
- The ground coordinates (X, Y, Z) of the required points (for stage, the image coordinates of the chosen points were identified,
gaps areas only not for all object). where there are two types of points, the common points, and the
Methodology. checkpoints. The common points are the points used to calculate
3
A. M. Eissa, I. F. Shaker, A.M. Abdel-Wahab et al. Ain Shams Engineering Journal 14 (2023) 101952
Fig. 4. Flow chart illustrating the followed methodology for both iPhone 12 Pro Max and iPhone 6s.
the DLT transformation parameters between the image coordinates used to calculate the DLT parameters, and points from 13 to 20 are
system and the laser scanner point cloud data coordinates, where checkpoints to evaluate the accuracy of using digital photogram-
this technique depends on the coordinates transformation from metry to complete any gap resulting from using the laser scanner.
image coordinates system to the ground coordinates system of The number of common points was chosen to be 12 points to test
the laser scanner and not on 3D models generated by photogram- the change in accuracy with changing the number of common
metry techniques. The checkpoints are the points used to compare points from 6 points (0 redundancy) to 12 points (100 % redun-
the transformed coordinates with the laser scanner ones. LDGCP dancy) with one point increment since 6 common points are the
was used to calculate the transformation parameters and then minimum needed number of common points to solve DLT model.
the transformed coordinates of the checkpoints from their image Therefore, there was no need to increase the number of common
coordinates. These two main stages were accomplished through points to be more than 12. For the check points, 8 check points
four field experiments illustrated in the following section. 20 were enough according to the number of common points used to
well-defined points (sharp points) were chosen as shown in achieve 95 % confidence level in the results. A façade without
Fig. 5, where points from 1 to 12 played the role of common points any problems as shown in Figs. 5 and 6 can be used as a reference
without any defect to compare there some of its areas which can be
compensated by other areas that come from the images. This
allowed anyone to assess the data extracted from image coordi-
nates to examine when there are gaps in the scans on other fea-
Fig. 5. The tested façade. Fig. 6. 3D model of the dome formed by the laser scanner.
4
A. M. Eissa, I. F. Shaker, A.M. Abdel-Wahab et al. Ain Shams Engineering Journal 14 (2023) 101952
Table 2
Trimble Tx5 Laser Scanner specifications after [23].
TLS Specification High-speed 3D laser scanner can measure at speeds of up to 976,000 pts/sec and up to range of 120 m
Includes an integrated color camera featuring an automatic 70 megapixels parallax-free color overlay.
Provides automated sensors (electronic compass-Altimeter).
Field of view (vertical/horizontal) 300°/360°. Ambient temperature 5 °C to 40 °C.
size 240 mm 200 mm 100 mm.
Max. vertical scan speed 5,820 rpm.
Wave length 905 nm. Beam diameter at exit 3.0 mm.
Ranging error 2 mm at 10 m and 25 m. Levels each scan with an accuracy of 0.015 and a range of ± 5°.
5
A. M. Eissa, I. F. Shaker, A.M. Abdel-Wahab et al. Ain Shams Engineering Journal 14 (2023) 101952
Table 3
RMS of the first field experiment at a distance of 10 m with different no. of images and common points.
From the third and fourth experiments, we can deduce that the
properties of the camera have a great effect on the resulting accu-
common points the accuracy gets better but increase, the time and racy for the iPhone 6 in the same condition for the iPhone 12 the
cost too without the need for that. RMS in this case of the iPhone 6 becomes approximately twice.
As for the number of images, the accuracy improved clearly The optimum number of common points between the laser
when using three images. scanner scans and the overlapped images is 9 and the optimum
No need to use more than four images as the model will be sat- number of the overlapped images is 4 related to the camera
urated with enough redundancy. accuracy.
6
A. M. Eissa, I. F. Shaker, A.M. Abdel-Wahab et al. Ain Shams Engineering Journal 14 (2023) 101952
Table 5
RMS of the third field experiment at a distance of 10 m with different No. of images and common points.
Table 6 scanner data. This integration depends on the DLT model using
RMS of the fourth field experiment. least squares adjustment through increasing redundancy by
Distance from façade (m) RMS (mm) increasing the number of common points and the number of over-
10 28
lapped images. The developed algorithm (LDGCP) is highly recom-
15 32 mended to calculate positions from the image coordinates and use
20 39 them to compensate for any gaps resulting from the laser scanner
to form a complete 3D model of the surveyed feature. The quality
of the camera has a great input in the resulting accuracy. The num-
The optimum distance between the camera and the surveyed ber of common points and images used depends on the required
feature is 10 m, if the camera used is low resolution the distance accuracy, but there is no doubt that with the increase in redun-
must be reduced between the camera and the objective (minimum dancy, the accuracy improves. Using eight common points and
distance not less than 5 m). three images gets a suitable accuracy related to resolution less
Finally, when using a less efficient camera, there is a need to than 2 cm (laser scanner accuracy), so for saving time and cost of
increase the number of common points and the number of used fixing points this number of points can serve the purpose, while
images to get higher accuracy through the high redundancy of increasing the number of common points improves the accuracy
data. but increases the time and cost too. Moreover, the distance
between the camera and the surveyed feature (the pixel size) plays
an important role in the accuracy, where as the distance decreases,
5. Conclusion
the accuracy increases and vice versa.
7
A. M. Eissa, I. F. Shaker, A.M. Abdel-Wahab et al. Ain Shams Engineering Journal 14 (2023) 101952