Suzuki, T. et al.

Paper:

3D Terrain Reconstruction by Small Unmanned Aerial Vehicle Using SIFT-Based Monocular SLAM
Taro Suzuki , Yoshiharu Amano , Takumi Hashizume , and Shinji Suzuki
Institute for Science and Engineering, Waseda University 17 Kikui-cho, Shinjuku-ku, Tokyo 162-0044, Japan E-mail: taro@power.mech.waseda.ac.jp Department of Aeronautics and Astronautics, School of Engineering, The University of Tokyo 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan [Received October 6, 2010; accepted February 7, 2011]
Research

This paper describes a Simultaneous Localization And Mapping (SLAM) algorithm using a monocular camera for a small Unmanned Aerial Vehicle (UAV). A small UAV has attracted the attention for effective means of the collecting aerial information. However, there are few practical applications due to its small payloads for the 3D measurement. We propose extended Kalman lter SLAM to increase UAV position and attitude data and to construct 3D terrain maps using a small monocular camera. We propose 3D measurement based on Scale-Invariant Feature Transform (SIFT) triangulation features extracted from captured images. Field-experiment results show that our proposal effectively estimates position and attitude of the UAV and construct the 3D terrain map.

Keywords: SLAM, SIFT, UAV, 3D reconstruction

1. Introduction
As aerial remote sensing using satellites, aircraft, and manned helicopters has become increasingly widespread, aerial laser surveys with laser scanners on aircraft and helicopters are being used in areas such as landslide measurement in natural disasters, quality work analysis in civil engineering, and river and levee management in ood control. Laser surveys by manned aircraft suitably measure broad terrain ranges, but are so costly to operate and ight procedures so time-consuming that they are not conveniently applied to measuring terrain demanding frequent observations. Another problem of aerial survey is the difculty in collecting high-resolution data due to aviation regulations prohibiting low-altitude ying in such aircraft. In this context, remote sensing using Unmanned Aerial Vehicles (UAVs) has attracted attention in recent researches [16]. UAVs are cheaper and easier to operate than manned craft and y at low altitudes when acquiring high-resolution data. Research using heavy unmanned helicopters has achieved precise three-dimensional (3D) terrain measurement [1]. Another report details 3D terrain 292

measurement combining cameras and laser scanners on a large UAV [2]. Large UAVs require runways and involve transport considerations that could potentially hinder practical application. In contrast, small xed-wing UAVs, which are light and portable and require a single operator, are highly mobile and safe enough for use in practical measurement. UAVs weighing several kilograms can be launched manually and collect aerial information at relatively low cost at whatever locations in whatever terrain. 3D terrain measurement using UAVs has problems, however. Small UAVs have such strictly limited weight that on-board equipment such as sensors must also have limited size and weight. This limits on-board installation of instruments such as laser scanners, which in turn prevents directly measuring 3D terrestrial coordinates. Related to the above restrictions preventing the use of high-precision sensors for UAV position and attitude, it is difcult to estimate accurate position and attitude of small UAVs essential for terrestrial measurement. Small UAVs are thus more often used for image monitoring than 3D terrestrial measurement. 3D topographic mapping with small UAVs requires that vehicle accuracy in estimating position and attitude be improved by integrating data from plural sensors. Frameworks must also enable 3D terrestrial features to be measured using small and lightweight sensors such as monocular cameras. Simultaneous estimation of self-positioning and mapping by mobile robots is called Simultaneous Localization And Mapping (SLAM). SLAM involves different approaches, particularly with wheel mobile robots [79]. Among related research using monocular cameras [3 6, 1012], monocular SLAM was designed by Davison et al. [10] and SLAM with terrestrial cameras by Williams et al. [11]. In such SLAM with monocular cameras, features are automatically tracked from camera videos to estimate camera movement and 3D positions of features using epipolar geometry. These approaches are mainly for 3D estimation at short distances and in particular for estimating positions and attitudes of cameras, making it difcult to apply them directly to 3D topographic mapping from small UAVs in a broad range of outdoor environments. Research on SLAM from UAVs [36, 12] includes Journal of Robotics and Mechatronics Vol.23 No.2, 2011

3D Terrain Reconstruction by Small Unmanned Aerial Vehicle

SLAM with rotary wing UAVs by Artieda et al. [3], in which UAVs with laterally xed cameras y at very low altitudes, making it difcult to produce broad-area 3D topographic maps. Bryson et al. studying SLAM combining Inertial Measurement Units (IMU) and near-infrared cameras [5, 6] used xed-wing vehicles with near-infrared cameras to observe markers on the ground and estimate position and attitude called Extended Kalman Filter (EKF) SLAM. SLAM with EKF has enabled vehicle position and attitude to be estimated by observing ground markers, although the need to place markers on the ground itself makes it difcult to apply this to mapping unknown environments. We propose a SLAM algorithm for small UAVs based on observation of natural features acquired through combined Global Positioning Systems (GPS), IMU, and monocular camera imaging. In our proposal, UAV position and attitude are estimated accurately using the EKF algorithm on Scale-Invariant Feature Transform (SIFT) features [13] extracted, tracked and observed from monocular camera imaging. In addition, such SIFT features are simultaneously estimated for 3D position to complete 3D topographic maps. In the sections that follow, we briey summarize small UAV use in our proposal and the proposed SLAM algorithm. We detail the proposed SLAM algorithm and its practice in actual ight tests, then evaluate accuracy by comparing topographic mapping results with the Digital Elevation Model (DEM) acquired in aerial laser surveys.

Fig. 1. UAV and conguration of onboard sensors.

Fig. 2. Typical ight plan for small UAV.

2. Small UAV and SLAM Algorithm Overview

2.1. Small UAV The UAV we used in this research, shown in Fig. 1, is 1.2 m long and 1.7 m wide and weighs 2 kg including a maximum carrying weight of 500 g. Because of weight limitations, equipment on board is limited to a GPS receiver module, MEMS inertial sensors for 3-axial accelerometers 3-axial gyroscopes, dynamic and static pressure sensors, avionics containing PCs for guidance control and computation, and a commercial digital camera. Vehicle, sensor data, and airborne video status are transmitted by radio to a Ground Control Station (GCS). Videos taken using a commercial digital camera looking down from the bottom of the vehicle are transmitted in VGA size (640 480, 30 fps) [14]. The UAV makes autonomous circumvolution around predesignated waypoints at altitudes and velocities designated prior to ight. A typical ight plan for a small UAV is shown in Fig. 2. The UAV is manually launched and ies along designated input routes automatically, landing via on-board parachutes to enable it to be safely recovered even in conned environs. In autonomous ights, the UAV ies at an altitude of 80120 m at 15 m/s for 30 min, enabling aerial information to be collected over a range of several kilometers.

2.2. EKF-SLAM Algorithm Overview The EKF-SLAM algorithm we propose estimates vehicle locations and topographic mapping using extended Kalman lters. Fig. 3 shows the overview of the proposed EKF-SLAM algorithm. Location and velocity are obtained by a single-frequency differential (D)-GPS receiver (4 Hz) on the UAV, together with acceleration and angular velocity (60 Hz) measured by MEMS inertial sensors for the SLAM algorithm. Continuous airborne ground imaging (30 Hz) taken by the digital camera is input to the algorithm to improve accuracy in estimating UAV position and attitude and to enable topographical mapping. The 1-step SLAM cycle is set to 60 Hz and measurement updated each time the sensor captures data. Topographic mapping is a set of landmarks with 3D coordinates. Environmental landmarks and features are automatically extracted by SIFT and initial 3D landmark coordinates determined from features tracked among plural frames added to EKF states (landmark initialization). Since our UAV cannot carry equipment such as laser scanners directly measuring distance, 3D landmark coordinates are determined based on triangulation using tracked features and previous data on vehicle position and attitude. Landmarks are observed by robust matching between already added states and extracted SIFT features (landmark observation). In EKF prediction, positions and attitude angles are calculated using acceleration and angular velocity. In EKF updating, observation is updated using position and attitude observed by GPS and landmark coordinates recognized from imaging and matched. Through these steps, UAV position and attitude and 3D landmark coordinates are thus estimated simultaneously. The proposed SLAM algorithm cannot be appropriately applied due to PC processing speed, so it is done therefore to be conducted with the PC in the GCS using sensor data and airborne imaging transmitted by radio.
293

Journal of Robotics and Mechatronics Vol.23 No.2, 2011