Reliable Spatial Relationship Constrained
Feature Point Matching of Oblique Aerial Images
Han Hu, Qing Zhu, Zhiqiang Du, Yeting Zhang, Yulin Ding
Abstract
This paper proposes a reliable feature point matching method
for oblique images using various spatial relationships and
geometrical information for the problems resulted by the
large view point changes, the image deformations, blurring,
and other factors. Three spatial constraints are incorporated
to filter possible outliers, including a cyclic angular ordering
constraint, a local position constraint, and a neighborhood
conserving constraint. Other ancillary geometric information,
which includes the initial exterior orientation parameters
that are obtained from the platform parameters and a rough
DEM, are used to transform the oblique images geometri-
cally and reduce the perspective deformations. Experiment
results revealed that the proposed method is superior to
the standard SIFT regarding both precision and correct
matches using images obtained by the SWDC-5 system.
Introduction
Beginning in 2000, oblique aerial camera systems garnered
attention from the photogrammetry community due to their
ability to capture the facades of buildings and their ability to
be briefly interpreted (Petrie, 2009). Many penta-view camera
systems that feature four 45
°
oblique cameras and one nadir
camera, including Pictometry (Gerke and Kerle, 2011),
MIDAS
(Madani, 2012), and
SWDC
-5 that are used in this paper, have
collected numerous datasets. However, traditional photo-
grammetry techniques and software are designed primarily
for nadir images and are difficult to adapt for oblique aerial
images. New challenges have been posed to photogrammetry
practitioners to integrate all of the images to extract more
compact and accurate information, especially for oblique
views (Jacobsen, 2009; Nyaruhuma
et al
., 2012; Fritsch and
Rothermel, 2013).
Exterior orientation (
EO
) parameters for the images are
necessary prior to 3D reconstruction (Gerke, 2009), texture
mapping (Wang
et al
., 2008), position measurement (Xiong
et
al
., 2014), and other mapping applications. Because all of the
cameras are installed on a rigid platform and held stationary
after manufacturing, the bore-sight angles and translations
(called platform parameters) between the nadir view and the
four oblique views are fixed in the ideal condition (Wiede-
mann and Moré, 2012) and are calibrated using retro-reflect
coded targets in the calibration field (Fraser, 1997). In theory,
only the
EO
parameters for the nadir images need to be esti-
mated in the bundle adjustment. However, due to reinstalla-
tions, limitations in mechanical manufacturing, and possi-
ble asynchronous exposures among different cameras, the
platform parameters are not stable and can only be considered
as fixed in a single flight (Jacobsen, 2009). In this situation, a
combined bundle adjustment with both the nadir and oblique
images in the same block are necessary, which requires suffi-
cient tie points to sew the block together.
It turns out that feature matching between nadir images
and oblique images is astonishingly difficult because of the
obvious difference in their appearances, which consists of
occlusions, perspective deformations, light conditions, and
blur that are caused by the wide baseline and large tilting an-
gles (Yao and Cham, 2007; Yang
et al
., 2012). Additionally, in
production practice, an existing solution with traditional soft-
ware is to process images for each camera separately and then
manually select enough inter-camera tie points to assemble
different blocks of images together. However, this solution is
not only time consuming, but it is also prone to inter-camera
inconsistencies due to the lack of accurate tie points.
To eliminate the drudgery of manual selection and to im-
prove the quality of bundle adjustment, we propose to amend
the standard feature matching process by injecting addition-
al spatial relationships of feature points into the process to
increase the reliability, rather than only using the appearance
information of the images. Specifically, we propose a cyclic
angular order constraint, a local position constraint and a
neighborhood conserving constraint. Furthermore, the initial
geometric information of the five cameras that is obtained
from the global positioning system (
GPS
) and inertial measure-
ment unit (
IMU
) onboard the aircraft, calibrated platform pa-
rameters, and a rough
DEM
are used to geometrically transform
the images, in order to reduce the perspective deformations.
In our previous work (Zhu
et al
., 2007), a filter strategy using
information content is proposed to improve the repeatability
of the interest points and the reliability of the matches, which
is also based on the appearance information. However, when
the images are essentially dissimilar in appearance, the spatial
Han Hu, Zhiqiang Du, and Yeting Zhang are with the State
Key Laboratory of Information Engineering in Surveying Map-
ping and Remote Sensing, Wuhan University, P.O. Box C310,
129 Luoyu Road, Wuhan, Hubei, 430079, P.R. China (huhan@
whu.edu.cn).
Qing Zhu is with the National-local Joint Engineering Lab-
oratory of Spatial Information Technology for High-speed
Railway Running Safety, Southwest Jiaotong University, P.R.
China; and the State Key Laboratory of Information Engi-
neering in Surveying Mapping and Remote Sensing, Wuhan
University, P.R. China.
Yulin Ding is with the State Key Laboratory of Information
Engineering in Surveying Mapping and Remote Sensing,
Wuhan University, P.R. China, and the Institute of Space and
Earth Information Science, The Chinese University of Hong
Kong, Shatin, N.T., Hong Kong.
Photogrammetric Engineering & Remote Sensing
Vol. 81, No. 1, January 2015, pp. 49–58.
0099-1112/15/811–49
© 2014 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.81.1.49
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
January 2015
49
