A Geometric Correspondence Feature
Based-Mismatch Removal in Vision-Based
Mapping and Navigation
Zeyu Li, Jinling Wang, and Charles Toth
Abstract
Images with large-area repetitive texture, significant view-
point, and illumination changes as well as occlusions often
induce high-percentage keypoint mismatches, affecting the
performance of vision-based mapping and navigation. Tra-
ditional methods for mismatch elimination tend to fail when
the percentage of mismatches is high. In order to remove
mismatches effectively, a new geometry-based approach is
proposed in this paper, where Geometric Correspondence
Feature (
GCF
) is used to represent the tentative correspon-
dence. Based on the clustering property of
GCFs
from correct
matches, a new clustering algorithm is developed to identify
the cluster formed by the correct matches.
With the defined quality factor calculated from the identi-
fied cluster, a Progressive Sample Consensus (
PROSAC
) process
integrated with hyperplane-model is employed to further
eliminate mismatches.
Extensive experiments based on both simulated and real
images in indoor and outdoor environments have demon-
strated that the proposed approach can significantly improve
the performance of mismatch elimination in the presence of
high-percentage mismatches.
Introduction
Keypoint matching plays an important role in vision based
mapping and navigation (Davison
et al
., 2007, Li
et al
., 2011,
Chowdhary
et al
., 2013). Typically through keypoint detec-
tion, description, and matching, matched keypoints can
provide the correspondence between two or more images of
the same scene. With the correct correspondence, a series of
fundamental components in vision based mapping and navi-
gation such as camera self-calibration, pose estimation and 3D
reconstruction can be achieved.
However, one common phenomenon for keypoint match-
ing is the potential for a mismatch (Gruen, 2012), which
means that the matched keypoints do not correspond to the
same physical object. Depending on the magnitude of mis-
match and overall geometry between the camera and physical
object, it may cause serious problems, such as large estima-
tion biases of camera relative or absolute position and orienta-
tion, and thus affecting integrity, continuity, and availability
of vision-based mapping and navigation.
Mismatch removal is challenging due to large-area repeti-
tive texture, significant viewpoint, and illumination changes
as well as occlusions. Even using the keypoints with a reason-
ably high performance such as
SIFT
(Lowe, 1999), the common
ratio of mismatches can be larger than 50 percent, and even
90 percent in some extreme cases as demonstrated later in the
Comparative Study Using Indoor Images Section
.
Over the past two decades, extensive research has been
performed to eliminate mismatches. Among the published
studies, the process of removing mismatches can be divided
into two basic and relative components, namely (a) corre-
spondence model establishment. and (b) model parameter es-
timation. The correspondence model for correct matches can
be further categorized into two types: general model and local
model. The former is to establish a general representation
that describes the overall relationship for the correct matches,
which includes affine transformation model (Fischler and
Bolles, 1981), epipolar geometry (Huang
et al
., 2007), non-
rigid model (e.g., thin-plate spline model) (Bookstein, 1989)
and collinearity equation (Ferraz
et al
., 2014, Li and Wang,
2014). The latter is to generate certain underlying or indirect
representations of the local correspondences. The representa-
tive local models include descriptor similarity, color, inten-
sity, and geometry information. For example, Liu
et al
. (2012)
developed the Restricted Spatial Order Constraints (
RSOC
)
that quantifies both local and global geometric relationship
to detect mismatches. Wang and Ma (2014) used a bipartite
graph to represent tentative matches and remove mismatches.
Yan
et al
. (2014) modeled the spatial relationship along with
the
SURF
descriptor (Bay
et al
., 2006) as the clue for mismatch
removal. However, no model is perfect as usually each model
has certain assumptions, thus merely a simplification for the
general case. For example, an affine transformation model
is applicable under the assumption of weak perspective and
planar assumption (Bebis
et al
., 1999). If this assumption is
violated, the model is no longer appropriate to describe the
characteristic of correct matches.
Using the selected model, the next step is model param-
eter estimation, which is to estimate correct parameters for
the model and filter out the mismatches. Three categories of
parameter estimation methods, namely statistic based meth-
ods, thresholding, and modern methods, are representative.
Among statistic based methods,
RANSAC
is the most com-
monly applied for parameter estimation, which uses random
sampling strategy to generate a hypothesis for the pre-selected
model (e.g., fundamental matrix or affine transformation
model), and identify the correct matches with the largest sup-
port. One important characteristic for measuring the statistic-
based methods is the breakdown point (Huber, 2011), which
Zeyu Li and Jinling Wang are with the School of Civil and
Environmental Engineering, UNSW Australia, Sydney, NSW
2052, Australia (
).
Charles Toth is with the Department of Civil, Environmental and
Geodetic Engineering, The Ohio State University, OH, 43212.
Photogrammetric Engineering & Remote Sensing
Vol. 83, No. 10, October 2017, pp. 693–704.
0099-1112/17/693–704
© 2017 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.83.10.693
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
October 2017
693
