construct a pair of initial triangulations on
the images. Next, feature points are matched
based on the triangulation constraint and
other constraints. The newly matched
points are inserted into the triangulations,
so that they are densified dynamically along
with the matching propagation. Because the
points with the most distinct textures are al-
ways the easiest and earliest to be matched,
the dynamic updating of triangulations is
self-adaptive to image textures. Accordingly,
it propagates the constraints from robust
matches with favorable textures to the
matching in areas with less favorable tex-
tures using triangular segmentation. Finally,
reliable matching results can be obtained.
Wu
et al
. (2012) further incorporated edge
matching into the
SATM
framework.
However, the application of
SATM
to
images of urban areas causes severe prob-
lems due to the aforementioned matching
difficulties. Therefore, the
SATM
+ proposed
in this paper incorporates the following
new components in an attempt to address
the challenges of image matching in ur-
ban areas: (1) occlusion filtering based on
image segmentation, (2) segment-adaptive
similarity measurement to reduce matching
ambiguity, and (3) local and regional dense
matching propagation to generate dense and
reliable matches.
Overview of the Approach
Figure 1 illustrates the workflow of the
SATM
+ approach. Here, robust feature
matches for a pair of high-resolution im-
ages of urban areas are obtained using the
previous
SATM
approach (Wu
et al.
, 2011
and 2012). Image segmentation of one of the
stereo images is then performed. The feature
matches are used to construct a pair of cor-
responding triangulations on the images,
and the disparities of the matched feature
points are used to interpolate a disparity
map. The image segmentation results are
used to fit the disparity map to planes with
constant disparities (e.g., flat building roofs
or ground patches) or smoothly varying dis-
parities (e.g., building facades). Meanwhile,
image matching is conducted under triangu-
lation constraint and other constraints (e.g.,
epipolar constraint). Once a matching candidate is obtained,
occlusion filtering is performed to determine whether the dis-
parity fits the surrounding disparity plane. If the fit is good,
the matching candidate is accepted as a successful new match
and inserted immediately into the triangulations. The sur-
rounding disparity plane is also updated simultaneously by
adding the disparity of the new match, and the corresponding
triangulations and disparity planes are dynamically updated
and densified along with the matching propagation. During
the matching process, a segment-adaptive similarity mea-
surement is used to reduce matching ambiguity in areas near
segment boundaries.
The completion of the above-described integrated image
matching and segmentation process yields a pair of densified
corresponding triangulations from the final matched results,
as well as updated and more accurate disparity planes. These
outputs are subsequently used to provide constraints for the
next dense matching step. Using the dense matching results,
3D point clouds can be derived using photometric space in-
tersection based on the image orientation parameters, and the
3D surface reconstruction results (e.g., digital surface models
(
DSMs
)) can be generated.
Image Segmentation Based on Edge Detection and Region Growing
Image segmentation is a primary component of the proposed
approach. In high-resolution images of urban areas, build-
ings are represented by roofs and facades and normally have
regular shapes with sharp edges. Our approach first uses a
well-known edge detection algorithm to detect the edges in
images. Subsequently, closed edges are used to form seg-
ments, and a region growing algorithm is used to reduce the
over-segmentation problem.
For edge detection, we used the
EDSION
(edge detection
with embedded confidence) algorithm (Meer and Georgescu,
2001), which is more stable and accurate than other algo-
rithms used for high-resolution remote sensing images (Li
et
al
., 2010). The
EDSION
algorithm includes three steps (gradient
Figure 1. Workflow of the integrated image matching and segmentation approach.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
March 2018
137
