Double Projection Planes Method for Generating
Enriched Disparity Maps from Multi-View
Stereo Satellite Images
Alaeldin Suliman and Yun Zhang
Abstract
The use of disparity information is computationally cost-
effective for 3D-assisted mapping applications. However,
off-nadir satellite images over dense urban areas suffer from
severe occlusion. Therefore, large gaps will be created due to
the unsuccessful matching in the occluded areas. Commonly,
gap interpolation produces misleading information that
destroys the quality of the subsequent information extraction
application. Hence, the more reliable solution is to fill the oc-
clusion gaps by supplementary data. However, disparity maps
are the co-relation of one stereo pair. Hence, supplementary
disparity maps cannot be directly generated and applied.
Thus, this paper introduces the Double Projection Plane (
DPP
)
method for constructing disparity proportionality among
multi-view stereo satellite images and calculating analyti-
cally the transferring scales required for producing supple-
mentary disparity data. Accordingly, based on multi-view
stereo satellite images, this method is promising to provide
gap-and-outlier-free disparity maps that have the potential
to replace the need for elevation models to some extent.
Introduction
The currently available very high resolution (
VHR
) satellite sen-
sors are able to provide along-track (inline) multi-view stereo
(
MVS
) images of sub-meter ground resolution, at low cost, and
with broad coverage. Digital elevation models (
DEMs
), ortho-
photos, and 3D rendered photo models are well-known prod-
ucts that can be reconstructed from stereo
VHR
satellite images
based on extracting the third dimension. Such products are
essential for many real-world applications including planning,
designing, and management of urban environments. Conse-
quently, stereo image processing of satellite images has become
an active research area within the remote sensing community.
The extraction of the third dimension from stereo images
is a well-established procedure in the field of Digital Photo-
grammetry. This procedure utilizes matching techniques to
provide stereo-based image measurements known as dispari-
ties before they are used to derive object-space elevations.
Hence, the third dimension may be represented by either the
disparity or the elevation information. Since extracting the
elevations photogrammetrically requires several calculation
steps after measuring the disparity data from stereo images,
the use of the disparity data instead is computationally cost-
effective in terms of the number of computational steps and
consequently the runtime.
An additional and essential advantage of using dispar-
ity maps over elevation models is the direct co-registration.
Unlike the orthographic elevation models, disparity maps,
as per definition, perfectly fit the same reference frame of the
reference image. Hence, pixel-level co-registration accuracy
between the selected reference image and the generated dis-
parity map is guaranteed without any further effort.
Moreover, because disparity maps accurately identify the
elevated objects in the co-registered image, these maps are
adequate for most 3D-assisted information extraction applica-
tions. In the case of off-nadir images, disparity maps provide
perfect co-registration by following the same line-of-sight of
the reference image. For 3D-assissted building detection in off-
nadir satellite images, the line-of-sight elevation model solu-
tion has been found promising. This solution was developed
by Suliman and Zhang (2015) to achieve accurate image-eleva-
tion co-registration even with off-nadir images. In contrast to
elevation models, this solution is available without extra effort
when the disparity maps are used instead of elevation models.
Occlusion areas remain a standing problem for all stereo
matching techniques since these areas are usually visible in
only one image of the stereo pair (known as half-occlusion).
Consequently, the generated disparity maps from stereo pairs
usually suffer from two major challenges: failed and false
matches. The failed matches are the unmatched pixels that
create no-data regions (gaps). On the other hand, the false
matches are the incorrectly matched points that create outliers.
These outliers are typically detected and removed from the
disparity map. Therefore, both of these unsuccessful matches
(failed and false matches) produce gaps that need to be filled.
In the case of matching stereo off-nadir satellite images
acquired over dense urban areas, the two challenges and
the resulting gaps are further exacerbated due to the exces-
sive building lean. This lean usually hides large areas in one
image of the stereo pair leading to unsuccessful matching.
Moreover, different building façades may have a similar ap-
pearance that confuses matching algorithms and hence create
additional false matches. Thus, disparity maps generated from
stereo off-nadir images over dense urban areas usually suffer
from large occlusion gaps. Figure 1 shows a case of stereo
images that contain an occluded area and a confusing similar-
ity that are usually encountered in
VHR
stereo images when
acquired off-nadir over urban areas. This pair will produce a
large number of failed and false matches.
Alaeldin Suliman is with the Department of Civil
Engineering, University of New Brunswick, 17 Dineen
Drive, Fredericton, NB, Canada, E3B 5A3; and formerly with
the Department of Geodesy and Geomatics Engineering,
University of New Brunswick
).
Yun Zhang is with the Department of Geodesy and Geomatics
Engineering, University of New Brunswick, 15 Dineen Drive,
Fredericton, NB, Canada, E3B 5A3; and with the Institute of
Remote Sensing and GIS at Peking University, and MIT Media
Lab at the Massachusetts Institute of Technology.
Photogrammetric Engineering & Remote Sensing
Vol. 83, No. 11, November 2017, pp. 749–760.
0099-1112/17/749–760
© 2017 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.83.10.749
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
November 2017
749
