In this paper, four different methods are adopted and re-
fined to improve the image quality and spatial resolution, one
of them is bicubic-upscale interpolation, and the other three
are
CNN
-based
SR
methods
SRCNN
,
VDSR
, and
SRMD
. To train a
robust
CNN
architecture specially for re
both natural images and aerospace ima
network training samples to retrain the
improving network adaptability.
Due to the special calculating operation of convolution, it
will cause serious information loss at image border, while it is
necessary to keep the same size between the input and output
images in this application. As used in many papers, border
padding before convolution operation is an effective way to
deal with the problem. Kim
et al.
(2016) padded zeros around
the original image before each convolution layer to make the
output have the same size as the input. However, the zero-
padding method inevitably bring some useless information or
even jeopardize the reconstruction of images. In our previous
work (Luo
et al.
2017), we calculated the size difference be-
tween the input and output according to convolutional kernel
and stride of each convolutional layer, and proposed to use
symmetric padding as a substitute to alleviate the information
loss at image border. In this paper, to further avoid informa-
tion loss and useless information taken-in for a better
DSM
, we
refined the
SR
models by applying symmetric padding before
each convolution layer for better capturing image information
rather than just used it at the very beginning.
DSM Generation
All the mentioned approaches regarding to
DSM
generation
are the previous achievements of our research team, which
makes a good foundation for the verification of the proposed
subpixel level
DSM
generation application. We will give a brief
overview on how to construct the
DSM
generation procedure
in this subsection.
As illustrated in Figure 1, this
DSM
generation procedure
contains four main steps: tie-point matching, mismatch detec-
tion, block adjustment, and dense image matching.
Firstly, tie-point matching is used to find corresponding
points in a pair of images (even in multiple images). Consider-
ing that images used for
DSM
generation may come from dif-
ferent remote sensing sensors, a tie-point matching algorithm
for multisource images (Ling
et al.
2016) is implemented for
corresponding points identification. After that, to produce reli-
able matching results, a region segmentation based matching
propagation strategy is designed in this algorithm, so that more
corresponding points for afterwards process can be obtained.
Secondly, considering the mismatch situation caused by
tie-point matching process, a mismatch detection method
should be brought into our
DSM
generation procedure. Wan
et
al.
(2017) proposed an effective mismatch detection method
for push broom high-resolution satellite images, named point-
to-line distance based (
P2L
) method. This method can distin-
guish most of the mismatches, even for the image pairs which
have a 54-degree intersection angle. Therefore, we take it for
our mismatch detection process.
At last, with the selected corresponding points, we con-
strain the elevation of each tie point for the sake of relative
geometric rigidity based on a
DEM
-Assisted rational function
model (
RFM
) (Zhang
et al.
2016). And a robust dense image
matching method named semiglobal vertical line locus match-
ing (
SGVLL
) is taken for the
DSM
generation (Zhang
et al.
2017).
Besides the aforementioned procedure, there is another
important step to this application, which is called rational
polynomial coefficient (
RPC
) regeneration.
RPC
files describes
the correlation between satellite images and three-dimension
objects and are necessary for image matching. As different
satellite images have different
RPC
files, the
RPC
files also need
to be regenerated when the images are super-resolved. The
procedure of regenerating
RPC
files is as follows:
1. With the original
RPC
file and
LR
image, the relationship
between the
LR
image and a virtual three-dimensional grid
can be established, so the geographic position of the grid
.
three-dimensional grid and new sample
ew
RPC
file can be regenerated.
Quality Assessment Metrics
Image quality refers to visually significant attributes of images.
As images with higher quality can offer better image texture
for subsequent process, it is necessary to evaluate the images
quality before taking them into the
DSM
generation process. To
evaluate the performance of the quality of reconstructed meth-
ods, peak signal-to-noise ratio (
PSNR
) and structural similarity
index (
SSIM
) are selected as quantitative metrics.
PSNR
(in dB)
and
SSIM
are calculated in Equations 2 and 3:
PSNR
=
−
(
)
10
2 1
10
log
MSE
n
(2)
where
MSE
is defined in Equation 1.
The calculation of
SSIM
is based on three relatively inde-
pendent indicators, namely luminance, contrast, and struc-
ture.
SSIM
is defined as follows:
SSIM
I I
C I I
C I I
C I I
l
c
s
,
,
,
,
( )
=
( )
( )
( )
α
β
γ
ˆ
ˆ
ˆ
(3)
Figure 1. Flowchart of the application to subpixel level
DSM
generation.
Figure 2.
CNN
architecture for
SR
.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
October 2019
767
