Improving Component Substitution
Pan-Sharpening Through Refinement
of the Injection Detail
Xiaohua Li, Hao Chen, Jiliu Zhou, and Yuan Wang
Abstract
This article presents a novel strategy for improving the well-
established component substitution-based multispectral
image fusion methods, because the fused results obtained
by component substitution methods tend to exhibit signifi-
cant spectral distortion. The main cause of spectral distor-
tion is analyzed and discussed based on the component
substitution method’s general model. An improved scheme
is derived from the sensitivity imaging model to refine the
approximate spatial detail and obtain one that is almost
ideal. The experimental results on two data sets show that
when it has been integrated into the Gram–Schmidt method
and the generalized intensity-hue-saturation method, the
proposed scheme allows the production of fused images of
the same spatial sharpness as standard implementations
but with significantly increased spectral quality. Quanti-
tative scores and visual inspection at full resolution and
spatially reduced resolution confirm the superiority of the
improved methods over the conventional algorithms.
Introduction
Pan-sharpening is a branch of image fusion which combines
a coarse-spatial-resolution multispectral (
MS
) image and a
co-registered fine-spatial-resolution panchromatic (
PAN
) image
to produce a fine-spatial-resolution
MS
image (Pohl and Van
Genderen 1998). Pan-sharpening can be useful for many prac-
tical applications, such as geometric correction, change detec-
tion, land cover classification, and earth
ingly, the remote sensing community is
attention (Aiazzi, Alparone and Baronti
In recent years, numerous pan-sharpening techniques have
been developed that can be divided into conventional and
modern methods. Conventional approaches are usually based
on simple fusion models and have been extensively studied
(Amro
et al.
2011; Vivone
et al.
2015). Modern methods are
usually based on more complex fusion models, such as total
variation (Aly and Sharma 2014; Wei, Dobigeon and Tourn-
eret 2015), compressive sensing (Zhu and Bamler 2013; He
et al.
2014; Deng, Feng and Tai 2019), convolutional neural
networks (Masi
et al.
2016; Yang
et al.
2017; Liu, Wang and
Liu 2018), and generative adversarial networks (Zhang, Li and
Zhou 2019). These modern methods can outperform con-
ventional techniques in some cases, but they require a large
number of training samples and suffer from a heavy computa-
tional burden.
The conventional methods can be further divided into
component substitution (
CS
)-based methods and multireso-
lution analysis (
MRA
)-based techniques (Aiazzi
et al.
2012).
The
CS
methods assume that the spatial detail information
of the
MS
image mainly lies in its structural component. The
structural component can be obtained by transforming the
multi-channel
MS
image into a new space. Then the structural
component is completely or partially replaced by the
PAN
image to increase the spatial detail information. Finally, the
fused
MS
image with finer spatial resolution is obtained by
inverse transformation. Given its high computational efficien-
cy, the
CS
method is extensively used in actual applications
(Aiazzi
et al.
2007). Methods that are based on intensity-hue-
saturation (
IHS
; Haydn
et al.
1982; Tu
et al.
2004), principal
component analysis (
PCA
; Chavez and Kwarteng 1989; Shah,
Younan and King 2008), the Gram–Schmidt process (
GS
;
Laben and Brower 2000; Aiazzi
et al.
2007), and the Brovey
transform (Gillespie, Kahle and Walker 1997) are the most
widely known
CS
methods. The
MRA
methods argue that the
missing spatial detail information in the
MS
image can be de-
rived from the high-frequency components of the
PAN
image
(Ranchin and Wald 2000). Therefore, the spatial detail infor-
mation can be extracted by employing a linear space-invariant
digital filter and then adding the coarse-resolution
MS
bands.
General implementations of such methods include ones based
on the wavelet transform (Núñez
et al.
1999; Amolins, Zhang
and Dare 2007), a generalized Laplacian pyramid (
GLP
; Aiazzi
et al.
2006), and the curvelet transform (ALEjaily, El Rube and
ally superior to
MRA
methods in terms
ial detail information. However, because
the spectral ranges covered by
PAN
and
MS
images do not
strictly overlap, the pan-sharpening results obtained by
CS
methods tend to be accompanied by a degree of spectral dis-
tortion (Aiazzi
et al.
2007). In this article, we propose a novel
strategy to mitigate the spectral distortion of
CS
methods.
Starting with the general scheme of the
CS
methods, the main
cause of spectral distortion is analyzed and discussed, and
a formula that refines the traditional spatial detail is derived
from the spectral sensitivity imaging model. To the best of our
knowledge, this is the first attempt to improve the
CS
method
by refining the injection detail based on the investigation of
the spectral sensitivity imaging model. We apply this strategy
to two conventional
CS
algorithms: the generalized intensity-
hue-saturation (
GIHS
) method and the Gram–Schmidt (
GS
)
method. Quantitative assessments of the two data sets under
full resolution and reduced resolution confirm the superiority
Xiaohua Li, Hao Chen, and Jiliu Zhou are with the College
of Computer Science, Sichuan University, Chengdu, People’s
Republic of China (
).
Yuan Wang is with the Key Laboratory of Radiation Physics
and Technology, Ministry of Education, Institute of Nuclear
Science and Technology, Sichuan University, Chengdu,
People’s Republic of China (
).
Photogrammetric Engineering & Remote Sensing
Vol. 86, No. 5, May 2020, pp. 317–325.
0099-1112/20/317–325
© 2020 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.86.5.317
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
May 2020
317
