linear interpolation model is introduced to integrate the spa-
tial and spectral information from the preprocessed input data
to produce more accurate transitional fused results. Then, a
residual dense network is used to learn the correspondence
relationship between the preliminary fused result and the real
Landsat image. This advantage of introducing the residual
dense network is based on extracting and using the hierarchi-
cal features from transitional fused data, thus, more detailed
spatial structure of the image can be captured in the fusion
results. In the prediction phase, a given Landsat-
MODIS
image
pair and a
MODIS
image on the prediction date are used to
predict the Landsat image on the prediction date. In addition,
the trained model can be saved and reused, it is very useful
for improving data processing efficiency when acquiring long-
term sequence images in the same study area.
Two data sets with different land surface changes are used to
conduct experiments. Compared with
STARFM
and
Fit-FC
algo-
rithms, the proposed method can predict phenological changes
more effectively and achieve better robustness for
CIA
data sets.
For LGC data sets, the proposed algorithm and
Fit-FC
algorithm
have similar fusion accuracy, but the proposed algorithm can
more effectively predict the spatial structure characteristics of
images. Therefore, the proposed method is an effective spatio-
temporal fusion method for remote sensing images.
In further studies, the network will be further optimized
to meet the actual needs, more training data set are needed to
improve the robustness of proposed method, and the perfor-
mance of the proposed method will be conducted on more
different remote sensing images fusions.
Acknowledgments
This research was supported by the National Natural Science
nder Grant 41971400, and in part by
arch Funds for the Central Universities
Y09. We would like to thank Dr. Qun-
ing access to
Fit-FC MATLAB
code, and Dr.
Feng Gao for making
STARFM
c code available, and the editor
and all reviewers whose insightful suggestions have signifi-
cantly improved this paper.
References
Aiazzi, B., L. Alparone, S. Baronti, R. Carlà, A. Garzelli and L.
Santurri, 2017. Sensitivity of pansharpening methods to
temporal and instrumental changes between multispectral and
panchromatic data sets. IEEE Transactions on Geoscience and
Remote Sensing 55 (1):308–319.
Aiazzi B., S. Baronti and M. Selva. 2007. Improving component
substitution pansharpening through multivariate regression of
MS + Pan data. IEEE Transactions on Geoscience and Remote
Sensing 45 (10):3230–3239.
Amorós-López, J., L. Gómez-Chova, L. Alonso, L. Guanter, R. Zurita-
Milla, J. Moreno and G. Camps-Valls. 2013. Multitemporal
fusion of Landsat/TM and Envisat/Meris for crop monitoring.
International Journal of Applied Earth Observation and
Geoinformation 23:132–141.
Chen, B., B. Huang and B. Xu. 2015. Comparison of spatiotemporal
fusion models: A review. Remote Sensing, 7 (2):1798–1835.
Cheng, Q., H. Liu, H. Shen, P. Wu and L. Zhang. 2017. A spatial
and temporal nonlocal filter-based data fusion method. IEEE
Transactions on Geoscience and Remote Sensing 55 (8):4476–4488.
Table 2. Accuracy evaluation of fusion results of
STARFM
,
Fit-
FC
, and the proposed algorithm in Experiment 2.
Band RMSE SSIM r
QI
ERGAS SAM
STARFM
1
0.0117
0.9568 0.8511 0.4632
1.2598 4.9599
2
0.0140
0.9482 0.8692 0.5132
3
0.0184 0.9209 0.8857 0.5322
4
0.0337 0.8556 0.9287 0.5264
5
0.0697 0.8123 0.7248 0.4986
7
0.0754 0.8046 0.6754 0.4735
Fit-FC
1
0.0121
0.9606 0.8472
0.4520
0.8616
4.4235
2
0.0144
0.9515 0.8648
0.4901
3
0.0179 0.9284 0.8952
0.5360
4
0.0327 0.8740 0.9339
0.5
5
0.0425 0.8151
0.8812
0.4
7
0.0344 0.8264
0.8922
0.4
Ours
1
0.0121 0.9591 0.8368
0.4780
0.8595
4.6001
2
0.0150 0.9473 0.8492
0.5164
3
0.0183 0.9257 0.8810
0.5428
4
0.0346 0.8729 0.9212
0.5533
5
0.0403 0.8455
0.8765
0.5344
7
0.0328 0.8546
0.8880
0.5424
Figure 8. Predication results on 29 January 2005 fused by (a) actual image, (b)
STARFM
, (c)
Fit-FC
, and (d) our method" to " (a)
actual image on 29 January 2005, and predication results fused by, (b)
STARFM
, (c)
Fit-FC
, and (d) our method
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
December 2019
913
