Urban Classification by the Fusion of
Thermal Infrared Hyperspectral and Visible Data
Jiayi Li, Hongyan Zhang, Min Guo, Liangpei Zhang, Huanfeng Shen, and Qian Du
Abstract
The 2014 Data Fusion Contest, organized by the Image
Analysis and Data Fusion (
IADF
) Technical Committee of the
IEEE
Geoscience and Remote Sensing Society, involved two
datasets acquired at different spectral ranges and spatial
resolutions: a coarser-resolution long-wave infrared (
LWIR
,
thermal infrared) hyperspectral data set and fine-resolution
data acquired in the visible (
VIS
) wavelength range. In this ar-
ticle, a novel multi-level fusion approach is proposed to fully
utilize the characteristics of these two different datasets to
achieve improved urban land-use and land-cover classifica-
tion. Specifically, road extraction by fusing the classification
result of the
TI-HSI
dataset and the segmentation result of the
VIS
dataset is first proposed. Thereafter, a novel gap inpaint-
ing method for the
VIS
data with the guidance of the
TI-HSI
data is presented to deal with the swath width inconsistency,
and to facilitate an accurate spatial feature extraction step.
The experimental results with the 2014 Data Fusion Contest
datasets suggest that the proposed method can alleviate
the multi-spectral-spatial resolution and multi-swath width
problem to a great extent, and achieve an improved urban
classification accuracy.
Introduction
Large remote sensing datasets for the study of urban land-use
and land-cover are very important for the study of human ac-
tivities and urbanization progress monitoring. Remote sensing
data processing and analysis for urban areas often benefits from
the integration of different information, such as different spec-
tral and spatial resolutions for image pan-sharpening (Guo
et
al.
, 2014), different spatial and spectral features for land-cover
classification (Huang and Zhang, 2011), and different temporal
observations for surface change detection (Huang
et al.
, 2014).
Among the above-mentioned fusion tasks, urban surface
land-cover and land-use classification has been the subject
of a great deal of interest. First, a lot of meaningful spatial
features have been designed to alleviate the discriminative
limitation of the spectral interpretability (Dalla Mura
et al.
,
2010). Based on the multiple features, both feature-level fu-
sion (Li
et al.
, 2014) and decision-level fusion (Huang and
Zhang, 2011), frameworks can be built for urban surface
classification. Second, object-oriented analysis techniques
utilizing the fusion of pixel-level labeling and a segmenta-
tion map can enhance the performance and stability in a
homogenous parcel, as well as speeding up the classification
progress (Hay and Blaschke, 2010; Li
et al.
, 2014). Meanwhile,
most of the current data fusion methods focus on utilizing
passive optical remote sensing data ranging from the visible
to the near-infrared (that is, from 400 nm to 1100 nm) (Zhang
et al.
, 2012). The radiant energy collected by a thermal infra-
red sensor can also contribute meaningful complementary
information to urban remote sensing land-cover classification.
The superiority of thermal infrared imagery is down to the
all-weather and all-time capability. Recently, a new airborne
measurement combination, involving thermal infrared hy-
perspectral imagery (
TI-HSI
) (ranging from 780nm to 1150 nm)
and a simultaneously acquired
VIS
dataset, was released by
the Image Analysis and Data Fusion (
IADF
) Technical Commit-
tee of the
IEEE
Geoscience and Remote Sensing Society (
GRSS
)
for the 2014 Data Fusion Contest (
.
it/IPRS/
IEEE
_
GRSS
_
IADF
TC_2014_Data_Fusion_Contest.htm
).
These datasets are suitable for urban land-use and land-cover
classification because the thermal infrared data can describe
surface-emitted energy differences caused by human activity
(Rodríguez-Galiano
et al.
, 2012); meanwhile, the drawback of
the coarse spatial resolution can be overcome by integration
with the very high spatial resolution (
VHSR
)
VIS
data.
With the aforementioned datasets released by the
IADF
Technical Committee, this article proposes a new fusion
framework for urban surface land-cover and land-use classifi-
cation. Among the recent state-of-the-art research, the win-
ners of the classification task in the 2014 Data Fusion Contest
designed a feature fusion approach by combining several of
the top principal components (
PC
s) of the
TI-HSI
data and some
of the spectral and spatial features of the
VIS
imagery. The run-
ners-up in the contest utilized various spectral-spatial features
as a filter to extract the urban land-use and land-cover classes
one by one. Details of the processing strategies of the other top
results can be found on the website of the Data Fusion Contest
/
IEEE
_
GRSS
_
IADF
TC_2014_
Classification_Contest_Results.htm
). It is notable that most
of these methods directly take the spectral information of the
TI-HSI
data into a supervised machine learning procedure,
without considering the specific discriminability of the differ-
ent urban objects in the thermal infrared spectral domain.
Interpretation of this new dataset combination with dif-
ferent resolutions and different swath widths is quite chal-
lenging. In view of this, the following issues must be taken
into consideration in the data analysis process. With regard
to the
TI-HSI
data: (a) the low energy and low signal-to-noise
ratio (
SNR
) in each band significantly affect the extraction of
discriminative features (Zhang
et al.
, 2014); (b) the high inter-
band correlation reveals significant spectral redundancy (Yan
and Niu, 2014); (c) there can be spectral variation in the same
Jiayi Li, Hongyan Zhang, Min Guo, and Liangpei Zhang are
with the State Key Laboratory of Information Engineering
in Surveying, Mapping, and Remote Sensing, and the
Collaborative Innovation Center for Geospatial Technology,
Wuhan University, P.R. China, 129 Luoyu Road, Wuhan,
Hubei, 430079 P.R. China
).
Huanfeng Shen is with the School of Resource and
Environment Science, Wuhan University, P.R. China, 129
Luoyu Road, Wuhan, Hubei, 430079 P.R. China.Qian Du is
with the Department of Electrical and Computer Engineering,
Mississippi State University, 406 Hardy Road, Mississippi
State, MS 39762.
Photogrammetric Engineering & Remote Sensing
Vol. 81, No. 12, December 2015, pp. 901–911.
0099-1112/15/901–911
© 2015 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.81.12.901
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
December 2015
901
