Integrating User Needs on Misclassification
Error Sensitivity into Image Segmentation
Quality Assessment
Hugo Costa, Giles M. Foody, and Doreen S. Boyd
Abstract
Commonly the assessment of the quality of image segmenta-
tions used in object-based land cover classification uses the geo-
metric match between the derived segmentation and a reference
dataset. This paper argues that a more appropriate assessment
of a segmentation is to also consider the thematic content of the
objects generated. This allows the assessment to be tailored to
the needs of the specific user. A new method for image segmen-
tation quality assessment is described, which combines a tradi-
tional geometric-only method with the thematic similarity index
(
TSI
), a metric that expresses the degree of thematic quality of
objects from a user’s perspective. The perspectives of two users
(a wolf researcher and a general user of land cover information)
were adopted in a case study to demonstrate the new method.
The results show that the new method allowed the production
of more accurate land cover classifications for the two users
than the use of the geometric-only approach.
Introduction
Land cover classification is one of the most commonly under-
taken analyses of remotely sensed images and is used in a wide
range of fields, such as ecology, climatology, and policy. There
is consequently a wide variety of map users and each may
differ greatly in their needs. Some, for example, may require
a very general representation based on broad classes (e.g., An-
derson
et al.,
1976; level I) while others require more thematic
detail. Similarly, the users may vary in their sensitivity to dif-
ferent types of misclassification (DeFries and Los, 1999).
It is well-known that the diverse needs of a range of map
users may result in a variety of perceptions about the qual-
ity of a specific land cover map and the methods used in its
production. Some methods have therefore been proposed to
assess the quality of land cover maps that can include the
user’s perspective. For example, Smits
et al.
(1999) propose a
protocol for quality assessment related to the economic cost
of misclassifications. Stehman (1999) presents map value
measures derived from either user’s accuracy or producer’s
accuracy, with weights assigned to each land cover class ac-
cording to the importance of that class in the eye of the user.
In addition, another measure may be derived from the infor-
mation in the confusion matrix which employs a weighting
scheme to reflect differences in the seriousness of misclassifi-
cations (Stehman, 1999). All of these methods are applied at
the end of the mapping process and were proposed at a time
when land cover classification from remotely sensed data was
performed mainly at the pixel level.
Recently geographic object-based image analysis (
GEO-
BIA
) has emerged as a new trend in land cover classification
(Blaschke
et al.,
2014). The main novelty of
GEOBIA
is the inclu-
sion of an image segmentation stage before the classification,
in which neighboring pixels are merged into discrete mutually
exclusive image objects according to some criteria, such as the
pixels’ spectral (dis)similarity and object size (e.g., Baatz and
Schäpe 2000). The objects ideally form a more appropriate spa-
tial unit than the pixel for land cover mapping applications.
At the classification stage, each object is typically assigned a
single land cover class as a function of, possibly among other
things, the aggregated proprieties of the pixels contained, such
as the mean and standard deviation of the pixel DNs for each
spectral band in use. Thus, segmentation has a large impact on
the map produced and its quality (Gao
et al.,
2011).
There are numerous algorithms able to segment remotely
sensed images. Typically, the algorithms have to be parameter-
ized before undertaking an image segmentation analysis, with,
for example, a threshold set to limit the degree of spectral
heterogeneity allowed inside objects (e.g., Baatz and Schäpe
2000). The variety of algorithms and respective parameteriza-
tion options allows the production of numerous candidate seg-
mentations. Some of the candidate segmentations will allow
the production of a highly accurate classification while many
others will not. It is not practical to generate a large number of
segmentations and use them to derive a set of classifications,
each of which is evaluated, so that the optimal segmentation
may be identified. Instead, the quality of a segmentation is
normally assessed before the classification stage. The goal is to
select the optimal segmentation, which allows the subsequent
classification to achieve the highest possible accuracy.
Traditionally, segmentation has been assessed through visu-
al inspection. However, as this approach is subjective and not
reproducible (Albrecht, 2010), objective methods have been
proposed. The latter can be grouped into two main categories:
goodness methods and discrepancy methods (Zhang, 1996),
also referred to as unsupervised and supervised, respectively.
Unsupervised methods use metrics to measure some desirable
properties of the segmentation (e.g., spectral homogeneity of
the objects obtained), thus measuring their goodness. Super-
vised methods compare the segmentation outputs to a refer-
ence dataset (assumed as the desired segmentation result).
Supervised methods are the focus of the present paper.
The vast majority of supervised methods focus on the
geometric characteristics of the objects obtained, essentially
quantifying the degree of match between the obtained seg-
mentation and that represented in a reference dataset based
on two features: area and position of their components. Clin-
ton
et al
. (2010) and Whiteside
et al
. (2014) provide a good
overview of supervised methods used in remote sensing to
University on Nottingham, School of Geography, University of
Nottingham, Nottingham, NG7 2RD, UK
(
).
Photogrammetric Engineering & Remote Sensing
Vol. 81, No. 6, June 2015, pp. 451–459.
0099-1112/15/451–459
© 2015 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.81.6.451
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
June 2015
451
