I
ntroduction
Most uses of remote sensing, re-
gardless of the application, require
the conversion of imagery into
information. This process can be
accomplished in multiple and often
interrelated ways that have evolved
over time. The authors believe, and
evidence presented here, supports the
view that today the remote sensing
community is not placing enough em-
phasis on image understanding when
developing tools and approaches to
extract information from imagery.
Visual interpretation historically
was the primary way to extract infor-
mation because of the simple black
and white aerial photography that
was available, along with limited
technology to assist in the process.
As the data became more complex
and digital, more machine based ap-
proaches were developed. Currently,
many of the information extraction
processes, especially in operational
applications, are a combination of
visual and machine techniques.
The concern of the authors is that
while visual interpretation methods
continue to be an extremely im-
portant and a common component
of information extraction in remote
sensing, they are not given the nec-
essary emphasis in the preparation
of the remote sensing workforce.
This paper addresses these concerns
and, to provide context, includes a
brief review of the history of remote
sensing textbooks, elements of
image interpretation, and examples
of the importance of visual methods.
The focus of this discussion will be
on land surface feature information
from airborne and spaceborne plat-
forms, but the primary issues can
be extended to other aspects and
applications of remote sensing.
V
isual
vs
M
achine
A
pproaches
The processes to convert remote sensing data to information have changed
considerably over time as have the platforms, sensors, and applications. There
are generally perceived to be two primary methods for information extraction.
In the first an analyst makes decisions by visual examination of the data and in
the second a machine/computer in essence makes the decisions (Franklin and
Wulder
Berberoglu and Akin
. The combination of the two methods
is also common, especially in an applied or operational effort where a machine
makes initial decisions and then an analyst reviews those decisions and makes
changes as appropriate.
An advantage of visual analysis by a skilled and qualified analyst is that the
context, site, situation, or association of a feature with its surroundings, are all
examined in the interpretation process. In addition, such an analyst can better
incorporate knowledge of the temporal and seasonal changes in surface features
such as crop calendar and natural vegetation phenology. A limitation of visual
analysis compared to machine or digital processing is that only a portion of the
data can be evaluated at a time as an analyst typically can only use a subset of
the spectral, spatial, and radiometric resolutions of the data. Furthermore, er-
rors can be introduced depending on the experience of the interpreter or whether
or not a systematic approach such as a dichotomous key is used.
Machine assisted information extraction in remote sensing has many different
components. Typically, it is a variation of spectral signature matching where the
first process is to obtain a set of appropriate signatures either from a spectral
library or more often from the image directly via supervised or unsupervised
procedures. Once suitable signatures are compiled, one of numerous statistical
decision rules is employed to determine, often on a pixel-by-pixel basis, which
signature each pixel is most similar to for classification of land use and land cov-
er (LULC) or other parameters. This approach considers the “color” or “tone” of a
feature and does not generally incorporate other interpretation factors. This can
lead to some interesting but erroneous results. For example, in the mid-1970s
land cover maps were created of the US portion of the Great Lakes Basin to es-
timate future population and pollution loadings. Automated methods were used
with Landsat MSS data without adequate attention to the local geography and
field verification. As a result of urban areas and open pit mines looking spectrally
much the same (i.e. they had similar signatures) on the early Landsat data, large
cities erroneously appeared in the land cover maps of northern Minnesota.
Among the advantages of machine or digital information extraction is the ability
to examine the full spatial, spectral, and radiometric resolutions of the data.
However, pixel based image analysis is generally limited by the lack of context,
texture, site or association that visual examination provides. A human has a
higher probability of understanding the landscape in most situations than a
machine. Machine assisted information extraction generally employs one of
numerous available commercial or open source image processing software. There
are many variations of the signature matching process and other approaches to
digital analysis. For example, one of the limitations of pixel by pixel classifica-
tion may be avoided by an object based approach where pixels are grouped into
objects or segments and the classification is for the entire object.
The visual and machine methods as well as a hybrid of the two can have ap-
propriate roles in remote sensing. They both have advantages and disadvan-
tages and often neither one nor the other can be argued to be more accurate or
objective
(Richards
. They both include subjective decisions either in the
796
December 2017
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Photogrammetric Engineering & Remote
Sensing, Vol. 83, No. 12, December
2017, pp. 795–806.
0099-1112/17/795–806
© 2017 American Society for
Photogrammetry and Remote Sensing
doi: 10.14358/PERS.83.12.795
