must be made before on-site inspections can be performed,
these choices could be informed using the products generated
through the automation of the damage detection process.
Previous studies such as Quackenbush, 2004 outline
general methods of linear feature (primarily roads) extraction
from remotely-sensed imagery using techniques including
mathematical morphology and template matching. Additional
works have examined the characteristics of asphalt pavement
deterioration (Herold and Roberts, 2005), and others (e.g.,
Dubois and Lepage, 2014) have attempted to make rapid esti-
mates of building damage (collapse) on the block level. How-
ever, based on survey of the literature, no studies were found
that attempted to identify fine-scale features in near real-time
based on pre- and post-event imagery, where new cracks are
not the only target features in the image. We find no examples
of an automated process to identify existing or new cracks
in road and bridge surfaces that uses very high resolution
(VHR), bi-temporal remotely sensed images of complex scenes
as input. Numerous studies, (e.g., Zou
et al.
, 2011; Amhaz
et
al.
, 2016; and Shi
et al.
, 2016) have developed state-of-the-
art, automated methods to detect cracks in pavement images.
However, these studies all use images that contain uniform
pavement surfaces that were captured at very close range as
input for analysis. They tend to have no requirement for filter-
ing or removing changes not of interest from actual cracks, as
cracks are the only feature present or change introduced. This
limits these methods from being deployed in an operational
post-hazard context as they have no ability to differentiate
different types of actual change or examine false detection.
The introduction of noise in the scene, i.e., additional features
other than cracks, would most likely severely limit the ability
of current methods to successfully delineate cracks in a scene
without returning a high number of false detections.
Through this study we attempt to fill this gap by automat-
ing the detection of fine-scale change(s) using bi-temporal im-
age pairs of more complex scenes containing multiple types
of surface changes in addition to new cracks, such as the
movement of cars, people, shadows, and vegetation, as well as
existing cracks (i.e., those already in the baseline image). We
then differentiate and classify changes as either a new crack
or a change feature not of interest. This study takes an opera-
tional approach to crack detection where equal importance is
given both to the accurate detection of cracks as well as the
minimization of false detections. To summarize, the objective
of this study is to develop, test and compare semi-automated
methods of detecting fine-scale damage to critical infrastruc-
ture, specifically manifested as new cracks in road and bridge
surfaces using images of complex scenes as input.
This study is the first comparative analysis of different
aerial, image-based crack detection methods targeting fine-
scale crack damage present in a complex scene. This is also
the first study that compares the relative effectiveness of
various image processing methods of fine scale crack damage
delineation using both bi-temporal (pre- and post-event) im-
age pairs, as well as single-date (post-event) images as input.
Various detection and delineation methods including pixel-
based, object-based image analysis (
OBIA
), and spatial contex-
tual analysis are tested to determine the optimal strategy for
detecting fine-scale damage. The bi-temporal image change
detection techniques build on the repeat station imaging (
RSI
)
approach to aerial image capture and registration, where im-
age capture from aircraft is location-based and repetitive, such
that sensor type and view geometry are replicated. Through
this replication of image capture location and view geom-
etry, geometric distortions caused by viewpoint change are
effectively eliminated and rapid registration is possible with
computationally simple methods (Coulter
et al.
, 2003; Coulter
et al.
, 2015; Stow
et al.
2016). It is important to note that the
most promising of the detection methods tested in this study
will be recommended for integration into time-sensitive
remote sensing systems (Lippitt
et al.
, 2014; Stow
et al.
, 2015)
for damage detection, for which
RSI
capture and registration is
necessary for effective application in near real-time.
As there are significant time and resource costs associated
with compiling and maintaining an up-to-date archive of
pre-event imagery of at-risk areas, the relative effectiveness of
bi-temporal versus single-date approaches is compared. To
justify these costs, the change detection products utilizing bi-
temporal imagery need to be more accurate than the detection
methods using only single-date images as input. As the exact
amount of improvement necessary to justify the use of bi-tem-
poral imagery will be subjective, the use of bi-temporal imag-
ery is deemed worth the cost if the composite accuracy metric
calculated as the product of the producer’s and user’s accu-
racy is higher than methods using the single-date approach.
Previous studies have compared the performance of manual
classification techniques using single-versus bi-temporal im-
ages for the detection of damage (Dong and Shan, 2013; Saito
et al.
, 2004; Yamazaki
et al.
, 2004), while this study compares
their benefits when applied to automated methods.
Through the testing of automated methods for detecting
and delineating fine-scale crack damage in near real-time us-
ing three band (
RGB
) images of complex scenes captured with
aerial digital frame camera systems, the following questions
have been addressed:
1. Do automated change detection methods utilizing bi-tem-
poral (pre- and post-event) imagery outperform methods
that rely on only single-date (post-event) imagery in the
detection and delineation of fine-scale crack damage fea-
tures? and,
2. What image change detection methodology or combina-
tion of approaches (
OBIA
, pixel-based or spatial contextual)
provide the most accurate results for detecting fine-scale
crack damage features; and what level of accuracy can be
achieved in the detection and delineation of fine-scale
damage features?
Methods
Data
Four sets of images were used to design and test varying
damage detection and delineation methods. Image sets were
captured with high-grade consumer
SLR
cameras mounted on
light sport or small unmanned aircraft, in the three visible
bands (red, green and blue (
RGB
)). The spatial resolution of
the images used vary between 2.5 cm and 7.5 cm. Bi-temporal
image pairs were captured using the
RSI
technique at varying
altitudes between one hour and one day apart with vary-
ing shadow conditions. Image pairs were directly registered
to one another and not georeferenced. As images depicting
desirable targets displaying actual damage features of inter-
est captured at the appropriate resolution for this study are
not generally available, these image sets depict scenes with
the addition or movement of objects over time, representing
damage. These proxy damage features were created by adding
black tape to road surfaces to mimic cracks.
Image datasets cover portions of four study areas: (a) Camp
Roberts California National Guard Post, (b) Lake Murray Com-
munity Park 1, (c) Lake Murray Community Park 2, and (d)
Bridge Blvd SW spanning the Rio Grande River in Albuquer-
que, NM. All bi-temporal image pairs were captured using the
RSI
technique to replicate sensor and view geometry (Coulter
et al.
, 2015). These images were then registered through a
point matching technique using
SIFT
and
RANSAC
Alignment
(
SARA
), an automatic image registration software routine
developed at San Diego State University. Figure 1 shows an
example image frame from each of the four study scenes.
76
February 2018
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
