Characterizing a Debris Field Using Digital
Mosaicking and CAD Model Superimposition
from Underwater Video
Jay M. Vincelli, Fatih Calakli, Michael A. Stone, Graham E. Forrester, Timothy Mellon, and John D. Jarrell
Abstract
Identifying submerged objects is critical for several disciplines
such as marine archaeology and search and rescue. However,
identifying objects in underwater searches presents many chal-
lenges, particularly if the only data available to analyze is poor-
quality video where the videographer did not plan for photogram-
metric techniques to be utilized. In this paper, we discuss the use
of adaptive sampling of the underwater video to extract sharp
still images for stitching and analysis, and creating mosaicked
images by identifying and matching local scale-invariant feature
transform features using computationally efficient algorithms.
Computer aided design models of suspected aircraft components
were superimposed, and a feature common in multiple mo-
saicked images was used to identify a common feature between
purported objects to assess goodness of fit. The superimposition
method was replicated using landing gear from a reference air-
craft and a rope of known dimensions, and favorably compared
against the remotely operated vehicle (
ROV
) analysis results.
Introduction
Documenting seabed environments, mapping the
seabed, and identifying submerged objects is critical for sever-
al disciplines including marine archaeology, geology, biology,
search and rescue, offshore drilling, and shipping industries.
Establishing positive identification of objects in underwater
searches presents many challenges. The costs involved in
search and recovery operations make false-positive identifica-
tions a pricey and protracted error. Traditionally, a dive team
or a remotely operated underwater vehicle (
ROV
) will search
the proposed area for pieces of potential wreckage in a pre-
determined search pattern (Lirman
et al
., 2007; Zhukovsky
et
al
., 2013). Underwater mapping is still largely carried out by
manual surveys and performing distance-based measurements
(Telem and Filin, 2013; Ruppé and Barstad, 2002). These
tasks become much more difficult when physical contact with
the studied objects is not possible, and when scaling markers
are absent. In these circumstances, an
HD
video recording will
often be used while executing search patterns (Negahdaripour
and Khamene, 2000). On its own, high-definition video can
be a powerful tool for identification but lacks many features
useful for characterizing a debris field (Campos
et al
., 2014).
For instance, the camera on an
ROV
can only capture a limited
view of the seabed. Even with wide-angle lenses, the field of
view in focus is only a small portion of the total surround-
ings. When the seabed contains many objects of interest over
a widely-spaced area, this can be problematic. Without an
expanded view of the proposed debris field, objects and their
ratiometric relationships to each other cannot be analyzed.
While it has been possible to extract still frames, or images,
from video for a long time, still frames can suffer from the ef-
fects of compression due to interlacing and distortion (Negah-
daripour and Khamene, 2000). Without intensive processing,
a comparison between single images can show qualitative
relationships between objects but often lacks meaningful
analytical observation. Furthermore, each image is taken at a
different point of view, skewing potential analysis. In some
cases, imagery from different perspectives is desired, such as
in stereo-photogrammetry, where common points are identi-
fied on each image, and rays can be constructed to each point
to triangulate the position of the points and allow for three-di-
mensional reconstruction. Using photogrammetry tools such
as Agisoft Photoscan (Agisoft, LLC, St. Petersburg, Russia)
allows for orthographic reconstruction in some situations.
Here, we present an alternate approach for instances when
the method of video collection, and conditions of the water,
mean that photogrammetry cannot successfully be employed.
We developed a method that allows us to retrospectively
identify objects and scale them in situations where no scaling
device was used during the video recording and where there
is erratic or unplanned tracking of the camera. This is typical
when the video recording was not intended to be used for
object analysis. We illustrate this approach using a case study,
in which underwater
ROV
footage with an irregular search
pattern and no method for scaling images was used (Figure
1). The approach uses image processing techniques to extract
frames from
HD
video, de-interlace, remove time stamps, and
stitch the remastered still frames together to create a mo-
saicked debris field with a single perspective. We further pre-
sented a strategy to use an object of known scale to measure
other proposed pieces of wreckage and aid in identification.
Background
In this case study, we describe using the combination of im-
age processing and mosaicking techniques using underwater
video as source data to assess the geometry of objects purport-
ed to be from the 02 July 1937 crash site of Amelia Earhart’s
lost airplane, a Lockheed Electra Model 10E, construction
number 1055, off of the island of Nikumaroro in the western
Jay M. Vincelli is with Materials Science Associates, 315
Commerce Park Rd., Unit 1, North Kingstown, RI 02893
(
).
Fatih Calakli, Michael A. Stone, and John D. Jarrell are with
Materials Science Associates, 315 Commerce Park Rd., Unit 1,
North Kingstown, RI 02893; and Brown University, 182 Hope
St., Providence, RI 02912.
Graham E. Forrester is with the University of Rhode Island,
Center for Biotechnology and Life Sciences, 120 Flagg Road,
Kingston, RI 02881.
Timothy Mellon is an independent consultant.
Photogrammetric Engineering & Remote Sensing
Vol. 82, No. 3, March 2016, pp. 223–232.
0099-1112/16/223–232
© 2016 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.82.3.223
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
March 2016
223
