Characterizing a Debris Field Using Digital
Mosaicking and CAD Model Superimposition
from Underwater Video
Jay M. Vincelli, Fatih Calakli, Michael A. Stone, Graham E. Forrester, Timothy Mellon, and John D. Jarrell
Abstract
Identifying submerged objects is critical for several disciplines
such as marine archaeology and search and rescue. Howev-
er, identifying objects in underwater searches presents many
challenges, particularly if the only data available to analyze
is poor-quality video where the videographer did not plan for
photogrammetric techniques to be utilized. In this paper, we
discuss the use of adaptive sampling of the underwater video to
extract sharp still images for stitching and analysis, and creating
mosaicked images by identifying and matching local scale-in-
variant feature transform features using computationally efficient
algorithms. Computer aided design models of suspected aircraft
components were superimposed, and a feature common in mul-
tiple mosaicked images was used to identify a common feature
between purported objects to assess goodness of fit. The super-
imposition method was replicated using landing gear from a
reference aircraft and a rope of known dimensions, and favorably
compared against the remotely operated vehicle (
ROV
) analysis
results.
Introduction
Documenting seabed environments, mapping the
seabed, and identifying submerged objects is critical for several
disciplines including marine archaeology, geology, biology, search
and rescue, offshore drilling, and shipping industries. Establishing
positive identification of objects in underwater searches presents
many challenges. The costs involved in search and recovery oper-
ations make false-positive identifications a pricey and protracted
error. Traditionally, a dive team or a remotely operated underwater
vehicle (
ROV
) will search the proposed area for pieces of poten-
tial wreckage in a pre-determined search pattern (Lirman
et al
.,
2007; Zhukovsky
et al
., 2013). Underwater mapping is still largely
carried out by manual surveys and performing distance-based
measurements (Telem and Filin, 2013; Ruppé and Barstad, 2002).
These tasks become much more difficult when physical contact
with the studied objects is not possible, and when scaling markers
are absent. In these circumstances, an
HD
video recording will
often be used while executing search patterns (Negahdaripour
and Khamene, 2000). On its own, high-definition video can be a
powerful tool for identification but lacks many features useful for
characterizing a debris field (Campos
et al
., 2014). For instance, the
camera on an
ROV
can only capture a limited view of the seabed.
Even with wide-angle lenses, the field of view in focus is only a
small portion of the total surroundings. When the seabed contains
many objects of interest over a widely-spaced area, this can be
problematic. Without an expanded view of the proposed debris
field, objects and their ratiometric relationships to each other
cannot be analyzed.
While it has been possible to extract still frames, or images,
from video for a long time, still frames can suffer from the ef-
fects of compression due to interlacing and distortion (Negah-
daripour and Khamene, 2000). Without intensive processing,
a comparison between single images can show qualitative
relationships between objects but often lacks meaningful
analytical observation. Furthermore, each image is taken at a
different point of view, skewing potential analysis. In some
cases, imagery from different perspectives is desired, such as
in stereo-photogrammetry, where common points are identi-
fied on each image, and rays can be constructed to each point
to triangulate the position of the points and allow for three-di-
mensional reconstruction. Using photogrammetry tools such
as Agisoft Photoscan (Agisoft, LLC, St. Petersburg, Russia)
allows for orthographic reconstruction in some situations.
Here, we present an alternate approach for instances when
the method of video collection, and conditions of the water,
mean that photogrammetry cannot successfully be employed.
We developed a method that allows us to retrospectively
identify objects and scale them in situations where no scaling
device was used during the video recording and where there
is erratic or unplanned tracking of the camera. This is typical
when the video recording was not intended to be used for
object analysis. We illustrate this approach using a case study,
in which underwater
ROV
footage with an irregular search
pattern and no method for scaling images was used (Figure
1). The approach uses image processing techniques to extract
frames from
HD
video, de-interlace, remove time stamps, and
stitch the remastered still frames together to create a mosa-
icked debris field with a single perspective. We further pre-
sented a strategy to use an object of known scale to measure
other proposed pieces of wreckage and aid in identification.
Background
In this case study, we describe using the combination of im-
age processing and mosaicking techniques using underwater
video as source data to assess the geometry of objects purport-
ed to be from the 02 July 1937 crash site of Amelia Earhart’s
lost airplane, a Lockheed Electra Model 10E, construction
number 1055, off of the island of Nikumaroro in the western
Jay M. Vincelli is with Materials Science Associates, 315
Commerce Park Rd., Unit 1, North Kingstown, RI 02893 (jvin-
).
Fatih Calakli, Michael A. Stone, and John D. Jarrell are with
Materials Science Associates, 315 Commerce Park Rd., Unit 1,
North Kingstown, RI 02893; and Brown University, 182 Hope
St., Providence, RI 02912.
Graham E. Forrester is with the University of Rhode Island,
Center for Biotechnology and Life Sciences, 120 Flagg Road,
Kingston, RI 02881.
Timothy Mellon is an independent consultant.
Photogrammetric Engineering & Remote Sensing
Vol. 82, No. 3, March 2016, pp. 223–232.
0099-1112/16/223–232
© 2016 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.82.3.223
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
March 2016
223
