Flexible Photogrammetric Computations Using
Modular Bundle Adjustment: The Chain Rule and
the Collinearity Equations
Niclas Börlin, Arnadi Murtiyoso, Pierre Grussenmeyer, Fabio Menna, and Erica Nocerino
Abstract
The main purpose of this articl
metric bundle-adjustment com
organized into modules. Furthe
used to simplify the computation of the analytical Jacobians
needed for the adjustment. Novel projection models can be
flexibly evaluated by inserting, modifying, or swapping the
order of selected modules. As a proof of concept, two vari-
ants of the pinhole projection model with Brown lens distor-
tion were implemented in the open-source Damped Bundle
Adjustment Toolbox and applied to simulated and calibra-
tion data for a nonconventional lens system. The results
show a significant difference for the simulated, error-free,
data but not for the real calibration data. The current flex-
ible implementation incurs a performance loss. However,
in cases where flexibility is more important, the modular
formulation should be a useful tool to investigate novel sen-
sors, data-processing techniques, and refractive models.
Introduction
Background
In photogrammetry and computer vision, bundle adjustment
(
BA
) is a fine-tuning estimation process where some or all pa-
rameters that describe a scene are estimated simultaneously.
It was introduced to photogrammetry some 60 years ago (D. C.
Brown 1958). The technique became popular in the computer-
vision community after an article by Triggs, McLauchlan,
Hartley, and Fitzgibbon (2000).
Photogrammetric
BA
uses the collinearity equations to de-
scribe the world-to-camera projection. The typical presenta-
tion of the function and its linearization has been as a single,
compact computation (see, e.g., Kraus 1993, section 5.3.2;
Wolf and Dewitt 2000, appendix D-4; Mikhail, Bethel, and
McGlone 2001, appendix C.3; Luhmann
et al.
2014, section
4.2.2). In contrast, the standard presentation of the world-
to-camera projection in the computer-vision community has
been as a sequence of small operations (Tsai 1987) with the
Jacobians described using the chain rule (M. Brown and Lowe
2003; Fusiello 2013). In both presentations, the lens distortion
is treated separately. Although both research communities
refer to the same D. C. Brown (1971) article, the details of the
lication have differed.
ince its introduction,
BA
has evolved to be a central algo-
m in photogrammetry and computer vision (D. C. Brown
1976; Triggs
et al.
2000; Luhmann
et al.
2014). Originally,
the procedure was used by a small research community that
understood the algorithm in detail. With a broadening user
base, the fraction of users with an intimate understanding of
the inner workings of
BA
has declined. This development has
further increased with the inclusion of
BA
into commercial,
black-box, software.
Related Work
The Damped Bundle Adjustment Toolbox (
DBAT
) was origi-
nally developed to illustrate the behavior of different damp-
ing schemes known from nonlinear optimization (Börlin and
Grussenmeyer 2013, 2014). Later work has focused on the use
to validate commercial black-box photogrammetric software—
e.g., PhotoModeler and PhotoScan (Börlin and Grussenmeyer
2016)—especially to provide detailed diagnostics of the
photogrammetric network (Dall’Asta
et al.
2015; Murtiyoso,
Grussenmeyer, and Börlin 2017; Murtiyoso, Grussenmeyer,
Börlin, Vandermeerschen, and Freville 2018). Recently, the
differential observation-weighting capabilities of
DBAT
were
used by Menna
et al.
(2018) to mitigate peripheral image-
quality degradation in underwater photogrammetry.
The chain rule has been used to simplify geometrical com-
putations in, e.g., computer vision (M. Brown and Lowe 2003;
Fusiello 2013), computer graphics (Barr 1984; Piponi 2004), im-
aging (Gallego and Yezzi 2015), robotics (Koditschek and Rimon
1990), and registration of terrestrial-laser-scanning point clouds
(Rabbani, Dijkman, van den Heuvel, and Vosselman 2007). In
photogrammetry, it has been used for, e.g., modeling of reflected
ray paths (Rupnik, Jansa, and Pfeifer 2015) and relative pose es-
timation (Cheng, Hao, and Li 2017). To our knowledge, the chain
rule has not previously been used to express the photogrammet-
ric world-to-camera projection model, including lens distortion.
Aim
The aim of this article is to apply the sequential computation
to the whole photogrammetric projection model, including
lens distortion, and to use the chain rule to modularize the
computations. Furthermore, to illustrate the flexibility of the
modular approach, a nonconventional optical system—a tilt-
shift lens—will be investigated.
A first version of this approach was presented by Börlin,
Murtiyoso, Grussenmeyer, Menna, and Nocerino (2018). The
current article is an extended version with an added empha-
sis on performance versus flexibility.
Niclas Börlin is with the Department of Computing Science,
Umeå University, Umeå, Sweden (
).
Arnadi Murtiyoso and Pierre Grussenmeyer are with the
Photogrammetry and Geomatics Group, ICube Laboratory
UMR 7357, INSA Strasbourg, France.
Fabio Menna is with the 3D Optical Metrology (3DOM) unit,
Bruno Kessler Foundation (FBK), Trento, Italy; and COMEX SA–
Innovation Department, COMEX, CS 80143, Marseille, France.
Erica Nocerino is with the LIS, I&M Team, Aix-Marseille
Université, Polytech Luminy, Marseille, France; and the Institute
of Theoretical Physics, ETH Zurich, Zurich, Switzerland.
Photogrammetric Engineering & Remote Sensing
Vol. 85, No. 5, May 2019, pp. 361–368.
0099-1112/18/361–368
© 2019 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.85.5.361
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
May 2019
361
