As mentioned, the low-cost
RGB-D
sensor is inferior to the
terrestrial laser scanner in terms of data quality. Thus, a noise
and outlier removal strategy (Wolff
et al.
2016) should be ap-
plied to the obtained point cloud model to allow the 3D map-
ping results (e.g., point clouds) to be generated and oriented
simultaneously. With the above method, the mapping range is
expected to reach 5–10 m in front of the sensor with typical
off-the-shelf
RGB-D
sensors (Tang
et al.
2017).
3D Mapping from RGB Images in Distant Ranges
RGB
sequences are collected simultaneously when carrying
out
SLAM
. Our work is based on an incremental
SfM
strategy
and additional promising input constraints from the
SLAM
re-
sults. Features are extracted from the input images by apply-
ing a common feature detector, such as
SIFT
(Lo and Siebert
2009), and binary features (Heinly
et al.
2012). Correspon-
dences can then be set up among the input images in the over-
laps based on the obtained features (Havlena and Schindler
2014; Johnson and Zhang 2014). A variety of approaches have
been used to address this problem,
obtaining the correspondences, the
geometric validation process to rem
dom sample consensus (
RANSAC
) (B
to improve the robustness of the obtained correspondences.
Moreover, as mentioned before, the corresponding depth
information is introduced into this offline process. Thus,
the point-to-pixel constraint (see Figure 2) can be applied as
a filter to further purify the obtained correspondences. The
algorithm provides a higher weight to correspondences with
corrected depth information. Starting from a reconstruction,
new images can be registered to the current model by solving
the Perspective-n-Point problem (Bolles and Fischler 1981)
using the gained correspondences. An efficient and robust
multiview triangulation method in
COLMAP
is implemented
in our algorithm to complete the transformation from two-
dimensional to 3D. Image registration and triangulation are
separate procedures even though their products are highly
correlated. Thus, bundle adjustment is carried out to solve a
nonlinear problem and to refine the pose-graph and orienta-
tions by minimizing the re-projection error
E
X P p
j
j k j
=
j
µ π
,
,
(
)
−
∑
2
2
(1)
where
π
is a function to transform a 3D point to image space,
μ
j
is a weight factor,
X
j
is the camera pose,
p
j
∈
R
2
is a pixel co-
ordinate in the image space, and
P
k
∈
R
3
is the corresponding
3D position in the object space. Based on the point-to-pixel
information, we add the following additional constraint:
α
P P x x
k j
k j
− = −
2
2
2
2
,
(2)
where
α
is a scale factor,
x
k
is a 3D coordinate from the point-
to-pixel, and
P
k
is its corresponding 3D coordinate from the
triangulation. As Schonberger and Frahm (2016) mentioned,
the additional steps of retriangulation and global
BA
can obvi-
ously improve the
SfM
results because
BA
is severely affected
by outliers and can be subsequently filtered.
After obtaining a sparse 3D model by implementing the
above process, we carry out an
MVS
process using
COMLAP
to
obtain the dense 3D image-based model. It should be noted
that although the
RGB
and
IR
cameras were calibrated before
the mapping task and the depth information can be obtained
in close ranges,
SfM
is a necessary step and cannot be skipped
before
MVS
. This is because the
SfM
provides sparse point
clouds to carry out
MVS
in distant ranges beyond the working
ranges of the depth sensor, and the global optimization of
SfM
can further provide more accurate geometric solutions.
Scale-Adaptive Registration of Point Clouds
Although the depth information from
SLAM
is introduced
e 3D model, the scale may not be
e two-point clouds due to data
r of points used. Thus, we need to
age-based model, register it in the
SLAM
model, and then further produce enhanced 3D mapping
results.
Point cloud registration is a key step in generating a com-
plete 3D model with a straightforward solution (i.e.,
ICP
) (Besl
and McKay 1992). However, in our case, we cannot directly
use
ICP
or even Scaled-
ICP
(Du
et al.
2007) to register the point
clouds because there are no initial values to start
ICP
or other
fine registration strategies (Ge and Wunderlich 2016). Specifi-
cally, although some
SLAM
points are introduced into the
SfM
solution to provide the scale estimation and an initial align-
ment, due to the data noise and limited distribution of those
points, there is no guarantee that the obtained alignment has
favorable quality. Therefore, in order to ensure the reliability
of the proposed approach, a scale-adaptive coarse registration
strategy is employed to provide more accurate transforma-
tion parameters between the two-point clouds. After that, a
Scaled-
ICP
is implemented for accurate coregistration of them.
We carry out an extended feature-based four-point congruent
sets (
4PCS
) method (Ge 2017; Aiger
et al.
2008) to minimize
the geometric errors
E
s
t
i
i
i
4
2
1
4
=
- ⋅
(
)
=
∑
V
,
(3)
where V is a transformation parameter set that includes the
rotation, translation, and scale and (
s
i
,
t
i
) represents a cor-
respondence from two-point clouds. To validate the trans-
formation parameter set obtained by Equation 3, a validation
Figure 2. An example of the depth image (left) and the corresponding
RGB
image (right).
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
September 2019
635
