solution of the minimization problem inf
u
F
(
u
). The gradient
of this energy is given by
∇
F
(
u
) = −
á
(
∇
2
u
)
η
,
η
ñ
= −
u
η
η
, where
u
η
η
stands for the second order derivative of
u
with respect to
η
and
∇
2
u
for the Hessian matrix. The minimization of F can
be done by gradient descent. If we cast it into a continuous
framework, we end up with the following equation to solve
our disocclusion problem:
∂
∂
− =
( )
×
=
(
)
=
(
)
u
t
R
u
T
u t
x y u x y
ηη
0 in
in
0
0
,
, ,
,
Ω
Ω
(4)
using the notations introduced earlier. We recall that
∆
u
=
u
η
η
+
u
η
T
η
T
, where
η
T
stands for a unitary vector orthogonal to
η
.
Thus, Equation 4 can be seen as an adaptation of the Gaussian
diffusion Equation 3 with respect to the diffusion prior in the
direction
η
.
Figure 10 shows a comparison between the original Gauss-
ian diffusion algorithm and our modification. The Gaussian
diffusion leads to an over-smoothing of the scene, creating an
aberrant surface, whereas our modification provides a result
that is more plausible.
The equation proposed in (4) can be solved iteratively. The
number of iterations simply depends on the size of the area
that needs to be filled in.
Results and Analysis
In this part, the results of the segmentation of various objects
and the disocclusion of their background are detailed.
Sparse Point Cloud
A first result is shown in Figure 12. This result is obtained
for a sparse point cloud (
≈
10
5
pts) of the KITTI database. A
pedestrian is segmented out of the scene using our proposed
segmentation technique (using the parameters previously in-
troduced) and a manual selection of the corresponding label.
This is used as a mask for the disocclusion of its background
using our modified variational technique for disocclusion.
Figure 12a shows the original range image. In Figure 12b, the
dark region corresponds to the result of the segmentation step
for the pedestrian.
For practical purposes, a very small dilation is applied to
the mask (radius of 2px in sensor topology) to ensure that no
outlier points (near the occluder’s silhouette with low accuracy
or on the occluder itself ) bias the reconstruction. Finally, Fig-
ure 12c shows the range image after the reconstruction. We can
see that the disocclusion performs very well as the pedestrian
has completely disappeared and the result is visually plausible
in the range image. Notice how the implicit sensor topology of
the range image has allowed here to use a standard 2D image
processing technique from mathematical morphology to filter
mislabeled and inaccurate points near silhouettes.
In this scene,
η
has a direction that is very close to the
x
axis of the range image and the 3D point cloud is acquired us-
ing a 3D LiDAR sensor. Therefore, the coherence of the recon-
struction can be checked by looking how the acquisition lines
are connected. Figure 13 shows the reconstruction of the same
scene in three dimensions. This reconstruction simply consists
(a) (b) (c)
Figure 10. Comparison between disocclusion algorithms. (a) is the original point cloud (white points belong to the object to be
disoccluded), (b) the result after Gaussian diffusion and (c) the result with our proposed algorithm (1500 iterations). Note that the
Gaussian diffusion oversmoothes the background of the object whereas our proposed model respects the coherence of the scene.
(a)
(b)
Figure 11. (a) is the definition of the different frames
between the LiDAR sensor (x
L
, y
L
, z
L
) and the real world (x
W
,
y
W
, z
W
), (b) is the definition and the visualization of
η
.
(a) (b) (c)
Figure 12. Result of disocclusion on a pedestrian on the
KITTI
database (Geiger
et al.
, 2013). (a) is the original
range image, (b) the segmented pedestrian (dark), (c) the
final disocclusion. Depth scale is given in meters. After
disocclusion, the pedestrian completely disappears from
the image, and its background is reconstructed accordingly
to the rest of the scene.
372
June 2018
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
