The main contribution of this paper is a novel methodol-
ogy for point cloud processing by exploiting the implicit
topology of various LiDAR sensors that can be used to infer
a simplified representation of the LiDAR point cloud while
bringing spatial structure between points. The utility of such
a methodology is here demonstrated by two applications.
First, a fast segmentation technique for dense and sparse
point clouds to extract full objects from the scene is presented
(Figure 1, center). Then, we introduce a fast and efficient
variational method for the disocclusion of a point cloud us-
ing the range image representation while taking advantage
of a horizontal prior without any knowledge of the color or
texture of the represented objects (Figure 1, right).
This paper is an extension of Biasutti
et al
. (2017) with
improved technical details of the methodology as well as a
complete validation of the proposed applications and a dis-
cussion about its limitations.
The paper is organized as follows: after a review on the state-
of-the-art of the two application scenarios, we detail how the
topology of various sensors can be exploited to turn a regular Li-
DAR point cloud into a range image. In the next Section, a point
cloud segmentation model using range images is introduced
with corresponding results and a validation on several datasets.
Then, a disocclusion method for point clouds is presented, as
well as results and validation on various datasets. Finally con-
clusions are drawn and potential future work is identified.
Related Works
The growing interest for
MMS
over the past decade has lead
to many works and contributions for solving problems that
could be tackled using range images. In this part, we present a
state-of-the-art on both segmentation and disocclusion.
Point Cloud Segmentation
The problem of point cloud segmentation has been extensive-
ly addressed in the past years. Three types of methods have
emerged: geometry-based techniques, statistical techniques,
and techniques based on simplified representations of the
point cloud.
Geometry-Based Segmentation
The first well-known method in this category is region- grow-
ing where the point cloud is segmented into various geomet-
ric shapes based on the neighboring area of each point (Huang
and Menq, 2001). Later, techniques that aim at fitting primi-
tives (cones, spheres, planes, cubes etc.) in the point cloud
using
RANSAC
(Schnabel
et al
., 2007) have been proposed.
Others look for smooth surfaces (Rabbani
et al
., 2006). Al-
though these methods do not need any priorknowledge about
the number of objects, they often suffer from over-segmenting
the scene resulting in objects segmented in several parts.
Semantic Segmentation
The methods in this category analyze the point cloud char-
acteristics (Demantke
et al
., 2011; Weinmann
et al
., 2015;
Landrieu
et al
., 2017). They analyze the geometric neighbor-
hood of each point in order to perform a point-wise classifi-
cation, possibly with spatial regularization, which, in turn,
yields a semantic segmentation. It leads to a good separation
of points that belong to static and mobile objects, but not to
the distinction between different objects of the same class.
Simplified Model for Segmentation
MMS
LiDAR point clouds typically represent massive amounts
of unorganized data that are difficult to handle. Different
segmentation approaches based on a simplified representa-
tion of the point cloud have been proposed. Papon
et al
. (2013)
propose a method in which the point cloud is first turned into
a set of voxels which are then merged using a variant of the
SLIC
algorithm for super-pixels in 2D images (Achanta
et al
., 2012).
This representation leads to a fast segmentation but it might fail
when the scale of the objects in the scene is too different. Geh-
rung
et al
. (2017) propose to extract moving objects from MLS
data by using a probabilistic volumemetric representation of the
MLS data in order to cluster points between mobile objects and
static objects. However this technique can only be used with 3D
sensors. Another simplified model of the point cloud is pre-
sented by Zhu
et al
. (2010). The authors take advantage of the
implicit topology of the sensor to simplify the point cloud in or-
der to segment it before performing classification. The segmen-
tation is done through a graph-based method as the notion of
neighborhood is easily computable on a 2D image. Although the
provided segmentation algorithm is fast, it suffers from the same
issues as geometry-based algorithms such as over-segmentation
or incoherent segmentation. Finally, an approach for urban ob-
jects segmentation using elevation images is proposed in Serna
and Marcotegui (2014). There, the point cloud is simplified by
projecting its statistics onto a horizontal grid. Advanced mor-
phological operators are then applied on the horizontal grid and
objects are segmented using a watershed approach. Although
this method provides good results, the overall precision of the
segmentation is limited by the resolution of the projection grid
and leads to the occurrence of artifacts at object borders.
Moreover, all those categories of segmentation techniques
are not able to treat efficiently both dense and sparse LiDAR
point clouds, i.e., point clouds acquired with high or low
sampling rates compared to the real-world feature sizes (e.g.,
macroscopic objects such as cars, pedestrians, etc.). For ex-
ample, one sensor turn in the
KITTI
dataset (Geiger
et al
., 2013)
corresponds to ×~10
5
points (sparse) whereas for a scene of
similar size in the Stereopolis-II dataset (Paparoditis
et al
.,
2012), the scene contains more than 4×10
6
points (dense). In
this paper, we present a novel simplified model for segmenta-
tion based on histograms of depth in range images by leverag-
ing grid-like topology without suffering from accuracy loss
that is often caused by projection/rasterization.
Disocclusion
Disocclusion of a scene has only been scarcely investigated
for 3D point clouds (Sharf
et al
., 2004; Park
et al
., 2005;
Becker
et al
., 2009). These methods generally work on com-
plete point clouds (with homogeneous sampling) rather than
LiDAR point clouds. This task, also referred to as inpaint-
ing, has been much more studied in the image processing
community. Over the past decades, various approaches have
emerged to solve the problem in different manners. Patch-
based methods such as the one proposed by Criminisi
et al
.
(2004) (and more recently Lorenzi
et al
. (2011) and Buyssens
et al
. (2015b)) have proven their strengths. They have been
extended for
RGB-D
images (Buyssens
et al
., 2015a) and to Li-
DAR point clouds (Doria and Radke, 2012) by considering an
implicit topology in the point cloud. Variational approaches
represent another type of inpainting algorithms (Weickert,
1998; Bertalmio
et al
., 2000; Bredies
et al
., 2010; Chambolle
and Pock, 2011). They have been extended to
RGB-D
images by
taking advantage of the bi-modality of the data (Ferstl
et al
.,
2013; Bevilacqua
et al
., 2017). Even if the results of the disoc-
clusion are quite satisfying, these models require the point
cloud to have color information as well as the 3D data. In this
work, we introduce an improvement to a variational disocclu-
sion technique by taking advantage of a horizontal prior.
Range Images Derived from the Sensor Topology
In this paper, we demonstrate that a simplified model of the
point cloud can be directly derived from it using the intrinsic
topology of the sensing pattern during acquisition. This section
introduces this sensor topology and how it can be exploited on
various kinds of sensors. Examples of its usages are presented.
368
June 2018
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
