A Learning Approach to Evaluate
the Quality of 3D City Models
Oussama Ennafii, Arnaud Le Bris, Florent Lafarge, and Clément Mallet
Abstract
The automatic generation of three-dimensional (3D) building
models from geospatial data is now a standard procedure.
An abundance of literature covers the last two decades, and
several solutions are now available. However, urban areas
are very complex environments. Inevitably, practitioners still
have to visually assess, at a city-scale, the correctness of these
models and detect frequent reconstruction errors. Such a
process relies on experts and is highly time-consuming, with
approximately two hours/km
2
per expert. This work proposes
an approach for automatically evaluating the quality of 3D
building models. Potential errors are compiled in a novel
hierarchical and versatile taxonomy. This allows, for the first
time, to disentangle fidelity and modeling errors, whatever
the level of details of the modeled buildings. The quality of
models is predicted using the geometric properties of build-
ings and, when available, Very High Resolution images
and Digital Surface Models. A baseline of handcrafted, yet
generic, features is fed into a Random Forest classifier. Both
multiclass and multilabel cases are considered: due to the
interdependence between classes of errors, it is possible to
retrieve all errors at the same time while simply predicting
correct and erroneous buildings. The proposed framework
was tested on three distinct urban areas in France with more
than 3000 buildings. 80%–99% F-score values are attained
for the most frequent errors. For scalability purposes, the
impact of the urban area composition on the error predic-
tion was also studied, in terms of transferability, generaliza-
tion, and representativeness of the classifiers. It showed the
necessity of multimodal remote sensing data and mixing
training samples from various cities to ensure a stability of
the detection ratios, even with very limit
Introduction
Context and Objectives
Three-dimensional (3D) urban models have a wide range of
applications. They can be used for consumer purposes (video
games or tourism) as much as they can be vital in more criti-
cal domains with significant societal challenges (e.g., disaster
control, run-off water, or microclimate simulation, urban
planning, or security operations preparation) (Musialski
et al.
2012; Biljecki
et al.
2015). Therefore, automatic urban recon-
struction from geospatial imagery (spatial/airborne sensors)
focuses efforts on both scientific research and industrial activi-
ties. 3D city modeling has therefore been deeply explored in
the photogrammetric, geographic information systems, com-
puter vision, and computer graphics literature with an empha-
sis on compactness, full automation, robustness to acquisition
constraints, scalability, inevitably, and quality (Müller
et al.
2006; Over
et al.
2010; Vanegas
et al.
2010; Lafarge and Mallet
2012; Poli and Caravaggi 2013; Stoter
et al.
2013; Zhou and
Neumann 2013; Cabezas
et al.
2015; Monszpart
et al.
2015;
Kelly
et al.
2017; Nguatem and Mayer 2017). However, the
problem remains partly unsolved (Sester
et al.
2011; Musialski
et al.
2012; Rottensteiner
et al.
2014). In fact, besides the seam-
less nature of reconstituted models, current algorithms lack of
generalization capacity. They cannot handle the high heteroge-
neity of urban landscapes. As such, for operational purposes,
human intervention is needed either in interaction within
the reconstruction pipeline or as a postprocessing refinement
and correction step. The latter is highly tedious: it consists in
an individual visual inspection of buildings (Musialski
et al.
2012). Consequently, automatizing the last step remains, for
all stakeholders (from researchers up to end-users), a criti-
cal step, especially in a production environment. It has been
barely investigated in the literature. This paper addresses this
issue by expanding earlier work (Ennafii
et al.
2019).
Qualifying 3D Building Models
Our work focuses on assessing polyhedral structured 3D mod-
els, representing building architectures (Haala and Kada 2010).
These models result from a given urban reconstruction meth-
od, that is unknown from our evaluation pipeline. We discard
triangle meshes that are standardly generated from multiview
images or point clouds with state-of-the-art surface reconstruc-
tion methods. Here, the studied objects are, by design, more
compact but less faithful to input data. In counterpart, they
hold more semantic information: each polygonal facet typical-
ly corresponds to a façade, a roof, or any other architecturally
atomic
feature of a building. 3D modeling algorithms tradi-
tionally build a trade-off between representation compactness
ut data (meshes or 3D points).
patial accuracy, the urban setting, and
n, the reconstituted result achieves
il (
LoD
) (Kolbe
et al.
2005). An
LoD
-1
model is a simple building extrusion (flat roof) (Ledoux and
Meijers 2011; Biljecki
et al.
2017). An
LoD
-2 model considers
geometric simplification of buildings, ignoring superstruc-
tures such as dormer windows and chimneys (Taillandier and
Deriche 2004). These are taken into account in
LoD
-3 (Brédif
et
al.
2007). The
LoD
rational is still open for debate (Biljecki
et
al.
2016b). Nevertheless, in this paper, we will follow the
LoD
categorization introduced above, which is also standard in the
computer vision and graphics literature.
A large body of papers has addressed the 3D building
modeling issue and subsequently tried to find the trade-off
between fidelity and compactness (Dick
et al.
2004; Zebedin
et al.
2008; Lafarge
et al.
2010; Verdié
et al.
2015). Conversely,
few works investigate the issue of assessing the quality of the
derived models, especially out of a given reconstruction pipe-
line (Schuster and Weidner 2003). Usually, quality assessment
Oussama Ennafii, Arnaud Le Bris, and Clément Mallet are
with the Univ. Paris-Est, LaSTIG STRUDEL, IGN, ENSG,
F-94160 Saint-Mandé, France.
Oussama Ennafii and Florent Lafarge are with INRIA, Titane,
06902 Sophia Antipolis, France.
Photogrammetric Engineering & Remote Sensing
Vol. 85, No. 12, December 2019, pp. 865–878.
0099-1112/19/865–878
© 2019 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.85.12.865
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
December 2019
865
