Results
To reflect the precision associated with both surveys, results
are rounded to two decimal places throughout this section.
In order to assess the accuracy of the resulting orthoimage
mosaics and
DEM
s, we first examined the discrepancy between
the field-measured coordinates and those derived from the
photogrammetrically-processed data. The results are shown
graphically using kernel density plots of the difference along
each axis line (Figure 2). Both plots were generated using a
Gaussian kernel and a bandwidth of 0.02
.
For the Evan Thomas survey, the X and Y errors were de-
rived from the 33 independent horizontal checkpoints. The er-
rors in X and Y were both found to be normally-distributed, ac-
cording to Shapiro-Wilk normality testing, with the distribution
clustering around zero in both cases. The
RMSE
x
and
RMSE
y
were
0.04 m and 0.03 m, respectively, giving an
RMSE
r
of 0.05 m, with
the average error being -0.01 m in X and 0.00 m in Y. Errors in Z
were derived from the 167 vertical checkpoints, none of which
were coincident with the horizontal checkpoints. The errors in
Z were also normally-distributed, peaking around -0.05 m, with
an
RMSE
z
of 0.06 m, and the average error being -0.02 m
.
For the Sunwapta survey, horizontal error was assessed us-
ing the 30 marked horizontal checkpoints. The X and Y kernel
density plots both showed bimodal distributions, with the peak
in X being at around -0.02 m, while the errors in Y showed a
positive trend, peaking at around 0.02 m. The bimodal shape of
the X and Y plots is not believed to be significant, but is most
likely due to the low number of sample points.
RMSE
x
and
RM-
SE
y
were both calculated to be 0.06 m, giving an
RMSE
r
of 0.08
m, with the average errors being -0.01 m in X and 0.01 m in Y.
Errors in Z were derived from the 172 vertical checkpoints, of
which 30 were coincident with the marked horizontal check-
points. Z errors were also normally distributed, with the peak
having a small positive offset of approximately 0.01 m.
RMSE
z
was 0.03 m, with the average error being 0.00 m.
Horizontal and Vertical Accuracy Classification Under ASPRS 1990
The surveys described provide examples of the types of
projects for which small
UASs
are ideally suited. The high
spatial resolution of the source photos is a direct consequence
of the low flying height, and this resolution made it possible
to create 3.5 cm resolution
DEM
s and orthoimage mosaics for
both surveys. This provided two baseline data sets to evaluate
using
ASPRS
accuracy criteria
.
Under
ASPRS
1990, the allowable
RMSE
x
or
RMSE
y
for Class
1 accuracy at a scale of 1:500 is 0.125 m, giving an allowable
RMSE
r
of 0.177 m. For the Evan Thomas survey, the
RMSE
r
as
determined from the 33 independent checkpoints was 0.05
m, falling well within the Class 1 category. For the Sunwapta
survey, the
RMSE
r
of 0.08 m over 30 checkpoints also fell well
within Class 1 accuracy limits for mapping at 1:500
.
Vertical accuracy for the Evan Thomas survey was assessed
using the 167 measured vertical checkpoints, giving an
RMSE
z
of 0.06 m. Under
ASPRS
1990, the allowable vertical error for
Class 1 accuracy is one third of the contour interval. This
RMSE
z
is therefore sufficient to meet Class 1 accuracy for 0.2 m
contours. For the Sunwapta survey, the
RMSE
z
calculated for
the 172 checkpoints was 0.03 m, which would be sufficient to
meet Class 1 accuracy levels for 0.1 m contours.
Horizontal and Vertical Accuracy Classification Under ASPRS 2015
ASPRS
2015 defines the horizontal accuracy class in terms of
RMSE
x
or
RMSE
y
, rather than relative to the pixel size of the de-
rived orthoimage (as in the draft published in December 2013),
or the specified map scale (as in
ASPRS
1990). For the Evan
Thomas survey,
RMSE
x
and
RMSE
y
values were 0.04 m, and 0.03
m, respectively. Referring to the list of common classes defined
by
ASPRS
2015 and listed in Table 1, this places the horizontal
accuracy in the 5 cm
RMSE
x
or
RMSE
y
class. For the orthoimage
mosaic produced for Sunwapta, the corresponding
RMSE
x
and
RMSE
y
values were both 0.06 m, placing accuracy in the 7.5 cm
RMSE
x
or
RMSE
y
class. It should be stressed that under
ASPRS
2015, horizontal accuracy classes are not rigidly defined, but
are rather specified in terms of the
RMSE
x
and
RMSE
y
appropri-
ate to the specific survey. However because surveys from small
UASs
are still comparatively new, there is no generally-accept-
ed understanding of what levels of accuracy are appropriate.
The common classes defined in Table 1 were therefore used to
determine horizontal accuracy for both surveys
.
ASPRS
2015 requires that the surveyed positions of horizon-
tal
GCP
s have
RMSE
x
and
RMSE
y
values that are less than a quar-
ter of the
RMSE
x
or
RMSE
y
values used to define the appropriate
accuracy class for the final orthoimage mosaic. Checkpoints
also need to have
RMSE
x
or
RMSE
y
e
r
rors which are less than
a third of those defining the accuracy class. For both studies,
the X, Y, and Z accuracy of the
GNSS
control surveys was esti-
mated at
≤
0.02 m, which means that the appropriate accuracy
class is ultimately determined by the accuracy of the control
survey. This would place both orthoimage mosaics in the 10
cm
RMSE
x
or
RMSE
y
class, as defined by Table 1
.
ASPRS
2015 replaces the contour interval with an absolute
value for vertical RMS error as shown in Table 2. In the case
of the
DEM
produced for the Evan Thomas survey,
RMSE
z
was
calculated to be 0.06 m. According to
ASPRS
2015, this cor-
responds to an
NVA
of 0.12 m, which would mean that the
survey meets the vertical requirements for mapping of non-
vegetated areas as defined by the 10 cm
RMSE
z
class shown in
Table 2. The accuracy requirements for
GCP
s and checkpoints
would also place the vertical accuracy in the 10 cm
RMSE
z
class. For Sunwapta, the
RMSE
z
of 0.03 m corresponds to an
NVA
of 0.06 m, which would suggest that the 5 cm
RMSE
z
class
would be appropriate. However for Sunwapta, the limiting
factor is likely to be the accuracy of the
GCP
and checkpoint
survey, with the result that the appropriate level of vertical
accuracy would again be defined by the 10 cm
RMSE
z
class
shown in Table 2. As with the horizontal accuracy classes,
vertical accuracy classes under
ASPRS
2015 are not rigidly
defined. However, in the absence of a generally-accepted un-
derstanding as to what levels of vertical accuracy are appro-
priate with small
UASs
, both surveys were assessed using the
common classes for vertical accuracy listed in Table 2.
Discussion and Conclusions
This project highlights the implications of changes in mapping
standards in the context of
UAS
data. When
ASPRS
1990 was
drafted, it was rare for surveys at scales larger than 1:500 to be
carried out photogrammetrically. Mapping at such scales would
most commonly be carried out using traditional ground surveys,
and would usually involve the production of physical printed
plans. In the digital landscape of today, new technologies, such
as
GPS
, ground-based lidar and small
UASs
have revolutionized
medium and large-scale mapping. This trend has been comple-
mented by a revolution in data-processing technology, with the
development of software packages to streamline the photogram-
metric workflow, allowing the production of
DEM
s and orthoim-
age mosaics to be carried out in a series of logical steps. Recent
developments in structure from motion software have also
made it easier to accommodate the highly variable image geom-
etry associated with low-resolution consumer cameras, allowing
DEM
s and orthoimages to be produced quickly, without the level
of specialized knowledge formerly required
.
It is apparent that
ASPRS
2015 has been developed to ac-
commodate a greater range of survey accuracies than was
previously available under
ASPRS
1990.
ASPRS
2015 avoids
specifically tying accuracy classification to pixel size or map
scale, and instead expresses accuracy in terms of the
RMSE
x
,
RMSE
y
, or
RMSE
z
deemed appropriate for the specific survey.
This approach is a recognition of the flexibility of modern sur-
vey methods, and also of the fact that high spatial resolution
792
October 2015
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
