120
February 2014
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
Runways, roads, tarmacs:
Number of samples: at least five
Length: at least 15 × NPS
Width: at least 15 × NPS
All points in all sampled areas should be included in the
calculation of the standard deviation. A single class from the
classified dataset should be used for this test.
Examining the standard deviation of the terrain elevation
from such surfaces provides a good indication of the relative
accuracy of one lidar point to another. Such a standard
deviation value according to the USGS specification should be
around 7 cm, assuming there is no elevation bias in the data.
However, examining the relative accuracy between swaths
is more complicated as it is difficult to agree on the best and
most economical way to achieve it. Lidar vendors or data
providers usually evaluate the elevation differences in the
entire overlap areas between flight lines. Such practice is
achieved through the evaluation of a color-coded raster map
built from the elevation differences, typically referred to as
the “Dz image.” The interpretation of such a map is either
achieved through visual inspection by an operator or through
an automated analytical evaluation or a hybrid between the
two. However, for end users or their consultants, examining
every single point in the overlap areas may not be practical
and can get very expensive very quickly. Statistically based
methods and sampling techniques have been proven to be
the most efficient method. In most cases, using a sample
can tell you just as much as an exhaustive testing routine.
Calculating the sample size involves a very complicated
statistical concept that may or may not work for geospatial
projects. Therefore, I suggest the following strategy:
1) Evaluate the Dz image, if it is available, to figure out the
unwanted occurrences of gross errors such as instrument
malfunction or other systematic problems.
2) Sample the data by selecting a certain number of overlap
areas between swaths. The sample size can vary by the
project and specifications; however, sampling one-third
the overlap areas between swaths but not less than five
samples may prove to be acceptable strategy. Use the
following formula to select the number of samples:
N = (n-1)/3 but not less than 5
Where,
N = is the number of samples
n = number of swaths or flight lines
3) In each of the sample overlap areas, select 20 locations
in open ground. The 20 locations are to be randomly
sampled using the bare-earth DEM to minimize the effect
of trees and buildings on statistics. The process can be
automated to reduce the manual labor required for the
selection process.
4) Evaluate the difference in elevation of the two flight lines
by examining a discrete point or along a short profile line.
5) Tabulate your results from the 20 samples and compute
RMSE.
6) Examine any sample that results in an RMSE that is
larger than three times the project-specified RMSE.
Discard it if it is due to vegetation or an explainable cause.
7) Evaluate the resulting RMSE from each sample (overlap
area). Examine the samples that results in an RMSE
higher than the threshold; in the case of the USGS
specifications it is 10 cm.
8) Calculate an RMSE from all the RMSEs of the samples.
9) Evaluate the final RMSE computed in step 8 against the
project specification.
10) Use the RMSE computed in step 8 to evaluate the final
relative accuracy against the project specifications.
As to whether to perform your assessment on the tiles base
or the swath base, I recommend the use of the individual
swaths as it prevents confusion from assuming the tiles
preserve the integrity of the final calibration of the data in
the form of swaths. LAS files for the individual swaths can
be added to the delivery; however, if such a deliverable is not
available, all bare-earth returns can be used from the tiles to
examine the relative accuracy between swaths, as the source
of each return is documented in the LAS file.
**Dr. Abdullah is Senior Geospatial Scientist at Woolpert,
Inc. He is the 2010 recipient of the ASPRS Photogrammetric
(Fairchild) Award.
The contents of this column reflect the views of the author, who is
responsible for the facts and accuracy of the data presented herein.
The contents do not necessarily reflect the official views or policies of
the American Society for Photogrammetry and Remote Sensing and/
or Woolpert, Inc.
“Statistically based
methods and sampling
techniques have been
proven to be the most
efficient method. In most
cases, using a sample can
tell you just as much as an
exhaustive testing routine.”
