29,689 pixels (28.2 percent) had a significant trend. Using
the Bonferroni correction, not a single pixel was found to
have a p-value <4.7578 * 10
-7
(0.05 / 105,090). Using the
FDR
of less than 5 percent (q <0.05), 9893 (9.4 percent) pixels
had a significant trend, about
⅓
of the 29,689 pixels that had
a significant trend with p <0.05. Based on chance alone, it
would be expected that around 17 percent (5,254 / 29,689) of
these significant trends would be false positives. Using a false
discovery rate of 5 percent, it is expected that only 494 (<0.5
percent total) pixels are false positive results. The difference
in the interpretation of the results is substantial; the differ-
ence in the area of found to have a significant trend between
the
FDR
and uncorrected p-values is over 1.2 million sq. km
.
Although the number of significant pixels changed sub-
stantially, the overall spatial patterns did not. The major areas
with substantial differences when
MCP
was accounted for were
within a latitudinal band from near Lake Chad east to Darfur,
Sudan, Northern Burkina Faso, and the border of Guinea and
Sierra Leone. Major areas that did become insignificant when
the multiple comparison problem was addressed included
locations in Senegal, Eritrea, and Central African Republic.
It should be noted that a higher or lower false discovery rate
may be used depending on the described tradeoff between the
chance of a false positive result and loss of potential true posi-
tive results. For example, Brown
et al
. (2012) used q <0.10.
Conclusions
Remote sensing does not embrace a single approach to quan-
tifying the relationships between remote sensing data and
observed phenomena. For example, statistical methods and
machine learning are both common. Therefore, the multiple
comparison problem does not affect every remote sensing
analysis. That said, repeated statistical tests do occur in remote
sensing for a wide range of applications. In these situations, the
multiple comparison problem must be addressed to provide
accurate interpretation of the results. The multiple compari-
son problem can greatly affect the interpretation of statistical
significance testing as demonstrated in the two case studies.
The multiple comparison problem is a fairly straightforward
concept with some easily implemented solutions available in a
number of statistical packages. This paper outlined four meth-
ods to account for the multiple comparison problem, but others
exist. Based on the results of this paper,
FDR
is suggested as it is
widely accepted across disciplines and relative easy to imple-
ment and interpret. It is strongly encouraged that all remote
sensing scientists and professionals explicitly consider the
impact of the multiple comparison problem in their analysis.
Acknowledgments
This research was funded in-part by the National Science
Foundation (award number 0927164).The Galapagos National
Park and Charles Darwin Research Station provided support
in the field for data collection. Special thanks to Stephen J.
Walsh, Conghe Song, Aaron Moody, Dean Urban, George P.
Malanson, Josh Gray, Brandon Wagner, Brian Becker, Rachel
Hackett, Russell Steele, and the anonymous reviewers for
their generous feedback on the manuscript.
References
Anyamba, A., and C.J. Tucker, 2005. Analysis of Sahelian vegetation
dynamics using NOAA- AVHRR NDVI data from 1981–2003,
Journal of Arid Environments
, 63(3):596−614.
Benjamini, Y., and Y. Hochberg, 1995. Controlling the false discovery
rate - A practical and powerful approach to multiple testing,
Journal of the Royal Statistical Society Series B- Methodological
,
57(1):289−300.
Bonferroni, C.E., 1936. Teoria statistica delle classi e calcolo delle
probabilit `a,
Pubblicazioni del R Istituto Superiore di Scienze
Economiche e Commerciali di Firenze
, 8:3−62.
Brenning, A., 2009. Benchmarking classifiers to optimally integrate
terrain analysis and multispectral remote sensing in automatic
rock glacier detection,
Remote Sensing of Environment
,
113(1):239−247: doi: 10.1016/j.rse.2008.09.005.
Brown, M.E., K. de Beurs, and A. Vrieling, 2010. The
response of African land surface phenology to large scale
climate oscillations,
Remote Sensing of Environment
,
114(10):2286−2296: doi: 10.1016/j.rse.2010.05.005.
Bruniquel-Pinel, V., and J.P. Gastellu-Etchegorry. 1998. Sensitivity
of texture of high resolution
images of forest to biophysical
and acquisition parameters,
Remote Sensing of Environment
,
65(1):61−85: doi: 10.1016/s0034-4257(98)00009-1.
Congalton, R.G., 1991. A review of assessing the accuracy of
classifications of remotely sensed data,
Remote Sensing of
Environment
, 37(1):35−46: doi: 10.1016/0034-4257(91)90048-b.
Curran-Everett, D., 2000. Multiple comparisons: Philosophies
and illustrations,
American
Journal of Physiology-Regulatory
Integrative and Comparative Physiology
, 279(1):R1−R8.
Delacourt, C., P. Allemand, E. Berthier, D. Raucoules, B. Casson, P.
Grandjean, C. Pambrun,
and E. Varel, 2007. Remote-sensing
techniques for analysing landslide kinematics: A
review,
Bulletin
De La Societe Geologique De France
, 178(2):89−100: doi:
10.2113/gssgfbull.178.2.89.
Dunn, O.J., 1961. Multiple comparison among means,
Journal of
the American Statistical Association
, 56(293):52−64: doi:
10.2307/2282330.
Dunnett, C.W., 1955. A multiple comparison procedure for comparing
several treatments with a
control,
Journal of the American
Statistical Association
, 50(272):1096−1121.
Eklundh, L., and L. Olsson, 2003. Vegetation index trends for the
African Sahel 1982–1999,
Geophysical Research Letters
,
30(8):1430−1433.
Hulme, M., R. Doherty, T. Ngara, M. New, and D. Lister, 2001. African
climate change: 1900– 2100,
Climate Research
, 17:145−168.
Gelman, A., J. Hill, and M. Yajima, 2012. Why we (usually)
don’t have to worry about multiple
comparisons,
Journal of
Research on Educational Effectiveness
, 5(2):189−211: doi:
10.1080/19345747.2011.618213.
Gray, J., and C. Song, 2012. Mapping leaf area index using spatial,
spectral, and temporal
information from multiple sensors,
Remote Sensing of Environment
, 119:173−183: doi: 10.1016/j.
rse.2011.12.016.
Groppe, D.M., T.P. Urbach, and M. Kutas, 2011. Mass univariate
analysis of event-related brain potentials/fields - I: A critical
tutorial review,
Psychophysiology
, 48(12):1711−1725: doi:
10.1111/j.1469-8986.2011.01273.x
Haralick, R.M. 1973. Glossary and index to remotely sensed image
pattern-recognition concepts,
Pattern Recognition
, 5(4):391-403:
doi: 10.1016/0031-3203(73)90029-0.
Heumann, B.W., J.W. Seaquist, L, Eklundh, and P. Jönsson, 2007.
AVHRR derived phenological change in the Sahel and
Soudan, Africa, 1982–2005,
Remote Sensing of Environment
,
108:385−392
Knudby, Anders, L. Ellsworth, and Alexander Brenning,
2010. Predictive mapping of reef fish
species richness,
diversity and biomass in Zanzibar using IKONOS imagery
and machine-learning techniques,
Remote Sensing of
Environment
,114(6):1230−1241, doi: 10.1016/j.rse.2010.01.007.
Jönsson, P., and L. Eklundh, 2002. Seasonality extraction by function
fitting to time-series of
satellite sensor data,
IEEE Transactions
on Geoscience and Remote Sensing
, 40:1824−1832.
Jönsson, P., and L. Eklundh, 2004. TIMESAT - A program for
analyzing timeseries of satellite
sensor data,
Computers &
Geosciences
, 30:833−845.
PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
December 2015
925
