PHOTOGRAMMETRIC ENGINEERING & REMOTE SENSING
November 2015
825
PROFESSIONAL
INSIGHT
Dr. Oliver Kreylos is a researcher with
the UC Davis W.M. Keck Center for
Active Visualization in the Earth Sciences
(KeckCAVES). Oliver received an M.Sc.
in Computer Science from the Karlsruhe
Institute of Technology, Germany, in
1999, and an M.Sc. and a Ph.D. in
Computer Science from the University
of California, Davis, in 2001 and 2003,
respectively.
Oliver has been researching scientific
applications of virtual reality (VR) since
1998, and is the primary developer of most KeckCAVES software. His
related research interests are low-cost or commodity VR hardware,
and development of software infrastructure to create widely portable
VR applications. He developed the Augmented Reality Sandbox as
part of an NSF-funded informal science education project.
I N T E R V I E W
What is the Augmented Reality Sandbox and why was
it designed?
The Augmented Reality Sandbox, or AR Sandbox, is a
self-contained augmented reality system that combines a
physical input modality, in this case a box full of sand,
with a virtual topographic model of the sand surface,
which in turn is projected back onto the sand to create a
dynamic topographic map.
We developed the AR Sandbox as part of an NSF-fund-
ed project to teach lake science concepts in an informal
science education context. Concretely, we saw a video
showing a similar AR Sandbox idea developed by a group
of Czech students, immediately realized how well a more
advanced version would fit the overall goals of our project,
and set out to design and implement such a version.
What is the technology based on?
The AR Sandbox is based on a closed feedback loop con-
sisting of sand, a 3D scanner (in this case, an off-the-shelf
Microsoft Kinect-for-Xbox-360 game controller), a data
projector, and custom software. As users manipulate the
sand surface, the 3D scanner detects the current topog-
raphy of the surface, feeds it into a virtual terrain mod-
el, generates a colorized topographic map with elevation
contour lines, simulates the flow of virtual water over the
virtual surface, and projects the results back onto the sand
surface, calibrated in such a way that projected virtual
features exactly match physical features in the sandbox.
Has mainstream technology, such as X-box Kinect and
in other cases a Wii Remote, advanced VR technology?
If so, how?
Absolutely. Before the current mainstream “VR renais-
AUGMENTED REALITY SANDBOX
sance” started by the highly successful Oculus Rift Kick-
starter project about three years ago, VR was a niche en-
deavor requiring specialized hardware, which, as a result
of the very small market volume, was very expensive.
Mainstream technology not originally intended for VR,
such as Nintendo’s Wiimote, Sony’s Playstation Move con-
troller, Microsoft’s Kinect, 3D televisions and 3D projec-
tors, have allowed us to create and experiment with low-
cost VR systems by misappropriating such mainstream
technology. Our first such low-cost device consisted of a
standard 3D TV, an optical motion capture system intend-
ed for indie movie production, and a Wiimote.
In a larger sense, the current VR renaissance itself has
been enabled by mainstream technology, primarily cheap
high-resolution smartphone displays and sensor packages.
How has this assisted in teaching Earth sciences?
Primarily by making educational VR software available to
a wider audience. We realized early on that VR is a very
powerful teaching tool for the same reasons it is a very
powerful research tool. However, the cost of traditional
VR displays severely limited the number of people that
could be exposed to it. Using commodity technology, VR
displays can now be installed in smaller laboratories and
even classrooms.
Photogrammetric Engineering & Remote Sensing
Vol. 81, No. 11, November 2015, pp. 825–830.
0099-1112/15/825–830
© 2015 American Society for Photogrammetry
and Remote Sensing
doi: 10.14358/PERS.81.11.825
