Immersive and Experiential Learning Based on Multi-Technology Integration: Bringing UAVs, Augmented Reality and Virtual Reality into the Classroom

Abstract:

We introduce system concepts, and report on integration efforts, recent enhancements/expansions and results of the incorporation of advances in geospatial technologies, including unmanned aerial systems (UAS), augmented reality and virtual reality, into teaching and learning in the geosciences. System design, integration and implementation are presented as innovative tools to enhance instructional materials in classrooms. The Disruptive Geospatial Technologies Laboratory (DiGTL) and the 3D Immersion and Geovisualization (3DIG) system, implemented at the Center for Geospatial Research at the University of Georgia incorporate in-house and augmented commercial-off-the-shelf solutions for data acquisition, visualization and human-machine interaction. Through DiGTL and the immersive capabilities of 3DIG, over 6000 students, faculty and staff at UGA have been involved in data acquisition, processing, analysis and manipulation. System integration relies on data streams, with emphasis to virtual model generation/manipulation and remote sensing applications, including multispectral data acquisition, structure-from-motion based point-cloud/model generation, elevation model and texture extraction, and orthomosaics. Products are used with Geographic Information Systems (GIS), game engines (Unreal Engine and Unity), and virtual and mixed reality holographic devices to create realistic multi-scale multi-theme 3D reconstructions of planet Earth, landscapes and/or man-made objects. An augmented reality digital sandbox equipped with two depth cameras integrates 3DIG and is used to support experiential learning and experimentation based on scaled down replicas of landscapes or user-defined topographies. Integration with GIS allows for quantitative analyses, modeling and what-if scenarios, in addition to fast representation of landscape changes (near-real time response) and fluid flow over modified landscapes. The 3DIG system has been incorporated into classwork and system contributions have been evaluated. We report on the interconnected and complementary technologies of 3DIG; present lessons learned; introduce system implementation and recent integration of new components and describe system use for hands-on and immersive experiential learning.