VResin: Externalizing spatial memory into 3D sketch maps

Xiao T.; Kim K.G.; Krukar J.; Subramaniyan R.; Kiefer P.; Schwering A.; Raubal M.

Abstract

An intuitive way to externalize spatial memory is to sketch it. Compared to traditional paper-based sketches, virtual reality (VR) creates new opportunities to investigate the 3D aspect of spatial memory as it empowers users to express 3D information on a 3D interface directly. The goal of this study is to design a 3D sketch mapping tool for researchers and non-expert users without sketching expertise that enables externalizing memories of spatial information after some 3D-critical tasks. There exist 3D sketching tools using VR, but there are two issues with the current mid-air 3D sketching approach: (1) distortion of sketches due to depth perception errors and (2) increased cognitive and sensorimotor demands due to an increased degree of freedom and absence of physical support. To address these problems, we implemented VResin, a novel sketching interface that synergizes 3D mid-air sketching with 2D surface sketching to scaffold 3D sketching into a layer-by-layer process. An experimental study with 48 participants on multi-layer building scenarios showed that VResin supports users in creating less distorted sketches while maintaining the level of completeness and generalization compared to mid-air sketching in VR. We also demonstrate the potential applications that can benefit from 3D sketch maps and the suitability of VResin for a variety of building shapes.