Remote Deictic Communication: Simulating Deictic Pointing Gestures across Distances Using Electro Muscle Stimulation

Medrano SN, Pfeiffer M, Kray C

Abstract

Deictic expressions – such as uttering ”this tool” and pointing at a hammer – are important and frequent elements of everyday communication. They are used, for example, when conveying location and orientation information, or when identifying objects. Pointing gestures exist among all known cultures, languages and ethnicities, but they depend on parties to be collocated. When communicating across distances, deictic expressions are usually lost. In this paper, we introduce an approach for communicating deictic pointing across distances by simulating the sender hand posture in the receiver’s hand using electrical muscle stimulation (EMS). We describe our prototypical implementation of this approach and report on a lab-based user study evaluating it. Our results indicate that it is technically feasible to use EMS for communicating deictic pointing gestures across distances. Participants in our study were successful in resolving deictic expressions consisting of verbal and gestural components. Furthermore, our results provide initial insights into user behavior, user satisfaction and specific challenges in this context. Our contributions can inform the design of future interfaces for naturally transmitting pointing gestures across distances.